fmpeg h265 and banding / blocking artefacts on dark scenes

[alex.brown111@hotmail.com]

Hi,

I have tried to compare the result of h265 and h264 codec and while h265 is much better in terms of quality and size reduction, unfortunatenly in my test introduce much banding and block on dark gradient.

Here some commands that I have tried:

(h264)
ffmpeg -y -i title00.mkv -an -t 80 -b:v 4M ouput_h264_no_special_options.mkv

This command produce a poor quality result and intruduce grain, but no banding

(h265)

ffmpeg -y -vsync 0 -i title00.mkv -an -sn -c:v:0 hevc_nvenc -rc:v:0 vbr -cq 27 -qmin 22 -qmax 40 -b:v:0 10M -maxrate:v:0 15M -gpu 0 -vf format=yuv420p10le -rc-lookahead 32 -spatial-aq 1 -aq-strength:v 15 -level:v 4.1 -t 80 -tune hq output_banding.mkv

The quality is much better but introduce banding on dark scene expecially with grey gadient. If I decrease the -cq to 22 the banding persist and seem a bit more evident.

(h265)

ffmpeg -y -i title00.mkv -c:v libx265 -an -aq-mode 3 -aq-strength 16 -deblock -2:-2 -g 120 -keyint_min 120 -t 80 -b:v 4M ouput_h265.mkv

Very very slow with the some banding artefacts no more no less.

My impression is that the h265 have this side effect by default.

If I reduce the gamma the banding is less visible, but the the video is more dark.

I have tried different noise redection via ffmpeg without get good result.

Someone have similar experience about this ?

There is a way to reduce/remove the banding on h265 ?


Thanks !

[microchip8]

10-bit x265 helps a bit in this case. Also, try setting psy-rd to 4 and psy-rdoq to 15. Report back if you still get banding

[alex.brown111@hotmail.com]

10-bit x265 helps a bit in this case. Also, try setting psy-rd to 4 and psy-rdoq to 15. Report back if you still get banding

Many many thanks I have adden this code "-x265-params psy-rd=4:psy-rdoq=15" in my command I don't see banding !! excellent !

I see banding only when I set too match compression but this is normal.

So your suggest work:

Last question: the codec libx265 is very slow I get 0.5x (CPU) while when I use CUDA (NVIDIA GPU) I get 16x.

typically I use a command like this:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i title00.mkv -an -sn -c:v:0 hevc_nvenc -rc:v:0 vbr -cq 27 -qmin 22 -qmax 40 -b:v:0 10M -maxrate:v:0 15M -gpu 0 -rc-lookahead 32 -spatial-aq 1 -aq-strength:v 15 -force_key_frames expr:gte(t,n_forced*1) -level:v 4.1 -vf hwdownload,format=nv12,hwupload_cuda -tune hq -t 80 output_ottimale_cuda.mkv

there is a CUDA equivalent of "-x265-params psy-rd=4:psy-rdoq=15" or another way to cuda ?

Thanks !

[microchip8]

Many many thanks I have adden this code "-x265-params psy-rd=4:psy-rdoq=15" in my command I don't see banding !! excellent !

I see banding only when I set too match compression but this is normal.

So your suggest work:

Last question: the codec libx265 is very slow I get 0.5x (CPU) while when I use CUDA (NVIDIA GPU) I get 16x.

typically I use a command like this:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i title00.mkv -an -sn -c:v:0 hevc_nvenc -rc:v:0 vbr -cq 27 -qmin 22 -qmax 40 -b:v:0 10M -maxrate:v:0 15M -gpu 0 -rc-lookahead 32 -spatial-aq 1 -aq-strength:v 15 -force_key_frames expr:gte(t,n_forced*1) -level:v 4.1 -vf hwdownload,format=nv12,hwupload_cuda -tune hq -t 80 output_ottimale_cuda.mkv

there is a CUDA equivalent of "-x265-params psy-rd=4:psy-rdoq=15" or another way to cuda ?

Thanks !

GPU encoders are a magnitude of order faster than CPU ones (like x265) but deliver quality not on-par with CPU ones

x265 does not have CUDA, as I recall. You may want to lower some settings to get a better compromise between compression and speed.

The below is what I use for my day-to-day encodings (I don't rely on presets) on an AMD Ryzen 7 3700X. With the settings below, I usually get between 3.5 to 7 FPS encoding speed (depends on content)


ref=4:hme=1:hme-search=umh,umh,star:subme=4:bframes=8:rd=4:rd-refine=0:qcomp=0.65:fades=1:strong-intra-smoothing=1:ctu=32:qg-size=32:sao=0:selective-sao=0:cu-lossless=0:cutree=1:tu-inter-depth=4:tu-intra-depth=4:max-merge=2:rskip=2:rskip-edge-threshold=2:rc-lookahead=80:aq-mode=1:aq-strength=1.0:rdoq-level=1:psy-rd=4.0:psy-rdoq=15.0:limit-modes=0:limit-refs=0:limit-tu=0:deblock=-4,-4:weightb=1:weightp=1:rect=1:amp=0:wpp=1:pmode=0:pme=0:b-intra=1:b-adapt=2:b-pyramid=1:tskip=1:tskip-fast=0:fast-intra=0:early-skip=0:splitrd-skip=0:min-keyint=24:keyint=240:transfer=bt709:colorprim=bt709:colormatrix=bt709"

[benwaggoner]

--aq-mode 3 is exactly to reduce banding/blocking in dark parts of the image. It does increase average bitrate at a given CRF, though.

[microchip8]

--aq-mode 3 is exactly to reduce banding/blocking in dark parts of the image. It does increase average bitrate at a given CRF, though.

Anything above aq-mode 1 sucks. That's my experience. They're supposed reduce banding but what they do is increase bitrate for no improvement at all

[alex.brown111@hotmail.com]

Thank you !

My CPU is a bit slow Intel I5 6600T with this command (that is similar to what you use):

ffmpeg -y -i title00.mkv -c:v libx265 -an -x265-params ref=4:hme=1:hme-search=umh,umh,star:subme=4:bframes=8:rd=4:rd-refine=0:qcomp=0.65:fades=1:strong-intra-smoothing=1:ctu=32:qg-size=32:sao=0:selective-sao=0:cu-lossless=0:cutree=1:tu-inter-depth=4:tu-intra-depth=4:max-merge=2:rskip=2:rskip-edge-threshold=2:rc-lookahead=80:aq-mode=1:aq-strength=1.0:rdoq-level=1:psy-rd=4.0:psy-rdoq=15.0:limit-modes=0:limit-refs=0:limit-tu=0:deblock=-4,-4:weightb=1:weightp=1:rect=1:amp=0:wpp=1:pmode=0:pme=0:b-intra=1:b-adapt=2:b-pyramid=1:tskip=1:tskip-fast=0:fast-intra=0:early-skip=0:splitrd-skip=0:min-keyint=24:keyint=240:transfer=bt709:colorprim=bt709:colormatrix=bt709 -t 80 -b:v 4M ouput_h265_forum.mkv

I get 0.1X average. There is somthing wrong ?

Thanks !

[microchip8]

You may have to disable some CPU hungry options like hme=1 (set it to 0 or remove completely along with hme-search). Are you encoding Full HD or UHD content? If the latter, your CPU is too weak to handle it. Same for mine.

[alex.brown111@hotmail.com]

You may have to disable some CPU hungry options like hme=1 (set it to 0 or remove completely along with hme-search). Are you encoding Full HD or UHD content? If the latter, your CPU is too weak to handle it. Same for mine.

It is a FullHD movie h264 content (a blu ray ripped with makemkv).

[Boulder]

--limit-refs 3 will give a nice speed boost and you probably won't notice any difference in quality. --tu-inter-depth 3 and --tu-intra-depth 3 should be enough coupled with --limit-tu 4.

And yes, for high-bitrate and high quality encodes, aq-mode 1 is the one to choose.

[alex.brown111@hotmail.com]

--limit-refs 3 will give a nice speed boost and you probably won't notice any difference in quality. --tu-inter-depth 3 and --tu-intra-depth 3 should be enough coupled with --limit-tu 4.

And yes, for high-bitrate and high quality encodes, aq-mode 1 is the one to choose.

unfortunatenly don't help too much (I stil get 0.5x)

A question:

I have intel CPU I5 6600T (Skylake) can be a good idea use intel hardware accellerator:

https://trac.ffmpeg.org/wiki/Hardware/QuickSync

I have never used it and I don't known if can be a good way ..

[benwaggoner]

Anything above aq-mode 1 sucks. That's my experience. They're supposed reduce banding but what they do is increase bitrate for no improvement at all
They vary in utility. But --aq-mode 3 IS helpful to reduce artifacts in low luma, as it lowers QP in those regions.

There is plenty of content where the only command-line level way to reduce banding is to use 10-bit.

[Blue_MiSfit]

unfortunatenly don't help too much (I stil get 0.5x)

A question:

I have intel CPU I5 6600T (Skylake) can be a good idea use intel hardware accellerator:

https://trac.ffmpeg.org/wiki/Hardware/QuickSync

I have never used it and I don't known if can be a good way ..

The Intel hardware HEC encoder on Skylake is pretty basic, but it can be useful in certain cases. It's totally unrelated to x265, though, and will have significantly worse compression capability (since its use cases are very different).