Advanced best tu-inter-depth:tu-intra-depth:max-tu-size settings,Please check sample

[Kill3rWolf]

Could you tell me the best value/settings for tu-inter-depth:tu-intra-depth:max-tu-size with ctu 32 (& other related settings)? For live action Blu-ray movie.

I want to achieve sharpness with accurate color without increasing gamma, without losing the details & less color banding.

Please specify the old era live camera movie encode settings & present era settings. I know present-era movies don't have noise unless they put it intentionally. (I'm going to encode Last.Night.In.Soho.2021)

Here https://forum.doom9.org/showthread.php?t=173187 they specified it for animations encoding, can it be used for live-action too!?

[benwaggoner]

Alas, plenty of new movies have grain added as well for stylistic reasons, and to make VFX compositing easier.

Generally, the less noise, the more value in deeper TU. They are particularly valuable with cel animation, graphics, and text.

A TU depth of 4 is the maximum, as that gets down to the minimum with --ctu 64. So with --ctu 32, 3 is the practical maximum (you can set 4, but it does exactly the same thing as 3).

If you do a --csv-log-level 2, the log file will show how many of each size and type of CU you'll get, which is a good indication of how useful higher TU depths will be with a given source.

My default settings for quality-over-speed 1080p content are:
--tu-inter-depth 4
--tu-inter-depth 4
--limit-tu 4 (which really reduces the performance hit of using higher depth values)

[Kill3rWolf]

Originally Posted by benwaggoner

My default settings for quality-over-speed 1080p content are:
--tu-inter-depth 4
--tu-inter-depth 4
--limit-tu 4 (which really reduces the performance hit of using higher depth values)

I think you mistakenly typed --tu-inter-depth 4 --tu-inter-depth 4, maybe another one is --tu-intra-depth 4

& what about max tu size?

Which Psycho-visual options should I increase & decrease for this movie with TU settings?

[benwaggoner]

Originally Posted by Kill3rWolf

I think you mistakenly typed --tu-inter-depth 4 --tu-inter-depth 4, maybe another one is --tu-intra-depth 4

Correct, --tu-intra-depth. AFAIK inter/intra should always use the same value, although perhaps there are oddball scenarios where different values might have some advantage.

& what about max tu size?

I've always used 32, which is quality/efficiency optimal. Lower values would only make sense if really emphasizing encoding speed over compression efficiency. I don't recall ever seeing that parameter set to anything other than 32, not even in --preset ultrafast[/QUOTE]

Which Psycho-visual options should I increase & decrease for this movie with TU settings?

Psychovisual settings are pretty orthogonal to TU settings as far as I've ever seen. I've not found examples where changing one based on the other made sense. I can kind of imagine how smaller min TU sizes could maybe allow for reduced blocking/ringing of sharp edges in some content, and thus allow for slightly higher psy-rd and psy-rdoq. But it's not something I've ever tested or really considered much before you asked.

[Kill3rWolf]

I run some sample tests before doing the final encode.

Duration: 37 sec.

Sample 1:

no-sao:selective-sao=0:strong-intra-smoothing=0:rect=0:aq-mode=3:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=20000:vbv-bufsize=40000:rskip=1:ctu=32

x265 [info]: Main profile, Level-5.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias : 24 / 240 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 1 / 0
x265 [info]: References / ref-limit cu / depth : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init : 40000 / 20000 / 0.900
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra lslices=5 deblock

Download link: https://drive.google.com/file/d/1wKN...ew?usp=sharing

Full Media Info: https://pastebin.com/6DRzdNGK

Sample 2:

no-sao:selective-sao=1:strong-intra-smoothing=0:rect=0:aq-mode=3:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:rskip=1:ctu=32:csv-log-level=2

x265 [info]: x265 [warning]: SAO turned ON when selective-sao is ON
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra lslices=5 deblock sao selective-sao

Download link: https://drive.google.com/file/d/1M4z...ew?usp=sharing

Full Media Info: https://pastebin.com/wym7dgBB

Sample 3:

limit-sao:selective-sao=1:strong-intra-smoothing=0:rect=0:aq-mode=3:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:rskip=1:ctu=32:csv-log-level=2:tu-inter-depth=4:tu-intra-depth=4:limit-tu=4

x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 limit-tu=4
x265 [info]: tools: signhide tmvp b-intra lslices=5 deblock sao selective-sao

Download link: https://drive.google.com/file/d/1SxW...ew?usp=sharing

Full Media Info: https://pastebin.com/yHuuzJeJ

Sample 4:

deblock=-1:-1:limit-sao:selective-sao=1:strong-intra-smoothing=0:rect=0:aq-mode=3:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:rskip=1:ctu=32:csv-log-level=2:tu-inter-depth=4:tu-intra-depth=4:limit-tu=4:rdpenalty=2

x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Intra 32x32 TU penalty type         : 2
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 limit-tu=4
x265 [info]: tools: signhide tmvp b-intra lslices=5 deblock(tC=-1:B=-1) sao
x265 [info]: tools: selective-sao

Download link: https://drive.google.com/file/d/1yHo...ew?usp=sharing

Full Media Info:https://pastebin.com/c9RTTj0t

Sample 5:

deblock=-1:-1:no-sao:selective-sao=0:strong-intra-smoothing=0:rect=0:aq-mode=3:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:ctu=32:csv-log-level=2:tu-inter-depth=4:tu-intra-depth=4:limit-tu=4:rdpenalty=2:rskip=2:rskip-edge-threshold=2:rd-refine

x265 [warning]: --rd-refine disabled, requires RD level > 4 and adaptive quant
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Intra 32x32 TU penalty type         : 2
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=2
x265 [info]: tools: rskip-edge-threshold=0.02 limit-tu=4 signhide tmvp b-intra
x265 [info]: tools: lslices=5 deblock(tC=-1:B=-1)

Download link: https://drive.google.com/file/d/1OjY...ew?usp=sharing

Full Media Info:https://pastebin.com/c9RTTj0t

Sample 6:

deblock=-1:-1:no-sao:selective-sao=0:strong-intra-smoothing=0:rect=0:aq-mode=4:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:ctu=32:csv-log-level=2:tu-inter-depth=4:tu-intra-depth=4:limit-tu=4:rd=4:rdpenalty=2:rskip=2:rskip-edge-threshold=2:rd-refine

x265 [warning]: --rd-refine disabled, requires RD level > 4 and adaptive quant
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 4 inter / 4 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Intra 32x32 TU penalty type         : 2
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 4 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=4 psy-rd=2.00 early-skip rskip mode=2
x265 [info]: tools: rskip-edge-threshold=0.02 limit-tu=4 signhide tmvp b-intra
x265 [info]: tools: lslices=5 deblock(tC=-1:B=-1)

Download link: https://drive.google.com/file/d/1OjY...ew?usp=sharing

Full Media Info: https://pastebin.com/kywf1CU1

Sample 7:

deblock=-1:-1:no-sao:selective-sao=0:strong-intra-smoothing=0:rect=0:aq-mode=4:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:ctu=32:csv-log-level=2:rd=4:rdpenalty=2:rskip=2:rskip-edge-threshold=2:rd-refine

x265 [warning]: --rd-refine disabled, requires RD level > 4 and adaptive quant
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Intra 32x32 TU penalty type         : 2
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 4 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=4 psy-rd=2.00 early-skip rskip mode=2
x265 [info]: tools: rskip-edge-threshold=0.02 signhide tmvp b-intra lslices=5
x265 [info]: tools: deblock(tC=-1:B=-1)

Download link: https://drive.google.com/file/d/1erB...ew?usp=sharing

Full Media Info: https://pastebin.com/5n61qMde

Sample 8:

deblock=-1:-1:limit-sao:selective-sao=1:strong-intra-smoothing=0:rect=0:aq-mode=4:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:ctu=32:rd=4:rdpenalty=2:rskip=2:rskip-edge-threshold=2:rd-refine:csv-log-level=2

x265 [warning]: --rd-refine disabled, requires RD level > 4 and adaptive quant
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Intra 32x32 TU penalty type         : 2
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 4 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=4 psy-rd=2.00 early-skip rskip mode=2
x265 [info]: tools: rskip-edge-threshold=0.02 signhide tmvp b-intra lslices=5
x265 [info]: tools: deblock(tC=-1:B=-1) sao selective-sao

Download link: https://drive.google.com/file/d/14i9...ew?usp=sharing

Full Media Info: https://pastebin.com/5n61qMde

Sample 9:

no-sao:selective-sao=1:strong-intra-smoothing=0:rect=0:aq-mode=3:aq-strength=1.0:qcomp=0.60:ref=5:bframes=8:me=2:merange=25:subme=7:rc-lookahead=40:vbv-maxrate=12000:vbv-bufsize=20000:rdpenalty=2:rskip=1:ctu=32:csv-log-level=2

x265 [warning]: SAO turned ON when selective-sao is ON
x265 [info]: Main profile, Level-4.1 (Main tier)
x265 [info]: Thread pool created using 2 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(26 rows)
x265 [info]: Coding QT: max CU size, min CU size : 32 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge         : umh / 25 / 7 / 3
x265 [info]: Keyframe min / max / scenecut / bias  : 24 / 240 / 40 / 5.00
x265 [info]: Intra 32x32 TU penalty type         : 2
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 0
x265 [info]: References / ref-limit  cu / depth  : 5 / off / on
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 3 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-18.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 20000 / 12000 / 0.900
x265 [info]: tools: rd=3 psy-rd=2.00 early-skip rskip mode=1 signhide tmvp
x265 [info]: tools: b-intra lslices=5 deblock sao selective-sao

Download link: https://drive.google.com/file/d/14uz...ew?usp=sharing
Full Media Info: https://pastebin.com/mTi1HXSB

Frame by Frame Comparision:

1. https://slow.pics/c/jzcSpmIK (Source Not Included)
2. https://slow.pics/c/HGXk7ukn (Source Included)
3. https://slow.pics/c/FpA59Ik3 (Source Not Included)

If you get enough relaxation time & patience, then please check all 9 images...I mean you have to click 9 times every time.

Folder: https://drive.google.com/drive/folde...ZR?usp=sharing

[Kill3rWolf]

For Everyone:

From 9 samples, which one's looks better & worst? & why? Which one got more details & sharpness? Can you see/identify the difference between aq-mode=3 & aq-mode=4? If not, then why?

Is there any chance to improve more? What should I do? Or everything is already almost fine...

Is there any command-line error? which one? Anything missing?
--hevc-aq missing from command-line?

I don't why some last test results provided a low bit rate & file size compared to some 1st beginning tests.

I think & observe that limit-sao:selective-sao=1 works well when you are using a low CRF value. Even better than no-sao:selective-sao=0. But when you are increasing CRF value with limit-sao:selective-sao=1 it adds smoothness to the face & details lost.

Guys, Does no-sao:selective-sao=0 have any playback issues? I mean ridiculously faster scene change compared to the source during fast movement.

no-sao:selective-sao=1
x265 [warning]: SAO turned ON when selective-sao is ON

But I saw that it provided slightly better than no-sao:selective-sao=0, I don't know the behind the reason.

Originally Posted by benwaggoner

If you do a --csv-log-level 2, the log file will show how many of each size and type of CU you'll get, which is a good indication of how useful higher TU depths will be with a given source.

HandBrake-1.5.1-x86_64-Win_GUI.exe

--csv-log-level 2, the log file nowhere found.

[benwaggoner]

Originally Posted by Kill3rWolf

From 9 samples, which one's looks better & worst? & why? Which one got more details & sharpness? Can you see/identify the difference between aq-mode=3 & aq-mode=4? If not, then why?

They are your samples, you get to decide which looks best!

Is there any command-line error? which one? Anything missing?
--hevc-aq missing from command-line?

Don't use --hevc-aq. The goodness of that is included in --aq-mode 4.

I don't why some last test results provided a low bit rate & file size compared to some 1st beginning tests.

Different modes have different bitrate allocation and rate control, so file sizes will be different. If you want to compare quality at a given file size, you should use 2-pass VBR encoding.

I think & observe that limit-sao:selective-sao=1 works well when you are using a low CRF value. Even better than no-sao:selective-sao=0. But when you are increasing CRF value with limit-sao:selective-sao=1 it adds smoothness to the face & details lost.

SAO is good at reducing visible artifacts at the cost of some detail. If you're encoding with enough bits that you weren't getting artifacts without SAO, using it will lose detail.

Guys, Does no-sao:selective-sao=0 have any playback issues? I mean ridiculously faster scene change compared to the source during fast movement.

It could have a minor improvement for software decoders (don't need to apply it on playback), but should have none for hardware decoders.

no-sao:selective-sao=1
x265 [warning]: SAO turned ON when selective-sao is ON

But I saw that it provided slightly better than no-sao:selective-sao=0, I don't know the behind the reason.

selective-sao 0 is the same as no SAO. For your content, having SAO on IDR frames is helpful. Selective-sao 2 is often the sweet spot.

--csv-log-level 2, the log file nowhere found.

You specify the file path with --csv

Lots of your questions are answered here: https://x265.readthedocs.io/en/master/cli.html

[Kill3rWolf]

Originally Posted by benwaggoner

They are your samples, you get to decide which looks best!

So, you haven't clicked any link? Not even slow.pics Frame by Frame Comparision? Okay, doesn't matter...waiting for any person gonna check those link.

Originally Posted by benwaggoner

They are your samples, you get to decide which looks best!

Actually I'm here for setings by setings review & practical explainations.