[HELP] best x264 settings for NLE-oriented sources

[PatchWorKs]

Hi everyone,
I need to avisynth-preprocess 1080i50/AVC @17Mbps sources and reencode into AVC "visually lossless" before using into an NLE.

I've asked some LLMs that suggests these parameters:
x264 --crf 15 ^
--preset veryslow ^
--tune film ^
--profile high --level 4.2 ^
--fps 50 ^
--keyint 25 --min-keyint 25 --scenecut 0 ^
--bframes 6 --b-adapt 2 --b-pyramid normal --weightb ^
--ref 4 ^
--me tesa --merange 32 --subme 10 ^
--trellis 2 --no-fast-pskip --no-dct-decimate ^
--psy-rd 1.0:0.15 ^
--aq-mode 2 --aq-strength 0.8 ^
--deblock -1:-1 ^
--direct auto --rc-lookahead 60 ^
--ipratio 1.40 --pbratio 1.30 ^
--qpmin 10 --qpmax 30 ^
--vbv-maxrate 20000 --vbv-bufsize 40000 ^
--no-open-gop ^
--videoformat pal ^
--colormatrix bt709 --colorprim bt709 --transfer bt709 ^
--aud --repeat-headers --output-csp i420 ^
--threads auto ^
-o OUTPUT.mp4 INPUT.vpy

It might seem quite reasonable but, on closer inspection, there are some perhaps "inconsistent" (or at least improvable) parameters: what do you think about?

[rwill]

Where is the option for MBAFF ?

[Z2697]

--tune fastdecode
Don't use VBV (not that low at least)

[Z2697]

Where is the option for MBAFF ?

Maybe he deinterlaced the video in the script.

I would also disable B frame completely, and save most if not all of the slow options, compensate with extra bits.

[PatchWorKs]

Sorry for the extreme delay in reply, gyuz (I was in vacation time).

@rwill QTGMC (and some other "correction" filters) ahead, of course...

@Z2697 fastdecode tune is totally right but can you argue about bframes ?

[Z2697]

With bframes the frame reordering is necessary.
Some not so well designed software may have trouble with frame accuracy.
Nothing I can readily call out, just that gut feeling... probably doesn't matter.

fastdecode, or the most essential part of this tune, cabac=0, is slower when trellis is on (1 or 2), which is the default.
That's because trellis is designed to be used with CABAC, without it the encoder will do some approximation...
And the file will be bigger.
But the decoding speed, especially when bitrate is high (normal for intermediate), can be twice as fast or more.
So ideally trellis should be disabled if both encoding and decoding speed is required.
With trellis disabled it's better to lower the deadzone... https://forum.doom9.org/showthread.php?p=2027985#post2027985

So my recommendation will be:

fast

--preset faster --tune film --no-dct-decimate [--bframes 0] --no-cabac --trellis 0 --deadzone-intra 6 --deadzone-inter 6 [--bitrate 50000]

normal (meh)

--preset faster --tune film --no-dct-decimate [--bframes 0] [--bitrate 50000]

good?

--preset faster --tune film --no-dct-decimate [--bframes 0] --trellis 2 --subme 8 [--bitrate 50000]


trellis and subme is what I consider to be the "more important" parameters in x264, other things can be more easily compensated by bitrate (yes, we are talking about intermediate after all).

*and again, "subme" in x264 is a lot of things combined, not just subpixel motion search

[PatchWorKs]

Updated commandline:


x264 --crf 18 ^
--preset veryslow ^
--tune film ^
--profile high --level 4.2 ^
--fps 50 ^
--keyint 25 --min-keyint 25 --scenecut 0 ^
--bframes 0 ^
--no-cabac ^
--ref 4 ^
--me umh --merange 32 --subme 8 ^
--trellis 0 --no-fast-pskip --no-dct-decimate ^
--psy-rd 1.0:0.15 ^
--aq-mode 2 --aq-strength 0.8 ^
--deblock -1:-1 ^
--direct auto --rc-lookahead 60 ^
--ipratio 1.40 ^
--qpmin 10 --qpmax 30 ^
--vbv-maxrate 20000 --vbv-bufsize 20000 ^
--no-open-gop ^
--videoformat pal ^
--colormatrix bt709 --colorprim bt709 --transfer bt709 ^
--aud --repeat-headers --output-csp i420 ^
--threads auto ^
-o OUTPUT.mp4 INPUT.vpy

As you suggested too, the tune parameter has been reverted to "film" anyway (some LLM outputs isn't so bad).

I still wanna remain in 20Mbps target, since source in 17Mbps.

[Z2697]

Ever heard about generational loss?
And there's qpmax...
Does this thing even runs on your computer... and not spit out error of --repeat-headers' non-existence.
Whatever... as long as I'm not using this.

[excellentswordfight]

Updated commandline:



As you suggested too, the tune parameter has been reverted to "film" anyway (some LLM outputs isn't so bad).

I still wanna remain in 20Mbps target, since source in 17Mbps.
If the source is of high quality (i.e. if 17Mbps is the first lossy-compression pass) and of at somewhat normal complexity; 20Mbps max bitrate for 1080p50 will not be enough for visually lossless. Especially with 0.5s closed GOP, without scencuts and no b-frames.

I would propably try something like this:

--preset slow --profile high --level 4.2 --crf 15 --tune film --keyint 50 --min-keyint 50 --rc-lookahead 50 --no-open-gop --ref 2 --bframes 2 --no-scenecut --b-adapt 0 --b-pyramid strict --vbv-bufsize 50000 --vbv-maxrate 50000 --colormatrix bt709 --colorprim bt709 --transfer bt709

It should also be noted that as this 8bit 4:2:0, most nle should use hw-decoder for these files anyway? If so, I dont think a lot of these things would be as important. It should mostly be to optimize for fast seeking right?

[PatchWorKs]

If the source is of high quality (i.e. if 17Mbps is the first lossy-compression pass) and of at somewhat normal complexity; 20Mbps max bitrate for 1080p50 will not be enough for visually lossless. Especially with 0.5s closed GOP, without scencuts and no b-frames.

I would propably try something like this:



It should also be noted that as this 8bit 4:2:0, most nle should use hw-decoder for these files anyway? If so, I dont think a lot of these things would be as important. It should mostly be to optimize for fast seeking right?
1st of all thanks for your efforts.

Your commandline recommendation is interesting, but can you explain the keyint/lookahead value (50) ? Wouldn't be better to have 0.5s for editing ?

Yes, mutch likely NLEs (Vegas Pro in my case) uses hardware decoding so the main goals it to preserve sources quality (of course) while have manipulable&manageable (read transferible) files.


At the moment I'm testing this real world commandline (which encodes from the pretreatment vs pipeline in staxrip) that produces pretty usable results:

x264.exe --crf 15 --preset veryslow --tune film --trellis 0 --psy-rd 1:0 --no-fast-pskip --deadzone-inter 6 --deadzone-intra 6 --aq-mode 2 --aq-strength 0.8 --vbv-maxrate 20000 --vbv-bufsize 40000 --qpmin 10 --qpmax 30 --subme 8 --merange 32 --bframes 0 --ref 4 --scenecut 0 --keyint 25 --min-keyint 25 --no-dct-decimate --videoformat pal --colorprim bt709 --colormatrix bt709 --transfer bt709 --alternative-transfer bt709 --fps 50 --opencl --aud --no-cabac --synth-lib

I can rise vbv up to 25000/50000 but I really can't go higher (to get an idea I must transcode 90+ Gb source video streams of a multicam concert shootings to upload on a remote server).

EDIT
note I'm using the staxrip-provided AVC Encoder x264 core 165 r3223+28 cfee1f8 [Mod by Patman]: it's the best choice for my needs ?

[Z2697]

--tune film and --psy-rd 1:0...
Bro you never mentioned that you are like transfering it over some network, just forget about absolute everything and use preset placebo tune film and set keyint.
If you are still trying to consult LLM on this then I guess AI overlord has won.

[PatchWorKs]

--tune film and --psy-rd 1:0...
Bro you never mentioned that you are like transfering it over some network, just forget about absolute everything and use preset placebo tune film and set keyint.
Consider I've copied the staxrip commandline...

...anyway testing this now:
x264.exe --crf 15 --preset veryslow --tune film --psy-rd 1:0.05 --no-fast-pskip --deadzone-inter 6 --deadzone-intra 6 --aq-strength 0.8 --vbv-maxrate 25000 --vbv-bufsize 50000 --qpmin 10 --qpmax 35 --subme 9 --merange 32 --b-pyramid strict --bframes 2 --ref 4 --scenecut 0 --keyint 25 --min-keyint 25 --no-dct-decimate --videoformat pal --colorprim bt709 --colormatrix bt709 --transfer bt709 --alternative-transfer bt709 --fps 50 --opencl --aud --no-scenecut --synth-lib
(note: --no-open-gop flag causes x264.exe: unknown option crash)

If you are still trying to consult LLM on this then I guess AI overlord has won.
I don't trust LLMs much, but in some cases helped me to better understand parameters' influence on encoding.

[Z2697]

OK, considering x264 never hesitate to add more --ref and can break some hardware decoder, maybe a little addition of --ref 6 will be better.
Other than that, placebo should give you what you want.

[Atak_Snajpera]

Why can't you just use original untouched source files? Decoding ancient h264 format 1080i 25fps 17 Mbps is a piece of cake for modern cpus. I was doing that kind of conversion for NLE when I still had Intel core Q6600 ~20 years ago.

[Z2697]

It's for the AVS processing probably.

Anyways without the context of being somewhat bandwidth constrained, most of us would though that a intermediate file could/should have higher bitrate and encoding speed.
First of all i assume that you want reasonable speed, and in general speed and visually lossless

It matters. the context.
If that constrain is super important more important than computational cost (energy+time) max out on x264 or even go x265.

[Atak_Snajpera]

It does not matter if it is for AVS or some other NLE program. h264 standard was finished in ~2004. 22 years later we have CPUs with much higher IPC + clocks + cores.
Also EVERY GPU on the market can also offload decoding of h264/AVC streams. Re-encoding h264 makes zero sense. You will always lose quality and make files larger. It would make more sense if we were talking about 4k/8k h.266 format.

This table nicely shows how much progress was made when it comes to just pure IPC (when all data is in L1 cache).

https://i.postimg.cc/z8J12Dsx/Flops-CPUv2.png


Quick decoding test on 8c/16t old SandyBridge xeon @ 3GHz

Source from camcorder

General
ID : 0 (0x0)
Complete name : C:\Users\Dave\Documents\_BACKUP_SAMSUNG_HDD\_Video_Samples\HDcam\112.MTS
Format : BDAV
Format/Info : Blu-ray Video
File size : 169 MiB
Duration : 58 s 996 ms
Overall bit rate mode : Variable
Overall bit rate : 24.0 Mb/s
Maximum Overall bit rate : 24.0 Mb/s
Frame rate : 25.000 FPS
Recorded date : 2013-06-30 16:03:38+02:00
Writing application : Canon

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 2 frames
Format settings, GOP : M=3, N=12
Codec ID : 27
Duration : 58 s 960 ms
Bit rate mode : Variable
Bit rate : 22.7 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan type, store method : Separated fields
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.438
Stream size : 160 MiB (95%)

Audio
ID : 4352 (0x1100)
Menu ID : 1 (0x1)
Format : AC-3
Format/Info : Audio Coding 3
Commercial name : Dolby Digital
Codec ID : 129
Duration : 59 s 40 ms
Bit rate mode : Constant
Bit rate : 256 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Delay relative to video : -80 ms
Stream size : 1.80 MiB (1%)
Service kind : Complete Main
Dialog Normalization : -31 dB
compr : -0.28 dB
dialnorm_Average : -31 dB
dialnorm_Minimum : -31 dB
dialnorm_Maximum : -31 dB



AVSMeter 3.0.6.0 (x64), (c) Groucho2004, 2012-2020
AviSynth+ 3.7.4 (r4273, 3.7, x86_64) (3.7.4.0)

Number of frames: 1476
Length (hh:mm:ss.ms): 00:00:59.040
Frame width: 1920
Frame height: 1080
Framerate: 25.000 (25/1)
Colorspace: i420

Frames processed: 1476 (0 - 1475)
FPS (min | max | average): 117.3 | 617.2 | 326.1
Process memory usage (max): 134 MiB
Thread count: 33
CPU usage (average): 50.3%

Time (elapsed): 00:00:04.526


Average decoding speed is 13x faster than real time.

[Z2697]

Then it's just the matter of using NLE with AVS support.