Which x264 settings are incompatible?

[DeeGee]

I wanted to test all the brand spanking new options together, but seems like some of them are incompatible, because I can't get a complete movie encode to finish. This is probably the 3rd try at this encode. The first time I had --me umh and --trellis 2. Had to take them off as the encodes failed and didn't have all week to try all settings... (fps was under 1 with those...) The x264 builds used were the newest Sharktooth builds available at the start of the encoding (from 364a to 367).

2 Pass Compression
1st Pass

D:\DVD>REM "x264.exe" --pass 1 --bitrate 834 --stats ".stats" --bframes 3 --subm
e 1 --weightb --analyse none --me dia --progress --no-psnr --sar 64:45 --zones 1
45000,154893,q=30 --output "D:\DVD\Star Wreck - In the Pirkinning\VIDEO_TS\Star
Wreck_1stpass.mp4" "D:\DVD\Star Wreck - In the Pirkinning\VIDEO_TS\Star Wreck.av
s"
2nd Pass

D:\DVD>"x264.exe" --pass 2 --bitrate 834 --stats ".stats" --bframes 3 --b-pyrami
d --ref 15 --filter -2:-2 --analyse all --weightb --me hex --subme 7 --b-rdo --m
ixed-refs --8x8dct --trellis 1 --aq-strength 0.5 --cqmfile "C:\Program Files\x26
4\soulhunter_v2.cfg" --sar 64:45 --progress --zones 145000,154893,q=30 --output
"D:\DVD\Star Wreck - In the Pirkinning\VIDEO_TS\Star Wreck.mp4" "D:\DVD\Star Wre
ck - In the Pirkinning\VIDEO_TS\Star Wreck.avs"
avis [info]: 720x432 @ 25.00 fps (154894 frames)
x264 [info]: using SAR=64/45
x264 [info]: using cpu capabilities MMX MMXEXT SSE 3DNow!
mp4 [info]: initial delay 2 (scale 25)
encoded frames: 32218/154894 (20.8%), 2.73 fps, eta 12:27:51
D:\DVD>pause
Press any key to continue . . .

[hpn]

Even if you input incompatible or contradictory ones like --b-rdo plus --subme 5, or --ref 1 plus --mixed-refs for example, x264 will silently fix the command line for you and the encoding should finish without a problem, so options incompatability shouldn't be a reason for a crash. Trellis 1/2, --subme 7, --b-rdo are still experimental and not tweaked enough, so it's normal to crash. I also manage to crash some of my encodes with combinations of those, but I can't find any "rule" when it will crash or not. Sometimes even changing some of the old and supposedly well tested options (like changing -f 0:0 to -f +-x:x for example) when used with a constant set of the new ones will trigger a crash.

[berrinam]

There is a thread about Adapitve Quantization crashing here (http://forum.doom9.org/showthread.php?t=101727&page=1&pp=20). No conclusive results were found, other than that it did seem to cause a crash.

Also, many problems in x264cli seem to be caused by bad frame-rates. Try adding the nicefps filter to the end of your avs script (it can be found with a search).

Otherwise, see if you can narrow down the options leading to a crash by slowly removing options until you no longer have a crash, and so on until you have worked out what leads to the crash.

[Sharktooth]

please report back if you find what's the option that cause the crash.

@berrinam: the latest aq patch seems to have fixed the crashing problem. however as manao said on irc it's still "latent".

[DeeGee]

Tried to do a new try with NiceFps() but failed at the same frame as the example. I guess I just need to start checking out all the settings. Might take a while, as encoding of 32218 frames takes a while with 1-2fps...

[Sharktooth]

Start with removing adaptive quantization and see if it fixes it.

[DeeGee]

Ok, I'll try that next. With --me hex and --trellis 0 x264 stopped already at frame 465. (otherwise the command line is identical to the first example).

[DeeGee]

Ok, your suggestion worked. Without Adaptive Quantization everything vent fine.

[Sharktooth]

Thanks.