Non deterministic with trellis or subme 6 with crf?

[mitsubishi]

Is this right? It cropped up in another thread and I think I've narrowed the non-deterministic behaviour to using subme 6 or trellis. Only in CRF though, I've tried it in 2-pass and it isn't like this.

E6300 B2 and x264 v663 and v667

So is it right or broken?

OT, but is it also intended behaviour that using a path that doesn't exist means the encode takes place, but no output is created, rather than just creating the folder?

[akupenguin]

I think I've narrowed the non-deterministic behaviour to using subme 6 or trellis. Only in CRF though, I've tried it in 2-pass and it isn't like this.

So is it right or broken?
I can reproduce some nondeterministic behaviour. But starting from your commandline I got it down to: -b1 -m6 -8 --threads auto. Ratecontrol method doesn't matter, nor does trellis.
This is strange, because removing either B-frames or rdo makes the problem go away, but rdo isn't enabled in B-frames.

is it also intended behaviour that using a path that doesn't exist means the encode takes place, but no output is created, rather than just creating the folder?
Happens only for mp4...

[mitsubishi]

Happens only for mp4...
Thanks, I'll bare that in mind.

Anyway, my results seemed different this morning. Some options seemed to fail quicker than others. I've written a batch script to compare automatically. It requires md5 (http://www.fourmilab.ch/md5/) because I don't know another way to compare files and get the result in a variable. It takes commands from a text file, one to a line without input/output or ratecontrol.

set results="results.txt"
set x264="x264-667-gcc3.4.6.exe"
set input=gs2.avs
set bitrate=800
set crf=20
set commands=commands.txt

del %results%

for /F "delims=;" %%n in (%commands%) do call :parse %%n

goto :EOF

:parse
echo %* >> %results%
%x264% --crf %crf% %* --output "temp.264" %input%

md5 temp.264 > md5temp.txt
for /F %%h in (md5temp.txt) do set md5=%%h


%x264% --crf %crf% %* --output "temp.264" %input%

md5 temp.264 > md5temp.txt
for /F %%h in (md5temp.txt) do set md5b=%%h

if %md5%==%md5b% (
echo CRF: PASS >> %results%
) ELSE (
echo CRF: FAIL >> %results%
)

%x264% --bitrate %bitrate% --pass 1 %* --output NUL %input%

md5 x264_2pass.log > md5temp.txt
for /F %%h in (md5temp.txt) do set md5=%%h


%x264% --bitrate %bitrate% --pass 1 %* --output NUL %input%

md5 x264_2pass.log > md5temp.txt
for /F %%h in (md5temp.txt) do set md5b=%%h

if %md5%==%md5b% (
echo 1sP: PASS >> %results%
) ELSE (
echo 1sP: FAIL >> %results%
)

%x264% --bitrate %bitrate% --pass 2 %* --output temp.264 %input%

md5 temp.264 > md5temp.txt
for /F %%h in (md5temp.txt) do set md5=%%h


%x264% --bitrate %bitrate% --pass 2 %* --output temp.264 %input%

md5 temp.264 > md5temp.txt
for /F %%h in (md5temp.txt) do set md5b=%%h

if %md5%==%md5b% (
echo 2sP: PASS >> %results%
) ELSE (
echo 2sP: FAIL >> %results%
)

[mitsubishi]

Hmm yeah I must have a made a cockup because ratecontrol doesn't make any difference now.

--ref 3 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --filter -2,-1 --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-psnr
CRF: FAIL
1sP: FAIL
2sP: FAIL
--ref 3 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --filter -2,-1 --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-psnr
CRF: FAIL
1sP: FAIL
2sP: FAIL
--ref 3 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --filter -2,-1 --subme 1 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-psnr
CRF: PASS
1sP: PASS
2sP: PASS
--ref 3 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --bime --weightb --filter -2,-1 --subme 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-psnr
CRF: PASS
1sP: PASS
2sP: PASS
--ref 3 --mixed-refs --no-fast-pskip --b-pyramid --b-rdo --bime --weightb --filter -2,-1 --subme 6 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-psnr
CRF: FAIL
1sP: FAIL
2sP: FAIL
--subme 6 --trellis 1 --8x8dct --threads auto --thread-input --progress --no-psnr --no-ssim
CRF: PASS
1sP: PASS
2sP: PASS
--bframes 3 --subme 6 --trellis 1 --threads auto --thread-input --progress --no-psnr --no-ssim
CRF: PASS
1sP: PASS
2sP: PASS
--bframes 3 --subme 6 --trellis 1 --threads auto --8x8dct --thread-input --progress --no-psnr --no-ssim
CRF: FAIL
1sP: FAIL
2sP: FAIL

[akupenguin]

Also, be sure to run the encode more than twice before calling it a pass. In my experiment (2000 frame clip), I could encode, say, 10 times and get only 4 different files. So there's a chance that 2 will be the same. (Is this what you mean by fail quicker?)

[mitsubishi]

Yeah, by quicker I meant a longer sample had to be taken to make sure it failed, I was using a 1000 frame sample to test the script, but moved to a 3500 to run it. At this length it's consistent across the 3 tests it runs.