Question regarding 2 pass encode through cli

[djesteban]

Hi,

So this is the first time I launch a job through cli for x264, but I am having issues, so I hope someone can help me.

First, this is my x264 settings:
x264.exe --profile high --preset fast --pass 2 --bitrate 1000 --stats "test.stats" --thread-input --deblock -1:-1 --bframes 3 --b-adapt 2 --direct auto --ref 8 --me umh --subme 8 --partitions all --trellis 2 --output "D:\Hatsukoi\_encoded\test.264" "D:\Hatsukoi\video.avs"

If I try to launch the above, it will throw out an error: ratecontrol_init: can't open stats file
Now I realize that this is due to the fact that if I specify --pass 2 right away and that it won't create the stats file... if I specify -- pass 1, it starts the encode.

So my question is, can someone show me how I can use the above to actually create a 2 pass encode? Do I have to wait until the first pass finished before I launch the second pass manually? Or is it as simple as this:
x264.exe --profile high --preset fast --pass 1 --bitrate 1000 --stats "test.stats" --thread-input --deblock -1:-1 --bframes 3 --b-adapt 2 --direct auto --ref 8 --me umh --subme 8 --partitions all --trellis 2 --output "D:\Hatsukoi\_encoded\test.264" "D:\Hatsukoi\video.avs"
x264.exe --profile high --preset fast --pass 2 --bitrate 1000 --stats "test.stats" --thread-input --deblock -1:-1 --bframes 3 --b-adapt 2 --direct auto --ref 8 --me umh --subme 8 --partitions all --trellis 2 --output "D:\Hatsukoi\_encoded\test.264" "D:\Hatsukoi\video.avs"

So please let me know how I can use the settings above to complete a 2 pass encode!

Thanks in advance!

[LoRd_MuldeR]

If you use "--pass 2" then you tell x264 to read the existing stats file. So this is what you use for the second pass of a 2-Pass encode, NOT for the first one.

For the first pass of a 2-Pass encode, you must use "--pass 1", which tells x264 to write to the stats file.

And of course you must complete the first pass, before you can start the second one. That's the whole idea behind 2-Pass encoding! You cannot run both passes in parallel.

However you don't need to sit in front of the screen and wait. Use a simple batch file ;)


BTW: Why do you start with "--preset fast" and then you manually overwrite most speed-relevant options with "slow" settings ???

Better try like this:
x264.exe --preset slow --tune film --pass 1 --bitrate 1000 --stats "test.stats" --output NUL "D:\Hatsukoi\video.avs"
x264.exe --preset slow --tune film --pass 2 --bitrate 1000 --stats "test.stats" --output "D:\Hatsukoi\_encoded\test.264" "D:\Hatsukoi\video.avs"

(I recommend you look at "x264.exe --fullhelp" to see what the presets actually do ^^)

[djesteban]

If you use "--pass 2" then you tell x264 to read the existing stats file. So this is what you use for the second pass of a 2-Pass encode, NOT for the first one.

For the first pass of a 2-Pass encode, you must use "--pass 1", which tells x264 to write to the stats file.

And of course you must complete the first pass, before you can start the second one. That's the whole idea behind 2-Pass encoding! You cannot run both passes in parallel.

However you don't need to sit in front of the screen and wait. Use a simple batch file ;)


BTW: Why do you start with "--preset fast" and then you manually overwrite most speed-relevant options with "slow" settings ???

Better try like this:
x264.exe --preset slow --tune film --pass 1 --bitrate 1000 --stats "test.stats" --output NUL "D:\Hatsukoi\video.avs"
x264.exe --preset slow --tune film --pass 2 --bitrate 1000 --stats "test.stats" --output "D:\Hatsukoi\_encoded\test.264" "D:\Hatsukoi\video.avs"

(I recommend you look at "x264.exe --fullhelp" to see what the presets actually do ^^)
Thanks for answering Lord! I appreciate! :)
Now, to answer your question, I started with "--preset fast" simply because I based myself on one of Sharktooth's preset in MeGUI; Unrestricted 2pass Extra Quality:
program --profile high --preset fast --pass 2 --bitrate 1000 --stats ".stats" --thread-input --deblock -1:-1 --bframes 3 --b-adapt 2 --b-pyramid
--direct auto --b-bias 0 --scenecut 40 --ref 8 --rc-lookahead 40 --no-mbtree --aq-mode 1 --aq-strength 1.0 --merange 16 --me umh --subme 8
--partitions all --trellis 2 --psy-rd 1.0:0 --output "output" "input"
Now, the only thing that I changed in there is that I removed b-pyramid and replaced it by mb-tree and also removed (what I think are) unnecessary flags which have the same value than the default value indicated in the x264 settings wiki page (http://mewiki.project357.com/wiki/X264_Settings).

In your last post you are telling me to use this instead:
--preset slow --tune film
I don't disagree :P ...but if I look at the x264 settings page, I cannot find a qualitative description that explains what is the difference between all of the preset (between ultrafast and placebo)... so this is another reason why I copied that old Sharktooth's setting :)
The --tune setting that you included is a little more straightforward, but still, the x264 settings wiki doesn't really explain when you should use one or the other.
Do you know a place where I could get information about those two flags and how I should use them to get something similar to the settings I have in my original post?

Hope you can help! thanks again!

[nurbs]

"x264 --fullhelp" yields this:
--preset Use a preset to select encoding settings [medium]
Overridden by user settings
- ultrafast:
--no-8x8dct --aq-mode 0 --b-adapt 0
--bframes 0 --no-cabac --no-deblock
--no-mbtree --me dia --no-mixed-refs
--partitions none --ref 1 --scenecut 0
--subme 0 --trellis 0 --no-weightb
--weightp 0
- veryfast:
--no-mbtree --me dia --no-mixed-refs
--partitions i8x8,i4x4 --ref 1
--subme 1 --trellis 0 --weightp 0
- faster:
--no-mbtree --no-mixed-refs --refs 2
--subme 4 --weightp 1
- fast
--rc-lookahead 30 --ref 2 --subme 6
- medium
Default settings apply.
- slow
--b-adapt 2 --direct auto --me umh
--rc-lookahead 50 --ref 5 --subme 8
- slower
--b-adapt 2 --direct auto --me umh
--partitions all --rc-lookahead 60
--ref 8 --subme 9 --trellis 2
- veryslow
--b-adapt 2 --bframes 8 --direct auto
--me umh --me-range 24 --partitions all
--ref 16 --subme 10 --trellis 2
--rc-lookahead 60
- placebo
--bframes 16 --b-adapt 2 --direct auto
--slow-firstpass --no-fast-pskip
--me tesa --me-range 24 --partitions all
--rc-lookahead 60 --ref 16 --subme 10
--trellis 2
--tune Tune the settings for a particular type of source
Overridden by user settings
- film:
--deblock -1:-1 --psy-rd <unset>:0.15
- animation:
--bframes {+2} --deblock 1:1
--psy-rd 0.4:<unset> --aq-strength 0.6
--ref {Double if >1 else 1}
- grain:
--aq-strength 0.5 --no-dct-decimate
--deadzone inter 6 --deadzone-intra 6
--deblock -2:-2 --ipratio 1.1
--pbratio 1.1 --psy-rd <unset>:0.25
--qcomp 0.8
- psnr:
--aq-mode 0 --no-psy
- ssim:
--aq-mode 2 --no-psy
- fastdecode:
--no-cabac --no-deblock --no-weightb
--weightp 0
- zerolatency:
--bframes 0 --rc-lookahead 0
--sync-lookahead 0 --sliced-threads
- touhou:
--aq-strength 1.3 --deblock -1:-1
--partitions {p4x4 if p8x8 set}
--psy-rd <unset>:0.2
--ref {Double if >1 else 1}
--slow-firstpass Don't use faster settings with --pass 1
All the defaults are also listed.

[Carpo]

hmm informative, guess i can change mine from

x264.exe --profile high --level 4.1 --preset veryslow --tune film --pass 2 --bitrate 9132 --stats "stats" --thread-input --b-adapt 2 --ref 4 --partitions all --me umh --direct auto -o gran.torino.mkv gran.torino.avs

to

x264.exe --profile high --level 4.1 --preset veryslow --tune film --pass 2 --bitrate 9132 --stats "stats" --thread-input -o gran.torino.mkv gran.torino.avs

or is the thread-input not required also ?

[LoRd_MuldeR]

It's not required.

[djesteban]

All right! Now I got this:
--preset slower --tune film --pass 1 --bitrate 1000 --stats "encode.stats" --output NUL "D:\Hatsukoi\video.avs"
--preset slower --tune film --pass 2 --bitrate 1000 --stats "encode.stats" --output "D:\Hatsukoi\_encoded\test.264" "D:\Hatsukoi\video.avs"
Now, some little questions remain.
- When I'll get my calculated bitrate, I should replace --bitrate 1000 in both line with the correct bitrate right? (just wondering if it's only necessary in the 2nd pass).

- With the settings above, how would I go to create a batch file to launch them in one go? Can someone provide me with an example please! thanks!

- I saw in the x264 settings page that there was a --pass 3 ... I'm just curious on how and/or when you would use such a pass. Even in the fullhelp that's not very clear.

Thanks!

*EDIT* Ohhh, also, Lord, in the first line you told me to put NUL as the output... is that intended? what exactly does that do? Creates a temp file?!?

[LoRd_MuldeR]

- When I'll get my calculated bitrate, I should replace --bitrate 1000 in both line with the correct bitrate right? (just wondering if it's only necessary in the 2nd pass).

You can use a different bitrate in the second pass, but it's not recommended. The more the second-pass bitrate differs from the first pass, the higher the expected quality degradation!

Anyway, x264 should always hit the bitrate that you specify in the second/final pass...

- With the settings above, how would I go to create a batch file to launch them in one go? Can someone provide me with an example please! thanks!

Open Notepad, enter the following lines and save the result to a .cmd file ;)

@echo off
"C:\Path to Tools\x264.exe" --preset slower --tune film --pass 1 --bitrate 1000 --stats "encode.stats" --output NUL "D:\Hatsukoi\video.avs"
"C:\Path to Tools\x264.exe" --preset slower --tune film --pass 2 --bitrate 1000 --stats "encode.stats" --output "D:\Hatsukoi\_encoded\test.264" "D:\Hatsukoi\video.avs"
pause

- I saw in the x264 settings page that there was a --pass 3 ... I'm just curious on how and/or when you would use such a pass. Even in the fullhelp that's not very clear.

"--pass 3" tells x264 to update the stats file. You would use that for the second pass of a 3-Pass encode. And before you ask: Nope, 3-Pass is not better than 2-Pass. It's a placebo!

Ohhh, also, Lord, in the first line you told me to put NUL as the output... is that intended? what exactly does that do? Creates a temp file?!?

Yes, it was intentionally. We don't need the output from the first pass. We'd overwrite it in the second pass anyway! All we need from the first pass is the .stats file itself.

(NUL is the "null" device on M$ Windows or DOS, similar to /dev/null (http://en.wikipedia.org/wiki//dev/null) on Linux/Unix. All data written to the "null" device is discarded)

[rack04]

- When I'll get my calculated bitrate, I should replace --bitrate 1000 in both line with the correct bitrate right? (just wondering if it's only necessary in the 2nd pass).

Use the calculated bitrate in the first and second pass.

- With the settings above, how would I go to create a batch file to launch them in one go? Can someone provide me with an example please! thanks!

Copy the following into notebook and save as a .bat file. Set the file paths to mediainfo, x264, and working directory.

@SET MediaInfo_PATH=[PATH\]MediaInfo.exe
@SET x264_PATH=[PATH\]x264.exe
@SET Working_Directory=[PATH]

@Echo off
set /p INPUT_VIDEO="Full Path to Input File: "

Echo General;%%FileName%%>"%Working_Directory%\temporary-generalfilename.txt"
"%MediaInfo_PATH%" --Inform=file://"%Working_Directory%\temporary-generalfilename.txt" "%INPUT_VIDEO%">"%Working_Directory%\temporary-analyzegeneralfilename.txt"
Echo General;%%FolderName%%>"%Working_Directory%\temporary-generalfoldername.txt"
"%MediaInfo_PATH%" --Inform=file://"%Working_Directory%\temporary-generalfoldername.txt" "%INPUT_VIDEO%">"%Working_Directory%\temporary-analyzegeneralfoldername.txt"
type "%Working_Directory%\temporary-analyzegeneralfilename.txt">"%Working_Directory%\temporary-analyzegeneralfilename.tmp"
type "%Working_Directory%\temporary-analyzegeneralfoldername.txt">"%Working_Directory%\temporary-analyzegeneralfoldername.tmp"
set /p SOURCE_FILENAME=<"%Working_Directory%\temporary-analyzegeneralfilename.tmp"
set /p SOURCE_FOLDER=<"%Working_Directory%\temporary-analyzegeneralfoldername.tmp"

DEL "%Working_Directory%\temporary-generalfilename.txt"
DEL "%Working_Directory%\temporary-analyzegeneralfilename.txt"
DEL "%Working_Directory%\temporary-analyzegeneralfilename.tmp"
DEL "%Working_Directory%\temporary-generalfoldername.txt"
DEL "%Working_Directory%\temporary-analyzegeneralfoldername.txt"
DEL "%Working_Directory%\temporary-analyzegeneralfoldername.tmp"

set /p ENCODE="Ratecontrol Method (bitrate/crf): "

if "%ENCODE%" EQU "crf" goto crf

set /p VIDBITRATE="Desired Bitrate (kbps): "
goto preset

:crf
set /p CRF="Constant Ratefactor: "

:preset
set /p PRESET="x264 Preset: "
set /p TUNE="x264 Tune: "

if "%ENCODE%" EQU "crf" goto crf

Echo.
Echo.
Echo.[ %TIME% ] Encoding pass 1 ...
Echo.

@Echo on
"%x264_PATH%" --preset %PRESET% --tune %TUNE% --pass 1 --bitrate %VIDBITRATE% --stats "%SOURCE_FOLDER%\%SOURCE_FILENAME%.stats" --output NUL "%INPUT_VIDEO%"
@Echo off

Echo.
Echo.
Echo.[ %TIME% ] Encoding pass 2 ...
Echo.

@Echo on
"%x264_PATH%" --preset %PRESET% --tune %TUNE% --pass 2 --bitrate %VIDBITRATE% --stats "%SOURCE_FOLDER%\%SOURCE_FILENAME%.stats" --output "%SOURCE_FOLDER%\%SOURCE_FILENAME%-output.264" "%INPUT_VIDEO%"
@Echo off
goto end

:crf
Echo.
Echo.
Echo.[ %TIME% ] Encoding CRF ...
Echo.

@Echo on
"%x264_PATH%" --preset %PRESET% --tune %TUNE% --crf %CRF% --output "%SOURCE_FOLDER%\%SOURCE_FILENAME%-output.264" "%INPUT_VIDEO%"
@Echo off

:end
Echo.
Echo.
Echo.[ %TIME% ] Finished !
Echo.
Echo.
Pause

- I saw in the x264 settings page that there was a --pass 3 ... I'm just curious on how and/or when you would use such a pass. Even in the fullhelp that's not very clear.

I see no benefit of using 3 passes.

Ohhh, also, Lord, in the first line you told me to put NUL as the output... is that intended? what exactly does that do? Creates a temp file?!?

It does exactly what it says. Outputs nul.

[nurbs]

While we are at command lines you can also shutdown your computer at the end of encoding
i.e.
shutdown -s -t 60
for a 60 second countdown.

It can be aborted with
shutdown -a

[djesteban]

Thanks! You guys are awesome!

Ok, so I have launch the 1st pass alone as a test and it start encoding fine, but I have noticed something weird; here are the first 2 lines that appears before it actually start enconding:
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
x264 [info]: profile Main, level 4.1
Now it says profile Main... isn't the default suppose to be high!?! I mean, even if I specify --profile high to force it, it still displays the line above!
Is this something I should matter about?

[Dark Shikari]

Thanks! You guys are awesome!

Ok, so I have launch the 1st pass alone as a test and it start encoding fine, but I have noticed something weird; here are the first 2 lines that appears before it actually start enconding:
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
x264 [info]: profile Main, level 4.1
Now it says profile Main... isn't the default suppose to be high!?! I mean, even if I specify --profile high to force it, it still displays the line above!
Is this something I should matter about?The first pass automatically uses faster settings during encoding, with the assumption that you don't care about the output of the first pass. As such, it disables High Profile features (8x8dct).

[LoRd_MuldeR]

Also note that x264 automatically chooses the suitable profile. That is: The lowest profile to which your resulting stream will comply.

Hence if you don't use any "high" profile features (and in the first pass you don't, for the reasons explained by Dark Shikari), the "main" profile is adequate for your stream.

[djesteban]

The first pass automatically uses faster settings during encoding, with the assumption that you don't care about the output of the first pass. As such, it disables High Profile features (8x8dct).

Cool, so I should assume that the second pass will indeed get a high profile by default right?

[LoRd_MuldeR]

Yes. With "--preset slow" definitely ;)

[djesteban]

Also note that x264 automatically chooses the suitable profile. That is: The lowest profile to which your resulting stream will comply.

Hence if you don't use any "high" profile features (and in the first pass you don't, for the reasons explained by Dark Shikari), the "main" profile is adequate for your stream.

Well, I knew it would auto-guess the best level for my settings, but I just want to make sure that with my settings, it will put a high profile for my second pass :):p

[djesteban]

Yes. With "--preset slow" definitely ;)

Nice, with that being said, I can come and answer to your question in the simple x264 launcher!
I still think there's a bug!!! hehe :)
Talk to you there

Thanks to all for your answers, really is a great forum!

[Carpo]

It's not required.

:thanks: LoRd_MuldeR :)