Please critique my x264 settings!

[Mixer73]

OK seeing as there's no 'right' and 'wrong' I'm looking for some areas where I can improve the efficiency of my encoding, while maintaining device compatibility, and I'm looking for some suggestions from people who know better than I.

Some background:

I have a long background with compression technologies, having been involved with companies like Macrovision, Fast, Quadrant, DPS, Darim, Optibase, Sonic Foundry, Canopus, Pioneer. I would have considered myself an expert in MPEG2 and early MPEG4 compression, however I don't have the time professionally to keep up those skills for present methods.

Currently I work with products from Viewcast but the needs for realtime compression are very different to what I am trying to achieve here.

My aim is to create nearly lossless results from fairly poor broadcast MPEG2, for playback either on DXVA or PS3/Xbox360. Primarily I playback on 7MCE, but I often take files to a friend's house who has a PS3. I also have a PSP so I'd like to understand what settings I might have to change to use that (apart from resolution), or whether I can do a one-size-fits-all profile.

Contrary to the above I do store a lot of TV so I want to get the files as small as possible while maintaining the quality. Hence quality-based 3-pass encoding. SD encodes I do at 720 width, HD encodes at 960 width. I've used deblocking 1:1 because the source material seems overly sharp, and I feel 1:1 delivers a very watchable picture.

My GUI is Xvid4PSP which isn't as well known as the others, and also doesn't support the standard preset system.

What I am hoping for is whether changes to things like ref frames or other more complex settings might give me more efficiency without terrible time penalties, and most importantly maintaining compatibility. I'm encoding on a Q6600, I can't wait until I can afford an i7.

So the challenge, give me some feedback. I'm very happy with the results I'm getting. Typically my files are half the size of a "scene" rip with equivalent quality.

Please tell me:
1. Anything you suggest that might realise smaller files
2. Anything I'm using that might be overkill, ie not worth the tradeoff time vs filesize
3. Anything else of relevance?

video codec:
x264

video cli:
--pass 1 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60
--pass 3 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60
--pass 2 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60

[Dark Shikari]

video cli:
--pass 2 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60
1. Your commandlines won't even work. You can't use multi-pass with CRF.
2. Use the presets and dump everything in bold. The presets are your friend.
3. I have no idea why you are lowering qcomp--and I doubt you do either. For deblocking, why not run a preprocessing filter, like fft3dfilter or dfttest? x264's deblocking is not meant to fix a crappy source.
4. You said "near lossless"; crf 24 is hardly that, though perhaps with your crappy source you could get away with it.
5. The VBV settings don't at all match the level you're using. Why did you pick 24 megabit?
My GUI is Xvid4PSP which isn't as well known as the others, and also doesn't support the standard preset system.How can it "not support" the preset system? Presets are just commandline options. Anything that offers an interface to let you add arbitrary options will clearly let you use presets.

[Mixer73]

Actually I should clarify, these are not actually x264 command lines, its the way my GUI stores the settings. Xvid4PSP uses its own presets and doesn't have the normal presets you see with Staxrip or MeGUI.

I might go and pull out the command lines once I've started an encode, that might be clearer.

[Dark Shikari]

Actually I should clarify, these are not actually x264 command lines, its the way my GUI stores the settings. Xvid4PSP uses its own presets and doesn't have the normal presets you see with Staxrip or MeGUI."Normal presets"? The presets are available natively in x264 as --preset. Staxrip and MeGUI's "presets" are unrelated to these. See x264 --help for details.

[Mixer73]

Well actually looks like those ARE the command lines.

That whole thing is made by Xvid4PSP, I have changed certain things but nothing I didn't understand, so I can't say why some things are set ;)

If I use those presets are they just single pass presets, I would still have to manually manage the multiple passes?

I'm not so sure on this stuff, this is why I need a GUI.

[Dark Shikari]

If I use those presets are they just single pass presets, I would still have to manually manage the multiple passes?CRF is single-pass only, so I'm not sure what your worry is here. For that matter, the presets have precisely zero to do with whether you use single or multi-pass; they solely affect the speed vs quality tradeoff. Read the help.I'm not so sure on this stuff, this is why I need a GUI.You seem to be very interested in choosing an optimal commandline, yet you're trying to rely on GUIs that don't generate such. Why not pick a different one?

[Mixer73]

You seem to be very interested in choosing an optimal commandline, yet you're trying to rely on GUIs that don't generate such. Why not pick a different one?

Well I can only say the results I have been getting out of Xvid4PSP have been excellent, and the developers have been good with bug fixes and stuff.

I couldn't get the same result out of StaxRip, and I couldn't get a result at all out of MeGUI.

I'm sure Xvid4PSP is out of date re the presets but I am getting good results out of it now.

But perhaps its time to revisit StaxRip.

[Mixer73]

It does seem that with each pass, the bitrate is being refined for the next pass in the cmd line.

[Dark Shikari]

What's the point of asking for a critique of your settings if you're just going to throw them in the bin and use some other unrelated preset from Xvid4PSP anyways? Seriously, I don't understand the point of this thread. The settings you posted won't even work. Therefore, if they are working, you're not using those settings.

Also, I'm just going to make a little guess here, but I'm pretty sure I know more about x264 than the Xvid4PSP developers. Just a hunch.

[Mixer73]

What's the point of asking for a critique of your settings if you're just going to throw them in the bin and use some other unrelated preset from Xvid4PSP anyways? Seriously, I don't understand the point of this thread. The settings you posted won't even work. Therefore, if they are working, you're not using those settings.

Well I'm hoping that any settings I can learn can translate to whatever tool I use, if Staxrip provides a more up to date command line that makes more sense then perhaps I can use that and integrate the advice you have given me.

I have no doubt you know a whole lot more than anybody, but I'm not sufficiently expert to use the command lines on their own at the moment, I need a GUI to glue it all together for me.

So I'm hoping that with Staxrip I can sort all the rest, and get a working command line that DOES make sense so I can improve what I am doing.

I'm only using Xvid4PSP because its very easy to use, and whatever the go is with the strange command line, it is giving me a result.

However it sounds as if I may be able to get the same or better result, quicker using x264 more closely to how its meant to be used.

This is the whole output of a job in progress with Xvid4PSP if it makes anything clearer:

TASK
------------------------------
Format: MP4 PS3 or XBOX360
Duration: 01:47:13:560 (160839)
VideoDecoder: MPEG2Source
Resolution: 1440x1080 > 720x400
VCodecPreset: Custom
VEncodingMode: ThreePassQuality
VideoCodec: MPEG2 > x264
VideoBitrate: 11980 > Q24.0
Framerate: 25.000
SourceType: PROGRESSIVE
FieldOrder: UNKNOWN
AudioDecoder: bassAudioSource
AEncodingPreset: AAC-HE ABR 64k
AudioCodec: AC3 > AAC
AudioBitrate: 384 > 64
Samplerate: 48000
Channels: 2

SCRIPT
------------------------------
import("C:\Program Files (x86)\Winnydows\XviD4PSP5\dlls\AviSynth\functions\AudioFunctions.avs")
import("C:\Program Files (x86)\Winnydows\XviD4PSP5\dlls\AviSynth\functions\VideoFunctions.avs")
loadplugin("C:\Program Files (x86)\Winnydows\XviD4PSP5\apps\DGMPGDec\DGDecode.dll")
loadplugin("C:\Program Files (x86)\Winnydows\XviD4PSP5\dlls\AviSynth\plugins\bass\bassAudio.dll")

video = MPEG2Source("C:\Encode\MOVIE- Mr - Mrs Smith - Nine Digital HD (2009-12-08 20-20).index\MOVIE- Mr - Mrs Smith - Nine Digital HD (2009-12-08 20-20).d2v",cpu=0,info=3)
audio = bassAudioSource("C:\Encode\MOVIE- Mr - Mrs Smith - Nine Digital HD (2009-12-08 20-20).index\MOVIE- Mr - Mrs Smith - Nine Digital HD (2009-12-08 20-20) T80 2_0ch 384Kbps DELAY 0ms.ac3")
AudioDub(video, audio)

AutoYV12()
Spline16Resize(720, 400)



VIDEO ENCODING
------------------------------
Encoding video to: C:\Temp\0098.264
x264 Q24.0 720x400 25.000fps (160839 frames)
...first pass...

x264.exe: --pass 1 --crf 24.0 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60 --sar 1:1 --stats "C:\Temp\0098.log" --output "C:\Temp\0098.264" "C:\Temp\0098.avs"

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: cabac=1 ref=1 deblock=1:1:1 analyse=0x1:0 me=dia subme=2 psy=1 psy_rd=0.0:0.0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 chroma_qp_offset=0 threads=6 nr=0 decimate=1 mbaff=0 constrained_intra=0 bframes=3 b_pyramid=0 b_adapt=2 b_bias=0 direct=3 wpredb=1 wpredp=2 keyint=250 keyint_min=25 scenecut=40 rc_lookahead=60 rc=crf mbtree=1 crf=24.0000 qcomp=0.50 qpmin=10 qpmax=51 qpstep=4 vbv_maxrate=24000 vbv_bufsize=24000 ip_ratio=1.40 aq=1:1.00
x264 [info]: profile Main, level 4.1
Best bitrate for Q24.0: 456kbps
...second pass...

x264.exe: --pass 3 --bitrate 456 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60 --stats "C:\Temp\0098.log" --sar 1:1 --output NUL "C:\Temp\0098.avs"

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: cabac=1 ref=3 deblock=1:1:1 analyse=0x3:0x133 me=esa subme=9 psy=1 psy_rd=1.0:0.0 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 chroma_qp_offset=-2 threads=6 nr=0 decimate=1 mbaff=0 constrained_intra=0 bframes=3 b_pyramid=0 b_adapt=2 b_bias=0 direct=3 wpredb=1 wpredp=2 keyint=250 keyint_min=25 scenecut=40 rc_lookahead=0 rc=2pass mbtree=1 bitrate=456 ratetol=1.0 qcomp=0.50 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0 qblur=0.5 vbv_maxrate=24000 vbv_bufsize=24000 ip_ratio=1.40 aq=1:1.00
x264 [info]: profile High, level 4.1
...last pass...

x264.exe: --pass 2 --bitrate 456 --level 4.1 --deblock 1:1 --direct auto --b-adapt 2 --trellis 2 --qcomp 0.5 --vbv-bufsize 24000 --vbv-maxrate 24000 --partitions all --subme 9 --me esa --rc-lookahead 60 --stats "C:\Temp\0098.log" --sar 1:1 --output "C:\Temp\0098.264" "C:\Temp\0098.avs"

x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: cabac=1 ref=3 deblock=1:1:1 analyse=0x3:0x133 me=esa subme=9 psy=1 psy_rd=1.0:0.0 mixed_ref=1 me_range=16 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 chroma_qp_offset=-2 threads=6 nr=0 decimate=1 mbaff=0 constrained_intra=0 bframes=3 b_pyramid=0 b_adapt=2 b_bias=0 direct=3 wpredb=1 wpredp=2 keyint=250 keyint_min=25 scenecut=40 rc_lookahead=0 rc=2pass mbtree=1 bitrate=456 ratetol=1.0 qcomp=0.50 qpmin=10 qpmax=51 qpstep=4 cplxblur=20.0 qblur=0.5 vbv_maxrate=24000 vbv_bufsize=24000 ip_ratio=1.40 aq=1:1.00
x264 [info]: profile High, level 4.1

[juGGaKNot]

Whats with Doom10.org ?

[Dark Shikari]

However it sounds as if I may be able to get the same or better result, quicker using x264 more closely to how its meant to be used.Erm, then why not modify your settings in Xvid4PSP to do so accordingly? I'm not sure what's so hard about this.

You've gone out of your way to change the settings from the (rather reasonable) defaults of Xvid4PSP to include a pointless exercise in 3-pass encoding. What's stopping you from changing them back?

I'm completely confused at this point. I gave you suggestions, many of which are equally applicable regardless of what GUI you use. Yet you haven't implemented a single one of them--nor given me a good reason why you haven't--despite the fact that you started this thread for the express purpose of asking for suggestions.

I give up.

[Mixer73]

You've gone out of your way to change the settings from the (rather reasonable) defaults of Xvid4PSP to include a pointless exercise in 3-pass encoding. What's stopping you from changing them back?

Nothing particularly, I'm just concerned by the fact that the settings in Xvid4PSP seem to have drifted somewhat from x264s. I'm just wondering if I try StaxRip it might be simpler.

3-pass encoding is in the drop down boxes so I used it, and have been for some time.

I'm just hesitant to go in and change the command line settings because I don't understand very much of it at all.

When you said above "dump the settings in bold" did you mean ad the bolded settings to presets and then try that?

I do really appreciate your help, and I'm sorry if I seem dumb!

[Dark Shikari]

Nothing particularly, I'm just concerned by the fact that the settings in Xvid4PSP seem to have drifted somewhat from x264s. I'm just wondering if I try StaxRip it might be simpler.If you're looking for simple GUIs, there are vastly simpler than either of those, like that new Micro x264 GUI. Check the GUI board...
3-pass encoding is in the drop down boxes so I used it, and have been for some time.Using more than one pass with CRF is stupid. Got that? Now don't do it.I'm just hesitant to go in and change the command line settings because I don't understand very much of it at all.If you don't understand an option, don't use it. Your commandline is full of options that you don't understand, so you shouldn't be using them.When you said above "dump the settings in bold" did you mean ad the bolded settings to presets and then try that?Dump. Remove. Eliminate. Drop. Excise. Discard. Eject. Eradicate...

[Mixer73]

Dump. Remove. Eliminate. Drop. Excise. Discard. Eject. Eradicate. :p

Right OK, I made the 'mistake' of x-posting this to Doom10 as well and I got other "advice" over there which confused me more!

Funny that I posted there so you would see it and was expecting others to help me here :P

So 2-pass and remove the stuff in bold, check.

[Dark Shikari]

So 2-pass and remove the stuff in bold, check.No, as I said above, more than one pass with CRF is pointless.

Anyways, I'm tired of repeating myself. I've said "don't use more than one pass with CRF" in four separate ways in this thread already.

[Mixer73]

Thanks, I'll go do some testing.

What I didn't expect to find out was that my cmd lines are largely a balls up!

[Mixer73]

Got the message regarding multiple passes.

So how does this look:

--preset veryslow --tune animation --crf 24.0 --level 4.1 --deblock 1:1

I found out that Xvid4PSP can save custom command lines so this could be a lot easier than I thought.

[nurbs]

--tune animation already contains --deblock 1:1

[LoRd_MuldeR]

Mixer73, you can type "x264.exe --fullhelp" for a complete list of all presets/tunes and the parameters they actually set.

[Mixer73]

Thanks, I was wondering about that.

I've just done a test encode, I gotta say the quality vs speed is stunning. Around 2x realtime on my Q6600, wow.

[RunningSkittle]

huh 2x realtime with --preset veryslow ? I dont think so...

[Groucho2004]

huh 2x realtime with --preset veryslow ? I dont think so...

Maybe overclocked to 20 GHz :D

[MatLz]

I beat him:
Preset veryslow at 6x realtime on my crapold dualcore...
My secret: pointresize(4,4)
:D

[Mixer73]

huh 2x realtime with --preset veryslow ? I dont think so...

The whole thing ran at 66FPS. SD resolution. I'm changing GUIs so I at least have all the current settings at hand.

Now that I have more comments about multi pass quality-based encoding and such I will need to spend some time doing some more tests before I can come back with a 'new' setting that people can critique.

Thanks all for your feedback.

[MatLz]

(Return to serious mode)multi pass quality-based encodingAfter all the explanations you got on this thread...I hope you simply forgot the ' , ' ? Am I right?

[Mixer73]

(Return to serious mode)After all the explanations you got on this thread...I hope you simply forgot the ' , ' ? Am I right?

Well comments = people saying it doesn't work. Xvid4PSP is certainly doing something with its multiple passes, even if its just basically selecting a bitrate for me. Obviously I'm the only freak around using these settings.

Well I'm going to spend some time playing with Staxrip so I can get a feel for the current settings, my previous GUI doesn't seem really relevant to x264 as its currently implemented so if I'm going to use x264 well I need to take Dark Shikari's advice and use the presets rather than re-inventing the wheel.

I think the job for me now is to try single pass quality encoding vs 2pass encoding and see how the results stack up vs filesize.

I'm very encouraged by what I have learned so far so thanks all for your help and I'm sorry I was so far off the page :)

[Chengbin]

Well comments = people saying it doesn't work. Xvid4PSP is certainly doing something with its multiple passes, even if its just basically selecting a bitrate for me. Obviously I'm the only freak around using these settings.

Well I'm going to spend some time playing with Staxrip so I can get a feel for the current settings, my previous GUI doesn't seem really relevant to x264 as its currently implemented so if I'm going to use x264 well I need to take Dark Shikari's advice and use the presets rather than re-inventing the wheel.

I think the job for me now is to try single pass quality encoding vs 2pass encoding and see how the results stack up vs filesize.

I'm very encouraged by what I have learned so far so thanks all for your help and I'm sorry I was so far off the page :)

Let me make this very clear.

There is ABSOLUTELY NO DIFFERENCE in quality between multi pass and CRF given the same bitrate and settings. You are WASTING TIME if you do 2 pass encoding when you don't care about having a strict final size.

Got that?

[Mixer73]

There is ABSOLUTELY NO DIFFERENCE in quality between multi pass and CRF given the same bitrate and settings.

I understand this, but I am hoping that by using the quality modes I might be able to achieve smaller files than by setting the bitrate explicitly.

Its my understanding of compression in general that multi pass will let you get a lower bitrate for the same effective quality.

Also quality mode with a higher preset might be a better balance of quality vs time than 2-pass, this is what I want to test.

[LoRd_MuldeR]

I understand this, but I am hoping that by using the quality modes I might be able to achieve smaller files than by setting the bitrate explicitly.

That makes no sense. By using 2-Pass mode and setting the target bitrate explicitly, you already define final file size!

file_size = bitrate * duration

If you want a smaller file, simply lower the bitrate accordingly. Yes, it really is that easy :D

Its my understanding of compression in general that multi pass will let you get a lower bitrate for the same effective quality.

Wrong. 2-Pass mode gives you the best possible quality for a given target bitrate or a given target file size, but it doesn't generally compress more efficient than CRF mode!

In fact a video encoded with 2-Pass mode will have (almost) identical quality compared to a video encoded with 1-Pass CRF mode -- given that both videos have the same average bitrate (file size).

The difference is that with CRF mode the final file size is completely unpredictable, while the final size is known in advance when using 2-Pass mode ;)

Conclusion: If you need to hit a specific file size or a specific bitrate, then go with 2-Pass mode. But if you want to hit a specific level of quality and don't care about filesize, then use CRF mode.

[qyqgpower]

I think there're still some features working better in multi-pass than crf mode, like --direct auto.
But since the frame decision is done in 1st pass, unless one uses high precision 1st pass settings,
they would obtain worse decision with fast 1st pass+slow 2nd pass than simply slow crf mode.

[ajp_anton]

Simplified:
CRF: You choose a quality, x264 will encode.
2-pass: You choose a bitrate (filesize). On the 1st pass x264 finds out what quality will fit into that size, and the 2nd pass it encodes with that quality.

[Chengbin]

I think there're still some features working better in multi-pass than crf mode, like --direct auto.


You get such a negligible increase in quality you can probably say you don't get any.

[zamwow]

1. Your commandlines won't even work. You can't use multi-pass with CRF.

Anyone mind giving me the list of things that doesn't work with CRF?

[Dark Shikari]

Anyone mind giving me the list of things that doesn't work with CRF?

--pass > 1
--bitrate

[zamwow]

--pass > 1
--bitrate

oops, thought the things in bold needed more than 1 pass to work. My mistake. :o

[Carpo]

Ok i dont know if this is x264 related or dgindexnv but will try here first, I am using the latest x264 from x264.nl (at this time 1373) and i am using

D:\>x264.exe --profile high --level 4.1 --preset veryslow --tune film --pass 1 --bitrate 9132 --stats "stats" --slow-firstpass --thread-input --b-adapt 2 --ref 4 --partitions all --me umh --direct auto -o gran.torino.mkv gran.torino.avs

But it seems like x264 is stalling, the cpu useage is at 0%, CUVIDServer is running (used to pipe the dgv to avisynth) can anyone see anything wrong with the above?

Thanks, if this isnt a x264 issue i will post in nvtools thread

[J_Darnley]

I have experienced this a few times and after using VDub and avs2yuv. Make sure that there is not another instance of a script using DGSource. Close it if there is. If not, try restarting the cuvid server then running x264 again.

[LoRd_MuldeR]

Well, if you open the very same AVS file in VirtualDub, can you play it up to the point where x264 freezes and beyond that point?

[Carpo]

Well, if you open the very same AVS file in VirtualDub, can you play it up to the point where x264 freezes and beyond that point?

Yes plays fine, thing is it seems random, i started the encode last night, woke up this morning and it said 32% - 0% cpu usage, i thought the CUVIDServer had stopped but it was still there so i started the encode again, it stopped on 7.3% this time.

I wondering if its something in the above line that is causing x264 to kick out, and therefore i will need to correct it, or if its a dgindexnv issue

I have experienced this a few times and after using VDub and avs2yuv. Make sure that there is not another instance of a script using DGSource. Close it if there is. If not, try restarting the cuvid server then running x264 again.

only things running are the server avisynth and x264 and the only things that are in the avs file is:

loadplugin("D:\dgdecnv200b6\DGDecodeNV.dll")
dgsource("D:\gran.torino.dgv", resize_w=1280, resize_h=720)

[ronnylov]

Half a year ago or something like that, the vbv settings was said to work better when using 2-pass encoding compared to crf encoding. But later things was improved in x264 and the vbv settings work fine also when using crf.

I think I used "a similar method" at that time. First encode with crf mode to get a bitrate that corresponds to the chosen crf value. I then could use that bitrate with 2-pass encoding to get the vbv settings to work in a good way. Maybe that is what XviD4PSP is trying to do when Mixer73 use it? Maybe it is an old version of the program?

Anyway with later builds of x264 I beleive setting VBV limits should work very well also when using CRF mode so there is no need to use 2-pass encoding any more just to get vbv limitations. Have I understood this correctly? so historically it may have been a reason to use multipass encoding based on the bitrate you got from a crf encoding if it was important to respect the vbv limitations of a target device. I sounds like XviD4PSP is a prograim aimed for encoding to devices like PlayStationPortable.

[Mixer73]

Ronnylov, I do believe this is what Xvid4PSP is doing.

However I'm now using Staxrip and really liking it so far. I gotta experiment some more and then maybe I can ask some sensible questions of all the people here :)