x264 Settings for CRF vs Multi-Pass

[delanejenkins]

To the best of my understanding many of the settings for x264encoding such as trellis, b-adapt, b-frames etc, effect compression and quality and are utilized most efficiently when using multi-pass encoding or trying to maintain a certain file size and bitrate. Is it best to use these same multi-pass settings when using CRF encoding and not worried about file size, or will any of the settings not be needed?

[Dark Shikari]

To the best of my understanding many of the settings for x264encoding such as trellis, b-adapt, b-frames etc, effect compression and quality and are utilized most efficiently when using multi-pass encodingNo, they work just fine in CRF mode.

[delanejenkins]

Thanks Dark. So all of the settings should be left the same whether we're using crf or multi-pass?

[tph]

Thanks Dark. So all of the settings should be left the same whether we're using crf or multi-pass?
Avoid using VBV with one-pass encodes, that's about it I think.

[delanejenkins]

what is the reasoning for not using vbv? Shoud it just be set to 0?

[nm]

what is the reasoning for not using vbv? It should just be set to 0?
If you need VBV, use it and encode in two-pass mode if possible. It doesn't work well in single-pass modes. if you don't need VBV, simply don't set vbv-maxrate and vbv-bufsize.

[delanejenkins]

Thanks. I am just trying to look at my encoding process step by step because no matter what I do I continue to get errors in my encodes. I was wondering if I need to change some settings for crf to avoid errors or just switch over to multipass and quit using crf

[kemuri-_9]

Thanks. I am just trying to look at my encoding process step by step because no matter what I do I continue to get errors in my encodes. I was wondering if I need to change some settings for crf to avoid errors or just switch over to multipass and quit using crf

errors is a very vague term and can mean lots of different things.
you should clarify what the problem you're having is and then we can help out to solve the problem.

[delanejenkins]

By errors I mean: the video plays fine then suddenly has green or other multi-colored blocks everywhere then goes back to playing fine. Sometimes the video will just freeze and the audio will continue and then the video will restart. This happens on my pc and my media player. It happens whether I mux into mkv or ts. I have been using megui with crf for my encodes but have been experimenting now with ripbot because I'm wondering if megui is the problem. All of my settings are within h.264 specs and dxva compliant. I do not have any codec packs installed. I strictly use ffdshow, haai, mpc. I have a 2.8ghz quad core processor so I know it isn't a lack of power. I am not sure what else to look for as a remedy.

[nm]

I have a 2.8ghz quad core processor so I know it isn't a lack of power.
Perhaps you have too much power there and the system overheats --- try running x264 with only one or two threads. Or there's something wrong with your RAM chips --- run Memtest86.

Have you had other stability problems, for example random lockups of some program or the whole system?

[Blue_MiSfit]

Yes this sounds very much like a hardware or possibly source content issue. What build of x264 are you using?

[delanejenkins]

I have run stress tests on my system and never had a heat issue. I have no lockup issues either but I will run a memtest. It's not an x264 build issue because it's happened with several builds. My sources are straight from original DVDs ripped with anydvd and DVD decrypter

[delanejenkins]

Ok guys I ran a memtest and everything came out alright. Any ideas what else could be the problem?

[Dark Shikari]

If the error occurs repeatedly at the same place every time you encode, and occurs even with an unpatched build of x264 (e.g. from x264.nl), collect the smallest possible input sample which, when encoded, generates the broken stream, the settings which do so, and post both.

[delanejenkins]

It doesn't occur at the same place each time. It happens at random points in random videos.

[Dark Shikari]

It doesn't occur at the same place each time. It happens at random points in random videos.So if you encode the same video twice with the same settings, it happens at two different points, even if multithreading is off?

I'm going to blame your CPU in that case.

[delanejenkins]

I have not tried to encode with multi-threading off. Also, I was using megui which I believed you mentioned at another time does use a patched build. I will run a test again to see if I can reproduce the errors and try another gui like ripbot that doesn't use a patched build. My cpu is only about 2 months old. I would hope that is not the problem

[JEEB]

Basically to try out where the problem is you can:

Download the unpatched build from x264.nl and put it into megui\tools\x264.
Try running the encode that way.
If the problem still persists, it's not the x264 build.
Clean up your avs script as much as possible and start testing with just command line instead of meGUI (check for any multithreading and/or other stuff that might give different results - I've had black lines come etc. with multithreading; also usually when meGUI tells you to use directshowsource it's usually better to use ffms2 [clicky (http://forum.doom9.org/showthread.php?t=127037)]).
Lastly, try changing settings and encoding on another PC.
If all this fails, or if some settings give you errors no matter what system it was run on I guess people would like a sample as Dark Shikari said.


As for playback, you could test with Kovensky's mplayer (clicky (http://kovensky.project357.com/)) f.ex. to check the streams without any muxer/directshow intervention - also make sure your meGUI is updated as well.

Personally I haven't had anything like that happen to me with my 32bit builds so I'm guessing it's not that, but if you end up blaming me I'll make sure to fix it as soon as possible :3 Also may I remind you that overclocking is not a nice thing to do when encoding. Hurf.

[delanejenkins]

After doing a little more reading, I am starting to wonder if my problem is the fact I have been using vbv with crf with all my encodes. Several people said it was causing severe macroblocking and broken I-frames. Do you think this could be the problem?

[Forteen88]

To the best of my understanding many of the settings for x264encoding such as trellis, b-adapt, b-frames etc, effect compression and quality and are utilized most efficiently when using multi-pass encoding or trying to maintain a certain file size and bitrate. Is it best to use these same multi-pass settings when using CRF encoding and not worried about file size, or will any of the settings not be needed?Just --direct auto
and VBV (as said) benefits from multiple pass.

[delanejenkins]

Thanks for all the help guys. Now I have to decide if I will continue using crf but remove vbv or switch over to 2pass. My last question is this: I have to keep my encodes compliant with h264 levels for hardware acceleration with my nmt so I do need to to cap off my bitrate. After going back through about 100 of my encodes, according to mediainfo, my avg bitrates are around 7500-8000 for 720p and 2500-3000 for 480p when using CRF 18. This is not near my max bitrates of 17.5k for High L3.1 and 12.5k for High L3.0. Would I be safe continuing to use CRF with no vbv values or should switch over and use 2pass with the above avg bitrates?

[Chengbin]

That depends on your value of qcomp. If it is at default of 0.6, then you should be fine.

The more important thing to look in mediainfo is the peak bitrate.

[delanejenkins]

Thanks Chengbin. I have not made any alterations to the qcomp in megui and I'm not sure the setting ripbot uses. Im sure it is set to default though. When looking in mediainfo I only see one bitrate listed under video. Is this the peak bitrate? I was assuming it was the avg bitrate.

[Chengbin]

Oh, I forgot, you can't check peak bitrate with MKV files on mediainfo, only MP4s.

Use something called bitrate viewer to check peak bitrate. It takes a little longer (must read through every frame) though.

http://www.softpedia.com/get/Multimedia/Video/Other-VIDEO-Tools/Bitrate-Viewer-winhoros.shtml

[delanejenkins]

Yeah I actually just ran some of my encodes through bitrate viewer before I received your last reply. None of the 480p encodes had peaks that came close to the level 3.0 maxes. My 720p encodes did have peaks that went beyond the level 3.1 maxes though so I guess if I bump my 720p encodes up to level 4.0 or 4.1 I should be ok. My networked media tank can handle up to level 4.1 High profile. If my understanding of h264 levels are correct as long as you stay within the maximum reference frames and bitrate for that particular level you stay compliant. Does this sound right to you?

[Chengbin]

Yeah I actually just ran some of my encodes through bitrate viewer before I received your last reply. None of the 480p encodes had peaks that came close to the level 3.0 maxes. My 720p encodes did have peaks that went beyond the level 3.1 maxes though so I guess if I bump my 720p encodes up to level 4.0 or 4.1 I should be ok. My networked media tank can handle up to level 4.1 High profile. If my understanding of h264 levels are correct as long as you stay within the maximum reference frames and bitrate for that particular level you stay compliant. Does this sound right to you?

Levels has nothing to do with the encode.

What I mean is if a video is L3.0, then L4.1 or L5.1 is the same thing. x264 will choose the lowest level needed for your settings unless you choose to use a higher level.

[delanejenkins]

So basically a video tagged as a level 4.1 for example is compliant as long as it stays up to the max bitrate and ref frames of level 4.1 or below. However a video tagged as level 3.0 is compliant at that level specs or below but not above it?

[nm]

So basically a video tagged as a level 4.1 for example is compliant as long as it stays up to the max bitrate and ref frames of level 4.1 or below.Yes.
However a video tagged as level 3.0 is compliant at that level specs or below but not above it?
Um, no. A video tagged as level 3.0 is compliant at that level as long as it stays within the level specs (as you said above for L4.1). Then it is also compliant with higher levels, but not necessarily with lower levels.

[delanejenkins]

Thanks NM. I think we're on the same page. Say you encode a video and tag it as a level 3.0 but the max bitrate spikes up to say 15000 it is no longer compliant. But if that same video is tagged as a level 3.1, 4.0 or 4.1 it is now compliant? (well assuming the ref frames and other specs are within 3.1, 4.0, 4.1) So if this is true we are better picking a higher level for our encodes to ensure we stay compliant?

[Dark Shikari]

So if this is true we are better picking a higher level for our encodes to ensure we stay compliant?No, to ensure you stay compliant, you set the max bitrate and bufsize accordingly. x264 will yell loudly if you pick settings that are incompatible with your chosen lvel.

[nm]

So if this is true we are better picking a higher level for our encodes to ensure we stay compliant?
Use the highest level that your target playback device supports.

[delanejenkins]

I am actually trying to no longer use vbv with crf as you guys instructed. I'm trying to choose my levels to encode to. After running many of my prior crf 18 encodes through bitrate viewer last night, it appears I can safely encode my 480p to level 3.1 and 720p to level 4.1 without any bitrate spikes coming close to the max bitrate of each level. Is my logic correct here to ensure compliancy for my media player which can handle up to level 4.1 high profile?

[nm]

Well, you can't ensure compliancy that way. Most of the encodes would probably work fine, but some sources will exceed those limits when encoded with CRF=18 without VBV. Try encoding white noise, for example.

Why don't you do it right and run a second pass with VBV? You can use CRF in the first pass along with faster encoding options to determine a good average bitrate. This will only be 10--50 % slower than 1-pass CRF unless you do heavy AviSynth filtering that dominates the encode time.

After doing a little more reading, I am starting to wonder if my problem is the fact I have been using vbv with crf with all my encodes. Several people said it was causing severe macroblocking and broken I-frames. Do you think this could be the problem?
I don't think your problem with corrupted encodes comes from CRF&VBV. I bet you get the same corruption without VBV and in two-pass mode with VBV. Still, you should try both and see if it helps.

[delanejenkins]

Nm, so run a crf pass like i have been but cut my settings back to the minimum. I then take the avg bitrate from this pass and plug it in as my bitrate for another abr pass with my vbv settings?

Well although I'm not feeling much confidence from you over there NM, I sure as hell wish this would fix my problem haha :)

[nm]

Nm, so run a crf pass like i have been but cut my settings back to the minimum. I then take the avg bitrate from this pass and plug it in as my bitrate for another abr pass with my vbv settings?
Yep. Run the first pass with --pass 1 --crf 18 to generate a stats file, and the second pass with --pass 2 --bitrate X --vbv-bufsize Y --vbv-maxrate Z, where X is the average bitrate from the first pass and Y and Z are your VBV parameters. There are some settings that should be kept for both passes. Check this thread for suggestions on what to change: http://forum.doom9.org/showthread.php?t=141202

[delanejenkins]

Nm, can I accomplish what you're telling me to do by running a crf encode in ripbot or megui with the suggested settings from the thread you linked then take the avg bitrate of that encode and plug it into another ripbot or megui single pass abr encode?

I may be way off and what you're suggesting will need to be done with command line and not a GUI to utilize the stats log you're talking about.

[nm]

Nm, can I accomplish what you're telling me to do by running a crf encode in ripbot or megui with the suggested settings from the thread you linked then take the avg bitrate of that encode and plug it into another ripbot or megui single pass abr encode?
Perhaps, but I don't know how those frontends manage the stats file. If they automatically remove it after the encode, it won't work.

There are some GUIs that support this kind of 2-pass encoding (http://forum.doom9.org/showthread.php?p=1286841#post1286841) based on a CRF first pass; sx264 and HDConvertX at least. Of course command line works, manually or through a script that parses the bitrate automatically.

[delanejenkins]

Thanks alot NM for all your help. Thanks everyone else who offered advice as well. Now I'm off to do some testing and see if by the grace of god this remedies my problem. Here's hoping. I will keep you posted.

[delanejenkins]

Well I did my first test run with CRF with vbv off. No errors at all through an entire movie. I'm hoping it was not just a lucky run. I guess now I will just keep my fingers crossed, keep encoding and hope this was the cure. Time will tell!