How to get x264 to vary the bitrate more?

[Chengbin]

I have this dilemma. I'm starting to use CRF now, and I got a problem with it.

I would like to start off by asking for a definition of CRF.What does it actually mean? (Don't ask me to use search, because there are lots of misunderstanding about the meaning of CRF)

My dilemma is that x264 doesn't use enough bitrates for high motion scenes (for example a fight). Most scenes with little to moderate motion doesn't require much bitrate to look good. I don't want to use high bitrates, and I would like to stay in the 500-700Kbps area.

Most of the video look very good using CRF 25, but a few high motion scenes have some blocking (some, but noticeable). I have to use a CRF value of 22 to 23 for those scenes to look good, but the rest of the scenes have little noticeable improvement. I have to almost double the bitrate so those parts look good.

Is there a way to get x264 to spend like around 600Kbps for little to moderate motion scenes and spend 1500Kbps for the 20 seconds of high motion stuff?

CRF doesn't work very well for anime. I use a CRF 25, and it looks very good. But with anime, I use CRF 25 and resulted in a 600Kbps file. I have encoded a 325Kbps video of the same video, and comparing those, I see very little difference on my monitor and no difference on my Archos 5.

[Dark Shikari]

If you want more bitrate in high motion scenes, raise qcomp. That's what it's there for.

[Amato]

CRF doesn't work very well for anime. I use a CRF 25, and it looks very good. But with anime, I use CRF 25 and resulted in a 600Kbps file. I have encoded a 325Kbps video of the same video, and comparing those, I see very little difference on my monitor and no difference on my Archos 5.

Then use a higher CRF setting for anime.

[Chengbin]

I'm assuming that's quant compression at 0.6

I'm reading a x264 settings guide, and it says a qcomp of 1 = constant quantizer. It says CQ mode produces a file bigger than CRF at the same visual quality. But as I raise qcomp, I'm making it closer to CQ, and does that mean I lose compression efficiency?

Is a qcomp value of 0.8 good, or is it overkill?

Is there a reason qcomp's default is 0.6? Wouldn't it be better to leave it at some high value and let the encoder choose if it wants to use a higher bitrate.

[Dark Shikari]

I'm assuming that's quant compression at 0.6

I'm reading a x264 settings guide, and it says a qcomp of 1 = constant quantizer.Constant frame quantizer, not constant macroblock quantizer.But as I raise qcomp, I'm making it closer to CQ, and does that mean I lose compression efficiency?Stop taking every damn sentence you read for gospel and judge with your eyes.

[Chengbin]

I tried 0.8, and tried 1, and I choose to stick with 1. Now my encodes are perfect. Thanks a lot Dark Shikari.

I hope my Archos 5 doesn't mind over 7Mbps peaks...

Why is qcomp not defaulted at 1? Wouldn't that help quality?

[Sagekilla]

Because qcomp 1 means "vary the bitrate as much as necessary to maintain constant quality." Using qcomp 0 would mean constant bitrate (= variable quality). Values in between give a nice medium between bitrate variability (think VBR mp3) and constant bitrate.

[akupenguin]

Values in between give a nice medium between bitrate variability (think VBR mp3) and constant bitrate.
Do you mean "VBR mp3 isn't constant quality, it too is a compromise between quality and predictable bitrate", or "VBR mp3 is an example of constant quality, qcomp is a compromise between that and CBR"? Both are wrong.

I would explain qcomp as: There's a continuum of bitrate variability, which looks like (not to scale)
| <- true constant bitrate
|
| <- VBV-based constant bitrate
|
| <- constant perceptual quality
|
| <- constant quantizer
|
| <- constant PSNR
qcomp is not a compromise between CBR and CQP, it's an attempt to optimize for an entirely different target (psy) which happens to be between them. But like any psy, the optimal value depends on the viewer, and it's hard to tune.

[ncahammer]

Because qcomp 1 means "vary the bitrate as much as necessary to maintain constant quality." Using qcomp 0 would mean constant bitrate (= variable quality). Values in between give a nice medium between bitrate variability (think VBR mp3) and constant bitrate.

Suppose a CRF with qcomp=1 encode reaches a *very* complex scene which finally encodes using a bitrate 10 times the final (average) bitrate of the video. Meaning if the final bitrate of all the video is 1000Kbps, this scene peaks at 10000Kbps

Now suppose we start for the same video a first pass encode specifying as bitrate, the previus average (1000Kbps)

Will this also give 10 times the specified bitrate for the same scene or it will cut it down ?
If the answer is the latter, will the 2nd pass correct this ?
I strongly doubt, which leads me to the final question, is anyway that 2pass encode, will 'distibute' the bitrate among different scenes the same way that a CRF does ?

[Dark Shikari]

Suppose a CRF with qcomp=1 encode reaches a *very* complex scene which finally encodes using a bitrate 10 times the final (average) bitrate of the video. Meaning if the final bitrate of all the video is 1000Kbps, this scene peaks at 10000Kbps

Now suppose we start for the same video a first pass encode specifying as bitrate, the previus average (1000Kbps)

Will this also give 10 times the specified bitrate for the same scene or it will cut it down ?
If the answer is the latter, will the 2nd pass correct this ?
I strongly doubt, which leads me to the final question, is anyway that 2pass encode, will 'distibute' the bitrate among different scenes the same way that a CRF does ?2-pass uses (for practical purposes) the same algorithm for bitrate distribution that CRF does.

[ncahammer]

2-pass uses (for practical purposes) the same algorithm for bitrate distribution that CRF does....but with different objectives.

And the objectives of a 2pass encode are dictated by an average bitrate and some (limited?) variation from it
Will the 2pass, encode the 10Mbps-scene at 10Mbps, when you have specify a 1Mbps as average bitrate ? My tests show that is not

[Dark Shikari]

Will the 2pass, encode the 10Mbps-scene at 10Mbps, when you have specify a 1Mbps as average bitrate ?Probably

[akupenguin]

...but with different objectives.
With exactly the same objectives.
If you have the equation y=f(x), you can either fix y and solve for x, or fix x and solve for y. You will in general get different answers, unless you happened to pick a y in the one case that corresponds to the x in the other case, but you're solving the same equation. To be concrete, x is ratefactor and y is filesize. CRF can evaluate f() directly, so it works in 1 pass. If you want to invert f(), that's a bit more complex, and involves 2 passes.

[ncahammer]

DS, aku : Thanks, I think I understand it now, specially after this (http://forum.doom9.org/showthread.php?p=1230509#post1230509) and this (http://forum.doom9.org/showthread.php?p=1270647#post1270647) posted in the other thread 2-pass with bitrate from the CRF pass should theoretically give better results than either CRF or pure 2-pass at the same bitrates,I'll make some tests, I think a CRF first-pass followed by a second pass, should give the best results

[Chengbin]

I got a 897Kbps video with a maximum peak bitrate of 5960Kbps. I'm curious, for high motion scenes, is it averaging say like 2000-3000Kbps for that 10 seconds of high motion?

[poisondeathray]

I got a 897Kbps video with a maximum peak bitrate of 5960Kbps. I'm curious, for high motion scenes, is it averaging say like 2000-3000Kbps for that 10 seconds of high motion?

How do you expect anyone to answer that?

Why don't you check with a bitrate viewer or stream analyzer?

[Chengbin]

What I'm curious is, is x264 peaking bitrates for a second or 2 or consistently throughout the high motion scene.

[poisondeathray]

What I'm curious is, is x264 peaking bitrates for a second or 2 or consistently throughout the high motion scene.

Why don't you check? Here's a free one

http://www.winhoros.de/docs/bitrate-viewer/index.html

[Chengbin]

Woah, thank god for VBR. In a 326Kbps video, the bitrate ranges from 4Kbps to 2942Kbps!!! I was shocked on how many frames were only 4-20Kbps (not exactly shocked, but shocked that it takes that little).

[Chengbin]

I just found out my Archos 5 does not like the peaks. The highest successful peak I got it to play without lag is 3870Kbps for 16x9 videos. So if I set VBV to 3900 it will prevent bitrate peaks above 3900?

[Sharktooth]

yes, but be also sure to set the correct buffer size and remember VBV works best with multiple passes.

[Chengbin]

I'm using CRF, so uh oh!

How do I choose a buffer size?

I just tested a video (720x304, or 2.35), with a 4500Kbps peak and it played perfectly fine. Since 720x480=345600 and 720x304=218880, and the 720x480 has 1.58 times more pixels that 720x304, does that mean that theoretically it has a maximum playback peak of a little over 6000Kbps?

Also, the accuracy or VBV in CRF, what is its tendency? Does it have the tendency to go over when it is in CRF (less accurate)

[LoRd_MuldeR]

I'm using CRF, so uh oh!

No problem. You can run the first pass of a 2-Pass encode in CRF mode ;)

How do I choose a buffer size?

--vbv-bufsize

[Chengbin]

I'm not sure what you mean. I don't want to use 2 pass because it takes a long time.

Sharktooth told me to set a buffer size. --vbv-buffsize is not a value.

[LoRd_MuldeR]

I'm not sure what you mean.

First pass:
--crf 22 --pass 1 --stats foo.stat

Second pass:
--bitrate <resulting_bitrate_of_first_pass> --pass 2 --stats foo.stat --vbv-maxrate <integer> --vbv-bufsize <integer>

I don't want to use 2 pass because it takes a long time.

Then better forget about VBV ;)

Sharktooth told me to set a buffer size. --vbv-buffsize is not a value.

Uhm? :confused:

[Chengbin]

Read the first post of page 2. Sharktooth said "yes, but be also sure to set the correct buffer size and remember VBV works best with multiple passes. "

So no CRF + VBV? I have to limit peak bitrate. Will it at least do something?

I don't see why VBV has a degree of accuracy with it. The purpose is to limit peak bitrate to x bitrate. So if the peak bitrate allowed is 5000Kbps, and x264 wants to use 6000Kbps, then x264 just reencode that frame at 5000Kbps.

[LoRd_MuldeR]

Read the first post of page 2. Sharktooth said "yes, but be also sure to set the correct buffer size and remember VBV works best with multiple passes. "

Yes, Sharktooth was right. VBV is known to work a lot better in 2-Pass mode. And the buffer size can be specified using the "--vbv-bufsize" option.

So no CRF + VBV? I have to limit peak bitrate. Will it at least do something?

You can use 2-Pass VBV with CRF by running a 2-Pass encode and using CRF mode for the first pass. See my previous post.

Of course you can also use 1-Pass VBV, but quality will suffer more than it does with 2-Pass VBV. Artifacts may occur...

I don't see why VBV has a degree of accuracy with it. The purpose is to limit peak bitrate to x bitrate. So if the peak bitrate allowed is 5000Kbps, and x264 wants to use 6000Kbps, then x264 just reencode that frame at 5000Kbps.

Nope, the purpose of VBV is not (only) to limit the peak bitarte. It's purpose is to make sure that the buffer will never overflow and never underflow!

Once VBV runs out of buffer, it may not have enough bits left to code the current frame at an acceptable quality. And that may result in ugly artifacts or even VBV violations.

So VBV must look ahead and must be able to predict the bit cost very accurately. This of course is much more reliable in 2-Pass mode...

[Chengbin]

I'm curious, if I set a VBV value of 25000Kbps, and say the video never hits that, say it only hits a maximum of 5000Kbps. Is VBV basically unused in the encoding?

I should probably lower qcomp to like 0.8 or 0.9 instead of 1.

[LoRd_MuldeR]

I'm curious, if I set a VBV value of 25000Kbps, and say the video never hits that, say it only hits a maximum of 5000Kbps. Is VBV basically unused in the encoding?

In that case VBV would not be "unused". It would still take care that no buffer overflows/underflow occur.

Of course less restrictive VBV settings (huge buffer + high maxrate) make it more unlikely that VBV will degrade the visual quality...

[Chengbin]

So VBV has been degrading the quality of my encodes all this time??? I've had VBV of 25000 and buff size of 14000 for ALL my encodes!!!

[Sharktooth]

no, unless your encodes had very high bitrate spikes.

[ajp_anton]

You can't simply set a maximum bitrate. Frames are of very different sizes, and most of your I-frames are probably above your limit.
The max bitrate has to be set for a specific interval, and this is where bufsize comes in. That interval is simply [bufsize]/[max bitrate] seconds (not that you ever need to calculate this...)

[Arcueid]

If I want to get a good bitrate from pass 1 which encoded in crf mode, should I use the same settings and filters in both pass?

[LoRd_MuldeR]

So VBV has been degrading the quality of my encodes all this time??? I've had VBV of 25000 and buff size of 14000 for ALL my encodes!!!

VBV potentially hurts the quality, even with a relatively large buffer. Therefore you really shouldn't use VBV unless you must do so (for hardware compatibility).

And to say it once again: 2-Pass VBV should be preferred over 1-Pass VBV...

[LoRd_MuldeR]

If I want to get a good bitrate from pass 1 which encoded in crf mode, should I use the same settings and filters in both pass?

Whenever you do two passes, you should use an identical source (avs script) and the same encoder settings for both passes, except for settings that don't effect the second pass.

Just look what options MeGUI's "Turbo" mode lowers/disables in the first pass. These options won't effect the final encode (second pass) in a noticeable way.

[Esurnir]

I think maybe you don't understand what the VBV model is Chengbin, so let me explain it to you :

You set a buffer size and a bitrate, what the encoder imagine is that the stream it output is sent bit-after-bit at a constant rate feeding a buffer of a fixed size. The encoder simulate the buffer as stocking the raw bits of the stream then once a frame is supposed to be decoded it deplete the buffer from the bits relevant to that frame. Each "frame is being displayed" time mark represents deadlines at which the bits relevant to that frame must be in buffer. Which is why most of the time the buffer contain bits of future frame, if a big frame is being downloaded (I frame) the player can then decode the frame it has "in stock" while the I frame is being downloaded.

A vbv-buffsize equal to a vbv-maxrate means that on average the buffer can stock "up to" 1 second of data.

What x264 try to avoid is having no decodable frame at the time where a frame is supposed to be displayed. A buffer overflow is supposed to be avoided just by inserting blank and dummy packet (handled by the transport stream) "pausing" the transmission of the h.264 stream and are thus not managed by the encoder.

As such the vbv will clamp the max size of frame under the buffsize (if a frame can't be stored in the buffer, then it can't be displayed) an avoid having spike of bitrate that could endanger a buffer underflow (a spike means that maybe some frame will arrive "too late" to be displayed)

If you have a buffer the size of the movie(/0.9) you can have whatever maxrate you want even 2kbit/s since the movie will only play once entirely downloaded.

A good illustration of when a buffer is in play is when viewing a streaming movie with windows media player, it will store the movie in buffer -even if it has playable frame- until the player "feel" that it could play the entire movie without having it "stall" (consequence of a buffer underflow) or in some case once it reach a trigger buffer size (30 second for example) that trigger the start of the player.

I hope it's clearer now.

[Chengbin]

I forgot to ask a question. When you raise qcomp, it allows x264 to vary bitrates more, and you much higher peak bitrate. It allows higher peak bitrate, but does it also do the opposite? Using less bitrate on low motion scenes?

[movmasty]

If you want more bitrate in high motion scenes, raise qcomp. That's what it's there for.
There are a lot of settings in x264, set them, That's what they are there for.

Only qcomp just transforms CRF into CQP or ABR, specially if our friend has to set it to 1...

[movmasty]

I forgot to ask a question. When you raise qcomp, it allows x264 to vary bitrates more, and you much higher peak bitrate. It allows higher peak bitrate, but does it also do the opposite? Using less bitrate on low motion scenes?

yess, quantizer becomes constant, so quants lower than 25 disappear, except for some I frame

[movmasty]

I tried 0.8, and tried 1, and I choose to stick with 1. Now my encodes are perfect. Thanks a lot Dark Shikari.

I hope my Archos 5 doesn't mind over 7Mbps peaks...

Why is qcomp not defaulted at 1? Wouldn't that help quality?
So you get a pure constant quantizer movie, not the top of the art....

Then deblocking and Adaptive quant are not doing their work??

Try something different,
eg. to raise the minimum quant to 22-23, lower the I factor to 1,3,
set a stronger AQ and, specially, deblocking,
higer motion extimation
all this with a qcomp of max 0,75.

Then see if this matrix can help against blocks:


INTRA4X4_LUMA =
12,16,17,18,
16,17,18,19,
17,18,19,20,
18,19,20,22

INTRA4X4_CHROMAU =
12,16,17,18,
16,17,18,19,
17,18,19,20,
18,19,20,22

INTRA4X4_CHROMAV =
12,16,17,18,
16,17,18,19,
17,18,19,20,
18,19,20,22

INTER4X4_LUMA =
16,16,16,16,
16,16,16,17,
16,16,17,18,
16,17,18,20

INTER4X4_CHROMAU =
16,16,17,18,
16,17,18,19,
17,18,19,20,
18,19,20,22

INTER4X4_CHROMAV =
16,16,17,18,
16,17,18,19,
17,18,19,20,
18,19,20,22

INTRA8X8_LUMA =
12,16,17,17,18,18,18,18,
16,16,17,17,18,18,18,19,
17,17,17,17,18,18,19,20,
17,17,17,17,18,19,20,21,
18,18,18,18,19,20,21,22,
18,18,18,19,20,21,22,23,
18,18,19,20,21,22,23,24,
18,19,20,21,22,23,24,26

INTER8X8_LUMA =
16,16,16,16,16,16,16,16,
16,16,16,16,16,16,16,17,
16,16,16,16,16,16,17,18,
18,16,16,16,16,17,18,19,
16,16,16,16,17,18,19,20,
16,16,16,17,18,19,20,21,
16,16,17,18,19,20,21,22,
16,17,18,19,20,21,22,24

[Dark Shikari]

So you get a pure constant quantizer movie, not the top of the art....

Then deblocking and Adaptive quant are not doing their work??Raising qcomp to 1 has nothing to do with adaptive quantization :rolleyes:

[movmasty]

...but with different objectives.

And the objectives of a 2pass encode are dictated by an average bitrate and some (limited?) variation from it
Will the 2pass, encode the 10Mbps-scene at 10Mbps, when you have specify a 1Mbps as average bitrate ?
NO, two pass is 'democratic', not dictatorial,

as the encoder knows all frames and can redistribuite bytes exactly,

In xvid at least two passes was even too similar to a constant quantizer pass,
dont think x264 is different, and i dont see neither a curve compression option....

[movmasty]

Because qcomp 1 means "vary the bitrate as much as necessary to maintain constant quality." Using qcomp 0 would mean constant bitrate (= variable quality). Values in between give a nice medium between bitrate variability (think VBR mp3) and constant bitrate.
qcomp 1 means 'keep the quant spefied in CRF constant, and dont care about the bitrate nor the quality'

And since we dont specify any bitrate in CRF, qcomp 0 means just to compress with high quantizer when high bitrates are found, w/o to know if after will be higher or lower , with unpredictable results,

think that settings out of the range 0,5-0,75 will lose all the CRF finalities

[movmasty]

Raising qcomp to 1 has nothing to do with adaptive quantization :rolleyes:

Maybe has something to do with blockness? that is the reason Chengbin used qcomp 1

[movmasty]

Originally Posted by Chengbin
Sharktooth told me to set a buffer size. --vbv-buffsize is not a value.
-


Uhm? :confused:
Chengbin needs to know which value to put in vbv buffer, and where it comes from, like a function of max rate,

btw i found that also 'vbv initial occupancy' has a noticeable effect on quantizer distribution, try to set it to 98%....

[Dark Shikari]

qcomp 1 means 'keep the quant spefied in CRF constant, and dont care about the bitrate nor the quality'

And since we dont specify any bitrate in CRF, qcomp 0 means just to compress with high quantizer when high bitrates are found, w/o to know if after will be higher or lower , with unpredictable results,

think that settings out of the range 0,5-0,75 will lose all the CRF finalitiesMaybe has something to do with blockness? that is the reason Chengbin used qcomp 1Please stop flooding this thread with incorrect information. You have absolutely no clue what you are talking about.

[movmasty]

Please stop flooding this thread with incorrect information. You have absolutely no clue what you are talking about.
i did tests with qcomp 1, this is the result
http://img5.imageshack.us/img5/469/t34qp1.gif

and is also true that in one pass the encoder doesnt know what will find in the future...

Then the x64 features are there to avoid blokness in CRF with qcomp 0.6,
if this happens the general settings have to be adjusted for that particular movie at that particular CRF quantizer for the desidered final size

[Dark Shikari]

i did tests with qcomp 1, this is the result
http://img5.imageshack.us/img5/469/t34qp1.gifAvinaptic only looks at frame quantizers, which are practically meaningless in x264.

[movmasty]

Avinaptic only looks at frame quantizers, which are practically meaningless in x264.
Well, this is also true, but then the developers should write an appropriate analyser for h264......:p

But what is not meaningless are the blocks in the encoding of Chengbin with the standard settings, resolved making pratically a constant quantizer encoding, possible also in 3.11 ;-)

[movmasty]

Please stop flooding this thread with incorrect information. You have absolutely no clue what you are talking about.Raising qcomp to 1 has nothing to do with adaptive quantization :rolleyes:
"qcomp Default: 0.60
Quantizer curve compression factor. 0.0 => Constant Bitrate, 1.0 => Constant Quantizer. Note that adaptive quantization partially replaces the effect of qcomp and x264 will internally raise qcomp to compensate based on the adaptive quantization strength.

http://mewiki.project357.com/wiki/X264_Settings#qcomp