DeathTheSheep United: x264 Guide and VfW Builds

[DeathTheSheep]

DeathTheSheep United (http://DeathTheSheep.Uni.cc): The ultimate x264 VFW spot has been born.

Download new VfW builds here (http://komisar.gin.by/) or here (Bugmaster) (http://sourceforge.net/projects/x264vfw/)! Special thanks to Komisar!

New x264 VfW builds, comprehensive x264 settings guide [server currently down], small community forum.

[Caroliano]

Don't ever use me-range higher than 32... it will only low the PNSR and increase the encoding time. ( http://forum.doom9.org/showthread.php?t=96636 )

And the B-frame reduction for anime is intersting... I think it can work, but don't tested yet. In that case I can use B-frames as references?

[berrinam]

the "Video Quality (PSNR)":"filesize" ratio will increase.I'm a bit surprised by this. Are you saying that you compared two files with different filesizes and different PSNRs? AFAIK, PSNR doesn't scale linearly with filesize.

Anyway, except for that, thank you for all your work and your guide. :thanks:

[DeathTheSheep]

Don't ever use me-range higher than 32... it will only low the PNSR and increase the encoding time.
Ah, thanks for that. You're right about it increasing the encoding time. As for the filesize, though, it will actually decrease with -esa (where it was mentioned), but PSNR values may be sacrificed in some cases. Due to the apparent variability of these results, I'll have this upper bound removed.

I'm a bit surprised by this. Are you saying that you compared two files with different filesizes and different PSNRs? AFAIK, PSNR doesn't scale linearly with filesize.
Actually, I just meant that you will achieve better PSNR for your file size (presumably, the same filesize proffers heightened PSNR with the higher-quality approach-- and vise versa).
I'll clarify that in my guide. Thanks! :)

[Sirber]

Good! I will try the SMEQ at 6 and bframe reduction 50%! :D

[neo_anderson]

i use 1 b-frame, 3 ref. frames and +512 to-512 range in nero digital avc max. def. and i get great quality!i see the average bitrate in info. in recode 2 and i use 20% of that bitrate in 2-pass encoding!

[DeathTheSheep]

Good news. What type of source material do you typically use? Animation, "real" movies, etc?

You mentioned how you got 20% of "that bitrate" when encoding in Nero. What exactly do you mean by this? ex: Have you achieved 20% better compression with Nero Recode than with x264 using the above method? Or did you simply use 20% of the average bitrate Nero reported to you?

My regards,
DTS

[DeathTheSheep]

The Guide is really long now...In fact, it can't get much longer at all due to the post size limit... *gulps in anxiety*

ADDED:

<> Deblocking Guide

<> Bitrate Variability Section

<> Streaming sections

NEED:

<> Cosmetics Work

<> More space for the rest of the guide (there's a 16000-char. limit here)!

[CiNcH]

//<>\\ When they become available for use, take advantage of custom H.264 quantization matrices. These nifty little numerical spreadsheets control how and where different frames are quantized. Typically, these matrices are not in widespread circulation and are still being tweaked to provide the best quality possible. The x264 codec currently (rev. 285) does not support this feature.

It actually does. Guess that it is just not part of the VfW interface.

CLI:
--cqm <string> Preset quant matrices ["flat"]
- jvt, flat
--cqmfile <string> Read quant matrices from a JM-compatible file
Overrides any other --cqm* options.
--cqm4 <list> Set all 4x4 quant matrices
Takes a comma-separated list of 16 integers.
--cqm8 <list> Set all 8x8 quant matrices
Takes a comma-separated list of 64 integers.
--cqm4i, --cqm4p, --cqm8i, --cqm8p
Set both luma and chroma quant matrices
--cqm4iy, --cqm4ic, --cqm4py, --cqm4pc
Set individual quant matrices

Problem is that ffdshow still doesn't support CQM. Have to use Nero Digital AVC Decoder instead.


Interesting information on motion search engine/motion search algorithms in x264: Motion Search Method (http://forum.doom9.org/showthread.php?t=98092)

[Sirber]

We could make that sticky :D

[neo_anderson]

Good news. What type of source material do you typically use? Animation, "real" movies, etc?

You mentioned how you got 20% of "that bitrate" when encoding in Nero. What exactly do you mean by this? ex: Have you achieved 20% better compression with Nero Recode than with x264 using the above method? Or did you simply use 20% of the average bitrate Nero reported to you?

My regards,
DTS
My sources are widescreen DVD movies, and yes, i use 20% of the bitrate recode reports, with range as +-512.5,3 ref. frames, 1 b-frames, psycho off, color optimization on, and using -1 deblocking!

check out these screenshots:
DVD Screen : http://img223.imageshack.us/img223/4628/riddickdvd1vg.jpg
Nero Max-Def AVC Screen:http://img223.imageshack.us/img223/56/riddickmp46wg.jpg

[DeathTheSheep]

Cool. :cool:

The thing that frazzles me is that the people who use Nero often confuse the "range:32" of x264 and the "range:512" of Recode2. I think some "range" clarification is in order for us poor folks (motion vector range : motion estimation range).

neo_anderson:
Slightly OT... But btw, in the screenshots, the guy looks more pleasant and clean-shaven in the AVC encode...He looks more menacing/dirty in the DVD frame with all his stubble. Gotta love that MPEG-4 smoothing! ;)

It actually does [already support CQM]. Guess that it is just not part of the VfW interface.
...
Problem is that ffdshow still doesn't support CQM. Have to use Nero Digital AVC Decoder instead.

Indeed. This is precisely the reason I left it out of my guide. I changed the wording a bit, though. :rolleyes:

Interesting information on motion search engine/motion search algorithms in x264: Motion Search Method (http://forum.doom9.org/showthread.php?t=98092)
Yes, my guide is fully compliant with that thread. :cool: I just wrote up the best methods in my guide to illimate the need for users to poke about.

Is there any way to increase the post size limit? (From 16000 characters to, say... 25000?)

[neo_anderson]

umm..., i used chronicles of riddick r1 director's cut dvd, so it's the best quality available for this movie!

[DeathTheSheep]

I know-- The source DVD looks very good in your screenshot :). I meant that the AVC encode smooths out his whiskers and makes him look less feirce.

But maybe only I think that.... :p

[neo_anderson]

hey, thank god u're online, please can u tell me some other way for finding out bitrate to be used,along with -1 deblocking, and 5 ref. frames? also, should i use more than 1 b-frames?

[DeathTheSheep]

To find out what bitrate to use, start out by asking yourself what the maximum should be. For instance, if I wanted to encode The Matrix 3, I'd probably set aside 2 CDs to acheive the maximum quality. Then use a bitrate calculator (like the one in xvid).

But, if you wanted a more exact bitrate based on quality, try to take a sample of the source (make it a fairly large) and encode it with your desired QP. With the resulting file, perform this calculation:

{ (# of MB in file x 1024) / (# of seconds in file) } x 8 = (Average Bitrate)

This will give you an average bitrate for the quality you desired.

An example: I take out 10 minutes of a cool scene in the middle and encode it with a quantizer of 22, 26, and 28. I find that the file with the quantizer of 26 fits my liking, so I apply my formula to it. With the resulting bitrate, I encode the entire movie in 2-Pass mode with the "High Speed, High Quality" mode above.

Of course, if you knew your target size (ex: you wanted to fit your content fit to two CDs), use the following formula:

( {(target MB x 1024) / (# of seconds in video)} x 8 ) - (audio bitrate) = (bitrate needed)

Cheers.

[neo_anderson]

thks DTS Dude, pls tell me what are ur thoughts or recommendations on Psycho-Visual Enhancements in Nero Digital AVC!

[_E_]

MeGUI comes with a bitrate calculator ;) You don't need to do any calcuations. It detects the type & size of audio used and it generates the average bitrate/target filesize based on the inputted framerate, total frames number, b frames mode on/off and desired container. So far it's giving accurate results.

[DeathTheSheep]

Psychovisual Enhancements? Okay, I added the section. The guide is shaping up, right? ;)

Well, I'll really have to do something about that accursed 16000 character limit in this forum... I had to tone down the guide already, and it's already full to the max! LOL

[CiNcH]

Here's my interpretation of Psychovisual Modeling / Psychovisual Heuristics:

Psychovisual modeling algorithms exploit the peculiarities of the Human Visual System (HVS). This knowledge allows the encoder to more efficiently allocate video data, helping it to increase perceptual quality. This area is full of possibilities which are continuously being explored and developed.

On FrameLevel data is reduced for a video sequence at full frame rate in a way that the HVS does not notice.

On MacroblockLevel distortions are masked in dark/bright (luminance masking) or highly textured areas of the picture (in other words bit spend is reduced) where the HVS does not notice it, details in flat areas, where the human eye is most sensitive, are enhanced.

Although psychovisual heuristics appear to improve quality the PSNR score drops, so the PSNR metric does not have a good correlation with subjective perception of video quality.


Psy-Vis in latest Ateme AVC Encoder Beta (I am not a tester so I am not aware of its effects)

There are 4 psychovisual levels, which are independant from the other settings.

* psy 0 -> no psychovisual heuristics
* psy 1 -> psychovisual heuristics on a frame level
* psy 2 -> psychovisual heuristics on a macroblock level
* psy 3 -> enhanced psychovisual heuristics on a macroblock level. Some people prefer it over psy 2, others don't.

[Sharktooth]

added Death The Sheep guide (link to this thread) to x264 builds sticky.

[DeathTheSheep]

I saw the doom9 guide section and couldn't help but wonder: how could I go about submitting this guide to the main Doom9 site? If I did that, I would have more space, a cleaner layout, and room to put in screenshots! How's that for a high-quality space for a high-quality guide? :p :D

[TheBashar]

---Bitrate Variability---
The bitrate variability feature controls how much your datarate can fluxuate at any given time. That is, if you plan on encoding at 500kbps and set this value to 40, the maximum amount of bitrate given to more complex scenes is 700kbps. In a nutshell, the lower you set this value, the better still/non-complex scenes look, but high-motion/complex scenes will look more shabby and garbled. The higher you set this value, the more equal the overall quality will become: still scenes would look worse than with a low value, and high-motion/complex scenes would look a lot better.

DTS,

I may have misunderstood two of the x264 parameters (ratetol and qcomp) but I believe qcomp is responsible for deciding how much more high motion scenes are compressed (aka lower bitrate) than static scenes. I think ratetol is an additional constraint on how much you allow qcomp to do its job.

I believe that ratetol is the appropriate parameter to set/modify if you have streaming or hardware playback concerns. However, the low/high motion compensation that you are describing in the above quoted text would, I believe, be more appropriately handled by adjustments to qcomp in conjunction with a sufficiently permissive ratetol.

I am a little confused why ratetol defaults so low. I would think you would want to default it as high as possible, and then adjust qcomp for motion compensation issues.

Anyway, just my perhaps misguided thoughts.

[DeathTheSheep]

Qcomp, in a quite literal sense, is a form of Bitrate Variability--actually, it was simply renamed in the VFW and multiplied by 100 for integer simplicity.

The "ratetol" parameter, as you mentioned, controls the amount of bitrate explicitly awarded to the video, in turn effecting the quantizers used on it. It does so as part of a bitrate regulation -- to ensure that the content reaches a certain bitrate without fluxuation.
raising ratetol allows greater fluctuations in bitrate (thus it would be more similar to 2-pass) at the expense of precision in aiming the target bitrate
It therefore defaults to a 1, a low value, in order to more accurately hit a given bitrate in ABR (1-pass variable bitrate) mode.
For this "fluxuation insurance," high-motion scenes would obviously require a higher quantizer than do still scenes in order to maintain a certain steady bitrate. However, qcomp directly controls these quantizers (A quantizer curve compression, or quantizer regulation), which in turn impacts the bitrate, serving either to give it absolute stability (0) or allow infinite fluxuation (1) based on what quantizers are used on each frame in the video.
Control diagram:

<> qcomp -> quantizers -> bitrate variability
<> ratetol -> bitrate variability -> quantizers

I think ratetol is an additional constraint on how much you allow qcomp to do its job.
Yes, I would assume so.
It is therefore safe to assume that with Qcomp in play, ratetol serves as a sort of "peak cutoff," allowing the video to conform to the quantizer specifications in qcomp so long as they don't exceed the bounds specified in ratetol. Yes, ratetol would be an excellent parameter to set when encoding streaming content (if using the cli, of course), because of these solid bounds.

Their purposes seem to check and balance one another and provide more explicit control.

[jellysandwich]

all but the 8x8 transform (for MP-compliant streams).

When you say this, are you referring to P8x8, B8x8, I8x8 High Profile, or a combination?

js

[DeathTheSheep]

Merely the high profile transform. I apologize for any inconvenience.

Yours,
DTS

[Revgen]

I have a couple questions that I haven't seen here.

1) What is "B-Frame Bias"? Does this setting determine what direction (Future frames/Past Frames) that a B-Frame takes into account? What kind of sources would this be useful for?

2) Is the "Chroma ME" setting useful for encoding B&W sources? I'm assuming it isn't, but I don't know for sure.

Thanks.

[Caroliano]

@revgen:

1) This was called "BVOP Sensitivity" in Xvid. B-frame Bias let you tweak the amount of Bframes the encoder will put in the clip. 0 is the defaut b-frame decision algorithm. A positive value increases the amount of B-frames; a negative value decreases it. It's better you don't touch in that option. Normaly the defauts are always better, unless in a speed by quality trade of.

Warning: It isn't the same as max consecutive B-frames.

2) I can't guarantee too...

[bond]

If you wish your video to maintain a specific, constant quality the whole way through, use only the Constant Quality feature of 1-pass mode.
<> Do not use a quantizer of under 15 unless you are working for archive/reproduction quality.
<> Also, do not use anything over 40: the quality is simply unbearable unless you are encoding from an extremely sharp source of high-contrast edges and plan on streaming it over the internet.
<> A good bet for most people interested in high quality video would be the range of 22-30 (or, more specifically, 24-28). Of course, this varies depending on individual taste and how much free space you have.
<> On animated content with few detailed textures, consider using a higher quantizer.
<> On "real-life" content, especially that with many dark scenes and important subtle textures, consider using a much lower quantizer value.where do these values come from? any screenshots?

why do you think that its a bad idea to trust x264 on deciding what quant to use? any samples which show that x264 does a bad decision on that?

To stream x264 video, you need to have a player capable of handling streaming; nearly all players I can think of at the moment support this. I suggest using Media Player Classic, but Windows Media Player 9/10 works just as well for many people as long as the ffdshow decoder is installed. Make sure to configure the player to automatically start a stream with the desired file extension (the most commonly made x264 files use the ".avi" extension). you are sure that dshow players will be able to play streamed h.264 .avi files? somehow i cant believe this. do you have a sample stream link i can try?

<> Use 0 references. Reference frames cause a lot of hop-skipping around the stream. If a certain frame's reference is far off into the file, the entire file becomes unplayable until the video is downloaded up to that reference.i doubt you mean 0 references, as this would mean using an i-frame only stream. what you mean here is using 1 reference frame

<> Use a maximum of 1 B-frame. It is NOT recommended to use consecutive B-frames on very-low bitrate content. I have proven this with the thread entitled "The Coolness of B-Frames." The WORST place to use a B-frame in H.264 (or many other formats for that matter) is on low-bitrate, low framerate, streaming, animated content.well i wouldnt call this to be proven at all, after all the purpose of b-frames is helping in low bitrates too...

<> 4-5 references are the typically accepted maximum. However, if you have a little bit of extra time on your hands (or a beefy computer), consider using up to 8 references (which decode just about as fast as 5, according to my Pocket PC AVC benchmarks). my psnr comparisons showed that references over 5 dont really bring any quality increase but will slow down decoding

<> B-frame reduction (if you choose to use B-frames) should be adjusted according to the desired datarate. For most DVD backups, the default B-frame reduction of 30% seems to be sufficient. However, with certain animated medium bitrate content, I suggest use of up to 50% B-frame reduction.why dont you think that the default values of x264 are ok? any sample files which show that default setting in x264 does a bad job or that your proposal does a better job?

<> Don't ever use CAVLC... My benchmarks prove that decoding CABLC streams is only slightly faster than CABACnot true, cabac will decrease decoding speed clearly, as i described here (http://forum.doom9.org/showthread.php?t=99131)

<> Personally, I find that the keyframe threshold of 45% and keyframe QP boost: 20 to be the optimal in achieving quality at medium bitrates. However, if you wish to produce Ultra-Low-Bitrate video (DVD backups less than one CD), consider reducing this to 0, for your video's quality (PSNR) per BM will increase.again my request for samples that show that x264s defaults perform worse than with your proposal

However, if you have a particularly complex source (or if you really want to scrimp for the best possible quality), try Uneven Multi-Hexagon with a ME range of 16-32hm didnt pengvado write somewhere that it doesnt make sense to use a value higher than 16 and that it even could decrease quality!?

Generally lower framerates (12-15 as opposed to 23.976-29.97) need a higher estimation range.why that?

<> In maximum-quality content, I find that the keyframe threshold of 35% and keyframe QP boost of either: 30 or 0 to be the optimal in achieving quality at medium-high bitrates in my "maximum possible quality" mode. again my request for samples that show that x264s defaults perform worse than with your proposal

<>\ When they become more widely available for use, take advantage of custom H.264 quantization matrices. The x264 codec currently (rev. 285) does not support this feature.it does

*B-frames can be activated in "pyramid" mode, which allows B-frames to serve as references. If you wish to use a lot of references (and thereby increase quality slightly), consider selecting the "Use as references" checkbox.b-pyramid doesnt really have much to do with the number of reference frames

However, this option may cause some crashes with both the encoder and the decoder (if either is old enough). For this reason, I recommend NOT using B-frame references.doesnt make sense, instead you should recommend to always use the latest version of the codec

<> For deblocking strength, try not go out of bounds of the -3 to 3 range. Generally, any more than 3 will turn your result into mush while decreasing PSNR and actually increasing the output file slightly. Any less than -3 may cause the result to look a bit too blocky and any lack of texture will merely become more apparent as all smoothing is taken out.positive values of loop will tend to remove details -> smaller filesize

Present in certain AVC encoders, this is a method of lumi (+chroma) masking in which very light or dark areas of a scene are encoded in terrible quality, which gives more bitrate to the rest of the video.i wouldnt talk about "terrible quality" here, as the point here is to encode with lower bitrate where the eye cant see the difference anyways, so for the eye the quality will stay the same

<> The main determining factor in whether or not you would like to enable Psychovisual Enhancements is the darkness of your source. In films with very dark scenes (dungeons, tunnels, night, shadow), turn this OFF for good measure. If the source isn't very dark, I'd recommend you try light enhancement. Strong psychovisual enhancement can lead to problems (the codec might think important semi-dark things are "dark enough" to heavily reduce the quality, which may be obvious to the viewer). Check out CiNcH's post below for more info... you are talking here mainly about nerodigital as x264 doesnt do psy, any link to information about how psychovisual enhancments work in nerodigital so i am able to find out whether nerodigital really does the lumimasking you describe and not something else?

[IgorC]

Deatsheep isn't a dev of x264 that's why his arguments far to be best. I apreciate his contribution :rolleyes: but induction and deduction aren't the same things. Particular case doesn't define situation in general.

[akupenguin]

The bitrate variability feature controls how much your datarate can fluxuate at any given time. That is, if you plan on encoding at 500kbps and set this value to 40, the maximum amount of bitrate given to more complex scenes is 700kbps.
No. See some explanations I gave to Chen.

Generally lower framerates (12-15 as opposed to 23.976-29.97) need a higher estimation range.
why that?
Because at lower framerates, objects have moved farther between consecutive frames. => need to search farther on average to find an object. OTOH, I can't think of any situation where I would reduce framerate below 24 without also reducing resolution, and low resolutions can use smaller search range. So it may cancel out.

If a certain frame's reference is far off into the file, the entire file becomes unplayable until the video is downloaded up to that reference.
No, multiple references have no effect on seekability. (BTW, they also have no effect on AVI compatibility.) All reference frames would have to decoded anyway, because they're between the current frame and the previous keyframe. (B-pyramid does add a few extra dependencies, but again it's independent of the number of reference frames.)

positive values of loop will tend to remove details -> smaller filesize
Not necessarily. If the codec encodes some details, and then loopfilter wipes them away, and then it has to code the details again in the next frame...
I'm not saying that the net effect is to increase size at a given QP (I don't know offhand; all of my loopfilter comparisons were 2pass, because it's the only fair way), just that you can't deduce the global effect from the definition of loopfilter.

[foxyshadis]

<> 4-5 references are the typically accepted maximum. However, if you have a little bit of extra time on your hands (or a beefy computer), consider using up to 8 references (which decode just about as fast as 5, according to my Pocket PC AVC benchmarks).
my psnr comparisons showed that references over 5 dont really bring any quality increase but will slow down decoding
It's quite concievable that different processor architectures on PPC vs PC could lead to different results. I'm not defending them, just saying.

<>\ When they become more widely available for use, take advantage of custom H.264 quantization matrices. The x264 codec currently (rev. 285) does not support this feature.
it does
True, but decoders don't. (I think ffdshow's support for it was buggy when I tried last month, and no others worked at all.)

<> Use a maximum of 1 B-frame. It is NOT recommended to use consecutive B-frames on very-low bitrate content. I have proven this with the thread entitled "The Coolness of B-Frames." The WORST place to use a B-frame in H.264 (or many other formats for that matter) is on low-bitrate, low framerate, streaming, animated content.
well i wouldnt call this to be proven at all, after all the purpose of b-frames is helping in low bitrates too...
B-frames help to a certain point, however below a certain framerate & bitrate they lower subjective quality. That section's about ultra-low bitrate streaming encodes, where assumptions from low-to-mid quality encodes don't all hold. (By that point the whole video looks like trash to me, but I'm not the target audience.) A few threads have come up about this.

[DeathTheSheep]

Bond:
As a renowned doom9 moderator with 5.7 thousand posts, your scrutiny is indeed commendable, justifying this vast post number while casting my rather insignificant 250 forum contributions into the realms of near-obscurity.

To Whom It May Concern:
Such a large number of doubts, questions, and concerns will not go unheeded by me, as I will attempt to explain myself for each of my points. I do this for the sake of both my guide and the doom9 community, which I hope will partake the reaping of its bounty.

For the sake of simplicity, I will display the questioned excerpt from my guide and explain myself briefly for each of the points bond (and perhaps other members) brought up about them in preceding posts.

---- M Y ----
D E F E N C E
-------------
I make the following statements to justify the content of my guide.

If you wish your video to maintain a specific, constant quality the whole way through, use only the Constant Quality feature of 1-pass mode.
<> Do not use a quantizer of under 15 unless you are working for archive/reproduction quality.
<> Also, do not use anything over 40: the quality is simply unbearable unless you are encoding from an extremely sharp source of high-contrast edges and plan on streaming it over the internet.
<> A good bet for most people interested in high quality video would be the range of 22-30 (or, more specifically, 24-28). Of course, this varies depending on individual taste and how much free space you have.
<> On animated content with few detailed textures, consider using a higher quantizer.
<> On "real-life" content, especially that with many dark scenes and important subtle textures, consider using a much lower quantizer value.
bond: why do you think that its a bad idea to trust x264 on deciding what quant to use? any samples which show that x264 does a bad decision on that?
DTS: However you may have conceived the preposterous notion that I intended to redefine x264’s quantizer utilization mechanism, I assure you, this is not the case at all. As is relatively obvious given the context of this quote, I was discussing the method to achieve high quality video using the “Constant Quality” 1-Pass mode of x264, where the user’s input literally is the codec’s control over the quantization. I also gave examples of when to use this feature. Please, read the guide carefully before jumping to conclusions based on fallacious/preconceived notions.


<> Use 0 references. Reference frames cause a lot of hop-skipping around the stream. If a certain frame's reference is far off into the file, the entire file becomes unplayable until the video is downloaded up to that reference. bond: i doubt you mean 0 references, as this would mean using an i-frame only stream. what you mean here is using 1 reference frame
DTS: Perhaps. However, you seem to be incorrect about your I-stream assumption; it appears as if you haven’t tried to input a value of ‘0’ for reference frames in x264, which outputs a perfectly normal IPB stream. It could very well be that 0 and 1 perform the same task in x264 when input into the VFW GUI, for which this guide was intended. In my guide, I was explicitly aiming for the least possible reference frames, so I used the value of 0 for clarity. Recall that I am not a developer of x264, nor was I discussing its internal workings—I was merely proffering input values which yield the highest output quality, based on various empirical and observational tests. Have you ever tried to input 0? Perhaps it wouldn’t hurt. After all,
IgorC: induction and deduction aren't the same things.
Here is a sample stream with “0” input for the number of reference frames: http://www.thefilehut.com/userfiles/xsquaredx/0refs.avi

<> Use a maximum of 1 B-frame. It is NOT recommended to use consecutive B-frames on very-low bitrate content. I have proven this with the thread entitled "The Coolness of B-Frames." The WORST place to use a B-frame in H.264 (or many other formats for that matter) is on low-bitrate, low framerate, streaming, animated content.
bond: well i wouldnt call this to be proven at all, after all the purpose of b-frames is helping in low bitrates too...
DTS: Although I might come off as a bit abrupt on my part, this statement may well prove nothing more than a lack of experience in the ultra low bitrate area on your part. The section of the guide from which the above quote was pulled was from an ultra-low bitrate section.
foxyshadis: B-frames help to a certain point, however below a certain framerate & bitrate they lower subjective quality. That section's about ultra-low bitrate streaming encodes, where assumptions from low-to-mid quality encodes don't all hold… A few threads have come up about this.
IgorC: induction and deduction aren't the same things. Particular case doesn't define situation in general.
DTS: Now, without looking into the matter further, it is not entirely safe to make such assumptions. Consider the following examples of standard, low bitrate, low-framerate content, one with and the other without B-frames. Which looks better to you? Remember, even if it seems that both results look shabby to you, you may not be of the target audience.
With: http://www.thefilehut.com/userfiles/xsquaredx/bframes1.avi
Without: http://www.thefilehut.com/userfiles/xsquaredx/bframes0.avi
These files were encoded with exactly the same settings (standard settings except for 2 references and 6-RDO refinement). The file with 2 consecutive B-frames is actually larger, and there is much more block noise during movement, and movement isn’t accurate (see how the character’s face looks to be falling off as he walks towards the camera). To further illustrate my point, I will refer you to a certain quote which attempts to explain such a phenomenon:
skal: Let's look at 3 frames at ~8fps (the upper ones in the following -ugly- sketch), and watch the coding in progress:

http://skal.planet-d.net/munch_me.jpg

Arghh! When the encoder is coding frame P1, look at what he has as reference! It's P0, coded at QP=45. A mere block feast!
Question: How do you expect to ME to find a good match between frame P0 and frame P1 ??? The first one is a block soup, but P1 still has lot of details. One could pick any block from the first frame and match it with the third, with a SAD of ~51340986542 (roughly) ;)
The motion vector field will be erratic, and the middle b-frame will find it difficult using the mighty BDirect mode (not to mention that it'll be hard to match the frame P0 / P1, too).
RadicalEd: possible for B frames to increase size indirectly, especially if they're not being overquantized with respect to the Ps. This is because a B frame essentially creates a void in prediction and the next P frame has to jump behind it and use the P (or I) frame before the B to predict from. Therefore P frames get bigger as a result of B frames. This is probably more prominent with anime as well, so maybe that's what sheep is experiencing.

<> 4-5 references are the typically accepted maximum. However, if you have a little bit of extra time on your hands (or a beefy computer), consider using up to 8 references (which decode just about as fast as 5, according to my Pocket PC AVC benchmarks).
bond: my psnr comparisons showed that references over 5 dont really bring any quality increase but will slow down decoding
DTS: At a constant quality quant=30, which I believe is the “fairest” way to test the quality of different settings on a filesize basis (it employs a uniform quality quantization throughout), see the following for yourself. Each having nearly identical quality to the other, which file is smallest?
5 refs http://www.thefilehut.com/userfiles/xsquaredx/5refs.avi
10 refs http://www.thefilehut.com/userfiles/xsquaredx/10refs.avi
16 refs http://www.thefilehut.com/userfiles/xsquaredx/16refs.avi
In my guide, I say 5 is the “typically excepted maximum.” But even so, it doesn’t take a leap of imagination to understand that there is a clear 14% filesize decrease between 5 references and 10 references. It isn’t a good idea to generalize on all content based on a single sect of testing; I have done tests with many bitrates and types of content, and there clearly a quality advantage at 10 references (over 5), but no real advantage over 10. My guide’s “absolute maximum” in this section is 8.

<> Don't ever use CAVLC... My benchmarks prove that decoding CABLC streams is only slightly faster than CABAC
bond: not true, cabac will decrease decoding speed clearly, as i described here (http://forum.doom9.org/showthread.php?t=99131)
DTS: I apologize, but I cannot take the word of that thread over the testing I have done, which has revealed much evidence to the contrary. Frankly, as IgorC pointed out below:
IgorC: Particular case doesn't define situation in general.
DTS: My results speak for themselves; I’ve no need to argue with anyone about my own findings. Consider these results at a constant quality of quant=35:
Cabac - http://www.thefilehut.com/userfiles/xsquaredx/cabac.avi
Cavlc - http://www.thefilehut.com/userfiles/xsquaredx/cavlc.avi
Benchmark them with this (ensure you set the screen size to 100%): http://cc.serveftp.org:8884/TCPMP_0.66e_for_Win32.exe
The results are clear: with the same exact source and exact same settings, cabac clearly underperforms cavlc in terms of decoding speed with the high-speed, high-quality ffmpeg H.264 decoder.

[i]However, if you have a particularly complex source (or if you really want to scrimp for the best possible quality), try Uneven Multi-Hexagon with a ME range of 16-32
bond: hm didnt pengvado write somewhere that it doesnt make sense to use a value higher than 16 and that it even could decrease quality!?
DTS: Perhaps you should have looked back upon said remark and the context in which it resided so as to attain a more complete understanding of the properties of such a motion estimation range. For instance, if you had more thoroughly read this thread, you would have found a thread in which I specified results to the contrary: http://forum.doom9.org/showthread.php?p=698397#post698397
akupenguin: dia and hex cap merange at 16. It slightly reduces code complexity, and if the real mv is more than 16 pixels from the prediction, they won't find it anyway.
DTS: Essentially, however, the range 32 uneven multi-hexagon search, because of its nature, would search a more exhaustive area within its range based on the principle that its core is larger, even if it is incapable of predicting past the 16 ME boundary (at which hex is capped). Again, I am not a developer, but I have seen consistent, concrete results using a umh ME range of 32.

Generally lower framerates (12-15 as opposed to 23.976-29.97) need a higher estimation range.
bond: why that?
akupenguin: Because at lower framerates, objects have moved farther between consecutive frames. => need to search farther on average to find an object.
DTS: This is also true of animated content, which by very definition is “animated;” drawings are typically spaced far enough apart from one another to render a higher ME range beneficial. This is also where B-frame users run into more trouble—due to the discrepancy between certain pairs of frames in low-framerate/animated content, bidirectionally predicted frames may not be able to accurately fill the gap easily, and even so, the subsequent P-frame would have to look 2 frames behind it for prediction (unless B-pyramid is activated). Look above for more information containing some B-frame analysis. My guide, however, states when to employ (or when not to employ) both B-frames and a high ME range, and it does so based on this “rule.”

<>\ When they become more widely available for use, take advantage of custom H.264 quantization matrices. The x264 codec currently (rev. 285) does not support this feature.
bond: it does
DTS: And if you’d read the entire thread, you will see that I acknowledge this, but due to the fact that decoder-side support of this is very limited at the moment (not to mention the lack of the current VFW’s lack of support), I chose to leave this out of my guide.
foxyshadis [to bond’s response]: True, but decoders don't. (I think ffdshow's support for it was buggy when I tried last month, and no others worked at all.)
DTS [earlier quote in this thread, to CiNcH]: Indeed. This is precisely the reason I left it out of my guide.

*B-frames can be activated in "pyramid" mode, which allows B-frames to serve as references. If you wish to use a lot of references (and thereby increase quality slightly), consider selecting the "Use as references" checkbox.
bond: b-pyramid doesnt really have much to do with the number of reference frames
DTS: True, but I still don’t really see much of a problem with my statement; B-pyramid, by very definition, is the usage of B-frames as “references” for following frames. By using B-frames as references, you are, in a sense, adding a type of references to the stream, thereby increasing quality slightly—due both to the fact that more references increase the video quality, and due to the fact that frames following the B-frames can use the B-frame for prediction (therefore, they would serve as references). Remember, I’m not a developer, so I don’t know the internal code, but try the guide—the quality attained using my tested method really speaks for itself.

However, this option may cause some crashes with both the encoder and the decoder (if either is old enough). For this reason, I recommend NOT using B-frame references.
bond: doesnt make sense, instead you should recommend to always use the latest version of the codec
DTS: It makes perfect sense. You see, the latest codec at the time of writing (rev. 285) consistently caused crashes for me when too many references were used with B-frame references. Why? I don’t know, but the fact is, the problem persists even today with revision 291 on all of my machines using the newest hardware and software. Whether the cause resides in the input colorspace, the codec’s internals, or a seemingly infinite number of other factors, it still frequently crashes on both of my brand-new computers and many others I know of, giving it my definition of “unstable” and therefore warranting a few words of my caution. It is for this reason that I’ve worded my statement so; perhaps when my guide is read at a later date, this option may have become more stable for all machines (or, at very least, when all major decoder-side components are stable—ffdshow has only begun to become stable for me within the past few months, to say nothing of others). But otherwise, I’d have to say that, yes, it is common knowledge to always use the newest version of a program/codec unless a newer version is known to have introduced sizeable flaws. Therefore, I shall also employ this advice in the upcoming, newer version of my guide.

[My statement continues in the following post]

[DeathTheSheep]

[This is a continuation of the statement started in the previous post]

<> For deblocking strength, try not go out of bounds of the -3 to 3 range. Generally, any more than 3 will turn your result into mush while decreasing PSNR and actually increasing the output file slightly. Any less than -3 may cause the result to look a bit too blocky and any lack of texture will merely become more apparent as all smoothing is taken out.
bond: positive values of loop will tend to remove details -> smaller filesize
(akupenguin [to bond’s statement]: Not necessarily.)
DTS: It is not safe to conclude this. If your statement is true, then I assume that at a fixed quantizer (and therefore with uniform quality throughout), filesize will decrease if loop filter is more heavily applied. Take a look at these sample encodings, where any difference is clearly discernable. I disabled B-frames, but all other settings are default (and quantizer = 30). You see, this statement is simply not true for most cases, and like akupenguin said, “you can’t deduce the global effect from the definition.”
Strength= 0; http://www.thefilehut.com/userfiles/xsquaredx/deblock0.avi
Strength=-6; http://www.thefilehut.com/userfiles/xsquaredx/deblock-6.avi
Strength= 6; http://www.thefilehut.com/userfiles/xsquaredx/deblock6.avi

And here are some 2-pass samples, which show such a phenomenon less, due to the fact that they are the same size because of the nature of 2-pass mode (but PSNR was lower for the -6 and 6 deblocked clips, as I predicted). In part due to the ultra, ultra low bitrate (unfortunately, I don’t have my own dsl line with which I can upload large files), and in part due to the animated nature of my content (I have qualms about uploading parts of Hollywood movies), the subjective quality stays about the same for the DB: 0 and DB: 6 encodes, but as the bitrate gets higher, difference is more capable of being clearly differentiated—the highly deblocked tends to wash too much detail.
http://www.thefilehut.com/userfiles/xsquaredx/deblockX_2pass.avi (replace the X with 0, 6, or -6).
All in all, though, it appears as if increasing or decreasing deblocking intensity by too far a margin will hurt both quality and filesize, as I clearly stated in my guide.

Present in certain AVC encoders, this is a method of lumi (+chroma) masking in which very light or dark areas of a scene are encoded in terrible quality, which gives more bitrate to the rest of the video.
bond: i wouldnt talk about "terrible quality" here, as the point here is to encode with lower bitrate where the eye cant see the difference anyways, so for the eye the quality will stay the same
DTS: It is true that I have seen fairly prominent flaws in low-lighted content not present in identical encodes with psychovisual enhancements disabled. Nevertheless, I was speaking of “quality” in terms of bitrate allocation; that is, how accurately dark areas are encoded. If I’d have said “visual quality,” I would be implying otherwise. Nevertheless, I can see your point, and I shall clear this up in my next version of the guide (let’s call the next version a “beta.”)
As to another issue relating to psychovisual enhancements, I’d like to make it clear that I am not a developer of NeroDigital AVC, and do not know what each psychovisual enhancement does internally, but I can be sure that the test results I have attained with ND AVC in very dark (dungeon) sources are of noticeably less visual quality than without psychovisual enhancements (most notably in textured areas), and it was indeed “noticeable.” If you insist, you may test this for yourself on a very dark source and see for yourself. This is most relevant at higher bitrates (so I can’t upload an indicative source).

<> B-frame reduction (if you choose to use B-frames) should be adjusted according to the desired datarate. For most DVD backups, the default B-frame reduction of 30% seems to be sufficient. However, with certain animated medium bitrate content, I suggest use of up to 50% B-frame reduction.

<> Personally, I find that the keyframe threshold of 45% and keyframe QP boost: 20 to be the optimal in achieving quality at medium bitrates. However, if you wish to produce Ultra-Low-Bitrate video (DVD backups less than one CD), consider reducing this to 0, for your video's quality (PSNR) per BM will increase.

<> In maximum-quality content, I find that the keyframe threshold of 35% and keyframe QP boost of either: 30 or 0 to be the optimal in achieving quality at medium-high bitrates in my "maximum possible quality" mode.
bond: where do these values come from? any screenshots? again my request for samples that show that x264s defaults perform worse than with your proposal
DTS: First and foremost, I’d like to clear up one massive point of obvious confusion: The above values, as were the constant values attained elsewhere in my guide, came to be as a combined result of my own empirical testing and material observation of visual results. The “evidence” is also within the results of this method.
Indeed, If you disagree with the guide without even attempting to try it, you cannot be certain of its fallacious nature. Remember, this is a guide (and one in “pre-alpha” stage at that), and as such, it deserves to be treated as one; give it a chance first and then compare the results to those of any other method you choose. Recall the fact that it is, at this point, you contesting me, or more correctly, the aforementioned results. However, I’m not about to argue with speculation-- speculation is easy and fun, isn’t it?—but rather, if the results I consistently achieve are in question, it is the duty of the challenger to provide counterevidence, which will indeed be addressed. In the case that you may have pondered the reason for my frequent requests for more forum space (or an alleviation of the 16k character limit), it was to provide extended services such as screenshots, a better graphic scheme, and more information—some of which, perhaps, serving to justify my conclusions. (Perhaps if you had read the entire thread, you might have come to this realization.)


------- M Y -------
C O N C E S S I O N
-------------------
I make the following statements to admit errors on my part.

The bitrate variability feature controls how much your datarate can fluxuate at any given time. That is, if you plan on encoding at 500kbps and set this value to 40, the maximum amount of bitrate given to more complex scenes is 700kbps.
akupenguin: No. See some explanations I gave to Chen.
DTS: I am not a developer, so I apologize for the misinformation regarding the internal workings of this tool. However, if the bitrate variability tool does not work as I have so erroneously stated, it still controls the amount by which the video stream fluctuates by means of it controlling QP curve compression, right?
Again I apologize, but I don’t really know who “Chen” is, much less where the aforementioned explanations were given. Any assistance?


To stream x264 video, you need to have a player capable of handling streaming; nearly all players I can think of at the moment support this. I suggest using Media Player Classic, but Windows Media Player 9/10 works just as well for many people as long as the ffdshow decoder is installed. Make sure to configure the player to automatically start a stream with the desired file extension (the most commonly made x264 files use the ".avi" extension).
bond: you are sure that dshow players will be able to play streamed h.264 .avi files? somehow i cant believe this. do you have a sample stream link i can try?
DTS: I understand your confusion. In fact, the wording wasn’t clear here in my guide. When I mention “desired file extension,” I’m not referring explicitly to .avi. On the contrary, I am referring to any format capable of streaming media, such as .asf or mp4, if you so choose. I also point out that the .avi file extension is most commonly used for the storage of x264 files, implying that at least a container switch could be in order if streaming is desired. I agree, though, that perhaps a bit more than the wording is unclear, and I apologize. Perhaps if I had more room, I could elaborate, but I will nevertheless either change the guide’s final wording or remove the streaming section entirely (but leave the streaming settings). However, it does appear as if I was mistaken about ffdshow being able to play streaming H.264 content as of yet; in fact, only with certain codecs was I able to attain streaming capability in WMP. Indeed, I should have stated that a streaming-enabled decoder is necessary, but I erroneously assumed the new versions of ffdshow could handle such content. I apologize; I’m clearly not an expert at making streams (do I have my own streaming server?), but my guide is intended for high-quality AVC settings, not a streaming how-to. Here is a sample asf stream attained using the beta ASF muxer. http://www.thefilehut.com/userfiles/xsquaredx/trythis2.asf
Perhaps the Moonlight H.264 Decoder & Streaming Pack could be of use in this situation, in which case it would supposedly be able to stream certain AVC-MP4 streams.


If a certain frame's reference is far off into the file, the entire file becomes unplayable until the video is downloaded up to that reference.
akupenguin: No, multiple references have no effect on seekability. (BTW, they also have no effect on AVI compatibility.) All reference frames would have to decoded anyway, because they're between the current frame and the previous keyframe...
DTS: You are absolutely correct, and I was absolutely (and rather foolishly) mistaken. I assumed (see, there’s the mistake) that somehow reference frames had an impact on streaming x264. I therefore assumed (there we go again...) that there must have been some sort of “backwards referencing” going on, which, looking back on it, now strikes me as preposterous. The reason I made these assumptions was that I could find so very little streaming x264 content online to perform comprehensive testing on. It was an imprudent decision to put any sort of streaming directions in my guide, for as I said above, my guide is intended for high-quality AVC settings advice, not a streaming how-to.

-----------
Final Words
-----------
By placing this preliminary version of a guide in a forum, I was obviously asking for responses and input, and I am thankful for all you have given me. Indeed, I greatly appreciate what I have learned and what the community has learned in the process. And this learning, I might add, will not be wasted on me, for I will release a so-called “beta” version of my highly updated and expanded guide in due time.

However, I can’t shake the feeling that there was some sort of negative connotation in your post, bond, based on the way you posed some of your questions. True, we all do different tests, and we can all make mistakes, and I’m sure you realize this too. We simply have to learn from them and grow. It never hurts to expand your knowledge-- my own, for my part. I thank you for being part of my own betterment in addition to part of your own, and I wish the rest of the community luck in utilizing this guide to its full potential.

I look forward to more comments and suggestions; after all, this is your guide, too!

DTS

[Revgen]

akupenguin: No. See some explanations I gave to Chen.
DTS: I am not a developer, so I apologize for the misinformation regarding the internal workings of this tool. However, if the bitrate variability tool does not work as I have so erroneously stated, it still controls the amount by which the video stream fluctuates by means of it controlling QP curve compression, right?
Again I apologize, but I don’t really know who “Chen” is, much less where the aforementioned explanations were given. Any assistance?


I believe he was reffering to this (http://forum.doom9.org/showthread.php?t=98386) thread.

[TheBashar]

DTS,

In the context of 2pass (or more) high quality encodes, I humbly suggest testing the parameters "--ratetol inf" and "--qpstep 40". In my testing I have had good results with these. Sorry, I don't know what they are referred to in the VfW encoder as I use CLI. They represent "Bitrate Variance" and "Maximum Quantizer Delta" in MeGUI.

Cheers!

[bond]

huh lots of things, i will pick out the most important ones :D

first of all thx everyone for the given info

- nero and moonlight support custom quant decoding
- when setting 0 reference frames in x264 it will encode with 1 reference frame (or 2 reference frames if b-frames are enabled). that is because x264 is smarter than the user i assume :D
- if you have crashes with b-ref and large numbers of mref you should start an own thread about this and tell the x264 devs
- "most x264 encodes use .avi" - where do you see that? on your harddisc? ;)

[numaios]

My sources are widescreen DVD movies, and yes, i use 20% of the bitrate recode reports,
I still don't understand :confused: So you don't use a fixed final size? And then you use the 20% of the source bitrate? Or the 20% of the bitrate that Nero Recode recommends you?

The thing that frazzles me is that the people who use Nero often confuse the "range:32" of x264 and the "range:512" of Recode2. I think some "range" clarification is in order for us poor folks (motion vector range : motion estimation range).
I don't understand, so is it the same or it isn't? "Maximum vector range" in Recode has the same effect as "Motion estimation range" in x264?

:thanks:

[bond]

I don't understand, so is it the same or it isn't? "Maximum vector range" in Recode has the same effect as "Motion estimation range" in x264?its not the same, x264 always uses 512 for vector range afaik?

[numaios]

I see, so we should always set 512 in Recode if we'd want the accuracy of x264, is that correct?

[max-holz]

Is it possible to update the guide with some explanation about the use of adaptive quantization?

[unmei]

What surprised me a bit is that the adaptive quantisation seems to work by "upping" the quality on bright areas (as compared to lower the quality in dark areas like i thought XviD did it).
I suspected this after doing a CQ (21) test with and without AQ. And while the one withou came out at 800 kbit/s the AQ one was 1400kbit/s (!). So i played the file and set ffdshow to show quantizers - and no longer surprisingly - a P frame supposed to be Q21 has the highest quantizer at 21 and the lowest one 12. A I frame (supposedly 18) even had blocks with quant 10 (that is the lowest allowed by the set Q range!).


Encoding with CQ 26 and AQ gave me a file with almost the same bitrate as Q21 and no AQ (800.52 vs 800.75 kbit/s). Interestingly the one with out AQ has a mean PNSR almost 1dB higher and global PSNR ~1.2dB higher than the one with AQ.

Also AQ seems to considerably raise the number of direct B macroblocks (2%->14%) and lower the number of skip B macroblocks (72%->42%). This is comparing the encodes with the same CQ (the one with adapted CQ 26 has 12% direct and 62% skip).

[Sharktooth]

one word: multipass.

[unmei]

i know that word :D

But would you elaborate a bit more.. will AQ only work correctly in a multipass scenario? Because this seems not to be the case..
Of course in CQ with AQ the file size becomes kind of even less predictable, but from what i saw so far, CQ with AQ also seems to be an option when i raise the "constant" quant to compensate for the lower quants in some macroblocks.

[Haali]

AQ should be used only when you have noticeable blocks in flat backgrounds. It will allocate lower qp to such areas. Adding some more bitrate won't hurt as well, so the rest of the frame doesnt lose too much quality.

[Teegedeck]

AQ seems to work fine for strong compression. I tested with various constant-quantizer encodings of the same clip and found that encoding at quantizer=27 without AQ yielded a filesize just above that produced by quant=29 + AQ. The encode at quant=27 without AQ exhibited extensive blocking (deblock=-1) while the one with AQ didn't and thus looked very acceptable, I daresay 'good'.

This should mean AQ makes the higher quantizers in x264 really worthwhile for the first time - because with AQ it's not necessary anymore to use strong deblocking that turns the whole picture into jelly.

Well done, Haali!

[IgorC]

--chroma-qp-offset <integer> QP difference between chroma and luma.
I couldn't fine any information about it.
When it's usefull to use this feature? At low bitrates? What are admissible integers numbers of interval --chroma-qp-offset [x,y]?

[akupenguin]

range: [-12,12]
I have no idea when it might be usefull. I just implemented it because it's one of those obscure headers that someone might want to play with.

[unmei]

Maybe this is totally obvious to anyone but me, but since it was not for me i thought i would mention it: Adaptive Quantisation will not only produce quantisers in the set QP range (QPmin, QPmax).

Probably i just had the wrong concept of these and they do not represent the actual quantizer limits but the limits of average quantizer in P frames (or something along that line).

I'm currently playing with the AQ and CQ vs CRF settings and let the QP range at default [10,51]. What i found so far are Q8 blocks in all frame types (I,P,B) ..and they are not rare at all, they might make up like a third of the frame.


Note that this is not a complaint, just a statement ;)


So far i really like the one movie i did completely with AQ and CQ 26 and as long as the result looks good i am not "against" low quantised blocks. Tho i might turn down the AQ strength a bit for the next movie (0.3 maybe), that is i do these tests right now to get a feeling of the AQ parameters.

[Kenshin5]

And here goes the first complaining about x264... When i pressed x264 uninstall it started deleting all my system32 dll files and the windows recovery thing showed up, next thing i restart windows is not working or shit. Fix this issue, i don't intend to reinstall windows for this.