Refferences and max B-frames

[SvenBent]

first of all thanx to DeathTheSheep for his guide.

i hate no to know jus a little about the settinfs i use.
and it relly helped understand and tweak my settings.

but i have to questions

1:
Can i increas the max b-frames without it migh hurt quality ?
i have adaptive b-frames on and if the only negative about going from 3 to 5 max b-frames is encoding time and if i could reveal just a sligtly qualiyy increase in seldom cases. I would go increase my current settgins of 3 to 5.

2
The same questions goes for the refference.
Currently i'm using 5 for "real" content and 8 for animated og 3D rendered movies.
Again... can i increase it to 16 without loss of quality?


I dont mind the extra encoding time.. if it's to slow i'll just upgrade

Thanx


P.S everything else is set after "DeathTheSheepAVCguide.pdf"

[foxyshadis]

I always use 8 b, and most of the time the codec uses 1-3. I figure for long easy pans and stills, I might as well let it code everything it wants as b-skips. It doesn't even seem to slow down much.

As for references, I use the rule of thumb, every reference back provides half as much benefit as the last for a constant extra slowdown (which depends on your me method). It's not exactly true, but it's the general idea; 16 references back almost nothing would correspond to the current video frames anyway, unless it's extremely cyclic, like some low-budget or old animation.

Something I'd be interested in is a debugging log of what references x264 uses, so that I can find out for sure where the optimal tradeoff usually is - in my copious free time(tm) maybe I'll try to add it (if it isn't there already).

[SvenBent]

Thanx for the answer
Since x264 is mult threaded i can just add more cpu's for the job :-)

however i cannot seem to set the b-frams to 8, max is 5.
Nut as you wrote, it probally want do much.
Snyway its going to be 5 from now on.

[foxyshadis]

The codec's max is 16, it might be the gui you're using limiting it (probably a vfw limitation, makes some sense for vfw though). Oh, and I remembered that it doesn't slow second-pass down at all, since all decisions are made on the first pass.

[Sirber]

IIRC RealAnime limits to 5 max. I usualy use 3. Is the use of more than 3 is recommended on anime?

[DeathTheSheep]

1:
Can i increas the max b-frames without it migh hurt quality ?
i have adaptive b-frames on and if the only negative about going from 3 to 5 max b-frames is encoding time and if i could reveal just a sligtly qualiyy increase in seldom cases. I would go increase my current settgins of 3 to 5.

x264 has mechanisms which decide whether additional B-frames will increase quality or not (make sure "adaptive" b-frames are enabled). You can set this to as high a number as you want, and x264 will use more if it believes it will increase quality. So yes, setting it to a higher number can indeed help quality slightly in rare cases, because it is giving x264 more freedom to decide to use more b-frames based on predicted quality. However, people using the VFW reported a lag in audio when using too many B-frames, so my guide is set to 3. Do what works best for you!

2
The same questions goes for the refference.
Currently i'm using 5 for "real" content and 8 for animated og 3D rendered movies.
Again... can i increase it to 16 without loss of quality?
Yes. As a general rule, the higher the better (stated in guide :P), so you can indeed increase the number of references to 16, which will highly reduce encoding speed and very slightly increase quality.

Happy encoding :)

[akupenguin]

Something I'd be interested in is a debugging log of what references x264 uses, so that I can find out for sure where the optimal tradeoff usually is - in my copious free time(tm) maybe I'll try to add it (if it isn't there already).
You mean, besides
x264 [info]: ref P 62.3% 10.1% 6.2% 3.3% 3.3% 2.9% 2.5% 1.6% 1.8% 1.1% 0.9% 0.8% 0.8% 0.8% 0.9% 0.7%
x264 [info]: ref B 70.6% 8.4% 4.9% 3.2% 2.2% 2.1% 1.6% 1.2% 1.1% 1.1% 0.8% 0.7% 0.7% 0.7% 0.7%
(ref #0 is on the left, ref #15 is on the right)
? Admittedly, a log per frame could be interesting too, because reference usage is not constant over the length of a movie. Someday I'll try making the number of refs searched adaptive based on the number used recently, and I'll need per-frame logs to tune that.
Note: even though some refs have about the same usage, there is still diminishing returns in the compression gain. e.g. if frames 8 and 9 are similar, either one would work almost as well, so the distinction between them doesn't add much.

[Blue_MiSfit]

Just a suggestion, perhaps something like the XviD status window?

I wish I could program...

~Misfit

[guada 2]

Hello :) ,

The codec's max is 16, it might be the gui you're using limiting it (probably a vfw limitation, makes some sense for vfw though). Oh, and I remembered that it doesn't slow second-pass down at all, since all decisions are made on the first pass.
Yesterday 04:15

Yes. As a general rule, the higher the better (stated in guide :P), so you can indeed increase the number of references to 16, which will highly reduce encoding speed and very slightly increase quality.


Interesting..
But, i often use VDM with x264.
How is it possible to increase this limitation of 5 out of the programming?

[foxyshadis]

No one but celtic_druid works on vfw gui, and he's indicated he's not interested in the "status window"... but the project's always willing to take patches. Unless you mean an elaborate text-mode version, but likely the same deal.

guada, my post referred to b-frames, 16 references will really slow down your second pass. And you'd probably have to patch vfw and recompile it, if you can live with the audio desync for extra b-frames.

[GodofaGap]

Perhaps it is noteworthy that I have seen occurances where the adaptiveness of b-frames is at least not PSNR safe (e.g. -b 3 gives higher PSNR than -b 5). I think anything above -b 3 is pointless in the majority of the cases anyway, so just don't bother with it. The desynch for b-frames in VFW can be overcome by deleting the drop-frames afterwards. So that shouldn't be a worry.

As for multiple references, they don't always are that a good option speed/quality wise. I would choose trellis 1 or/and subme 6 before using multiple references.

[guada 2]

Thank you foxyshadis. :)

But you don't think that it is preferable to increase the quality and to find an alternative to the speed; as it was the case of the Wavelet. especially when we has already a decoder (Paying certainly but useful).

I wonder if it was possible to force NERO DIGITAL to pass the cape of 8?
Is the speed of the NERO codec inspired it of this analysis?

Yes. As a general rule, the higher the better (stated in guide :P), so you can indeed increase the number of references to 16, which will highly reduce encoding speed and very slightly increase quality.

Does it mean that the second is more important than the first, whereas the 2 passes are important (said me AKUPENGUIN in a post)?

[DeathTheSheep]

Both passes are considered to be equally important, though the fast first pass is more of an analysis round than the actual full-blown encoding. The major encoding process, which takes place in pass 2, is based on the information gathered from the first pass, without which its performance would be severely impaired. It is thus best to allow the first pass to most accurately obtain this crucial information, which is best accomplished when the settings (such as the number of references) stay the same for both passes.

In my personal experience, if not based on actual metric testing, increasing the subpixel refinement or "partition decision" (--me[x], x=1-7) leads to more of a quality gain and less of a speed sacrifice than the addition of more than 5 references in real-life content. Thus, one can relatively safely conclude that an increase in subpixel refinement quality leads to better overall quality in less encoding time than a similar quality improvement yielded by increasing reference frames beyond 5, at least with x264's current implementation of reference encoding.

[guada 2]

:thanks: DeathTheSheep :) .

Another thing;
Which is the low limit critical and the high limit critical of the codec x264, in term of bitrate?

[foxyshadis]

1 kbps and 1 Gbps.

You'll have to be a little more specific if you want a useful answer.

[akupenguin]

Both passes are considered to be equally important, though the fast first pass is more of an analysis round than the actual full-blown encoding. The major encoding process, which takes place in pass 2, is based on the information gathered from the first pass, without which its performance would be severely impaired. It is thus best to allow the first pass to most accurately obtain this crucial information, which is best accomplished when the settings (such as the number of references) stay the same for both passes.
Right reasoning, wrong conclusion.
The 1st pass only has to predict the size at a given QP, while the 2nd pass has to figure out exactly which mvs, coefficients, etc achieve that size. Thus, all settings can be reduced, as long as the fast version can predict how much beneift the slow version gives. Analysing 2 or 3 refs gives a very good approximation of the gain from 16 refs. Yes, there will be certain frames that gain much more from 16 refs than the prediction says, but that's negligible when amortized over the length of a movie.
The question is not "does increasing 1st pass settings improve quality". The question is "does increasing first pass settings improve quality more than spending the same extra time on the 2nd pass". And the answer is usually "no".
People use 1st pass subme1 ref1 analyse none, for a reason. If you want to relax those, subme is the first to increase, not refs.

[guada 2]

@foxyshadis
1 kbps and 1 Gbps.

Why did you say that?

Just a Look at this :

I have a file of: 1.26 Mo
I use VDM + X264 with a bitrate: 40 kbit/s
My script: the last Limitedsharpen + IIP (script SOCIO)

the final result:

final size: 286 ko
VDM: 2pass 1stpass: Done(warning)
VDM: 2pass 2stpass: Done(warning)

What do you think about it?
Then, what would say VDM if i use a such bitrate 1 kbit/s ?

@akupenguin
Thank you for these little precisions.

[SvenBent]

Thanx DeathTheSheep for you answers in #6
just the info i needed,

Seems i keep the B-frames at 3 until the issue is sorted out (if there is indeed a issue).
maybe i'll try to check it out myself.

and up goes the refferences :-)

-- edit --
may i sugest at benchmark in you next revision of you guide,
så that people coudl se ow muc the settigns would increase encoding time.

[DeathTheSheep]

Akupenguin: "There will be certain frames that gain much more from 16 refs than the prediction says."

My conclusion: "It is thus best to allow the first pass to most accurately obtain this crucial information, which is best accomplished when the settings (such as the number of references) stay the same for both passes."

Akupenguin: "Right reasoning, wrong conclusion."

In this situation where the benefit of more than 8 references is itself negligible, it is apparent that the absolute maximum quality level is being saught here, a level only attainable with the most accurate of predictions.
I still cannot see any difference between my conclusion and your statements...? :)

[akupenguin]

In this situation where the benefit of more than 8 references is itself negligible, it is apparent that the absolute maximum quality level is being saught here, a level only attainable with the most accurate of predictions.
Speed is never entirely ignored. If I decide that the optimal settings at one speed is 1st -r2, 2nd -r8, and I have some extra time to spend, I may increase it to 1st -r3, 2nd -r16. As negligible as the gain of 2nd pass ref16 is, 1st pass is even more so, but the time cost is the same.
The only sane arguments for 1st pass refs is scenecut detection and direct=auto, which may conceivably by influenced a little.

[foxyshadis]

Sven, it's informal and not entirely up to date, but:

http://forum.doom9.org/showthread.php?t=107699
http://forum.doom9.org/showpost.php?p=795658&postcount=35

Timing and ssim information included.

Guada, for all intents and purposes x264 has no minimum and maximum, there's only mins and maxes for individual videos. (IE, q51 to q1.) The VFW slider is just an arbitrary limit, you can still type anything into the bitrate box if you need, say, 12 mbps. I still have no idea exactly what you're actually asking about though, can you be a bit more clear?

[Caroliano]

final size: 286 ko
VDM: 2pass 1stpass: Done(warning)
VDM: 2pass 2stpass: Done(warning)

What do you think about it?
Then, what would say VDM if i use a such bitrate 1 kbit/s ? If you use an adequate source, like a sequence of black frames, probablily VDM would say nothing. All that bitrate thing depends on source. It's normaly better to ask about quantizers. Then I'd say that the range 15~40 is usable. Anything out of that, it's better change your bitrate/resolution/whatever.

I still cannot see any difference between my conclusion and your statements...?
I think you overlooked this: The question is not "does increasing 1st pass settings improve quality". The question is "does increasing first pass settings improve quality more than spending the same extra time on the 2nd pass". And the answer is usually "no".
And besides that, Daodan reached in very strange results in his tests: http://forum.doom9.org/showthread.php?p=791504#post791504

The 2-pass with turbo had an higher quality than the one with out that. Isn't that strange?

[guada 2]

foxyshadis & Caroliano,

It is strange but it is true.

With 1 Kbps:

My file: exactly 1.22 Mo
I use VDM (1.5.10.2) + X264 (Rev 518) with a bitrate: 1kbit/s
My script: the last Limitedsharpen + IIP (script SOCIO)

the final result:

final size: ????
VDM: 2pass 1stpass: Done
VDM: 2pass 2stpass: Error

and this:
requested bitrate is too low. estimated minimun 7 kbps


Report x264-2-1-1.stats notepad

#options: 720x576 fps=25/1 cabac=1 ref=16 deblock=1:-2:-2 analyse=0x3:0x113 me=esa subme=6 brdo=1 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 chroma_qp_offset=0 slices=1 nr=0 decimate=1 bframes=5 b_pyramid=0 b_adapt=1 b_bias=0 direct=2 wpredb=1 bime=0 keyint=250 keyint_min=25 scenecut=40 rc=abr bitrate=1 ratetol=4.0 rceq='blurCplx^(1-qComp)' qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 pb_ratio=1.30
in:0 out:0 type:I q:21.00 itex:132828 ptex:0 mv:15887 misc:4637 imb:1620 pmb:0 smb:0 d:-;
in:6 out:1 type:P q:24.00 itex:0 ptex:15384 mv:6087 misc:1233 imb:0 pmb:731 smb:889 d:-;
in:1 out:2 type:b q:26.00 itex:0 ptex:1260 mv:1211 misc:1057 imb:0 pmb:535 smb:1084 d:-;
in:2 out:3 type:b q:26.00 itex:0 ptex:1918 mv:2357 misc:1117 imb:0 pmb:567 smb:1050 d:-;
in:3 out:4 type:b q:26.00 itex:0 ptex:2531 mv:3253 misc:1216 imb:0 pmb:492 smb:1114 d:-;
in:4 out:5 type:b q:26.00 itex:0 ptex:2639 mv:3707 misc:1142 imb:0 etc.........

Your idea please...

[DeathTheSheep]

arguments for 1st pass refs is scenecut detection and direct=auto
Wow, that's exactly what I use! :D

[nurbs]

@guada 2:
You don't really expect you can encode a 720*576 video at 1kbps, do you? You tell the encoder to go to 1kbps, it sees that it can't reach that bitrate even with q=51 and so it gives you error messages.
If you want to find the minimum bitrate just do a const quant encode at q=51 or whatever the lowest setting for x264 is. I'm not sure what you are trying to do, since it won't give you watchable quality anyway.

[Caroliano]

Your idea please...
I said all depends on the source... a sequence of black frames in QCIF resolution maybe... but I won't continue discussing that. Foxyshadis only said this to show how your question was vague, nothing more. I already answered it in my perspective (quantizers instead of bitrate).

[foxyshadis]

On the other hand, 7 kbps is pretty awesome. It may well be a mess of flat 8x8 blocks, with a little inloop smearing, but the holy grail of fitting a dvd-res tv shows on a floppy may be here! (You can get about 25 minutes that way... of course you'd have to factor in an extra 1 kbps for speex audio.) This sounds like a breakthrough DeathTheSheep would be interested in! :p

guada, bitrate and quality are very hard to correlate in general. You'd be much better off using a crf to estimate size for a certain quality. I'll even make a little table of average quants for you.

0: lossless.
10: Huge. Inloop usually disabled.
15: Effectively transparent to most viewers, especially with inloop turned down. For DVD-R encodes.
20: Very good quality, not transparent but good enough for most people.
23: Often the best size/quality tradeoff for me.
25: Decent quality, if pressed for size.
30: Reminds one of Divx3.
35: Reminds one of Indeo3, smeared and a bit blocky.
40: Reminds one of RealVideo3 streamed clips, barely watchable.
45-51: Entirely unwatchable.

The bitrate you get from these is the important bitrate, not some theoretical maximum or minimum bitrates.

Also, at higher resolutions you can usually get by with higher quants, but since you're getting rid of details anyway, you might as well resize down and use a higher quality.

[guada 2]

can you be a bit more clear?
Today 20:11


@DeathTheSheep
Wow, that's exactly what I use!
Today 21:22

Don't laugh.
i think, it is normal and necessary to know, how to do understand itself: by using other methods to succeed an answer.
English is not my (native) language.

@nurbs
If you want to find the minimum bitrate just do a const quant encode at q=51 or whatever the lowest setting for x264 is.
:thanks:
that is the answer i was seeking.

I'm not sure what you are trying to do, since it won't give you watchable quality anyway..
It is not the real problem.
I use these settings because x264 is a very good codec on low-medium bitrate and surely on ultra low bitrate.

Finally, i seek an alternative between optimal script and x264optimization to exploit ultra low bitrate on big size.

Just this:
the script of socio is not the most advanced, for this I used for scripts different.

Thank you very much.
To soon.

[ChronoCross]

It is not the real problem.
I use these settings because x264 is a very good codec on low-medium bitrate and surely on ultra low bitrate.

Finally, i seek an alternative between optimal script and x264optimization to exploit ultra low bitrate on big size.

Just this:
the script of socio is not the most advanced, for this I used for scripts different.

Thank you very much.
To soon.

I'm sorry...seriously this is just a little retarded. x264 could be the greatest codec in existence and still wouldn't be able to hit 1kbps.

I would recommend not going below 100kbps for anything. it will barely look watchable even at that. Especially at that resolution. If your looking to do streaming or have a super low filesize for distribution your going to need to drop the resolution down else your not going to be able to fit it into whatever crazy size your aiming for.

[foxyshadis]

Well, if you like blurry blobby video that's your perogative. If you are looking for advice on dropping the bitrate, try dropping the framerate to half or 1/3 and resizing down some too. You'll probably want to raise the me range and pretty much maximize your other options too. (Which isn't all possible in VFW.) I think DTS's old guide pretty much went over all that, but I'm not sure if his new one does.

I still stand by my assertion that average qp of 40-44 is the minimum watchable quality.

[guada 2]

@ChronoCross & foxyshadis
:thanks: for your opinions.

If your looking to do streaming or have a super low filesize for distribution your going to need to drop the resolution down else your not going to be able to fit it into whatever crazy size your aiming for.

I 'm looking to do streaming with it, without changing the size, but finding the smallest value possible of bitrate ( of course not 1kbps) and combine it with four optimal scripts.
" A way to do a upscaling streaming "....
In my opinion i think it is possible but very difficult...

if i use it, which audio codec i must choose at 20 kbps.

Bye.

[foxyshadis]

Nero's new he-aac, there simply is no alternative that low. (There may be as soon as Aoyumi finishes his new q-negative tunings.) I tested several at "as low as it goes" quality earlier, nero's new aac was the best.

I made a whole anime episode at 25 kbps x264, 13 kbps AAC, and I could still follow it, so it's certainly possible... I resized to 568x320 and use 8 fps, so you could probably do D1, 8 fps, at 50 kbps, audio at 20, while skating the "barely watchable" q42. Hey, if you don't mind realmedia streaming quality...

[guada 2]

Thak you very much for your advice.
It wil be useful for me. ;)


Just a question:
I tested several at "as low as it goes" quality earlier, nero's new aac was the best.

Did you try Winamp aacv2 PS in your test?

[foxyshadis]

No, I only have 3gpp and faac, so I'll reserve judgement that winamp's PS-SBR might be better. Test and report! But I also compared the latest vorbis release and prerelease, wouldn't go as far down (needs a q-3 :D) and sounded a little worse. And speex, which munges the music something awful but actually sounds a lot better in the pure speech parts; but there was a lot of music so.... (I don't think there's really any other formats that don't just fall over and die at that bitrate.)

[guada 2]

A few example, just with BELIGHT:

At 20 kbps:
Winamp (CT) aacv2 PS vs 3gpR vs the latest vorbis ... vs new nero vs the latest lame.

Your verdict.

[DeathTheSheep]

I suggest the new CLI nero at 20kbps (maybe even 2pass if you need the extra few driblets of quality) :)

7 kbps is pretty awesome...of course you'd have to factor in an extra 1 kbps for speex audio. This sounds like a breakthrough DeathTheSheep would be interested in!
Oh, come on, 7 whole kbps is really stretching it... There's no way I'm going to buy more than one floppy disk to store my movie collection on...

:o

[guada 2]

@DeathTheSheep,

I suggest the new CLI nero at 20kbps

And WMA 10 pro?

[foxyshadis]

You'll have to test yourself, probably. Not too many people have the leaked beta. (I don't, and won't until the final.) It shouldn't be too hard to encode an episode in all the relevant codecs and see how they sound to you; the sounds I test on may not be representative at all of yours.

[DeathTheSheep]

Maybe WMA10Pro+ (SBR+PS), but standard WMA10Pro doesn't cut it (20kbps is too low for it, it seems)... But we'll see tomorrow!