The new B-frame decision (--bframes 16 and you)

[Dark Shikari]

One of the GSOC projects for the summer was to improve B-frame decision. This was the most important project, IMO... but the student on the task just disappeared after the first week.

So I decided to do his job for him, and with akupenguin's assistance on the algorithm side of things, spent this week writing an optimal trellis-like function for B-frame decision. This algorithm provides perfect B-frame decision--or more precisely, as perfect as one can get with the current cost approximation function used.

The downside is that its a bit slower, and moreover, its speed scales linearly with --bframes. While with the previous algorithm, --bframes 16 was only minimally slower than --bframes 3, here it will be a lot slower. And this new algorithm is going to be default, since it is considerably better. The old algorithm will still be available for those who need a fast but inaccurate B-frame decision method.

So, this is a warning to all those who use --bframes 16, MeGUI and otherwise; the new B-frame decision is coming soon, and when it does, in the interests of speed you may want to lower the --bframes value to 3-5 or something otherwise reasonable.

[LoRd_MuldeR]

Thanks for the info, looking forward :thanks:

BTW: Will this be available as another patch or will it be released directly in the git?

[Dark Shikari]

Thanks for the info, looking forward :thanks:

BTW: Will this be available as another patch or will it be released directly in the git?Almost certainly direct when I feel that it has been sufficiently tested. This one's easy to test, since I can just test raw PSNR (unlike psy optimizations).

[gav1577]

Well done Dark Shikari & akupenguin your work is always appreciated keep up the good work :thanks:

[Ranguvar]

Fantastic :D

I can't wait until this comes out; I've been looking forward to it since I heard it was a GSoC project.

Great work, Shikari and aku, as always! :)

[Sharktooth]

One of the GSOC projects for the summer was to improve B-frame decision. This was the most important project, IMO... but the student on the task just disappeared after the first week.

So I decided to do his job for him, and with akupenguin's assistance on the algorithm side of things, spent this week writing an optimal trellis-like function for B-frame decision. This algorithm provides perfect B-frame decision--or more precisely, as perfect as one can get with the current cost approximation function used.

The downside is that its a bit slower, and moreover, its speed scales linearly with --bframes. While with the previous algorithm, --bframes 16 was only minimally slower than --bframes 3, here it will be a lot slower. And this new algorithm is going to be default, since it is considerably better. The old algorithm will still be available for those who need a fast but inaccurate B-frame decision method.

So, this is a warning to all those who use --bframes 16, MeGUI and otherwise; the new B-frame decision is coming soon, and when it does, in the interests of speed you may want to lower the --bframes value to 3-5 or something otherwise reasonable.
Dont worry, MeGUI presets with 16 b-frames (except the insane presets) will be updated as soon as i see that commit in x264 git changelog, also the Devices, Standalones, DXVA presets have already a limited number of b-frames.
thanks for informing us.
can you tell us what will be the x264 option to select the old decision mode (so i can start implementing in megui)?

EDIT: I've just updated the new x264 presets for MeGUI...

[Dark Shikari]

Dont worry, MeGUI presets with 16 b-frames (except the insane presets) will be updated as soon as i see that commit in x264 git changelog, also the Devices, Standalones, DXVA presets have already a limited number of b-frames.
thanks for informing us.
can you tell us what will be the x264 option to select the old decision mode (so i can start implementing in megui)?--b-adapt will be the option, and 0 will be the same as --no-b-adapt, 1 will be the old method, 2 will be the new method.

--no-b-adapt will still exist for legacy compatibility.

[Sharktooth]

ok. is it worth keeping the old method?

[Dark Shikari]

ok. is it worth keeping the old method?If you want very fast encoding, the old method is probably still useful for that purpose. The new method has a significant speed penalty, albeit only on the first pass.

From my testing so far, the new B-frame decision gives about 3% more efficient encoding overall and 15% or more improvement in cases with lots and lots of fades.

[Sharktooth]

well, for very fast encoding we have presets with 0 or 1 b-frames... (like --crf 22 --bframes 1 --nf --no-cabac --subme 1 --partitions none --me dia --threads auto --thread-input --progress --no-psnr --no-ssim)
if you want uber speed i dont think quality/compression is so important... also almost all multipass presets include a fast first pass...

[Dark Shikari]

well, for very fast encoding we have presets with 0 or 1 b-frames...With the old B-frame algorithm, fewer B-frames are not necessarily faster. Zero is almost always faster than 1 or more, but 1 is not necessarily faster than 3.
if you want uber speed i dont think quality is so important... also almost all presets include a fast first pass...My point being that this will slow down that fast first pass a good bit ;)

[Sharktooth]

edited my previous post.
however, the idea is if you want fast encoding, you wont run multiple passes...

[Dark Shikari]

edited my previous post.
however, the idea is if you want fast encoding, you wont run multiple passes...A one-pass encode counts as a "first-pass" for purposes of being slowed down by B-frame decision.

[Sharktooth]

yep. in any case, you have to choose, speed or compression/quality.
with current HW, speed is becoming less important.

[Avenger007]

Deos this new algorithm improve b-pyramid as well?
Is the number of consecutive b-frames generally going to increase when compared with the old algorithm? The old algorithm almost never used more than 4 consecutive b-frames.

[Dark Shikari]

Deos this new algorithm improve b-pyramid as well?The new algorithm does have pyramid-specific code, so it should. It doesn't have any code for deciding between pyramid and non-pyramid though; pyramid is still forced by x264; it just takes pyramid into account when deciding number of B-frames.

Is the number of consecutive b-frames generally going to increase when compared with the old algorithm? The old algorithm almost never used more than 4 consecutive b-frames.That's because more than 4 is almost never useful ;)

I've found generally that the new algorithm, without pyramid, may raise or lower number of B-frames. With pyramid, it generally raises the number.

[cogman]

So, will the improvement of bframe 16 be pretty much non-existent over say bframe 6? I guess I'm just wondering if this will make bframes similar to the way reference frames are now (IE, 16 refs may give some imperceivable improvement but you are fine with far fewer).

Though, My quad is hungry, so I probably won't change my encoding settings :D

[Soichiro]

Unlike in live-action, where more than 3-4 bframes are rarely placed in a sequence, more than 4 bframes is very useful in animated sources. Previously, my testing showed that increasing bframes from 6 to 16 provided a 10% speed hit, but a fairly large quality boost on an animated test clip (much better than increasing ref frames, which costs much more speed). However, I'm wondering just how much of a speed hit this new method will cause, and if it will still be worth it...

[Comatose]

Me too... my encodes are already super slow, I'm not sure I want it to be a ton slower for only a slight improvement in quality :\
I guess I'll have to try it for myself to decide if it's worth it x_x

[Soichiro]

I did some test encodes.

http://kuroha.no-ip.org/New%20BFrame%20Test.mkv
http://kuroha.no-ip.org/Old%20BFrame%20Test.mkv

And the logfile: http://kuroha.no-ip.org/Stats.txt

(and stats files for those who want them: http://kuroha.no-ip.org/oldmethod.txt & http://kuroha.no-ip.org/newmethod.txt)

2nd pass encodes were nearly the same speed, but the first pass with the new bframe method was over 50% slower, and the thing that concerns me is that the new method actually creates a slightly worse PSNR and SSIM. The new method also seems to favor b-frames a lot more, but in a pattern that strikes me as a bit odd, with 20% of bframes being located in 15-frame sequences.

(Unrelated, but I totally need to edit that kara. Good thing I used this as a test encode <_<;; )

Edit: After watching through both encodes, I'd have to say that, without any careful comparison, the new b-frame system seems to look a bit better, despite the PSNR and SSIM.

[Dark Shikari]

This is one reason I'm not releasing it officially yet--there may still be some kinks to work out ;) In particular, I'm not entirely sure whether the method I used for B-pyramid heuristic is going to be problematic or not.

The only case I have found where the new algorithm is worse than the old algorithm is the case where you have a bunch of frames that are nearly 100% I-blocks, and not because of a fade. A couple scenes in "300" do this. This is a flaw in the cost algorithm, not really in the decision method.

Try without b-pyramid to see if the pyramid heuristic is the problem--since I haven't tested that as much.

My own tests show about a 0.15db improvement on everything, PSNR-wise, and about a 0.75db improvement on an animated clip with lots of fades. The extreme clip from "300" lost about 0.2db PSNR.

Edit: I already may have found a rather nasty bug in it, so don't complain just yet ;)

[iwod]

So you have just done what was suppose to be a whole summer of work in one week of your spare time?? :D

[Dark Shikari]

So you have just done what was suppose to be a whole summer of work in one week of your spare time?? :DEh, the project was supposed to cover a lot more than what I did (pyramid vs non-pyramid decision, better frametype cost function, multi-level b-pyramid, opengop, etc).

[burfadel]

As stated with psy-RDO etc etc, what looks better is more important than SSIM and PSNR, as they are not entirely accurate. Afterall, isn't it a measurement of the differences between the original and encoded, with a higher score for less difference? Now the differences for the old method may be slighly less, but the difference in the differences may be a lot more for the old method. That would mean the new method would look better...?

[Dark Shikari]

As stated with psy-RDO etc etc, what looks better is more important than SSIM and PSNR, as they are not entirely accurate. Afterall, isn't it a measurement of the differences between the original and encoded, with a higher score for less difference? Now the differences for the old method may be slighly less, but the difference in the differences may be a lot more for the old method. That would mean the new method would look better...?For everything other than psy optimizations, one has to make the assumption that a given change will not have a signiificant psychovisual impact--and therefore that one can use PSNR when optimizing the feature. Otherwise its too difficult to try to improve non-psy optimizations.

[burfadel]

I think I might have added my last post on the wrong thread :) oops! well, that's easy enough I guess with multiple similar tabs open!

Anyways, I was going to ask, is the slowdown and benefit of higher b-frames similar to that of with reference frames? Also, with psy-rdo I'm guessing the new decision mode may even show more gains, due to the nature of psy-rdo? I know they're not exactly related, but since psy-rdo changes the image in a particular manner the b-frame decision may be changed?... :)

[Dark Shikari]

It seems the new algorithm's slowness has drastically aggravated the threading problem with fast firstpass + b-adapt... I'm reconsidering making it default. I may wait until threaded slicetype decision is done before making it default.

[burfadel]

It seems the new algorithm's slowness has drastically aggravated the threading problem with fast firstpass + b-adapt... I'm reconsidering making it default. I may wait until threaded slicetype decision is done before making it default.

Does this affect CRF mode though? it could be made default for CRF but optional for multiple passes (such that CRF just implies that it is enabled).

[Dark Shikari]

Does this affect CRF mode though? it could be made default for CRF but optional for multiple passes (such that CRF just implies that it is enabled).Its just as bad in CRF--its just that in CRF, people normally use slower settings, while in the first pass of a 2pass encode, they use fast settings.

Anything that causes threading issues is going to be less problematic at slower setting levels.

[guada2]

@Dark Shikari

A small question:
Which is the adjustment of reference of this codec for multipass mode(1+2) at High bitrate and ultra low bitrate?

[vmrsss]

Hi Dark Shikari, and thanks for the good work.

(1) If the new algorithm scales linearly with the --bframes parameter, and we know that the final number of consecutive B frames varies a lot with the kind of source, it'll be useful to have a rule of thumb to decide what --bframes to use for each encode rather than going straight to --bframes 16. Or is it that the speed depends not on the parameter but on the number of consecutive B frames actually allocated during the encoding?

(2) Are the key aspect of the new algorithm described somewhere?

best

[akupenguin]

Actually cost scales with the square of max-B-frames, so you really don't want to use 16. We have discussed an optimization which could reduce it to linear, but that's not implemented yet.
If the new algorithm scales linearly with the --bframes parameter, and we know that the final number of consecutive B frames varies a lot with the kind of source, it'll be useful to have a rule of thumb to decide what --bframes to use for each encode rather than going straight to --bframes 16.
1) I just added a line to the stats summary you get at the end of every encode, so try out the algorithm and see how many you get.
2) If you don't have a good reason to think otherwise, 3 is a good number.
3) We will probably eventually add some heuristics to speed up trellis, but that comes after the generically optimal case.
Are the key aspect of the new algorithm described somewhere?
Good question. It wouldn't make sense to explain this instance of the algorithm in isolation (Trellis can be used for lots of things, and instantiating it only for frame types obscures the real structure), but I have wondered about a good introduction of Trellis to point people to. I personally consider it just a special case of Dijkstra's algorithm, but that probably doesn't help, because if you knew Dijkstra's shortest path algorithm this wouldn't need explaining.

[vmrsss]

2) If you don't have a good reason to think otherwise, 3 is a good number.

Ah, quite a few fewer than I expected...

3) We will probably eventually add some heuristics to speed up trellis, but that comes after the generically optimal case.

Hmm, so the new algorithm is an instance/variation of trellis? (Sorry if the question is dumb.) Sometimes you read trellis is not recommended with --crf (which is what i use most of the times), does this change with the new algorithm?

I have wondered about a good introduction of Trellis to point people to. I personally consider it just a special case of Dijkstra's algorithm, but that probably doesn't help, because if you knew Dijkstra's algorithm this wouldn't need explaining.

On the contrary, maybe I don't know which of Dijkstra's algorithms you're referring to and a general explanation of the context an specifics of the problems, together with a pointer to Dijkstra would be exactly sufficient.

[akupenguin]

Hmm, so the new algorithm is an instance/variation of trellis?
Trellis is exactly the same as trellis. It just so happens that "trellis quantization" was implemented before "trellis frametype decision", so the former got abbreviated.

Sometimes you read trellis is not recommended with --crf (which is what i use most of the times), does this change with the new algorithm?
That misconception has somehow persisted since long ago when XviD's implementation of trellis quantization had a bug. I have attempted to purge it from the internet, but of course that's impossible; I can only edit wikis, not individual people's memory.

On the contrary, maybe I don't know which of Dijkstra's algorithms you're referring to and a general explanation of the context an specifics of the problems, together with a pointer to Dijkstra would be exactly sufficient.

Dijkstra's shortest path algorithm. (http://en.wikipedia.org/wiki/Dijkstra's_algorithm)
We want to find a sequence of frame types (or DCT coefficient values, or other decisions) to minimize their total cost. (We of course have to measure cost somehow, so when I say the solution is optimal, I only mean with respect to the particular metric I picked.) The cost of each frametype at each position depends not only on the pixel content of that frame, but also on the types of the neighboring frames, but the amount of such history that's relevant can be condensed into a rather small context (in particular, just name which frames this one is predicted from).
So we construct a weighted directed graph. Each vertex represents a P-frame. Each edge represents an encoding of the destination P-frame predicted from the source P-frame. The weight of an edge is the cost of that encoding of that P-frame plus the costs of that encoding of all intervening B-frames.
Then we ask Dijkstra to find the path from start to end with the lowest cost.
The special case part is that we never really construct a graph, just do all the comparisons in the shortest path algorithm implicitly, and some can be reordered for convenience, and all array sizes are small and known in advance.

[Dark Shikari]

Simple explanation of algorithm using an example:

In this example, Max B-frames is 2. All paths must end with a P-frame.

Path length of 1 gives us P as our best path.

Path length of 2 gives us BP and _P. "_" is the best path of length 1.
This gives us BP and PP. Now, we calculate the actual costs.
Let's say, for the sake of this example, that BP is best.

Path length of 3 gives us BBP and _BP and __P. "__" is the best path of length 2.
This gives us BBP and PBP and BPP. Now, we calculate the actual costs.
Let's say, for the sake of this example, that BBP is best.

Path length of 4 gives us _BBP and __BP and ___P. "___" is the best path of length 3.
This gives us PBBP and BPBP and BBPP. Now, we calculate the actual costs.
Let's say, for the sake of this example, that BPBP is best.

Path length of 4 gives us __BBP and ___BP and ____P. "____" is the best path of length 4.
This gives us BPBBP and BBPBP and BPBPP.

Now--wait a minute--all of the paths agree that the first frame is a B! So the first frame is a B.

Process is repeated until they agree on a P-frame, and then encoding starts.

(Note: current algorithm does not actually use the "agreeing" method--instead it just goes for quite some distance into the future and uses the longest path as the tie-break.)

[LoRd_MuldeR]

This somehow reminds me of the "Knapsack problem" ;)

[lexor]

This somehow reminds me of the "Knapsack problem" ;)

I hope not, that sucker is NP-complete. If they were alike, we'd be paying way more than linear time per b-frame.

[Dark Shikari]

I hope not, that sucker is NP-complete. If they were alike, we'd be paying way more than linear time per b-frame.Yup, this problem appears NP-complete at first-glance, but in reality is just linear-time if you dynamic-program it.

[Inventive Software]

This might sound daft, but in the past with a 2-pass encode, I use the same number of B-frames in both passes because that way the stats file might be of some use in this. Does this now make more sense with the new algo?

[Dark Shikari]

This might sound daft, but in the past with a 2-pass encode, I use the same number of B-frames in both passes because that way the stats file might be of some use in this. Does this now make more sense with the new algo?You have to use the same number of B-frames on both passes--the B-frame decision isn't done in the second pass, its re-used from the first.

[skystrife]

Interesting development, not sure if anyone else has this problem.

When running my own patched build + the new bframe decision patch I don't achieve 100% cpu usage (I get about 25%-ish on 4 cores) as opposed to 100% on 4 cores with the patched build without the new bframe decision. Is this to be expected or is something borking in the threading department with the new bframe decision method?

[LoRd_MuldeR]

This might be because the frametype-decision, which is not multi-threaded yet, becomes the bottleneck?

BTW: Where can I get that patch? :)

[Audionut]

Current patches thread of course. ;)

[LoRd_MuldeR]

I don't think the new improved b-frame decision has been posted as a patch yet. If I'm mistaken, please point me to that post...

[skystrife]

I don't think the new improved b-frame decision has been posted as a patch yet. If I'm mistaken, please point me to that post...

http://forum.doom9.org/showthread.php?p=1163095#post1163095

[LoRd_MuldeR]

Ehm? I think this link is either broken or the patch has been removed :confused:

http://pastebin.com/d495ce2e

[Dark Shikari]

I set it up for only a day for a reason :p

Here's the latest version (http://pastebin.com/f75a7283), which reverts to --b-adapt 1 as the default.

[LoRd_MuldeR]

I set it up for only a day for a reason :p

Here's the latest version (http://pastebin.com/f75a7283), which reverts to --b-adapt 1 as the default.

Thank you, Sir.

[foxyshadis]

Interesting development, not sure if anyone else has this problem.

When running my own patched build + the new bframe decision patch I don't achieve 100% cpu usage (I get about 25%-ish on 4 cores) as opposed to 100% on 4 cores with the patched build without the new bframe decision. Is this to be expected or is something borking in the threading department with the new bframe decision method?

And you definitely have pthreads compiled in?

[skystrife]

Yes, pthreads is compiled in. The same build environment was used for both the new bframes decision build and the old bframes decision build.

[vmrsss]

Hi again

1) I just added a line to the stats summary you get at the end of every encode, so try out the algorithm and see how many you get.

There is one extra value in the stats, eg for --bframes 16 I get:

x264 [info]: consecutive B-frames: 63.3% 4.1% 6.9% 6.9% 13.3% 3.9% 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 1.5%

Can you please explain the 17th number?

In this case this tells me that about 95% of the B-frames occurs in sequences of length at most 5. Does this mean that forcing --bframes 5 carries corresponding to adding (0.05 * total number of B-frames) P-or I-frames? Another way to look at this is that the expected length of a B-frame sequence is 2.104, which may indicate that your previous suggestion --bframes 3 is sensible.

[Dark Shikari]

Can you please explain the 17th number?They're 0 to 16 B-frames, not 1 to 17.

[LoRd_MuldeR]

But why 16 consecutive b-frames were used in 1.5% cases, when 6-14 consecutive b-frames were used almost never?

[Dark Shikari]

But why 16 consecutive b-frames were used in 1.5% cases, when 6-14 consecutive b-frames were used almost never?There was clearly some sort of case where "more B-frames == better," according to the algorithm, and it capped at 16.

This could be, for example, at a completely static image.

[Ranguvar]

There was clearly some sort of case where "more B-frames == better," according to the algorithm, and it capped at 16.

This could be, for example, at a completely static image.
Would it be possible to set an option so that for speed purposes, there cannot be more consecutive b-frames than a set amount, as it is now, but x264 can use up to 16 b-frames in places it detects as the quoted text, like completely static images? Would the speed hit be worth the space saved by using more b-frames?

[Dark Shikari]

Would it be possible to set an option so that for speed purposes, there cannot be more consecutive b-frames than a set amount, as it is now, but x264 can use up to 16 b-frames in places it detects as the quoted text, like completely static images? Would the speed hit be worth the space saved by using more b-frames?More useful might be an extension to the current slicetype trellis method, as suggested by Manao, to cull paths that are not likely to be useful.

[LoRd_MuldeR]

I doubt that the "shortest way" algorithm can check 16 b-frames without having checked all the intermediate numbers as well...

[vmrsss]

They're 0 to 16 B-frames, not 1 to 17.

uh? sequence of 0 B-frames? From the code it appears clear that --counting from 0 to 15 -- the ith number is the proportion of B-frames occurring in sequences of length exactly i+1.

So is the 16th number the number of B-frames which would end up being in sequences longer than 16 if that weren't ruled out by the --bframes 16 option?

[LoRd_MuldeR]

Of course you can use a "sequence of 0 B-frames". Sometimes using only P-frames might be cheaper...

[vmrsss]

There was clearly some sort of case where "more B-frames == better," according to the algorithm, and it capped at 16.

This could be, for example, at a completely static image.

I suppose that may be that this feature starts with a relatively long sequence of non-scrolling titles where the frames are almost entirely black...

[vmrsss]

Of course you can use a "sequence of 0 B-frames". Sometimes using only P-frames might be cheaper...

yes, I understand that; but it wouldn't appear in the stats, as then you could count as many sequences of zero B-frames as you want between any two consecutive P-frames.

Also, do not forget that the stats does not give you the number of sequences of length k, but the number of **frames** in those sequences. And there are zero B-frames in sequences of length 0... (This is based on glimpsing at encoder.c)

[vmrsss]

I doubt that the "shortest way" algorithm can check 16 b-frames without having checked all the intermediate numbers as well...

that's right, it's dynamic programming.

[akupenguin]

Also, do not forget that the stats does not give you the number of sequences of length k, but the number of **frames** in those sequences. And there are zero B-frames in sequences of length 0... (This is based on glimpsing at encoder.c)
It counts the number P+B frames in a sequence of some Bs followed by one P.
I doubt that the "shortest way" algorithm can check 16 b-frames without having checked all the intermediate numbers as well...
Sure it can. We can lop out whatever edges we want from the graph. If I thought it was useful, I could test only 0 and 16 B-frames.
When/if we optimize pre-motion-search to merge the efforts of searching the same ref list in different versions of the frame, *then* it won't much cpu-time to ignore intermediate numbers. But that's independent of the dynamic programming part of the algorithm, and is rather part of the computation of the cost metric.

[skystrife]

Maybe I'm just doing it horribly wrong, but I'm consistently getting crashes when testing with --b-adapt 2.

r920 + psyRDO + hrd + progress indication.

Source: one of my fprofile samples, a 1080p sample of a crowd running (200 frames).

Command:

x264.920.modified.02.exe --pass 1 --bitrate 10000 --stats test.stats --ref 16 --mixed-refs --bframes 16 --b-adapt 2 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,1 --subme 7 --trellis 2 --partitions all --8x8dct --me umh --merange 24 --aq-strength 1.2 --threads 6 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL crowd.avs

Crashes at 65%-ish. It hangs there for a while at 25% cpu usage, and then suddenly quits when it moves on the next frame.

If I run the same command with --b-adapt 1, it completes successfully: no crash, no hanging.

Not quite sure what's going on here... maybe it's just not happy with one of the other patches...

[Dark Shikari]

It could also be a bug in the algorithm. Its certainly not bug-free, I would think, though it should be relatively easy to debug given the fact that it uses strings to represent paths.

[wyti]

skystrife Can you upload your build with psyRDO hrd etc....
I really want to test this patch, but now i do all my encodes with psy-RDO, and it will be easier for me to compare with a patched version.
I know it's unstable, but for testing, it's a good thing no ?

[skystrife]

skystrife Can you upload your build with psyRDO hrd etc....
I really want to test this patch, but now i do all my encodes with psy-RDO, and it will be easier for me to compare with a patched version.
I know it's unstable, but for testing, it's a good thing no ?

Sure.

Here you go (http://forum.doom9.org/showthread.php?p=1163720#post1163720).

[LoRd_MuldeR]

What am I doing wrong? :confused:

LoRd_MuldeR@MULDER_NEU /c/downloads/x264_git
$ git clone git://git.videolan.org/x264.git
remote: Generating pack...
remote: Done counting 6479 objects.
remote: Deltifying 6479 objects...
remote: 100% (6479/6479) done
remote: Total 6479 (delta 5149), reused 168 (delta 123)
Receiving objects: 100% (6479/6479), 1.60 MiB | 237 KiB/s, done.
Resolving deltas: 100% (5149/5149), done.
Initialized empty Git repository in c:/downloads/x264_git/x264/.git/
Initialize x264/.git

LoRd_MuldeR@MULDER_NEU /c/downloads/x264_git
$ cd x264

LoRd_MuldeR@MULDER_NEU /c/downloads/x264_git/x264
$ patch -p1 < ../b-frames.diff
patch unexpectedly ends in middle of line
patch: **** Only garbage was found in the patch input.

LoRd_MuldeR@MULDER_NEU /c/downloads/x264_git/x264
$ patch -p1 < ../x264-psyrd-0.5.diff
patching file `common/common.c'
patching file `common/common.h'
patching file `encoder/analyse.c'
patching file `encoder/encoder.c'
patching file `encoder/rdo.c'
patching file `x264.c'
patching file `x264.h'

LoRd_MuldeR@MULDER_NEU /c/downloads/x264_git/x264
$

[Conquerist]

So I tested the build posted here (http://forum.doom9.org/showthread.php?p=1163149#post1163149), and found some peculiar results. I encoded a ~10 minute animation clip (the short film Kigeki). Even though it's animated, there's lots of artificial grain and dust to make it look like old film.

--b-adapt 2 gave me a lower PSNR and for some reason seemed to favor 15 consecutive B-frames. Because of the grain, this might be the "extreme case" like 300.

Here is the command line:
x264.920.new.bframes.decision.01.exe --pass 2 --bitrate 2000 --stats ".stats" --b-adapt 1/2 --ref 5 --mixed-refs --bframes 16 --b-pyramid
--b-rdo --bime --weightb --direct auto --subme 6 --trellis 1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input
--progress --output "badapt1/2.mp4" "kigeki.avs"and here's the logs--b-adapt 1
x264 [info]: consecutive B-frames: 21.4% 23.8% 10.1% 12.1% 13.5% 14.1% 0.4% 0.3% 0.0% 0.2% 0.1% 0.0% 0.1% 0.3% 0.0% 0.0% 3.7%
x264 [info]: PSNR Mean Y:49.011 U:53.972 V:53.690 Avg:49.835 Global:48.842 kb/s:2002.42

--b-adapt 2:
x264 [info]: consecutive B-frames: 2.6% 7.6% 5.2% 32.8% 4.7% 2.7% 1.2% 1.0% 0.4% 1.1% 0.5% 0.8% 1.5% 1.4% 0.0% 33.1% 3.3%
x264 [info]: PSNR Mean Y:48.752 U:53.800 V:53.521 Avg:49.619 Global:48.496 kb/s:2001.27

A 1000 frame low-motion clip from 300 never used more than 3 consecutive B-frames with the old method, while --b-adapt 2 used 15 consecutive B-frames 63.0% (but strangely never 16 consecutive ones) of the time, so this is probably the same "extreme case".

Edit: I already may have found a rather nasty bug in it, so don't complain just yetIs that bug related to this?

[Dark Shikari]

1. Here's the working patch (not changed since previously posted) (http://pastebin.com/f7f9b7323).
2. I'm working on the issue described above ;)

[LoRd_MuldeR]

Yup, that one does work :)

Here is the result from a live music ("rock am ring" festival) video clip:
x264 [info]: consecutive B-frames: 0,3% 9,7% 18,7% 26,6% 5,7% 6,2% 0,4% 0,1% 0,3% 0,1% 0,5% 1,5% 2,0% 1,6% 0,0% 26,2%

And yes, it slow: The CPU usage on my Quadcore dropped from ~99% to 45-75% ...

[Sagittaire]

Little test on HPII trailer:


|--------------|---------|---------|---------|---------|---------|
| Codec | PProc | Bitrate | Size | OPSNR | SSIM 2 |
|--------------|---------|---------|---------|---------|---------|
| DivX6 ASP | PP4 | 446 | 6757 | 39.85 | 65.02 |
| XviD ASP | PP4 | 446 | 6756 | 39.66 | 64.33 |
| LAVC ASP | PP4 | 447 | 6775 | 39.71 | 63.82 |
| Ateme AVC | PP0 | 446 | 6746 | 41.21 | 72.10 |
| x264 AVC | PP0 | 447 | 6760 | 41.50 | 73.07 |
| Elecard AVC | PP0 | 446 | 6745 | 41.22 | 71.89 |
| VP7 | PP2 | 446 | 6760 | 40.84 | 70.05 |
| VC1 | PP1 | 446 | 6746 | 40.27 | 66.87 |
| RV10 | HF2 | 446 | 6743 | 39.84 | 64.60 |
| DivX3 | PP4 | 447 | 6771 | 38.70 | 58.56 |
|--------------|---------|---------|---------|---------|---------|
| DivX6 ASP | PP4 | 896 | 13493 | 42.78 | 77.99 |
| XviD ASP | PP4 | 896 | 13489 | 42.66 | 77.96 |
| LAVC ASP | PP4 | 897 | 13528 | 42.77 | 77.77 |
| Ateme AVC | PP0 | 896 | 13495 | 43.75 | 81.59 |
| x264 AVC | PP0 | 896 | 13509 | 44.10 | 82.44 |
| Elecard AVC | PP0 | 896 | 13481 | 43.71 | 81.13 |
| VP7 | PP2 | 897 | 13534 | 43.34 | 80.02 |
| VC1 | PP1 | 896 | 13510 | 43.06 | 78.75 |
| RV10 | HF2 | 896 | 13493 | 42.77 | 77.72 |
| DivX3 | PP4 | 896 | 13508 | 41.84 | 74.40 |
|--------------|---------|---------|---------|---------|---------|
SSIM 0: Lumimask Off
SSIM 1: Lumimask On (Original Lumimask)
SSIM 2: Lumimask On (One2Tech Patch)

[LoRd_MuldeR]

Sample encoded at CRF=22

Method Max. B-Frames Size

Fast 16 4.631.020
Trellis 3 4.417.454
Trellis 5 4.303.896
Trellis 8 4.153.670
Trellis 10 4.049.532
Trellis 12 3.951.452
Trellis 16 3.797.840

Here is my sample:
http://mplayer.somestuff.org/misc/compare.7z

[Soichiro]

Sample encoded at CRF=22

Method Max. B-Frames Size

Fast 16 4.631.020
Trellis 3 4.417.454
Trellis 5 4.303.896
Trellis 8 4.153.670
Trellis 10 4.049.532
Trellis 12 3.951.452
Trellis 16 3.797.840

Interesting results. It seems that this patch will have a much greater benefit at lower bitrates then, judging from these stats.

Though I'm surprised that even 3 bframes with the new patch beat out 16 bframes with the old version. I assume this was a live action source and not animated, correct? The old version rarely used more than 3 bframes in a row with live action, while the new version seems to favor bframes in almost every circumstances (though that may be a bug, judging from multiple people getting huge numbers of 15-bframe sequences).

[burfadel]

A significant improvement even at 3 :) interesting that the improvement is consistent right up to 16, did the output quality remain the same? I know PSNR etc isn't exact, but was wondering whether at 16 the PSNR and SSIM were lower than that at say 8, just out of interest (owing that the filesize is smaller) :). If it is truly as good as the above table shows, hopefully DS can do some of his magic and reduce some of that speed loss!

[LoRd_MuldeR]

There is a visual difference, but I cannot say which one looks better yet. I'll upload a sample...

[skystrife]

Just an update on my issue. I just tested using bob0r's build to verify that it wasn't a confliction with the other patches, and it's not. However, it still crashed on me in the same spot as my build did. It appears to be an issue outside of x264, though, as adding SetMemoryMax(128) to the top of the script fixed the crash.

Though it's odd that a script consisting of only imagesource() and converttoyv12() would manage to make x264 hit its max address space...

Either way, it's probably related to avisynth on my end, so I would probably say that the patch itself is working.

EDIT: with my plugins folder empty it still crashes, so it's not related to autoloading... odd.

[LoRd_MuldeR]

Okay, I have added my sample to the previous post. Frame-by-frame comparison "Fast" vs. "Trellis" B-Frame decision method, using max. 16 B-Frames.

*click* (http://forum.doom9.org/showpost.php?p=1163990&postcount=73)

[Sharktooth]

h264 in avi...... aaaaargh....
the new b-frames decision loks better to me, less blocky backgrounds and slighrly sharper image (edit: after several views, i actually take the latter back).

[LoRd_MuldeR]

h264 in avi...... aaaaargh....

That is FFv1 in AVI :p

I decoded both encodes with FFmpegSource(), interleaved them in Avisynth and encoded everything with VirtualDub/ffdshow...

[Sharktooth]

oh... my bad. i thought you used lossless h.264 for the final encode.

[vmrsss]

It counts the number P+B frames in a sequence of some Bs followed by one P.

OK, thanks. Let's me test if I got it now:


x264 [info]: consecutive B-frames: 63.3% 4.1% 6.9% 6.9% 13.3% 3.9% 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 1.5%


(1) the percentage refers to the total number of P+B frames in the encode

(2) the first number counts the number of PP sequences (0 B-frames), so it says that 63.3% of P+B frames occur as the second P in PP sequences

(3) the second number says that 4.1% of P+B frames occur (as the 2nd and 3rd element) in sequences PBP

(4) and so on till the 16th number.

Would it not be interesting --if possible-- to report how often a sequence BB... was terminated for having hit the --bframes limit? As a measure of how useful it may be to increase the --bframes parameter...

[akupenguin]

Would it not be interesting --if possible-- to report how often a sequence BB... was terminated for having hit the --bframes limit? As a measure of how useful it may be to increase the --bframes parameter...
The only way to know that is to run the algorithm for N+1 bframes, at which point the cpu-time is wasted if you don't actually use N+1 when it's chosen.

[skystrife]

Ok, I've done some more testing and I'm back to believing there's something in the patch. (For reference, I'm running a machine with 8GB of memory, so I think it has something to do with x264 running itself out of address space?)

http://forum.doom9.org/showthread.php?p=1158428#post1158428

I grabbed the referenced "sneak peak" and verified that it does in fact clamp to 1GB of memory usage (went on a video seeking spree in AvsP and watch the task manager).

So, I figured that this would solve my crashing error in the new patched x264. Well... not so much. x264 crashes (just your standard "x264 has stopped working," so I don't get any helpful info) after hitting a little over 1.3GB of memory.

I'm out of ideas, and it's weird that I'm the only one reporting this error.

Running the same command line (except with --b-adapt 1) with the same exact build completes perfectly fine, with memory usage peaking at about 1.35GB... more than what the --b-adapt 2 pass crashes at.

o_O

[check]

What is your input? Are you using a multithreaded avisynth modification?

[woah!]

i ran a full encode from 1080p POtC bluray to a 1280x720p bluray compatable dvd+rdl with these settings exactly the same for 1st and 2nd pass...

--b-adapt 2 --deadzone-inter 6 --deadzone-intra 6 -t 0 --aq-strength 1.0 -f -4:-3 --level 4.1 --keyint 24 --min-keyint 1 --ref 3 --mixed-refs --bframes 3 --b-rdo --bime --weightb --direct auto --subme 6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 30000 --vbv-maxrate 38000 --qcomp 0.5 --me umh --threads auto --thread-input --progress --no-psnr --no-ssim --mvrange 511 --aud --nal-hrd --sar 1:1


seems 1st pass was 11.67fps and 2nd pass was 16.3fps. but the result was very good for an avg bitrate of 6600.. no crashes on 206070 frames encoded...

[Sharktooth]

that's quite normal. D_S already said the b-frames decisions are made in the first pass.

[Comatose]

I don't think he was complaining, just providing input :P

[skystrife]

What is your input? Are you using a multithreaded avisynth modification?

Input is:


imagesource("pathtoimages\%05d.png", start=07410, end=07609)
converttoyv12()

No, I am not using any MT modification. I am NOW using AviSynth 2.58, but the same thing happens in AviSynth 2.57 (and earlier in this version too, interestingly enough).

Does anyone else have an absurdly large amount of ram (>= 4GB) in a machine that can test this? I think that is the cause of this, but I suppose I could be wrong. *shrug*

[burfadel]

Well running x264 from www.x264.tk and from skystrife there seems to be random crashes, although not frequent. This is the first build that has done that from my knowledge. I went away on 12 July, and at that time x264 seemed to be working fine! Its hard to test for too, its very random, and is not frequent. Encoding a lot of stuff that was on tv from when I was way, it happens only once every few shows or movies (it could go 10 hours without crashing or crash in 5 hours). If you run it again it encodes through the show where it crashed before.

Here are the options used:

--crf 24 --keyint 450 --ref 5 --mixed-refs --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --partitions all --8x8dct --trellis 2 --me umh --threads 4 --progress --thread-input --no-psnr --no-ssim

I'd like to point out that I am not using the b-frames patch, just the psy-rdo patch from the above mentioned builds. I though I'd mention it here since others are having crashes, which may be unrelated to the b-frames patch itself! With the above options, the RAM usage at 640x352 settles off at 1.5GB, which seems higher than previous x264 builds.

I'm running 4GB of RAM on Vista x64.

[Manao]

LoRd_Mulder : by your test methodology, using -b 16 --b-adapt 0 would yield even better results. Filesize alone means nothing

[Dark Shikari]

... and I found the bug that was causing long strings of 15/16 B-frames where there shouldn't be.

Fix will come up in a bit.

(Turned out the bug was in akupenguin's code) :p

[LoRd_MuldeR]

LoRd_Mulder : by your test methodology, using -b 16 --b-adapt 0 would yield even better results. Filesize alone means nothing

Sure. But a significant smaller file that looks at least as good as the reference (maybe even better) does mean something :)

[Sharktooth]

... and I found the bug that was causing long strings of 15/16 B-frames where there shouldn't be.

Fix will come up in a bit.

(Turned out the bug was in akupenguin's code) :p
now, it's only a matter of speed.
if you find a way to optimize the algo speed, then it can be used as default.

[akupenguin]

Sure. But a significant smaller file that looks at least as good as the reference (maybe even better) does mean something :)
No, really it doesn't. How good is your intuition of what a 5% bitrate difference is supposed to look like? If it's only "maybe" better, then do you know that it would have been out of he realm of maybe and beyond your error bounds if they had been encoded at the same bitrate?

There are only two valid comparisons: same bitrate and compare quality, or same quality and compare bitrate. And the latter only works with objective quality metrics, since psy quality isn't precise enough.
You can get lucky and find a case where encode B has both lower bitrate and higher quality than encode A, but then you can't quantify the amount of gain, so please don't post any numbers.

[LoRd_MuldeR]

No, really it doesn't. How good is your intuition of what a 5% bitrate difference is supposed to look like?

Well, if I visually compare the two files and the smaller file actually looks better (or equal at least), then I think this is definitely a good thing :)

Also I didn't claim that those "numbers" mean anything. I tested the new method with CRF, because that's how I'm gonna use it. And I post my results for discussion.

20% smaller file at same CRF definitely is an interesting observation, whether it means anything for judging the new method or not...


Doing a 2-Pass encode now ;)

[LoRd_MuldeR]

Running a 2-Pass encode I got this at 90% of the second pass:

*********** EXCEPTION **************
Registers:
EAX: 00000004 EBX: 00000003 ECX: 7D61F5ED EDX: 01120001 ESI: 77BA2C95
EDI: 00000000 ESP: 00228948 EBP: 00228C88 EIP: 77BC6B3E EFlags: 00000246

Exception Code: EXCEPTION_BREAKPOINT (80000003)
Exception Flags: 00000000

Origin:
C:\WINDOWS\syswow64\msvcrt.dll(assert+0x26B3E) [0x77BC6B3E]
*********** EXCEPTION **************

*********** BACKTRACE **************
Frame 0: D:\Avidemux 2.4.3\avidemux2_gtk.exe(Z16exceptionHandlerP17_EXCEPTION_RECORDPvP8_CONTEXTS1_+0x12B90E) [0x0052B90E]
Frame 1: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EC2A) [0x7D61EC2A]
Frame 2: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EBFB) [0x7D61EBFB]
Frame 3: C:\WINDOWS\system32\ntdll.dll(KiUserExceptionDispatcher+0x1EA36) [0x7D61EA36]
Frame 4: D:\Avidemux 2.4.3\libx264.dll(x264_slicetype_analyse+0x8F58) [0x6FC88F58]
*********** BACKTRACE **************

Edit: Same settings but b_adapt=1 will run through just fine. May this be related to the mentioned bug?

[Dark Shikari]

New patch (http://pastebin.com/m2a557e16)

Changes:

1. Bugfixes
2. Remove pyramid-specific code, it didn't seem useful
3. Precalculate intra cost once per frame instead of per slicetype call; can increase or reduce speed for --b-adapt 1 (depending on which phase the moon is in) and always increases speed on --b-adapt 2.
4. Add --weightb code; use --weightb on your first pass now. In a quick test, this greatly improved x264's detection and reaction to fades.

[LoRd_MuldeR]

Thanks for the new patch, but I still get the same crash, using 2-Pass mode:
*********** EXCEPTION **************
Registers:
EAX: 00000004 EBX: 00000003 ECX: 7D61F5ED EDX: 01460001 ESI: 77BA2C95
EDI: 00000000 ESP: 0022A278 EBP: 0022A5B8 EIP: 77BC6B3E EFlags: 00000246

Exception Code: EXCEPTION_BREAKPOINT (80000003)
Exception Flags: 00000000

Origin:
C:\WINDOWS\syswow64\msvcrt.dll(assert+0x26B3E) [0x77BC6B3E]
*********** EXCEPTION **************

*********** BACKTRACE **************
Frame 0: d:\Avidemux 2.4.3\avidemux2_gtk.exe(Z16exceptionHandlerP17_EXCEPTION_RECORDPvP8_CONTEXTS1_+0x12B90E) [0x0052B90E]
Frame 1: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EC2A) [0x7D61EC2A]
Frame 2: C:\WINDOWS\system32\ntdll.dll(RtlRaiseStatus+0x1EBFB) [0x7D61EBFB]
Frame 3: C:\WINDOWS\system32\ntdll.dll(KiUserExceptionDispatcher+0x1EA36) [0x7D61EA36]
Frame 4: d:\Avidemux 2.4.3\libx264.dll(x264_slicetype_analyse+0x8EB8) [0x6FC88EB8]
*********** BACKTRACE **************

There seems to be no problem for CRF mode though...

[Dark Shikari]

That "exception" report is utterly useless; I can't derive any useful information from it. How about a gdb backtrace on a build with --enable-debug?

[LoRd_MuldeR]

I'll try to get a debug build. Nevertheless it seems the new version doesn't show the behavior to prefer 16 B-Frames anymore:
x264 [info]: consecutive B-frames: 11,1% 30,1% 50,6% 3,3% 1,2% 3,0% 0,3% 0,1% 0,1% 0,1% 0,1% 0,0% 0,0% 0,0% 0,0% 0,0%

Now it would make sens to lower "Max. Consecutive B-Frames" to ~6, right ???

[Dark Shikari]

It appears that this patch correctly detects fades!

Before and after (blue == inter, red/orange == intra, green == direct, yellow == skip)

Blue/red/orange P-frame is before, the B-frame with lots of skips is after:

http://i34.tinypic.com/2vjo2f8.gif

Combined with weighted prediction, this should drastically decrease the bit cost of fades.

[Warpman]

It appears that this patch correctly detects fades!

Before and after (blue == inter, red/orange == intra, green == direct, yellow == skip)

Blue/red/orange P-frame is before, the B-frame with lots of skips is after:



Combined with weighted prediction, this should drastically decrease the bit cost of fades.
nice work!
i just tried one small sample (anime/not much motion/many fades)
and with a 500kbit bitrate... results are great ^^


--[NoImage] Job commandline: "C:\Programme\megui\tools\x264\x264.exe" --pass 2 --bitrate 500 --stats "G:\Anime\(DVDISO)Kanon
Vol.2\ff2.stats" --ref 10 --mixed-refs --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --partitions
p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --sar 1:1 --progress --output "G:\Anime\(DVDISO)Kanon
Vol.2\nobadapt2_2.mkv" "G:\Anime\(DVDISO)Kanon Vol.2\ff2.avs"
--[Information] [29.07.2008 21:13:08] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 720x480 @ 23.98 fps (3001 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: slice I:32 Avg QP:15.34 size: 28103 PSNR Mean Y:50.16 U:50.59 V:50.92 Avg:50.34 Global:49.60
---[NoImage] x264 [info]: slice P:691 Avg QP:15.77 size: 7407 PSNR Mean Y:48.67 U:49.13 V:49.46 Avg:48.84 Global:48.23
---[NoImage] x264 [info]: slice B:2278 Avg QP:16.87 size: 812 PSNR Mean Y:47.88 U:48.67 V:49.08 Avg:48.15 Global:47.81
---[NoImage] x264 [info]: consecutive B-frames: 4.3% 7.3% 5.7% 14.4% 16.0% 20.4% 12.5% 1.6% 1.2% 0.7% 1.1% 0.4% 1.3% 0.5% 0.0% 0.0% 12.6%
---[NoImage] x264 [info]: mb I I16..4: 6.2% 58.8% 35.0%
---[NoImage] x264 [info]: mb P I16..4: 1.0% 6.1% 2.0% P16..4: 33.1% 14.7% 9.6% 0.0% 0.0% skip:33.5%
---[NoImage] x264 [info]: mb B I16..4: 0.0% 0.1% 0.0% B16..8: 18.5% 0.6% 0.7% direct: 0.9% skip:79.1% L0:46.9% L1:49.7% BI: 3.3%
---[NoImage] x264 [info]: 8x8 transform intra:64.2% inter:50.5%
---[NoImage] x264 [info]: direct mvs spatial:94.4% temporal:5.6%
---[NoImage] x264 [info]: ref P L0 56.0% 12.2% 8.4% 4.6% 4.1% 3.4% 3.6% 2.2% 2.7% 2.8%
---[NoImage] x264 [info]: ref B L0 73.3% 10.5% 5.2% 2.4% 2.5% 2.0% 1.6% 1.5% 1.0%
---[NoImage] x264 [info]: ref B L1 93.3% 6.7%
---[NoImage] x264 [info]: SSIM Mean Y:0.9943368
---[NoImage] x264 [info]: PSNR Mean Y:48.086 U:48.795 V:49.185 Avg:48.335 Global:47.920 kb/s:502.84
---[NoImage] encoded 3001 frames, 32.56 fps, 502.95 kb/s



--[NoImage] Job commandline: "C:\Programme\megui\tools\x264\x264.exe" --pass 2 --bitrate 500 --stats "G:\Anime\(DVDISO)Kanon
Vol.2\ff2.stats" --ref 10 --mixed-refs --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --partitions
p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --sar 1:1 --progress --output "G:\Anime\(DVDISO)Kanon
Vol.2\badapt2_2.mkv" "G:\Anime\(DVDISO)Kanon Vol.2\ff2.avs" --b-adapt 2
--[Information] [29.07.2008 21:07:51] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 720x480 @ 23.98 fps (3001 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
---[NoImage] x264 [info]: slice I:30 Avg QP:15.55 size: 27491 PSNR Mean Y:50.08 U:50.51 V:50.88 Avg:50.26 Global:49.49
---[NoImage] x264 [info]: slice P:536 Avg QP:15.54 size: 7155 PSNR Mean Y:48.42 U:49.03 V:49.42 Avg:48.64 Global:48.22
---[NoImage] x264 [info]: slice B:2435 Avg QP:16.64 size: 1318 PSNR Mean Y:48.12 U:48.84 V:49.23 Avg:48.38 Global:47.99
---[NoImage] x264 [info]: consecutive B-frames: 0.6% 4.4% 10.3% 13.5% 7.6% 24.2% 3.3% 1.9% 0.0% 0.3% 0.0% 0.4% 1.8% 2.4% 0.0% 15.1% 14.3%
---[NoImage] x264 [info]: mb I I16..4: 6.7% 59.7% 33.7%
---[NoImage] x264 [info]: mb P I16..4: 0.9% 5.9% 2.3% P16..4: 30.9% 12.4% 8.9% 0.0% 0.0% skip:38.7%
---[NoImage] x264 [info]: mb B I16..4: 0.1% 0.4% 0.1% B16..8: 20.4% 1.0% 1.2% direct: 1.5% skip:75.4% L0:50.8% L1:44.8% BI: 4.4%
---[NoImage] x264 [info]: 8x8 transform intra:62.8% inter:51.0%
---[NoImage] x264 [info]: direct mvs spatial:93.8% temporal:6.2%
---[NoImage] x264 [info]: ref P L0 54.1% 11.8% 8.2% 5.1% 4.6% 3.8% 3.3% 2.7% 3.2% 3.2%
---[NoImage] x264 [info]: ref B L0 72.2% 10.3% 5.0% 3.3% 2.5% 1.7% 2.1% 1.6% 1.3%
---[NoImage] x264 [info]: ref B L1 92.6% 7.4%
---[NoImage] x264 [info]: SSIM Mean Y:0.9944345
---[NoImage] x264 [info]: PSNR Mean Y:48.197 U:48.895 V:49.277 Avg:48.442 Global:48.041 kb/s:503.00
---[NoImage] encoded 3001 frames, 31.80 fps, 503.11 kb/s


more SSIM and PSNR...
differences are really small but my eyes agree with the metrics here...

[Soichiro]

If the threading could be fixed with the new patch, this could be entirely usable. However, the test I'm running with the most recent version of the patch is only using 50% of the CPU, and going very slow, so until the threading issue is fixed, this isn't a very reasonable patch for everyday use.

I'll post stats/links when the encodes finish.

[Dark Shikari]

If the threading could be fixed with the new patch, this could be entirely usable. However, the test I'm running with the most recent version of the patch is only using 50% of the CPU, and going very slow, so until the threading issue is fixed, this isn't a very reasonable patch for everyday use.

I'll post stats/links when the encodes finish.I'm going to work on heuristics to prune the path tree of unlikely paths--this could drastically improve speed without much quality loss (and the faster slicetype decision is, the less of a threading issue there is).

It also probably helps a lot to use --bframes 2 or 3 instead of 16 ;)

[Warpman]

I'm going to work on heuristics to prune the path tree of unlikely paths--this could drastically improve speed without much quality loss (and the faster slicetype decision is, the less of a threading issue there is).

It also probably helps a lot to use --bframes 2 or 3 instead of 16 ;)
well i still get 15fps with --bframes 16 in the first pass so its stil fast enough(tm) for me ;)
2nd pass runs with 40fps tho xD

[Soichiro]

-[Information] Log for job1 (video, Pot 04.avs -> Pot 04 Old Bframes.mkv)
--[Information] [7/29/2008 6:14:59 PM] Started handling job
--[Information] [7/29/2008 6:14:59 PM] Preprocessing
--[NoImage] Job commandline: "C:\Documents and Settings\Soichiro\Desktop\x264.r920.bframepatch.exe" --bitrate 700 --keyint 400 --min-keyint 12 --ref 6 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 24 --threads auto --thread-input --sar 1:1 --progress --output "S:\Prince of Dumbness\04\Pot 04 Old Bframes.mkv" "S:\Prince of Dumbness\04\Pot 04.avs" --b-adapt 1
--[Information] [7/29/2008 6:15:01 PM] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 640x480 @ 23.98 fps (3001 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Slow
---[NoImage] x264 [info]: slice I:20 Avg QP:17.55 size: 22405 PSNR Mean Y:49.86 U:51.03 V:51.57 Avg:48.99 Global:43.09
---[NoImage] x264 [info]: slice P:1438 Avg QP:23.26 size: 6879 PSNR Mean Y:41.01 U:44.26 V:44.42 Avg:41.74 Global:38.67
---[NoImage] x264 [info]: slice B:1543 Avg QP:21.86 size: 1486 PSNR Mean Y:43.15 U:45.80 V:45.78 Avg:43.77 Global:40.01
---[NoImage] x264 [info]: consecutive B-frames: 22.5% 32.7% 10.4% 8.3% 13.1% 1.8% 1.6% 0.0% 0.9% 0.0% 1.1% 0.4% 0.4% 0.0% 1.0% 0.0% 5.7%
---[NoImage] x264 [info]: mb I I16..4: 24.8% 44.2% 31.0%
---[NoImage] x264 [info]: mb P I16..4: 4.8% 10.9% 2.3% P16..4: 39.7% 11.7% 8.5% 0.0% 0.0% skip:22.0%
---[NoImage] x264 [info]: mb B I16..4: 0.2% 0.2% 0.0% B16..8: 27.7% 1.0% 1.5% direct: 4.1% skip:65.2% L0:38.9% L1:52.0% BI: 9.2%
---[NoImage] x264 [info]: final ratefactor: 23.01
---[NoImage] x264 [info]: 8x8 transform intra:59.3% inter:57.8%
---[NoImage] x264 [info]: direct mvs spatial:96.4% temporal:3.6%
---[NoImage] x264 [info]: ref P L0 71.4% 12.2% 7.7% 3.1% 2.8% 2.9%
---[NoImage] x264 [info]: ref B L0 83.0% 9.7% 3.8% 2.0% 1.4%
---[NoImage] x264 [info]: ref B L1 95.2% 4.8%
---[NoImage] x264 [info]: SSIM Mean Y:0.9872354
---[NoImage] x264 [info]: PSNR Mean Y:42.167 U:45.097 V:45.166 Avg:42.831 Global:39.330 kb/s:807.44
---[NoImage] encoded 3001 frames, 5.01 fps, 807.52 kb/s
-[Information] Log for job3 (video, Pot 04.avs -> Pot 04 New Bframes.mkv)
--[Information] [7/29/2008 6:25:03 PM] Started handling job
--[Information] [7/29/2008 6:25:03 PM] Preprocessing
--[NoImage] Job commandline: "C:\Documents and Settings\Soichiro\Desktop\x264.r920.bframepatch.exe" --bitrate 700 --keyint 400 --min-keyint 12 --ref 6 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 24 --threads auto --thread-input --sar 1:1 --progress --output "S:\Prince of Dumbness\04\Pot 04 New Bframes.mkv" "S:\Prince of Dumbness\04\Pot 04.avs" --b-adapt 2
--[Information] [7/29/2008 6:25:05 PM] Encoding started
--[NoImage] Standard output stream
--[NoImage] Standard error stream
---[NoImage] avis [info]: 640x480 @ 23.98 fps (3001 frames)
---[NoImage] x264 [info]: using SAR=1/1
---[NoImage] x264 [info]: using cpu capabilities: MMX2 SSE2Slow
---[NoImage] x264 [info]: slice I:20 Avg QP:17.24 size: 23237 PSNR Mean Y:50.16 U:51.27 V:51.79 Avg:49.28 Global:43.40
---[NoImage] x264 [info]: slice P:1377 Avg QP:20.94 size: 6525 PSNR Mean Y:42.58 U:45.38 V:45.44 Avg:43.22 Global:39.87
---[NoImage] x264 [info]: slice B:1604 Avg QP:23.15 size: 1996 PSNR Mean Y:42.20 U:45.14 V:45.23 Avg:42.88 Global:39.55
---[NoImage] x264 [info]: consecutive B-frames: 15.0% 36.6% 19.7% 19.6% 2.5% 1.4% 3.1% 1.1% 0.3% 0.3% 0.4% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
---[NoImage] x264 [info]: mb I I16..4: 23.7% 44.7% 31.7%
---[NoImage] x264 [info]: mb P I16..4: 3.8% 9.4% 2.2% P16..4: 35.8% 10.6% 8.2% 0.0% 0.0% skip:29.9%
---[NoImage] x264 [info]: mb B I16..4: 0.6% 0.9% 0.1% B16..8: 30.6% 1.4% 1.9% direct: 6.4% skip:58.1% L0:39.1% L1:51.1% BI: 9.8%
---[NoImage] x264 [info]: final ratefactor: 22.47
---[NoImage] x264 [info]: 8x8 transform intra:59.0% inter:57.5%
---[NoImage] x264 [info]: direct mvs spatial:96.0% temporal:4.0%
---[NoImage] x264 [info]: ref P L0 70.4% 12.5% 8.0% 3.3% 2.8% 2.9%
---[NoImage] x264 [info]: ref B L0 81.7% 9.6% 4.8% 2.2% 1.6%
---[NoImage] x264 [info]: ref B L1 95.4% 4.6%
---[NoImage] x264 [info]: SSIM Mean Y:0.9879179
---[NoImage] x264 [info]: PSNR Mean Y:42.427 U:45.291 V:45.369 Avg:43.083 Global:39.710 kb/s:808.61
---[NoImage] encoded 3001 frames, 1.68 fps, 808.70 kb/s

Top is old, bottom is new method. SSIM is much better on the new method, but wow, that's so much slower that I don't even know if it's worth it.

[LoRd_MuldeR]

Looking at your B-Frames stats, more than 6 B-Frames are used only in very rare cases.
So restricting the max number of B-Frames to 6 should give some good speed-up with the new method...

[Dark Shikari]

Top is old, bottom is new method. SSIM is much better on the new method, but wow, that's so much slower that I don't even know if it's worth it.Did you even read any of the posts in this thread... like the ones saying "don't use --bframes 16 with the new method if you're going to whine about its speed"? :p

[Soichiro]

Maybe I was just running a test. Sorry for whining about a slow test run. Geez. :p

I'm kind of surprised the patched version didn't use 16 bframes though, since the original did in some places (since this is animation with some static scenes).

[Audionut]

I'm kind of surprised the patched version didn't use 16 bframes though, since the original did in some places.

... and I found the bug that was causing long strings of 15/16 B-frames where there shouldn't be.

http://forum.doom9.org/showthread.php?p=1164133#post1164133

[bkman]

Builds with the latest patch?

Much obliged :)

[Sharktooth]

head over the x264 current patches thread.

[LoRd_MuldeR]

That "exception" report is utterly useless; I can't derive any useful information from it. How about a gdb backtrace on a build with --enable-debug?

Here we go:

LoRd_MuldeR@MULDER_NEU /d/Avidemux 2.4.3
$ gdb avidemux2_gtk.exe
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32"...
(gdb) run

[..]

Program received signal SIGSEGV, Segmentation fault.
[Switching to thread 3096.0x528]
0x6fc9bcbd in block_residual_write_cabac (h=0x6526170, cb=0x177cd3b0,
i_ctxBlockCat=4, i_idx=21, l=0x6527ff2, i_count=15) at ./common/bs.h:228
228 return x264_ue_size_tab[(val+1)>>8] + 16;
(gdb) bt
#0 0x6fc9bcbd in block_residual_write_cabac (h=0x6526170, cb=0x177cd3b0,
i_ctxBlockCat=4, i_idx=21, l=0x6527ff2, i_count=15) at ./common/bs.h:228
#1 0x6fcae380 in x264_partition_size_cabac (h=0x6526170, cb=0x177cd3b0,
i8=<value optimized out>, i_pixel=3) at encoder/cabac.c:1089
#2 0x6fcae8ca in x264_rd_cost_part (h=0x6526170, i_lambda2=1462, i8=1,
i_pixel=3) at encoder/rdo.c:191
#3 0x6fcb95f6 in x264_me_refine_qpel_rd (h=0x6526170, m=0x177ce210,
i_lambda2=1462, i8=1) at encoder/me.c:955
#4 0x6fcb1ba1 in x264_macroblock_analyse (h=0x6526170)
at encoder/analyse.c:2370
#5 0x6fcd44e1 in x264_slice_write (h=0x6526170) at encoder/encoder.c:1160
#6 0x6fced884 in x264_stack_align () from d:\Avidemux 2.4.3\libx264.dll
#7 0x6fcd2afa in x264_slices_write (h=0x6526170) at encoder/encoder.c:1305
#8 0x611812fa in ptw32_threadStart@4 () from d:\Avidemux 2.4.3\pthreadGC2.dll
#9 0x77bcb530 in msvcrt!_endthreadex () from C:\WINDOWS\syswow64\msvcrt.dll
#10 0x7d4dfe21 in KERNEL32!GetConsoleOutputCP ()
from C:\WINDOWS\syswow64\kernel32.dll
#11 0x1177b358 in ?? ()
#12 0x00000000 in ?? ()
(gdb)


Are those the information you need? :confused:

[gav1577]

Sorry fo the noob question but is the --b-adapt 2 cmd only needed in the first pass of a 2 pass encode the reason i ask is because i read somewhere if i remember correctly all the b frame calculations are made in the first pass ? Thanks

[Dark Shikari]

That's... odd. That shouldn't happen.

Place assert( i_coeff > 0 ); on line 730 of encoder/cabac.c and see if the assert triggers.

Sorry fo the noob question but is the --b-adapt 2 cmd only needed in the first pass of a 2 pass encode the reason i ask is because i read somewhere if i remember correctly all the b frame calculations are made in the first pass ? Thankscorrect

[gav1577]

correct

Thanks :)

[LoRd_MuldeR]

That's... odd. That shouldn't happen.

Place assert( i_coeff > 0 ); on line 730 of encoder/cabac.c and see if the assert triggers.

correct

Okay. That's what I got during the compilation process. I used fprofiled.

avis [info]: 1280x720 @ 59.94 fps (200 frames)
x264 [warning]: psy RD is not compatible with trellis=1; use 0 or 2.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
Assertion failed: i_coeff > 0, file encoder/cabac.c, line 731

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

Well, it compiled anyway. But when encoding, I now see this:

http://img410.imageshack.us/img410/6701/gdbcrash3el0.th.png (http://img410.imageshack.us/my.php?image=gdbcrash3el0.png)

[Dark Shikari]

Hmm, this is very bizarre. Its crashing on a set of code that does have a CBP--the CBP check should ensure that a zero-set of coefficients never reaches the writing loop. I don't get a crash here either.

Do you have a sample that consistently generates the issue, and/or a set of encoding settings?

[LoRd_MuldeR]

Do you have a sample that consistently generates the issue, and/or a set of encoding settings?

Well, I run Avidemux with the newly compiled libx264.dll, load some MPEG-2 capture, load my usual x264 settings (from .js script) and save the video.

It will then hang for quite some time, which it didn't use to do, and then *boom*


[EDIT]

You can get Avidemux r4281 here: http://avidemux.razorbyte.com.au/#avidemux2.4
My x264 settings are here: http://pastebin.org/58864

Note: You'll need the Psy RDO patch or you must insert dummy fields for "i_psy_rd" and "f_psy_rd".
And if you want to use "b_adapt=2" with Avidmux, you'll need to hack the x264 defaults...

[Dark Shikari]

I need something that I can replicate with standalone x264, so x264 CLI settings would be more useful than a hexdump ;)

[LoRd_MuldeR]

I need something that I can replicate with standalone x264, so x264 CLI settings would be more useful than a hexdump ;)

Extracted from the file that crashed:
x264 - core 61 r0+922 4e7f5cb - H.264/MPEG-4 AVC codec - Copyleft 2003-2008 - http://www.videolan.org/x264.html - options: cabac=1 ref=8 deblock=1:0:0 analyse=0x3:0x133 me=umh subme=7 psy_rd=1,000000 brdo=1 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0 deadzone=21,11 chroma_qp_offset=0 threads=6 nr=0 decimate=1 mbaff=0 bframes=15 b_pyramid=1 b_adapt=2 b_bias=0 direct=1 wpredb=1 bime=1 keyint=500 keyint_min=25 scenecut=40(pre) rc=crf crf=24,0 rceq='blurCplx^(1-qComp)' qcomp=1,00 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1,40 pb_ratio=1,30 aq=2:1,00

[LoRd_MuldeR]

http://www.soadfans.de/forum/images/armageddon/smilies/wand.gif

After re-compiling my libx264.dll for about 30 times today, I finally got a working build. No idea what I did wrong all the time :confused:

Sorry for bothering you, but I must conclude the the problem was on my side :o

[LoRd_MuldeR]

Back to the 2-Pass crash that happens exclusively when b_adapt=2 is used:

LoRd_MuldeR@MULDER_NEU /d/Avidemux 2.4.3
$ gdb avidemux2_gtk.exe
GNU gdb 6.8
Copyright (C) 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-pc-mingw32"...
(no debugging symbols found)
(gdb) run
Starting program: d:\Avidemux 2.4.3/avidemux2_gtk.exe
[New thread 4144.0x12f0]
[New thread 4144.0x1380]
[New thread 4144.0x62c]
[New thread 4144.0xdfc]
[New thread 4144.0xe9c]
[New thread 4144.0x1114]
[New thread 4144.0xdec]
[New thread 4144.0xf04]
[New thread 4144.0x520]

[....]

[New thread 4144.0x13f4]
[New thread 4144.0x1290]
[New thread 4144.0xe2c]
[New thread 4144.0x13c4]
[New thread 4144.0x1098]

Program received signal SIGTRAP, Trace/breakpoint trap.
0x77bc6b3f in msvcrt!_assert () from C:\WINDOWS\syswow64\msvcrt.dll
(gdb) bt
#0 0x77bc6b3f in msvcrt!_assert () from C:\WINDOWS\syswow64\msvcrt.dll
#1 0x6fc9fc55 in x264_slicetype_analyse (h=0x7ed9290)
at encoder/slicetype.c:477
Backtrace stopped: frame did not save the PC
(gdb)

It always crashes near to the end of the second pass. When I lower b_adapt=1 then all runs through fine...

[Dark Shikari]

????

Slicetype_analyse isn't run on the second pass.

[LoRd_MuldeR]

????

Slicetype_analyse isn't run on the second pass.

Well, that is what I see. There must be some reason. I can reproduce it, with shorter and longer samples. Any ideas? :scared:

[Dark Shikari]

if( h->param.rc.b_stat_read )
{
/* Use the frame types from the first pass */
for( i = 0; h->frames.next[i] != NULL; i++ )
h->frames.next[i]->i_type =
x264_ratecontrol_slice_type( h, h->frames.next[i]->i_frame );
}
else if( (h->param.i_bframe && h->param.i_bframe_adaptive)
|| h->param.b_pre_scenecut )
x264_slicetype_analyse( h );

It is physically impossible for slicetype_analyse to be run if b_stat_read is set. b_stat_read is set, GUARANTEED, if pass is 2 or 3. Might I suggest you make sure that you're actually running a second pass?

[LoRd_MuldeR]

Just for info: x264 r927 creates a working DLL with MingW GCC 4.3.1-tdm-1 and "fprofiled" enabled again. That didn't work with r922 and caused a lot of headache...

[Ranguvar]

4. Add --weightb code; use --weightb on your first pass now. In a quick test, this greatly improved x264's detection and reaction to fades.
IIRC, you previously advised against using --weightb (second pass) with anime/cartoons. Does this mean we should enable --weightb on the first pass, but not the second? Or both passes now, or do I just have a bad memory?

Thanks for the great work!

[Dark Shikari]

IIRC, you previously advised against using --weightb (second pass) with anime/cartoons. Does this mean we should enable --weightb on the first pass, but not the second? Or both passes now, or do I just have a bad memory?

Thanks for the great work!Weightb isn't recommended too much for cartoons, at least if there isn't significant fading. The reason is because it slightly reduces the effectiveness of multiref.

However, if there is any significant amount of fades, its probably worth using it either way.

I didn't recommend it before because x264 didn't properly detect fades.

[Soichiro]

http://forum.doom9.org/showthread.php?p=1164133#post1164133

I know there was a bug, and I'm not talking about the old version of the patch, I'm talking about the non-trellis bframe method giving strings of 16 bframes in still scenes (where it makes sense to have them) and the new version not having any. Maybe there's some fancy technical reason, but it seems odd IMO.

[akupenguin]

I know there was a bug, and I'm not talking about the old version of the patch, I'm talking about the non-trellis bframe method giving strings of 16 bframes in still scenes (where it makes sense to have them) and the new version not having any.
And what makes you think the scene actually deserved 16 B-frames?
In a perfectly still frame it doesn't matter what frame types you pick, and in a non-perfectly still frame there is going to be some optimal number short of infinity. Why would the old simple algorithm have got it right?

[Comatose]

Well, logically, in a still scene where the only difference between frames is noise, wouldn't it be most efficient to look both ways (B)?
Assume the scene lasts 2 seconds at 24 fps.

[TL0]

For the options --b-adapt 2 --weightb, does it matter if these options are enabled for both passes? or are they only needed for first pass and keeping them enabled for 2nd pass causes slowdown in encoding?

I read the other replies but am still not sure what is correct currently. This new b-frame decision gives nice bitrate saving for crf :D

[woah!]

For the options --b-adapt 2 --weightb, does it matter if these options are enabled for both passes? or are they only needed for first pass and keeping them enabled for 2nd pass causes slowdown in encoding?

I read the other replies but am still not sure what is correct currently. This new b-frame decision gives nice bitrate saving for crf :D

http://forum.doom9.org/showpost.php?p=1164712&postcount=116

[skystrife]

Just an update for the problem happening on my end. I just compiled a r928 version with psy+hrd+prog and the new bframe decision patch (the third one), and I no longer have any crashing being exhibited! Thank you DS! =D

[LoRd_MuldeR]

I still get the crash in 2-Pass mode. But this might be an Avidemux problem. Nevertheless it does not happen with b_adapt=1, only with b_adapt=2 :confused:

http://img211.imageshack.us/img211/4214/bframecrashij3.th.png (http://img211.imageshack.us/my.php?image=bframecrashij3.png)

[kemuri-_9]

Just for info: x264 r927 creates a working DLL with MingW GCC 4.3.1-tdm-1 and "fprofiled" enabled again. That didn't work with r922 and caused a lot of headache...

gcc 4.3.1 is not an official mingw distribution and is not fully supported, so i believe your problems could be lying there.
there's a reason tdm's site has got 'Experimental' plastered on the top...
try compiling in gcc 3.4.5 (the official mingw distribution on sourceforge) and see if you still have problems
https://sourceforge.net/project/showfiles.php?group_id=2435&package_id=82723&release_id=428244

[MythCreator]

Just a small test:Source is Macross Frontier EP01 1080 BDRip,
--cqmfile "M4G_High_Detail_V3.1.cfg" --no-fast-pskip --no-dct-decimate --mixed-refs --scenecut 40 -I 90 -i 25 --me umh --merange 32 --sar 1:1 --threads 3 -r 6 --nr 0 -f -2:-1 -b 16 --b-adapt 2 --direct "auto" --bime --b-bias 0 --b-pyramid -w --deadzone-inter 4 --deadzone-intra 6 --aq-mode 0 --chroma-qp-offset 0 --direct-8x8 1 --8x8dct -A p8x8,p4x4,b8x8,i8x8,i4x4 -m 7 --b-rdo --psy-rd 1.0 --qpmin 10 --qpmax 51 --qpstep 4 --qcomp 0.900000 --ipratio 1.400000 --pbratio 1.300000 --vbv-init 0.9


use CRF=20.


1st:using --b-adapt 1:
x264 [info]: slice I:9 Avg QP:17.78 size: 98476 PSNR Mean Y:53.46 U:52.12 V:51.32 Avg:52.61 Global:52.28
x264 [info]: slice P:144 Avg QP:19.66 size: 44595 PSNR Mean Y:51.32 U:52.34 V:51.64 Avg:51.47 Global:51.20
x264 [info]: slice B:348 Avg QP:21.53 size: 12453 PSNR Mean Y:50.47 U:52.05 V:51.49 Avg:50.82 Global:50.57
x264 [info]: consecutive B-frames: 6.1% 14.2% 6.7% 16.3% 16.3% 35.4% 2.8% 0.0% 0.0% 0.0% 2.2% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 21.4% 73.8% 4.8%
x264 [info]: mb P I16..4: 7.2% 13.4% 0.5% P16..4: 50.4% 7.8% 5.1% 0.1% 0.0% skip:15.4%
x264 [info]: mb B I16..4: 0.5% 0.8% 0.0% B16..8: 31.0% 0.8% 0.8% direct: 1.4% skip:64.8% L0:46.3% L1:43.7% BI:10.0%
x264 [info]: 8x8 transform intra:65.2% inter:32.2%
x264 [info]: direct mvs spatial:96.6% temporal:3.4%
x264 [info]: ref P L0 73.7% 16.7% 5.4% 2.1% 1.2% 1.0%
x264 [info]: ref B L0 85.3% 10.3% 2.8% 1.2% 0.5%
x264 [info]: ref B L1 96.2% 3.8%
x264 [info]: SSIM Mean Y:0.9946263
x264 [info]: PSNR Mean Y:50.764 U:52.135 V:51.532 Avg:51.042 Global:50.771 kb/s:4457.01


encoded 501 frames, 0.86 fps, 4457.77 kb/s






2nd:using --b-adapt 2
x264 [info]: slice I:9 Avg QP:17.33 size:101944 PSNR Mean Y:53.71 U:52.34 V:51.52 Avg:52.84 Global:52.49
x264 [info]: slice P:221 Avg QP:19.40 size: 36761 PSNR Mean Y:51.69 U:52.93 V:52.23 Avg:51.92 Global:51.60
x264 [info]: slice B:271 Avg QP:21.22 size: 11365 PSNR Mean Y:51.07 U:52.67 V:52.12 Avg:51.43 Global:51.10
x264 [info]: consecutive B-frames: 7.5% 49.6% 26.2% 12.2% 2.0% 2.4% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0%
x264 [info]: mb I I16..4: 19.9% 74.8% 5.2%
x264 [info]: mb P I16..4: 5.4% 9.1% 0.4% P16..4: 52.8% 6.8% 4.8% 0.1% 0.0% skip:20.5%
x264 [info]: mb B I16..4: 0.3% 0.6% 0.0% B16..8: 30.1% 0.7% 0.7% direct: 1.4% skip:66.0% L0:43.9% L1:45.5% BI:10.6%
x264 [info]: 8x8 transform intra:63.8% inter:30.1%
x264 [info]: direct mvs spatial:99.3% temporal:0.7%
x264 [info]: ref P L0 71.9% 15.6% 8.0% 2.2% 1.3% 0.9%
x264 [info]: ref B L0 87.2% 9.0% 2.3% 1.0% 0.5%
x264 [info]: ref B L1 96.5% 3.5%
x264 [info]: SSIM Mean Y:0.9951781
x264 [info]: PSNR Mean Y:51.390 U:52.781 V:52.156 Avg:51.668 Global:51.340 kb/s:4640.69


encoded 501 frames, 0.17 fps, 4641.44 kb/s

[Sagittaire]

You have a big quality gain with small higher bitrate. But like always for make good comparision you must strictly have the same bitrate. Make comparison with crf mode is useless.

[G_M_C]

OK read through this tread, and the "current patches" thread; So --weightb is best used on the first pass only, and --b-adapt can best be used on both passes ? Or did I misunderstand this ?

I also understand that --weightb is best used on clips that have fades in them; But how must i judge this ? Is it good to use on stuff like "Sin Sity" with a few fades (but also only few colors) ? Or must there be "more fades than average" and defenately not black&white ? So how do you judge when to use it ?

Now i have to fugure out how to get Megui to do a different commandline for the first pass than the second pass. I've only been using it for 1 week you know :p

[Soichiro]

--weightb should be used on both passes, whereas before it was only useful on the second pass. The --b-adapt setting must be the same on both passes or it won't work right. And yes, --weightb does give the most benefit on a clip with many fades, but it's probably not a bad idea to enable it on other clips, as it shouldn't hurt anything.

[G_M_C]

--weightb should be used on both passes, whereas before it was only useful on the second pass. The --b-adapt setting must be the same on both passes or it won't work right. And yes, --weightb does give the most benefit on a clip with many fades, but it's probably not a bad idea to enable it on other clips, as it shouldn't hurt anything.

Thx, i'm gonna try encoding a clip this weekend to see if i get it right :)

Anyway guess it is easy to use on megui (for me as a Megui n00b), since both passes/commandlines can be the same ;)

[Sagittaire]

--weightb should be used on both passes, whereas before it was only useful on the second pass. The --b-adapt setting must be the same on both passes or it won't work right. And yes, --weightb does give the most benefit on a clip with many fades, but it's probably not a bad idea to enable it on other clips, as it shouldn't hurt anything.

No I think that frame type decision is only for first pass.

[burfadel]

A forward predictor and statistics would be good for b-frame decision to become automatic. This would work by scanning ahead say, 6 frames and detecting whole scene changes in luminance. This would impact on performance very little, and by ramping up the b-frames when a change in luminance is detected (fades), you won't have the overall performance penalty of a high b-frame number throughout. The default b-frame decision number as base could be 3! This would also benefit other scenes where there are constant changes in luminance. On normal scenes where the luminance detector hasn't detected fades, statistics could be gathered to determine the optimum b-frame number. On very clean sources 3 may be ideal where there aren't any fades, and on more noisy sources 5 may be more beneficial. this would give the benefits of the better b-frame decision where needed, without the overall speed compromise of setting 16 b-frames. On clean video where there aren't fades, this may lead to a very fast encode without compromising quality/bitrate, and for scenes with grain or lots of fades, give the benefits of the higher b-frames, however with the slower speed. This would of course be an automatic feature when adaptive b-frames is enabled. However, for those that want an auto feature whilst encoding a video with lots of fades and noise, but don't want the speed penalty of a maximum of 16 b-frames, an option to set the maximum b-frames even on those scenes would be good. Maybe make the range of the maximum from say, 6 up to 16. For those that don't want the auto feautre, setting the adaptive b-frames to '2' could be used...

Just an idea, I don't know how practical it would be. Just since more b-frames is beneficial on fades though it would make sense to have a ramp up feature when fades are detected. Likewise, on normal scenes if 3 or 4 b-frames is the most common used, and since on scenes without fades are usually fairly consistent in terms of noise etc, then having a low b-frame decision number makes sense in those cases. In heavy motion scences, since this is already detected with the motion estimation etc and I believe in such cases less b-frames are used (mostly I and P frames?), even setting the b-frame decision level to 1 (if in high motion cases b-frames aren't used then maybe even 0!) in heavy scenes or 2 in less heavy scenes would promote a faster speed whilst again without the penalty of a higher decision level since its not needed.

Would this work at all? or is it just a fanciful idea? :)

[G_M_C]

I think that a forward predictor allready exsists; b-frame decicions are made on forward en backward (reference)frames (as far as i understand this whole encoding-thing ;) ).

I do not understand your post fully, but i get the idea that your method depends on fades not having much movement (classic fade-outs for instance). But what would happen on scenes that have same caractaristics as a fade, but have much movement; For instance a scene where "the bad guy" is chased by car, and drives into an unlit cave or tunnel. In a scene like that there is much movement, but overall-scene-luminance fades away gradually; It should not be detected as a fade.

I am not really knowledgable on things like this, but i've got the impression that luminance, chrominance and movement should all be considerated and weighed separately while encoding; Offcourse , in reality, you cant really see them separate from eachother, and i think that is whats making this whole thing complicated.

[burfadel]

Thats too true, I never though of it like that :) I guess that would be the benefit of setting a maximum b-frame decision limit of say 6 :) the main idea was to limit the thorough detection if the simple detection detects there is little change, since as pointed out by DS himself (I think it was him), 3 b-frames would be a good standard number. I was thinking that 3 would be good on normal scenes, but in the case of fade-outs more may be more beneficial. Even in the situation you pointed out, ramping it up to 6 or 7 just for that short time, even if its not used, would still be beneficial in an overall context as you would still have the benefit where needed of 6 or 7 b-frames, but without the penalty in static scenes where less are used and having the decision higher has a high penalty in terms of speed :) the statistics idea would still be good even if the other idea wasn't, as a ramp down feature instead of a ramp up!

I'm not entirely sure of how its done myself, the concept could be completely flawed :D!

[G_M_C]

Thats too true, I never though of it like that :) I guess that would be the benefit of setting a maximum b-frame decision limit of say 6 :) the main idea was to limit the thorough detection if the simple detection detects there is little change, since as pointed out by DS himself (I think it was him), 3 b-frames would be a good standard number. I was thinking that 3 would be good on normal scenes, but in the case of fade-outs more may be more beneficial. Even in the situation you pointed out, ramping it up to 6 or 7 just for that short time, even if its not used, would still be beneficial in an overall context as you would still have the benefit where needed of 6 or 7 b-frames, but without the penalty in static scenes where less are used and having the decision higher has a high penalty in terms of speed :) the statistics idea would still be good even if the other idea wasn't, as a ramp down feature instead of a ramp up!

I'm not entirely sure of how its done myself, the concept could be completely flawed :D!

In the situation I pointed out the max number of b-frames would never be reached, and in static scenes even more could be usefull (static scenes with fades are more or less just frame-repeats where only the avg-luminance changes).

But the max number of b-frames you can use also depends on your object; If you're encoding for blu-ray's, you can use less b-frames then when you make MKV's on level 5.1. So in many cases you are limited in what you can do.

The algorithm DS is trying to make has (more or less) be usefull for all cases, MKV and BR alike; So i have the feeling it's not that easy to develop. But i think luminance-weiging is indeed a part of it :)

[Dark Shikari]

Explicit fade-detection is not necessarily ideal; most fades are not linear so you can't just make them into a bunch of B-frames.

[gav1577]

--weightb should be used on both passes, whereas before it was only useful on the second pass. The --b-adapt setting must be the same on both passes or it won't work right. And yes, --weightb does give the most benefit on a clip with many fades, but it's probably not a bad idea to enable it on other clips, as it shouldn't hurt anything.

Sorry getting a little confused now so should --weightb be in both passes? and should --b-adapt 2 be in just the first pass or both? :)

[burfadel]

Ah ok! thats fair enough then :) I'm currently using 6 b-frames with skystrife's build, the speed loss is quite significant. The cpu use is less, as suggested that it may be, but even if it were back to ~98 percent say, it would still be significantly slower than the current method. That said, the results are worth it! Are there many speed optimisations in the works for this patch?

[Dark Shikari]

Sorry getting a little confused now so should --weightb be in both passes? and should --b-adapt 2 be in just the first pass or both? :-)Weightb should be in both passes.

--b-adapt can be in both, but it'll only affect the first.

[gav1577]

Weightb should be in both passes.

--b-adapt can be in both, but it'll only affect the first.

Thanks Dark Shikari So having --b-adapt 2 in the second pass will have no benefit whatsoever ? :)

[LoRd_MuldeR]

Thanks Dark Shikari So having --b-adapt 2 in the second pass will have no benefit whatsoever ? :)

Since b-frame decision is made in first pass, using "--b-adpat 2" in second pass should have no effect at all...

[gav1577]

Since b-frame decision is made in first pass, using "--b-adpat 2" in second pass should have no effect at all...

Ok thanks LoRd_MuldeR ;)

[tetsuo55]

i have some questions, hope you can answer them and correct me if i am wrong.

First i am assuming that B-frames are not limited by Levels.

So from what i read in this thread i can conclude the following:

-The new decision method leads to better phychovisual results
-The new decision method leads to better psnr in most cases, regardless of this phycovisually it looks better
-The new decision method leads to a better compression rate
-All of this gets better and better as the number of b frames increase from 1-16
-This all comes at a base slowness cost that increases exponentially from 1-16 at every 4 steps.


If what i'm saying is correct the only reason not to always use b-frames 16 is the speed-hit, and if that could be fixed then b-frames 16 could be the default right?

[LoRd_MuldeR]

If what i'm saying is correct the only reason not to always use b-frames 16 is the speed-hit, and if that could be fixed then b-frames 16 could be the default right?

If you read the very beginning of this thread, it was clearly said that with the new method 16 b-frames will be a speed killer.
Also you will see that most sources won't use more then ~6 b-frames anyway, even with the new method.
Therefore it makes sens to use the new method, but limit the number of b-frames to a sane value, such as 6 b-frames.

Dark Shikari also mentioned that he might add some heuristics to speed-up the new method ...

[Dark Shikari]

-This all comes at a base slowness cost that increases exponentially from 1-16 at every 4 steps.Linearly, not exponentially.

[bkman]

So is the whole 1st pass 16x slower at 16 b-frames then, or just the b-frame portion of it?

Btw, I've done a full encode with Skystrife's latest build and b-adapt 2. No crashes :)

[Dark Shikari]

So is the whole 1st pass 16x slower at 16 b-frames then, or just the b-frame portion of it?Just the B-frame portion.

[akupenguin]

Quadratic, not linear. The B-frame part is 147 times slower with 16 B-frames than with 1.
It will be linear when I'm done reusing all the motion estimation data.

[kemuri-_9]

actually i'm experiencing a unique phenomenon on my computer for the increase of b-frames on --b-adapt 2:

well firstly, i got 7.37 fps for --bframes 6 which ended up with
x264 [info]: consecutive B-frames: 0.0% 0.4% 0.0% 0.0% 0.0% 0.0% 99.6%
so i got interested and ramped it up all the way to --bframes 16 to see what it would generate on this anime (there's numerous fades and --weightb is on)
and ended up with
avis [info]: 720x480 @ 23.98 fps (29731 frames)
x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:61 Avg QP:13.75 size: 25590
x264 [info]: slice P:1784 Avg QP:13.98 size: 16537
x264 [info]: slice B:27886 Avg QP:16.20 size: 3135
x264 [info]: consecutive B-frames: 0.0% 0.0% 0.0% 0.0% 0.0% 1.2% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 98.7%
x264 [info]: mb I I16..4: 14.1% 67.5% 18.5%
x264 [info]: mb P I16..4: 3.3% 20.0% 4.3% P16..4: 35.2% 22.1% 13.1% 0.2% 0.1% skip: 1.9%
x264 [info]: mb B I16..4: 0.4% 1.4% 0.3% B16..8: 31.5% 1.4% 1.8% direct: 6.7% skip:56.6% L0:51.6% L1:45.2% BI: 3.2%
x264 [info]: final ratefactor: 15.26
x264 [info]: 8x8 transform intra:68.7% inter:56.4%
x264 [info]: direct mvs spatial:99.8% temporal:0.2%
x264 [info]: ref P L0 42.3% 20.4% 11.3% 6.2% 4.1% 3.9% 2.5% 1.6% 1.3% 1.2% 0.9% 1.0% 0.8% 0.9% 0.8% 0.9%
x264 [info]: ref B L0 58.2% 14.9% 8.1% 5.1% 3.1% 2.5% 2.1% 1.2% 0.9% 0.9% 0.7% 0.6% 0.6% 0.7% 0.3%
x264 [info]: ref B L1 88.5% 11.5%
x264 [info]: kb/s:764.4

encoded 29731 frames, 9.61 fps, 764.42 kb/s

it encoded faster with more max bframes!

but i originally get about a 66% average cpu usage on the original with 6 bframes across my 4 cores and now i get about 85% usage with 16 bframes on my phenom 9850
(i have pthreads built in and have use -threads 4)
so.... even though it's actually faster it's also due to actually using more of my cores, so it's an interesting situation for me.

[Soichiro]

What confuses me more is that thing with 98.7% of bframes being in 16-frame sequences. What clip did you use? A still image repeated 29731 times? :p

In all seriousness though, are you sure you're using the latest version of the bframe patch? The old version had an odd bug that would cause long strings of bframes (though I had never seen 98.7% of a clip at 16 o.O).

[kemuri-_9]

it's an r2 dvdrip of the first episode of an anime called mushi-uta without the OP/ED since i plan on using matroska's segmentation feature to handle that.
I handed x264 an avs that loads an ffvh encoded lossless, this particular lossless is 3.67 GB
and yes, this is the 3rd revision of the patch that has been working for people as of late, which is the latest from what I've been reading as of yet.

[Ranguvar]

Any word yet on how and how much this increases subjective quality; anybody willing to upload sample clips? :D

I'm guessing that short clips will be much better than single frame comparisons for this patch :)

[tetsuo55]

thanks for the replies.


kemuri-_9's post basically explains why 16 can be a good idea, in sources that don't need them they simply aren't used.

As soon as the b-frame picker finds the upper limit, and is sure it could abort further efforts, just keep trying till it finds the upper limit.

In that case it would stop at 6 most of the time for live action and go all the way to 16 for static images and fades in antimated videos.

If done correctly assuming its even possible, the speed hit for 16 would only be noticeable on encodes that actually need that many....

Kinda like dynamic b-frames

[Dark Shikari]

thanks for the replies.


kemuri-_9's post basically explains why 16 can be a good idea, in sources that don't need them they simply aren't used.

As soon as the b-frame picker finds the upper limit, and is sure it could abort further efforts, just keep trying till it finds the upper limit.The algorithm doesn't work this way though; there's no obvious way to do such a thing.

[kemuri-_9]

to actually perform something like that, x264 would have to run through a full pass then would look at the produced consecutive b-frame statistics and if it's over some threshold percentage of the given max bframes, it would then rerun it again with an increased count of some arbitrary/semi-arbitrary amount, which the new pass may do the same thing again depending on the increase amount and whether the 16 max bframe maximum was encountered.

so in other wounds, not very feasible for x264 to do this itself.

it can't do this midstream either as it's still accumulating data with frames encoded in the future are not guaranteed to produce statistics similar to ones already encountered

all in all, it's best to do nothing and leave it to the user discretion.

[Avenger007]

Any advice for choosing B-frame mode (--direct spatial or auto) in CRF and 2-pass modes for the new B-frame decision?

[Dark Shikari]

Any advice for choosing B-frame mode (--direct spatial or auto) in CRF and 2-pass modes for the new B-frame decision?Auto is always best, of course.

[Sharktooth]

auto isnt so efficient for 1 pass encodings. i dont think this was ever changed.

[chainring]

auto isnt not so efficient for 1 pass encodings. i dont think this was ever changed.
Is that the reason for Spatial being chosen in the meGUI CRF presets?

[Sharktooth]

yes, and in all 1 pass presets.

[TheRyuu]

Any advice for choosing B-frame mode (--direct spatial or auto) in CRF and 2-pass modes for the new B-frame decision?

As said by DS himeself:
auto is almost surely going to be better
the speed vs quality tradeoff in onepass is not as good as is in twopass though

[kemuri-_9]

Quadratic, not linear. The B-frame part is 147 times slower with 16 B-frames than with 1.
It will be linear when I'm done reusing all the motion estimation data.

any progress coming along on that or have you been too busy from working on other aspects of x264?

[wata]

previously i am using bframes 16 with all my crf encode
so now i have update x264 to build 937 - Jarod's patched build,
should i stay at bframes 16 or lower to it
and to what value?

[DarkZell666]

previously i am using bframes 16 with all my crf encode
so now i have update x264 to build 937 - Jarod's patched build,
should i stay at bframes 16 or lower to it
and to what value?

Read the first post ... :mad:

[wata]

i have read dont understand
it said coming soon
so is the new b-frame decision in build 937 - Jarod's patched build?

[DarkZell666]

i have read dont understand
it said coming soon
so is the new b-frame decision in build 937 - Jarod's patched build?

http://forum.doom9.org/showthread.php?p=1171274#post1171274

Edit : + http://forum.doom9.org/showthread.php?p=1172240#post1172240

[kemuri-_9]

if you are using 16 bframes w/ b-adapt 1 (the default) then you don't have to worry about anything.
However, if you are using 16 bframes w/ b-adapt 2 (the new patch) then just be prepared for it to go incredibly slow:
i do encodes with 16 bframes w/ b-adapt 2 and it makes pass 1 go about 25% the speed of pass 2 on my quad-core system.
the code murders threading efficiency and the processing will mostly reside on a single core, or at least as far as I've seen from my own usage.

so if you are using b-adapt 2 with an 'absurdly high' bframe count, then i would recommend just using a single thread on pass1 to not have --pre-scenecut be active.
for 16 bframes there was only a minor speed decrease from using several threads to just 1 (again as far as my own usage has gone).

[burfadel]

Once the patch is finalised I'm guessing the threading issue will be (for the most part) resolved. DS has stated that the patch does require efficiency tuning, at the moment I'm guessing it to be classified as a 'proof of concept' patch. 'Proof of concept' means it just shows the capability of the patch, and is in no part tuned for efficiency in terms of speed! That said, with the new patch 16 will be slower than say 6 even with the optimisations, but ideally for those concerned with a good b-frame number without sacrificing too much speed 6 is the suggested consecutive b-frame number. If the speed optimisations go to the extent that a higher number doesn't have so much of an impact then a higher number may be suggested in the future for speed vs efficiency :)

[Avenger007]

ideally for those concerned with a good b-frame number without sacrificing too much speed 6 is the suggested consecutive b-frame number.
Since when?

From first post:
in the interests of speed you may want to lower the --bframes value to 3-5 or something otherwise reasonable.
--bframes 3 seems to be the most practical for most videos.
I'm guessing going from --bframes 3 to --bframes 6 would generally offer much less quality improvement when compared with --ref 3 to --ref 6, for example.

[Sharktooth]

no... refs gain is usually less than b-frames gain.
obviously much depends on the source.

[Sagekilla]

If you check the number of b-frames used, 6 is more or less the point where more B's is useless. Anything beyond that and I generally see around 0 - 0.5% usage.

[Dark Shikari]

no... refs gain is usually less than b-frames gain.
obviously much depends on the source.wrong again... bframes 3 to 6 give almost no benefit for the vast majority of sources, while ref 3 to 6 is guaranteed to help at least a bit.

[Sharktooth]

not from the test i did some time ago.
in low and very low motion movies (long scenes, almost no action...etc) b-frames are very helpfull.
obviously, as i said, much depends on the source and what you're used to encode.

[burfadel]

With the official b-frame decision there isn't much benefit going from 3 to 6, but with the new method there does seem to be a good benefit between 3 and 6 :) although as DS said it depends on the source. I concur that the speed loss of going higher than 6 is not worth it given the small benefit of doing so gives. Sunjectively I didn't notice any picture quality differences, but the file sizes were slighly smaller with each higher number of b-frames (say 6,8, 10, 12, 16) out of interest! This doesn't really affect crf mode, but may improve the quality ever so fractionally in ABR mode.

It was mentioned by someone somewhere that 6 was a good figure, and I agree with that! 6 for both ref and b-frames covers most sources, although animation can benefit from setting the ref frames even higher from my experience.

[kemuri-_9]

I mostly work with animation sources and have noticed significant percentage of 14+ bframe sections with the new patch often.

but yes, it's all source dependent, that fact has never changed

[Sagekilla]

In that case 16 B-frames is highly useful for you ;) I tend to use 16/16 on refs/B's for anime anyway, since I want to get it compressed as small as possible and I don't mind waiting the extra time.

[Soichiro]

I mostly work with animation sources and have noticed significant percentage of 14+ bframe sections with the new patch often.

but yes, it's all source dependent, that fact has never changed

I have reason to believe that's a bug, actually. It was supposedly fixed, but there was at least one report (post-bugfix) where something like 98% of b-frames were in 16-frame sequences, which is something that should really only occur in a 98%-still video.

And yes, b-frames are much more beneficial than ref frames on many sources (at least according to my tests), however once you get above 6 for each the benefits become unnoticeable except at low bitrates/high crf.

[Sagekilla]

It sounds unlikely to be a bug, as anime can easily use up a huge number of B's. If this was real life video, then I'd have reason to believe something is fishy.

[Avenger007]

Quadratic, not linear. The B-frame part is 147 times slower with 16 B-frames than with 1.
It will be linear when I'm done reusing all the motion estimation data.
Is efficiency the reason why the B-frame decision patch hasn't been committed as yet?

[wyti]

i think it's one of the reason, the second may be that --b-adapt-2 brake multithreading efficiency (about 60% on a quad core)

[Dark Shikari]

i think it's one of the reason, the second may be that --b-adapt-2 brake multithreading efficiency (about 60% on a quad core)Sure, if you insist on doing retarded things like --bframes 16 with it...

[Adub]

What's this? It's not smart to use --bframes 16? I was going under the impression that 16 bframes works quite well for anime (which is why I would use 16 bframes).

[Dark Shikari]

What's this? It's not smart to use --bframes 16? I was going under the impression that 16 bframes works quite well for anime (which is why I would use 16 bframes).Hint: read the original post in threads you respond to.

[Adub]

Ah, see I thought you were saying that it negatively affected quality or something. I knew about the speed hit, and I am perfectly okay with it. I have encountered numerous instances where the full 16 bframes were used a majority of the time, and I have time, so I let my encodes run on "retarded" settings from time to time.

[Comatose]

What are you running on? (CPU)

I was encoding at seconds per frame with my "retarded" settings :\ (C2D E6420)

[Sagekilla]

Sounds extremely excessive.. I use very, VERY slow settings on 720p sources with heavy prefiltering and I still get 2-3 fps on my Opteron 170 (2 GHz) and closer to 4+ on my C2D @ 2.16 GHz. Something is definitely wrong with your settings if you're going by seconds by frame.

[cogman]

Sure, if you insist on doing retarded things like --bframes 16 with it...

:P What, you should have made B-Frames multithreaded. 1 Thread for every b-frame... (Yes, I am kidding)

[Avenger007]

Is it possible to merge the old and new B-frame decision algorithms such that the new algorithm is applied to the first, say, 4 B-frames and any further consecutive B-frames would be determined by the old algorithm?

[DarkZell666]

Is it possible to merge the old and new B-frame decision algorithms such that the new algorithm is applied to the first, say, 4 B-frames and any further consecutive B-frames would be determined by the old algorithm?

That's just like asking the new algo to stop searching after 4 b-frames, since the old algo almost never goes past 3 anyway :p

[Warpman]

That's just like asking the new algo to stop searching after 4 b-frames, since the old algo almost never goes past 3 anyway :p
uhm ive got tons of encodes, mosty anime where the old algo frequently used all 16bframes...

[Dark Shikari]

Is it possible to merge the old and new B-frame decision algorithms such that the new algorithm is applied to the first, say, 4 B-frames and any further consecutive B-frames would be determined by the old algorithm?No, this is inherently impossible by the way B-frame trellis works.

However, I'd much rather have 4 B-frames with the new algorithm than 16 with the old algorithm.

[burfadel]

Wasn't there an issue with the old b-frame decision that selected b-frames where p-frames should be in certain circumstances? (with a resultant loss in visual quality)?

[akupenguin]

Wasn't there an issue with the old b-frame decision that selected b-frames where p-frames should be in certain circumstances?
Umm, yes? Otherwise we wouldn't be able to make b-adapt=2 better...

[IgorC]

I don't know if it was already discussed but for this source http://www.steves-digicams.com/2005_reviews/s2is/samples/MVI_0371.AVI
new b-adapt with 16 b-frames cause higher bitrate than expected (from 1500 to 1700). While with 3 bframes bitrate is normal.

[Dark Shikari]

I don't know if it was already discussed but for this source http://www.steves-digicams.com/2005_reviews/s2is/samples/MVI_0371.AVI
new b-adapt with 16 b-frames cause higher bitrate than expected (from 1500 to 1700). While with 3 bframes bitrate is normal.This is probably due to the increased delay introduced by b-adapt 2 (I suspect the longer the source, the less the effect). I'm not sure exactly how this would cause it, but it'd be my prime culprit.

[kemuri-_9]

anime DVD source 720x480


x264_r977 --pass 1 --bitrate 784 --stats x264.stats --min-keyint 1 --keyint 500 --scenecut 50 --ref 16 --mixed-refs
--deblock 0:0 --no-fast-pskip --b-adapt _x_ --bframes 16 --b-bias 0 --b-pyramid --b-rdo --bime --weightb
--direct auto --subme 7 --direct-8x8 -1 --trellis 2 --analyse all --8x8dct --qpmin 1 --qpmax 51 --qpstep 60
--pbratio 1.4 --ratetol 100.0 --qcomp 0.6 --me umh --merange 32 --psy-rd 0.0:0.0 --thread-input --threads 1
--progress --no-dct-decimate --level 5.1 --log-file x264.1pass.log --output NUL open_lossless.avs


b-adapt 1

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:248 Avg QP:13.92 size: 26696 PSNR Mean Y:49.85 U:51.77 V:51.95 Avg:50.33 Global:49.22
x264 [info]: slice P:22363 Avg QP:15.36 size: 4893 PSNR Mean Y:48.09 U:50.67 V:50.89 Avg:48.76 Global:48.08
x264 [info]: slice B:7120 Avg QP:22.01 size: 537 PSNR Mean Y:48.02 U:51.28 V:51.60 Avg:48.77 Global:46.66
x264 [info]: consecutive B-frames: 65.1% 10.1% 5.2% 5.3% 9.1% 3.9% 0.0% 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.1% 0.1% 0.1% 0.6%
x264 [info]: mb I I16..4: 30.2% 33.0% 36.8%
x264 [info]: mb P I16..4: 3.5% 5.6% 1.4% P16..4: 55.8% 8.5% 8.5% 0.0% 0.0% skip:16.7%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 23.7% 0.2% 0.2% direct: 0.9% skip:74.7% L0:41.7% L1:57.2% BI: 1.1%
x264 [info]: final ratefactor: 22.28
x264 [info]: 8x8 transform intra:51.2% inter:53.8%
x264 [info]: direct mvs spatial:98.2% temporal:1.8%
x264 [info]: ref P L0 72.4% 7.9% 5.2% 2.1% 1.9% 1.8% 1.7% 0.8% 0.9% 0.8% 0.8% 0.8% 0.7% 0.7% 0.8% 0.5%
x264 [info]: ref B L0 80.6% 7.8% 3.7% 1.5% 1.2% 1.0% 0.9% 0.6% 0.5% 0.4% 0.4% 0.4% 0.4% 0.4% 0.3%
x264 [info]: ref B L1 96.2% 3.8%
x264 [info]: SSIM Mean Y:0.9903388
x264 [info]: PSNR Mean Y:48.087 U:50.823 V:51.071 Avg:48.776 Global:47.701 kb/s:773.35
x264 [info]: encoded 29731 frames, 2.91 fps, 773.35 kb/s


b-adapt 2

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:248 Avg QP:13.72 size: 27347 PSNR Mean Y:50.03 U:52.08 V:52.21 Avg:50.49 Global:49.34
x264 [info]: slice P:18059 Avg QP:14.66 size: 5778 PSNR Mean Y:49.09 U:51.46 V:51.71 Avg:49.72 Global:48.42
x264 [info]: slice B:11424 Avg QP:20.34 size: 743 PSNR Mean Y:47.03 U:50.15 V:50.40 Avg:47.82 Global:47.13
x264 [info]: consecutive B-frames: 41.6% 9.1% 40.2% 3.6% 2.1% 0.8% 0.7% 0.4% 0.3% 0.3% 0.2% 0.1% 0.1% 0.1% 0.0% 0.1% 0.3%
x264 [info]: mb I I16..4: 30.3% 32.5% 37.2%
x264 [info]: mb P I16..4: 3.6% 5.7% 1.7% P16..4: 55.3% 9.7% 9.6% 0.0% 0.0% skip:14.4%
x264 [info]: mb B I16..4: 0.7% 0.7% 0.1% B16..8: 23.7% 0.2% 0.2% direct: 1.4% skip:73.0% L0:51.4% L1:47.4% BI: 1.2%
x264 [info]: final ratefactor: 21.29
x264 [info]: 8x8 transform intra:49.4% inter:50.9%
x264 [info]: direct mvs spatial:99.7% temporal:0.3%
x264 [info]: ref P L0 68.2% 9.3% 5.8% 2.5% 2.2% 2.0% 2.0% 1.0% 1.0% 0.9% 1.0% 0.9% 0.9% 0.8% 0.9% 0.6%
x264 [info]: ref B L0 81.9% 7.9% 3.1% 1.4% 1.1% 0.8% 0.7% 0.5% 0.5% 0.4% 0.4% 0.4% 0.3% 0.3% 0.2%
x264 [info]: ref B L1 94.5% 5.5%
x264 [info]: SSIM Mean Y:0.9905947
x264 [info]: PSNR Mean Y:48.306 U:50.962 V:51.208 Avg:48.995 Global:47.888 kb/s:771.61
x264 [info]: encoded 29731 frames, 2.02 fps, 771.61 kb/s


working on a threaded comparison now.
if this was the version of b-adapt when it was still a patch, then the b-adapt 2 would of been more around 1.50 fps

[kemuri-_9]

and the threaded comparisons...

x264_r977 --pass 1 --bitrate 784 --stats x264.stats --min-keyint 1 --keyint 500 --scenecut 50 --ref 16 --mixed-refs
--deblock 0:0 --no-fast-pskip --b-adapt _x_ --bframes 16 --b-bias 0 --b-pyramid --b-rdo --bime --weightb
--direct auto --subme 7 --direct-8x8 -1 --trellis 2 --analyse all --8x8dct --qpmin 1 --qpmax 51 --qpstep 60
--pbratio 1.4 --ratetol 100.0 --qcomp 0.6 --me umh --merange 32 --psy-rd 0.0:0.0 --thread-input --threads 4
--progress --no-dct-decimate --level 5.1 --log-file x264.1pass.log --output NUL open_lossless.avs


b-adapt 1: ~90% average CPU usage (across 4 cores)

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:347 Avg QP:13.38 size: 23731 PSNR Mean Y:51.19 U:53.47 V:53.57 Avg:51.63 Global:49.71
x264 [info]: slice P:22264 Avg QP:15.37 size: 4851 PSNR Mean Y:48.10 U:50.66 V:50.88 Avg:48.77 Global:48.12
x264 [info]: slice B:7120 Avg QP:21.99 size: 538 PSNR Mean Y:48.05 U:51.28 V:51.60 Avg:48.77 Global:46.70
x264 [info]: consecutive B-frames: 65.0% 10.2% 5.2% 5.4% 9.2% 3.9% 0.0% 0.1% 0.0% 0.0% 0.0% 0.0% 0.0% 0.1% 0.1% 0.1% 0.6%
x264 [info]: mb I I16..4: 32.8% 33.4% 33.8%
x264 [info]: mb P I16..4: 3.4% 5.5% 1.3% P16..4: 55.8% 8.5% 8.5% 0.0% 0.0% skip:16.9%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 23.8% 0.2% 0.2% direct: 0.9% skip:74.6% L0:41.8% L1:57.1% BI: 1.1%
x264 [info]: final ratefactor: 22.30
x264 [info]: 8x8 transform intra:50.8% inter:53.9%
x264 [info]: direct mvs spatial:98.2% temporal:1.8%
x264 [info]: ref P L0 72.4% 7.9% 5.2% 2.1% 1.9% 1.8% 1.7% 0.8% 0.9% 0.8% 0.8% 0.8% 0.7% 0.7% 0.8% 0.5%
x264 [info]: ref B L0 80.6% 7.9% 3.6% 1.5% 1.1% 1.0% 0.9% 0.6% 0.5% 0.4% 0.4% 0.4% 0.4% 0.4% 0.3%
x264 [info]: ref B L1 96.2% 3.8%
x264 [info]: SSIM Mean Y:0.9903609
x264 [info]: PSNR Mean Y:48.125 U:50.837 V:51.081 Avg:48.804 Global:47.750 kb/s:774.59
x264 [info]: encoded 29731 frames, 9.44 fps, 774.59 kb/s


b-adapt 2: ~60% average CPU usage (across 4 cores)

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:346 Avg QP:13.17 size: 24719 PSNR Mean Y:51.35 U:53.55 V:53.70 Avg:51.76 Global:49.81
x264 [info]: slice P:18174 Avg QP:14.75 size: 5692 PSNR Mean Y:49.10 U:51.48 V:51.72 Avg:49.73 Global:48.46
x264 [info]: slice B:11211 Avg QP:20.38 size: 697 PSNR Mean Y:47.00 U:50.10 V:50.35 Avg:47.78 Global:47.14
x264 [info]: consecutive B-frames: 42.2% 9.4% 39.9% 3.3% 1.9% 0.8% 0.7% 0.4% 0.3% 0.2% 0.1% 0.0% 0.1% 0.1% 0.0% 0.1% 0.3%
x264 [info]: mb I I16..4: 32.8% 32.5% 34.7%
x264 [info]: mb P I16..4: 3.6% 5.7% 1.7% P16..4: 55.4% 9.6% 9.4% 0.0% 0.0% skip:14.6%
x264 [info]: mb B I16..4: 0.5% 0.6% 0.1% B16..8: 23.8% 0.2% 0.2% direct: 1.2% skip:73.3% L0:51.3% L1:47.5% BI: 1.2%
x264 [info]: final ratefactor: 21.40
x264 [info]: 8x8 transform intra:49.3% inter:51.2%
x264 [info]: direct mvs spatial:99.8% temporal:0.2%
x264 [info]: ref P L0 68.2% 9.3% 5.9% 2.5% 2.2% 2.0% 2.0% 1.0% 1.0% 0.9% 0.9% 0.9% 0.9% 0.8% 0.9% 0.6%
x264 [info]: ref B L0 82.7% 7.9% 3.0% 1.3% 1.0% 0.7% 0.7% 0.5% 0.4% 0.3% 0.4% 0.3% 0.3% 0.3% 0.2%
x264 [info]: ref B L1 94.5% 5.5%
x264 [info]: SSIM Mean Y:0.9905993
x264 [info]: PSNR Mean Y:48.335 U:50.982 V:51.225 Avg:49.021 Global:47.922 kb/s:772.90
x264 [info]: encoded 29731 frames, 5.33 fps, 772.90 kb/s

[Sharktooth]

that's a good reason to not use -bframes 16 with b-adapt 2 ...

[kemuri-_9]

that's a good reason to not use -bframes 16 with b-adapt 2 ...

depends on what kind of quality you want!

[Ranguvar]

Looks to me like --bframes 3 is still the sweet spot :) Speed (not so much now with the improvements), PS3 compatibility, not much quality loss I'd guess from those stats :)

Thanks for the test!

[Quark.Fusion]

Looks to me like --bframes 4 is the sweet spot :) But I'm prefer 6 not to miss better compression.

kemuri-_9, can you test 2, 4 and 6 b-frames on this source (where 5 and 6 b-frames were used on 1% cases, and 2 on most cases)?

[Audionut]

and the threaded comparisons...


Looks like the speed loss is worth it to me.

Look at the resulting bitrate, quantizers and metrics of each file.

Could you run another test with b-frames at say 5?

TIA.

[Sapo84]

I tried to encode a particular anime source (Iblard Jikan, with is mostly consosed by static landscapes with little motion between frames), the results are somewhat interesting

b-adapt 1 (2 pass encode with turbo)
--pass 2 --bitrate 1460 --stats "G:\iblard.stats" --ref 4 --mixed-refs --bframes 16 --b-pyramid
--b-rdo --bime --direct auto --filter 1:0 --psy-rd 0.5:0.4 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct
--threads auto --thread-input --aq-strength 0.8 --sar 1:1 --progress --output "G:\iblard.mp4" "G:\iblard.avs"

x264 [info]: slice I:59 Avg QP:16.48 size:136488 PSNR Mean Y:47.67 U:50.26 V:51.03 Avg:48.38 Global:45.40
x264 [info]: slice P:3873 Avg QP:19.58 size: 13427 PSNR Mean Y:42.22 U:45.04 V:45.37 Avg:42.96 Global:42.18
x264 [info]: slice B:8935 Avg QP:20.29 size: 2060 PSNR Mean Y:44.12 U:46.40 V:47.29 Avg:44.70 Global:42.56
x264 [info]: consecutive B-frames: 17.9% 0.8% 0.9% 4.2% 15.3% 28.3% 7.3% 1.2% 1.3% 1.2% 0.7% 1.4% 0.7% 0.8% 0.1% 1.0% 16.9%
x264 [info]: mb I I16..4: 7.3% 43.8% 48.9%
x264 [info]: mb P I16..4: 0.2% 1.1% 0.2% P16..4: 53.0% 9.1% 14.5% 0.0% 0.0% skip:22.0%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 27.5% 0.5% 1.4% direct: 3.6% skip:66.9% L0:32.8% L1:54.4% BI:12.8%
x264 [info]: 8x8 transform intra:58.5% inter:46.8%
x264 [info]: direct mvs spatial:95.1% temporal:4.9%
x264 [info]: ref P L0 81.7% 8.0% 7.7% 2.7%
x264 [info]: ref B L0 65.8% 27.8% 6.4%
x264 [info]: ref B L1 81.4% 18.6%
x264 [info]: SSIM Mean Y:0.9839589
x264 [info]: PSNR Mean Y:43.565 U:46.005 V:46.730 Avg:44.197 Global:42.449 kb/s:1462.18
encoded 12867 frames, 27.37 fps, 1462.25 kb/s

b-adapt 2 (otherwise same settings)

x264 [info]: slice I:59 Avg QP:16.37 size:137576 PSNR Mean Y:47.76 U:50.35 V:51.13 Avg:48.47 Global:45.51
x264 [info]: slice P:6890 Avg QP:18.51 size: 8726 PSNR Mean Y:44.97 U:47.18 V:48.18 Avg:45.58 Global:42.83
x264 [info]: slice B:5918 Avg QP:21.58 size: 1733 PSNR Mean Y:41.52 U:44.41 V:44.85 Avg:42.31 Global:41.85
x264 [info]: consecutive B-frames: 28.2% 37.4% 8.5% 8.6% 0.8% 1.2% 1.9% 3.1% 0.4% 1.6% 3.3% 2.2% 0.2% 0.3% 1.2% 1.1% 0.0%
x264 [info]: mb I I16..4: 7.5% 43.8% 48.7%
x264 [info]: mb P I16..4: 0.1% 0.6% 0.1% P16..4: 44.5% 6.1% 10.2% 0.0% 0.0% skip:38.5%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 29.1% 0.5% 1.3% direct: 3.4% skip:65.8% L0:30.8% L1:56.5% BI:12.7%
x264 [info]: 8x8 transform intra:59.0% inter:45.0%
x264 [info]: direct mvs spatial:93.1% temporal:6.9%
x264 [info]: ref P L0 80.5% 9.3% 6.6% 3.6%
x264 [info]: ref B L0 62.1% 28.4% 9.6%
x264 [info]: ref B L1 85.6% 14.4%
x264 [info]: SSIM Mean Y:0.9835001
x264 [info]: PSNR Mean Y:43.400 U:45.919 V:46.662 Avg:44.091 Global:42.360 kb/s:1462.82
encoded 12867 frames, 29.89 fps, 1462.90 kb/s

Less b-frames and worse PSNR and SSIM.

b-adapt 2 with 5 b-frame (good enough speed)
x264 [info]: slice I:59 Avg QP:16.53 size:135574 PSNR Mean Y:47.61 U:50.22 V:51.00 Avg:48.32 Global:45.36
x264 [info]: slice P:7329 Avg QP:18.84 size: 8600 PSNR Mean Y:44.54 U:46.82 V:47.78 Avg:45.17 Global:42.54
x264 [info]: slice B:5479 Avg QP:21.76 size: 1351 PSNR Mean Y:41.38 U:44.26 V:44.70 Avg:42.16 Global:41.69
x264 [info]: consecutive B-frames: 28.3% 43.1% 9.4% 12.2% 1.9% 5.2%
x264 [info]: mb I I16..4: 7.4% 44.3% 48.3%
x264 [info]: mb P I16..4: 0.1% 0.5% 0.1% P16..4: 44.5% 6.6% 11.1% 0.0% 0.0% skip:37.1%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 28.9% 0.4% 1.1% direct: 2.5% skip:67.2% L0:31.8% L1:55.5% BI:12.7%
x264 [info]: 8x8 transform intra:59.1% inter:45.6%
x264 [info]: direct mvs spatial:92.4% temporal:7.6%
x264 [info]: ref P L0 76.4% 10.9% 8.1% 4.6%
x264 [info]: ref B L0 53.9% 24.6% 21.5%
x264 [info]: ref B L1 87.8% 12.2%
x264 [info]: SSIM Mean Y:0.9827461
x264 [info]: PSNR Mean Y:43.209 U:45.744 V:46.484 Avg:43.904 Global:42.162 kb/s:1461.51
encoded 12867 frames, 28.38 fps, 1461.59 kb/s

Then i tried something different, I slowed down the first pass by using better settings so to give better data to the algorithm

--pass 1 --bitrate 1460 --stats "G:\iblard.stats" --ref 2 --mixed-refs --bframes 16 --b-adapt 2
--b-pyramid --direct auto --filter 1:0 --subme 5 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me dia --threads auto
--thread-input --aq-strength 0.8 --sar 1:1 --progress --output NUL "G:\iblard.avs"

x264 [info]: slice I:59 Avg QP:16.38 size:136688 PSNR Mean Y:48.45 U:51.04 V:51.85 Avg:49.17 Global:45.61
x264 [info]: slice P:6087 Avg QP:18.57 size: 9705 PSNR Mean Y:45.80 U:47.57 V:48.44 Avg:46.20 Global:42.82
x264 [info]: slice B:6721 Avg QP:21.25 size: 1686 PSNR Mean Y:41.79 U:44.58 V:45.01 Avg:42.56 Global:42.05
x264 [info]: consecutive B-frames: 18.6% 41.4% 11.0% 9.7% 0.9% 1.2% 3.0% 3.4% 0.6% 1.6% 2.6% 1.9% 0.5% 0.2% 2.2% 1.0% 0.3%
x264 [info]: mb I I16..4: 8.7% 43.4% 47.9%
x264 [info]: mb P I16..4: 0.1% 0.6% 0.1% P16..4: 45.3% 6.6% 11.0% 0.0% 0.0% skip:36.3%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 27.5% 0.4% 1.2% direct: 3.5% skip:67.3% L0:30.5% L1:59.0% BI:10.6%
x264 [info]: 8x8 transform intra:58.4% inter:45.0%
x264 [info]: direct mvs spatial:94.5% temporal:5.5%
x264 [info]: ref P L0 80.0% 9.6% 6.8% 3.6%
x264 [info]: ref B L0 63.3% 27.6% 9.1%
x264 [info]: ref B L1 86.7% 13.3%
x264 [info]: SSIM Mean Y:0.9838412
x264 [info]: PSNR Mean Y:43.719 U:46.026 V:46.661 Avg:44.310 Global:42.410 kb/s:1462.31
encoded 12867 frames, 29.06 fps, 1462.39 kb/s

More b.frame (6700 vs 5900) and better quality.
It's hard to draw a clear conclusion with only one source but I think it may be interesting to test how much turbo first pass can degrade the quality with b-adapt 2.

I also tried a slower first pass (with the same setting as the one above) with b-adapt 1

x264 [info]: slice I:59 Avg QP:16.75 size:132272 PSNR Mean Y:48.12 U:50.76 V:51.56 Avg:48.85 Global:45.23
x264 [info]: slice P:3998 Avg QP:19.98 size: 13844 PSNR Mean Y:41.84 U:44.62 V:44.97 Avg:42.58 Global:41.81
x264 [info]: slice B:8810 Avg QP:20.29 size: 1738 PSNR Mean Y:44.40 U:46.42 V:47.19 Avg:44.91 Global:42.53
x264 [info]: consecutive B-frames: 12.3% 3.3% 12.1% 19.1% 18.0% 18.6% 8.0% 0.3% 0.7% 0.2% 0.3% 0.4% 0.1% 0.0% 0.2% 0.1% 6.2%
x264 [info]: mb I I16..4: 8.7% 43.8% 47.5%
x264 [info]: mb P I16..4: 0.2% 1.0% 0.2% P16..4: 55.2% 9.7% 15.5% 0.0% 0.0% skip:18.3%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 26.4% 0.4% 1.2% direct: 3.3% skip:68.7% L0:31.7% L1:59.0% BI: 9.4%
x264 [info]: 8x8 transform intra:58.5% inter:46.7%
x264 [info]: direct mvs spatial:97.0% temporal:3.0%
x264 [info]: ref P L0 79.2% 7.6% 9.9% 3.2%
x264 [info]: ref B L0 69.3% 24.9% 5.8%
x264 [info]: ref B L1 81.5% 18.5%
x264 [info]: SSIM Mean Y:0.9835551
x264 [info]: PSNR Mean Y:43.622 U:45.884 V:46.521 Avg:44.205 Global:42.302 kb/s:1462.16
encoded 12867 frames, 27.89 fps, 1462.24 kb/s

Similar number of b-frames and similar quality.

[Audionut]

Then i tried something different, I slowed down the first pass by using better settings so to give better data to the algorithm.
.....
More b.frame (6700 vs 5900) and better quality.
.

That's because --b-adapt makes all it's decisions on the first pass.

[Sapo84]

That's because --b-adapt makes all it's decisions on the first pass.
Exactly :)
And that's why I wanted to try a slower (but better) first pass, and the results speak for themself.
Moreover with b-adapt 2 and a high (10+) number of b-frames the first pass is slow anyway :devil:

[Sharktooth]

metrics do not measure quality...
so the result doesnt speak at all.

[Sapo84]

metrics do not measure quality...
We all know, but neither subjective opinions nor still frames really measure quality.
And I still think that posting metrics/opinions/still frames is better than posting nothing at all.
I could have easily posted a few frames, but they would have been nearly useless because they're too similar, and I fail to see any difference at all watching the video in motion (which is expected).
Nevertheless a small gain in metrics with the same settings for the second pass is, at last, something interesting ;)

[kemuri-_9]

imo there should be some raised eyebrows as to how sapo's b-adapt 2 passes came out faster than the b-adapt 1 pass
since b-adapt 2 is slower than b-adapt 1.
i don't think it could be completely attributed to caching

I'm running tests again on a for loop (easy way to do it) for bframes 2-6 since everyone seems to be interested on these values' results.
will post when they finish in several hours.

[Quark.Fusion]

imo there shouldn't be some raised eyebrows as lesser b-frames = faster encode. (Ok, there can be from someone who don't understand) :)

[Quark.Fusion]

Can you also test how same settings as in second pass affect --b-adapt 2?

[kemuri-_9]

imo there shouldn't be some raised eyebrows as lesser b-frames = faster encode. (Ok, there can be from someone who don't understand) :)

are you blind? look at his first two outputs: they're both on 16 bframes, the first on b-adapt 1, the second on b-adapt 2, the 2nd (b-adapt 2) is faster.

and then the b-adapt 2 w/ 5 bframes was slower than the b-adapt 2 w/ 16 bframes
that shouldn't happen either.

[Quark.Fusion]

b-adapt 1 (2 pass encode with turbo)
x264 [info]: slice B:8935

b-adapt 2 (otherwise same settings)
x264 [info]: slice B:5918who is blind? :)

[Quark.Fusion]

But why --b-frames 5 is slower that --b-frames 16 is interesting, I think it is source-dependant — maybe Dark Shikari can answer?

[DarkZell666]

The B-frame distribution is vastly different, and the source is anime. This might account for the wierd fps differences, but it's still contrary to what was announced by Dark Shikari weeks ago.
Maybe he was talking about a typical movie DVD-rip usecase implicitely ... ? :)

Edit : I think I'll have a try myself in fact ...

[DarkZell666]

Well, results were consistent for me even on an anime source (Edit : using x264 r979 from x264.nl) :

Z:\data>encode2

Z:\data>SET OPTS=-r 16 --mixed-refs --b-rdo --bime --weightb --subme 7

Z:\data>SET OPTS2= -t 2 -8 --me umh --psy-rd 0.8:0.8 --thread-input --threads 6 --progress

Z:\data>x264.979.exe -r 16 --mixed-refs --b-rdo --bime --weightb --subme 7 -t 2 -8 --me umh --psy-rd 0.8:0.8 --thread-input --threads 6 --progress --pa
ss 1 --b-adapt 1 --bframes 5 --bitrate 784 --stats x264_anime_1_5.stats --output NUL anime.avs
avis [info]: 704x396 @ 23.98 fps (3001 frames)
x264 [warning]: width or height not divisible by 16 (704x396), compression will suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4 Cache64
x264 [info]: slice I:26 Avg QP:15.23 size: 41329 PSNR Mean Y:49.88 U:56.31 V:56.39 Avg:51.11 Global:49.25
x264 [info]: slice P:1353 Avg QP:16.80 size: 6874 PSNR Mean Y:45.94 U:52.22 V:52.44 Avg:47.20 Global:46.10
x264 [info]: slice B:1622 Avg QP:17.26 size: 1923 PSNR Mean Y:45.93 U:52.68 V:52.89 Avg:47.22 Global:45.90
x264 [info]: consecutive B-frames: 25.5% 15.8% 7.2% 9.7% 9.4% 32.5%
x264 [info]: mb I I16..4..PCM: 17.8% 39.7% 42.5% 0.0%
x264 [info]: mb P I16..4..PCM: 3.1% 7.2% 2.2% 0.0% P16..4: 44.1% 7.5% 10.0% 0.0% 0.0% skip:25.9%
x264 [info]: mb B I16..4..PCM: 0.3% 1.9% 0.8% 0.0% B16..8: 24.1% 1.0% 1.6% direct: 3.6% skip:66.8% L0:47.8% L1:45.9% BI: 6.3%
x264 [info]: final ratefactor: 19.22
x264 [info]: 8x8 transform intra:56.6% inter:30.1%
x264 [info]: ref P L0 75.9% 10.2% 4.2% 2.4% 1.4% 1.4% 0.8% 0.7% 0.5% 0.5% 0.3% 0.4% 0.3% 0.4% 0.2% 0.2%
x264 [info]: ref B L0 78.0% 7.1% 4.5% 2.0% 1.7% 1.2% 1.2% 0.7% 0.7% 0.6% 0.5% 0.4% 0.5% 0.4% 0.5%
x264 [info]: SSIM Mean Y:0.9942085
x264 [info]: PSNR Mean Y:45.966 U:52.506 V:52.719 Avg:47.247 Global:46.010 kb/s:862.46

encoded 3001 frames, 29.62 fps, 862.56 kb/s

Z:\data>x264.979.exe -r 16 --mixed-refs --b-rdo --bime --weightb --subme 7 -t 2 -8 --me umh --psy-rd 0.8:0.8 --thread-input --threads 6 --progress --pa
ss 1 --b-adapt 2 --bframes 5 --bitrate 784 --stats x264_anime_2_5.stats --output NUL anime.avs
avis [info]: 704x396 @ 23.98 fps (3001 frames)
x264 [warning]: width or height not divisible by 16 (704x396), compression will suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4 Cache64
x264 [info]: slice I:26 Avg QP:15.15 size: 41581 PSNR Mean Y:49.99 U:56.31 V:56.38 Avg:51.22 Global:49.25
x264 [info]: slice P:1606 Avg QP:15.62 size: 5975 PSNR Mean Y:46.64 U:52.91 V:53.16 Avg:47.90 Global:46.77
x264 [info]: slice B:1369 Avg QP:18.74 size: 1889 PSNR Mean Y:45.32 U:52.21 V:52.33 Avg:46.62 Global:45.22
x264 [info]: consecutive B-frames: 25.4% 35.0% 22.5% 7.0% 4.0% 6.1%
x264 [info]: mb I I16..4..PCM: 17.7% 39.7% 42.7% 0.0%
x264 [info]: mb P I16..4..PCM: 2.3% 5.5% 2.1% 0.0% P16..4: 45.0% 6.2% 8.5% 0.0% 0.0% skip:30.3%
x264 [info]: mb B I16..4..PCM: 0.6% 1.9% 0.5% 0.0% B16..8: 24.3% 1.0% 1.5% direct: 3.6% skip:66.7% L0:45.1% L1:46.9% BI: 8.0%
x264 [info]: final ratefactor: 19.15
x264 [info]: 8x8 transform intra:55.0% inter:28.6%
x264 [info]: ref P L0 81.8% 7.4% 3.5% 1.7% 1.1% 1.0% 0.8% 0.5% 0.4% 0.4% 0.3% 0.2% 0.2% 0.2% 0.2% 0.1%
x264 [info]: ref B L0 76.6% 9.1% 4.4% 2.1% 1.9% 1.2% 1.1% 0.7% 0.6% 0.5% 0.5% 0.4% 0.4% 0.3% 0.3%
x264 [info]: SSIM Mean Y:0.9942401
x264 [info]: PSNR Mean Y:46.065 U:52.620 V:52.810 Avg:47.346 Global:46.007 kb/s:847.72

encoded 3001 frames, 27.58 fps, 847.83 kb/s

Z:\data>x264.979.exe -r 16 --mixed-refs --b-rdo --bime --weightb --subme 7 -t 2 -8 --me umh --psy-rd 0.8:0.8 --thread-input --threads 6 --progress --pa
ss 1 --b-adapt 1 --bframes 16 --bitrate 784 --stats x264_anime_1_16.stats --output NUL anime.avs
avis [info]: 704x396 @ 23.98 fps (3001 frames)
x264 [warning]: width or height not divisible by 16 (704x396), compression will suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4 Cache64
x264 [info]: slice I:26 Avg QP:15.24 size: 41270 PSNR Mean Y:49.87 U:56.33 V:56.38 Avg:51.11 Global:49.24
x264 [info]: slice P:1297 Avg QP:17.02 size: 7078 PSNR Mean Y:45.81 U:52.13 V:52.33 Avg:47.08 Global:46.00
x264 [info]: slice B:1678 Avg QP:17.24 size: 1938 PSNR Mean Y:45.93 U:52.67 V:52.86 Avg:47.22 Global:45.94
x264 [info]: consecutive B-frames: 24.7% 15.7% 6.9% 9.9% 9.4% 10.5% 11.1% 1.9% 1.8% 1.7% 0.7% 1.2% 0.9% 1.4% 0.0% 1.1% 1.1%
x264 [info]: mb I I16..4..PCM: 17.8% 39.6% 42.6% 0.0%
x264 [info]: mb P I16..4..PCM: 3.3% 7.5% 2.3% 0.0% P16..4: 43.4% 7.8% 10.3% 0.0% 0.0% skip:25.5%
x264 [info]: mb B I16..4..PCM: 0.3% 1.8% 0.7% 0.0% B16..8: 24.6% 1.0% 1.7% direct: 3.8% skip:66.0% L0:48.3% L1:45.4% BI: 6.3%
x264 [info]: final ratefactor: 19.18
x264 [info]: 8x8 transform intra:56.7% inter:30.3%
x264 [info]: ref P L0 75.3% 10.5% 4.3% 2.5% 1.5% 1.5% 0.9% 0.7% 0.5% 0.6% 0.3% 0.4% 0.3% 0.4% 0.2% 0.2%
x264 [info]: ref B L0 78.2% 7.1% 4.4% 2.1% 1.7% 1.2% 1.1% 0.7% 0.6% 0.5% 0.6% 0.4% 0.4% 0.4% 0.4%
x264 [info]: SSIM Mean Y:0.9941755
x264 [info]: PSNR Mean Y:45.915 U:52.469 V:52.660 Avg:47.195 Global:45.985 kb/s:863.15

encoded 3001 frames, 30.73 fps, 863.25 kb/s

Z:\data>x264.979.exe -r 16 --mixed-refs --b-rdo --bime --weightb --subme 7 -t 2 -8 --me umh --psy-rd 0.8:0.8 --thread-input --threads 6 --progress --pa
ss 1 --b-adapt 2 --bframes 16 --bitrate 784 --stats x264_anime_2_16.stats --output NUL anime.avs
avis [info]: 704x396 @ 23.98 fps (3001 frames)
x264 [warning]: width or height not divisible by 16 (704x396), compression will suffer.
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 PHADD SSE4 Cache64
x264 [info]: slice I:26 Avg QP:15.18 size: 41497 PSNR Mean Y:49.97 U:56.32 V:56.39 Avg:51.20 Global:49.24
x264 [info]: slice P:1531 Avg QP:15.59 size: 6041 PSNR Mean Y:46.66 U:52.91 V:53.18 Avg:47.92 Global:46.77
x264 [info]: slice B:1444 Avg QP:18.62 size: 2035 PSNR Mean Y:45.23 U:52.09 V:52.23 Avg:46.53 Global:45.15
x264 [info]: consecutive B-frames: 24.8% 33.5% 18.7% 6.7% 2.2% 0.4% 4.9% 0.8% 1.5% 2.4% 0.4% 0.0% 2.6% 0.0% 0.0% 0.5% 0.6%
x264 [info]: mb I I16..4..PCM: 17.9% 39.7% 42.5% 0.0%
x264 [info]: mb P I16..4..PCM: 2.4% 5.8% 2.1% 0.0% P16..4: 44.8% 6.2% 8.4% 0.0% 0.0% skip:30.4%
x264 [info]: mb B I16..4..PCM: 0.6% 1.8% 0.5% 0.0% B16..8: 25.0% 1.1% 1.8% direct: 3.9% skip:65.3% L0:45.0% L1:46.6% BI: 8.5%
x264 [info]: final ratefactor: 19.16
x264 [info]: 8x8 transform intra:55.0% inter:28.9%
x264 [info]: ref P L0 83.9% 6.4% 3.3% 1.5% 1.2% 0.8% 0.7% 0.4% 0.4% 0.3% 0.3% 0.2% 0.2% 0.2% 0.1% 0.1%
x264 [info]: ref B L0 74.3% 10.3% 6.0% 2.3% 1.8% 1.1% 0.9% 0.6% 0.6% 0.5% 0.4% 0.4% 0.3% 0.3% 0.3%
x264 [info]: SSIM Mean Y:0.9941612
x264 [info]: PSNR Mean Y:45.998 U:52.543 V:52.752 Avg:47.279 Global:45.925 kb/s:847.84

encoded 3001 frames, 14.07 fps, 847.94 kb/s
Z:\data>

[kemuri-_9]

Quote:
b-adapt 1 (2 pass encode with turbo)
x264 [info]: slice B:8935

b-adapt 2 (otherwise same settings)
x264 [info]: slice B:5918

who is blind? :)

i was talking about the speed of the encodes in respect to each other,
not the number of resulting bframes at all... pay attention plz kthx.

thanks for the confirmation of speeds DarkZell666 - they're at how they're supposed to be.

[kemuri-_9]

here comes some of the comparisons that finished:

x264_r977 --pass 1 --bitrate 784 --stats x264.stats --min-keyint 1 --keyint 500 --scenecut 50 --ref 16 --mixed-refs
--deblock 0:0 --no-fast-pskip --b-adapt _x_ --bframes _y_ --b-bias 0 --b-pyramid --b-rdo --bime --weightb
--direct auto --subme 7 --direct-8x8 -1 --trellis 2 --analyse all --8x8dct --qpmin 1 --qpmax 51 --qpstep 60
--pbratio 1.4 --ratetol 100.0 --qcomp 0.6 --me umh --merange 32 --psy-rd 0.0:0.0 --thread-input --threads 4
--progress --no-dct-decimate --level 5.1 --log-file x264.1pass.log --output NUL open_lossless.avs


b-adapt 1, bframes 2:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:347 Avg QP:13.29 size: 23913 PSNR Mean Y:51.27 U:53.50 V:53.62 Avg:51.71 Global:49.75
x264 [info]: slice P:23467 Avg QP:15.53 size: 4651 PSNR Mean Y:48.18 U:50.77 V:51.00 Avg:48.86 Global:48.05
x264 [info]: slice B:5917 Avg QP:22.05 size: 488 PSNR Mean Y:47.86 U:51.18 V:51.50 Avg:48.69 Global:46.72
x264 [info]: consecutive B-frames: 67.2% 10.3% 22.5%
x264 [info]: mb I I16..4: 33.0% 32.9% 34.1%
x264 [info]: mb P I16..4: 3.3% 5.2% 1.3% P16..4: 55.8% 8.3% 8.2% 0.0% 0.0% skip:18.0%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 24.1% 0.1% 0.1% direct: 0.4% skip:75.0% L0:42.0% L1:57.1% BI: 0.9%
x264 [info]: final ratefactor: 22.44
x264 [info]: 8x8 transform intra:50.5% inter:54.1%
x264 [info]: direct mvs spatial:97.9% temporal:2.1%
x264 [info]: ref P L0 71.4% 8.2% 5.6% 2.2% 2.0% 1.9% 1.7% 0.8% 0.9% 0.8% 0.8% 0.8% 0.8% 0.7% 0.8% 0.5%
x264 [info]: ref B L0 84.6% 6.8% 3.1% 1.0% 0.9% 0.8% 0.6% 0.3% 0.3% 0.3% 0.3% 0.2% 0.2% 0.2% 0.2%
x264 [info]: ref B L1 94.3% 5.7%
x264 [info]: SSIM Mean Y:0.9903959
x264 [info]: PSNR Mean Y:48.151 U:50.887 V:51.133 Avg:48.857 Global:47.764 kb/s:776.30
x264 [info]: encoded 29731 frames, 9.63 fps, 776.30 kb/s


b-adapt 2, bframes 2:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:346 Avg QP:13.17 size: 24596 PSNR Mean Y:51.34 U:53.53 V:53.69 Avg:51.76 Global:49.83
x264 [info]: slice P:18736 Avg QP:14.90 size: 5582 PSNR Mean Y:49.02 U:51.44 V:51.68 Avg:49.66 Global:48.37
x264 [info]: slice B:10649 Avg QP:20.38 size: 667 PSNR Mean Y:47.02 U:50.10 V:50.34 Avg:47.80 Global:47.23
x264 [info]: consecutive B-frames: 42.8% 11.3% 45.9%
x264 [info]: mb I I16..4: 32.7% 32.5% 34.8%
x264 [info]: mb P I16..4: 3.6% 5.7% 1.6% P16..4: 55.6% 9.5% 9.2% 0.0% 0.0% skip:14.8%
x264 [info]: mb B I16..4: 0.5% 0.6% 0.1% B16..8: 23.4% 0.2% 0.2% direct: 1.0% skip:74.0% L0:51.3% L1:47.5% BI: 1.2%
x264 [info]: final ratefactor: 21.52
x264 [info]: 8x8 transform intra:49.5% inter:51.6%
x264 [info]: direct mvs spatial:99.6% temporal:0.4%
x264 [info]: ref P L0 67.7% 9.5% 6.0% 2.6% 2.3% 2.0% 2.0% 1.0% 1.0% 0.9% 0.9% 0.9% 0.9% 0.8% 0.9% 0.6%
x264 [info]: ref B L0 83.9% 7.1% 3.0% 1.2% 0.9% 0.8% 0.6% 0.4% 0.4% 0.3% 0.3% 0.3% 0.3% 0.3% 0.2%
x264 [info]: ref B L1 94.9% 5.1%
x264 [info]: SSIM Mean Y:0.9906227
x264 [info]: PSNR Mean Y:48.332 U:50.983 V:51.226 Avg:49.020 Global:47.937 kb/s:775.41
x264 [info]: encoded 29731 frames, 9.47 fps, 775.41 kb/s



b-adapt 1, bframes 3:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:347 Avg QP:13.35 size: 23773 PSNR Mean Y:51.22 U:53.48 V:53.59 Avg:51.67 Global:49.71
x264 [info]: slice P:22766 Avg QP:15.43 size: 4759 PSNR Mean Y:48.19 U:50.76 V:50.98 Avg:48.86 Global:48.10
x264 [info]: slice B:6618 Avg QP:21.99 size: 553 PSNR Mean Y:47.80 U:51.15 V:51.47 Avg:48.61 Global:46.67
x264 [info]: consecutive B-frames: 66.0% 10.2% 5.2% 18.6%
x264 [info]: mb I I16..4: 33.0% 33.2% 33.8%
x264 [info]: mb P I16..4: 3.4% 5.3% 1.3% P16..4: 55.7% 8.4% 8.4% 0.0% 0.0% skip:17.4%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 27.4% 0.2% 0.2% direct: 0.4% skip:71.5% L0:39.2% L1:59.8% BI: 0.9%
x264 [info]: final ratefactor: 22.36
x264 [info]: 8x8 transform intra:50.6% inter:53.9%
x264 [info]: direct mvs spatial:98.0% temporal:2.0%
x264 [info]: ref P L0 72.1% 7.9% 5.4% 2.2% 1.9% 1.8% 1.7% 0.9% 0.9% 0.8% 0.8% 0.8% 0.7% 0.7% 0.8% 0.5%
x264 [info]: ref B L0 83.2% 6.6% 3.3% 1.4% 1.0% 0.8% 0.8% 0.5% 0.4% 0.3% 0.3% 0.4% 0.3% 0.3% 0.3%
x264 [info]: ref B L1 97.4% 2.6%
x264 [info]: SSIM Mean Y:0.9903856
x264 [info]: PSNR Mean Y:48.137 U:50.874 V:51.119 Avg:48.837 Global:47.754 kb/s:775.81
x264 [info]: encoded 29731 frames, 9.44 fps, 775.81 kb/s


b-adapt 2, bframes 3:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:346 Avg QP:13.16 size: 24688 PSNR Mean Y:51.35 U:53.54 V:53.70 Avg:51.76 Global:49.82
x264 [info]: slice P:18440 Avg QP:14.82 size: 5641 PSNR Mean Y:49.09 U:51.48 V:51.72 Avg:49.72 Global:48.42
x264 [info]: slice B:10945 Avg QP:20.38 size: 687 PSNR Mean Y:46.97 U:50.07 V:50.31 Avg:47.75 Global:47.17
x264 [info]: consecutive B-frames: 42.5% 10.2% 39.8% 7.5%
x264 [info]: mb I I16..4: 32.8% 32.5% 34.7%
x264 [info]: mb P I16..4: 3.6% 5.7% 1.7% P16..4: 55.4% 9.5% 9.3% 0.0% 0.0% skip:14.7%
x264 [info]: mb B I16..4: 0.5% 0.6% 0.1% B16..8: 24.2% 0.2% 0.2% direct: 1.1% skip:73.1% L0:51.2% L1:47.6% BI: 1.2%
x264 [info]: final ratefactor: 21.46
x264 [info]: 8x8 transform intra:49.4% inter:51.4%
x264 [info]: direct mvs spatial:99.5% temporal:0.5%
x264 [info]: ref P L0 67.9% 9.3% 6.0% 2.6% 2.2% 2.0% 2.0% 1.0% 1.0% 0.9% 0.9% 0.9% 0.9% 0.8% 0.9% 0.6%
x264 [info]: ref B L0 84.0% 7.4% 2.6% 1.2% 0.9% 0.7% 0.7% 0.4% 0.4% 0.3% 0.3% 0.3% 0.3% 0.2% 0.2%
x264 [info]: ref B L1 95.8% 4.2%
x264 [info]: SSIM Mean Y:0.9906133
x264 [info]: PSNR Mean Y:48.333 U:50.985 V:51.224 Avg:49.021 Global:47.928 kb/s:774.72
x264 [info]: encoded 29731 frames, 9.08 fps, 774.72 kb/s



b-adapt 1, bframes 4:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:347 Avg QP:13.37 size: 23730 PSNR Mean Y:51.20 U:53.45 V:53.58 Avg:51.65 Global:49.70
x264 [info]: slice P:22440 Avg QP:15.37 size: 4817 PSNR Mean Y:48.19 U:50.75 V:50.97 Avg:48.86 Global:48.13
x264 [info]: slice B:6944 Avg QP:22.13 size: 531 PSNR Mean Y:47.84 U:51.15 V:51.49 Avg:48.65 Global:46.66
x264 [info]: consecutive B-frames: 65.4% 10.2% 5.3% 5.4% 13.7%
x264 [info]: mb I I16..4: 32.9% 33.3% 33.8%
x264 [info]: mb P I16..4: 3.4% 5.4% 1.3% P16..4: 55.7% 8.5% 8.4% 0.0% 0.0% skip:17.2%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 23.9% 0.2% 0.2% direct: 0.5% skip:75.0% L0:42.6% L1:56.3% BI: 1.1%
x264 [info]: final ratefactor: 22.31
x264 [info]: 8x8 transform intra:50.7% inter:53.9%
x264 [info]: direct mvs spatial:98.0% temporal:2.0%
x264 [info]: ref P L0 72.3% 8.0% 5.3% 2.1% 1.9% 1.8% 1.7% 0.8% 0.9% 0.8% 0.8% 0.8% 0.7% 0.7% 0.8% 0.5%
x264 [info]: ref B L0 81.5% 7.5% 3.6% 1.4% 1.1% 0.9% 0.9% 0.5% 0.4% 0.4% 0.4% 0.4% 0.4% 0.3% 0.3%
x264 [info]: ref B L1 96.2% 3.8%
x264 [info]: SSIM Mean Y:0.9903619
x264 [info]: PSNR Mean Y:48.147 U:50.875 V:51.125 Avg:48.845 Global:47.752 kb/s:774.24
x264 [info]: encoded 29731 frames, 9.48 fps, 774.24 kb/s


b-adapt 2, bframes 4:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:346 Avg QP:13.17 size: 24672 PSNR Mean Y:51.34 U:53.54 V:53.70 Avg:51.76 Global:49.82
x264 [info]: slice P:18328 Avg QP:14.79 size: 5658 PSNR Mean Y:49.10 U:51.49 V:51.73 Avg:49.74 Global:48.43
x264 [info]: slice B:11057 Avg QP:20.37 size: 689 PSNR Mean Y:46.97 U:50.07 V:50.32 Avg:47.76 Global:47.17
x264 [info]: consecutive B-frames: 42.3% 9.5% 40.8% 4.0% 3.3%
x264 [info]: mb I I16..4: 32.8% 32.5% 34.8%
x264 [info]: mb P I16..4: 3.6% 5.7% 1.7% P16..4: 55.4% 9.5% 9.3% 0.0% 0.0% skip:14.7%
x264 [info]: mb B I16..4: 0.5% 0.6% 0.1% B16..8: 23.9% 0.2% 0.2% direct: 1.1% skip:73.4% L0:51.2% L1:47.6% BI: 1.2%
x264 [info]: final ratefactor: 21.43
x264 [info]: 8x8 transform intra:49.3% inter:51.3%
x264 [info]: direct mvs spatial:99.8% temporal:0.2%
x264 [info]: ref P L0 68.0% 9.4% 6.0% 2.5% 2.2% 2.0% 2.0% 1.0% 1.0% 0.9% 0.9% 0.9% 0.9% 0.8% 0.9% 0.6%
x264 [info]: ref B L0 83.3% 7.4% 2.9% 1.2% 1.0% 0.8% 0.7% 0.4% 0.4% 0.3% 0.4% 0.4% 0.3% 0.3% 0.2%
x264 [info]: ref B L1 94.4% 5.6%
x264 [info]: SSIM Mean Y:0.9906084
x264 [info]: PSNR Mean Y:48.335 U:50.984 V:51.225 Avg:49.023 Global:47.929 kb/s:773.22
x264 [info]: encoded 29731 frames, 8.65 fps, 773.22 kb/s


sets for 5 and 6 bframes still going, will post after i wake up.

[qyqgpower]

SSIM keeps dropping while bframes being increased.
b-adapt 1
bframes 2: 0.9903959
bframes 3: 0.9903856
bframes 4: 0.9903619
bframes 16: 0.9903609

b-adapt 2
bframes 2: 0.9906227
bframes 3: 0.9906133
bframes 4: 0.9906084
bframes 16: 0.9905993

From what I've learned, SSIM reflects more subjective quality than PSNR. So you get worse quality at same bitrate(2pass mode) with more bframes?

And why b-adapt 2 produce lower QP, larger size of bframes with lower PSNR than b-adapt 1?
x264 [info]: slice B:7120 Avg QP:21.99 size: 538 PSNR Mean Y:48.05 U:51.28 V:51.60 Avg:48.77 Global:46.70
x264 [info]: slice B:11211 Avg QP:20.38 size: 697 PSNR Mean Y:47.00 U:50.10 V:50.35 Avg:47.78 Global:47.14

[Sagekilla]

With more bframes is the bitrate being decreased? And what of PSNR for the different values? You have to realize that B's are typically encoded with a higher quant than a P or I frame, so that could be the reason for the slightly lower SSIM. Plus, that difference is too little to be anything of note. If you're seeing 0.99 --> 0.98 decrease, we have huge issues.

[IgorC]

SSIM keeps dropping while bframes being increased.
b-adapt 1
bframes 2: 0.9903959
bframes 3: 0.9903856
bframes 4: 0.9903619
bframes 16: 0.9903609

b-adapt 2
bframes 2: 0.9906227
bframes 3: 0.9906133
bframes 4: 0.9906084
bframes 16: 0.9905993

From what I've learned, SSIM reflects more subjective quality than PSNR. So you get worse quality at same bitrate(2pass mode) with more bframes?

And why b-adapt 2 produce lower QP, larger size of bframes with lower PSNR than b-adapt 1?
x264 [info]: slice B:7120 Avg QP:21.99 size: 538 PSNR Mean Y:48.05 U:51.28 V:51.60 Avg:48.77 Global:46.70
x264 [info]: slice B:11211 Avg QP:20.38 size: 697 PSNR Mean Y:47.00 U:50.10 V:50.35 Avg:47.78 Global:47.14
Looking at ssim result maybe bitrate is enough high and b-frames are rather harlmless. And the difference in ssim values isn't relevant.

[Sagekilla]

The difference in SSIM falls within the range of inaccuracy -- the 3 ending digits aren't too particularly useful for comparison.

[G_M_C]

@ DarkZell666;
What/whose x264-build are you using ?

[DarkZell666]

@ DarkZell666;
What/whose x264-build are you using ?



Ooops, good question, it's the r979 from x264.nl :)
(Post edited accordingly)

[Quark.Fusion]

SSIM keeps dropping while bframes being increased.
b-adapt 1
bframes 2: 0.9903959
bframes 3: 0.9903856
bframes 4: 0.9903619
bframes 16: 0.9903609

b-adapt 2
bframes 2: 0.9906227
bframes 3: 0.9906133
bframes 4: 0.9906084
bframes 16: 0.9905993

From what I've learned, SSIM reflects more subjective quality than PSNR. So you get worse quality at same bitrate(2pass mode) with more bframes?

And why b-adapt 2 produce lower QP, larger size of bframes with lower PSNR than b-adapt 1?
x264 [info]: slice B:7120 Avg QP:21.99 size: 538 PSNR Mean Y:48.05 U:51.28 V:51.60 Avg:48.77 Global:46.70
x264 [info]: slice B:11211 Avg QP:20.38 size: 697 PSNR Mean Y:47.00 U:50.10 V:50.35 Avg:47.78 Global:47.14
SSIM is almost the same, b-adapt slightly better.
Maybe b-adapt 2 produces more pyramids, they get lower QP, p-frames also get lower QP that b-frames.
Global PSNR is higher and PSNR is more useless that SSIM.

[Quark.Fusion]

As I didn't found dedicated b-pyramids thread can I request --bpratio switch to control b-pyramid QP here?

[Soichiro]

A bit late, but regarding the speed of the test encodes posted on the previous page, they're all the same/similar because those were 2nd pass stats, thus, b-adapt type wouldn't have any effect on the speed.

[Audionut]

here comes some of the comparisons that finished......

sets for 5 and 6 bframes still going, will post after i wake up.

Thanks.

[qyqgpower]

Another suggestion: since in 2pass mode, the result of b-adapt 2 is largely affected by the quality of 1st pass, and none of us(at least I think so:)) want to waste time on 1st pass.
We should find out a optimal 1st pass setting, which let b-adapt2 make ideal frame decision, without sacrifice more speed on unnecessary options.
It can be called "Turbo 1st pass for b-adapt 2" or something else ;)

[kemuri-_9]

b-adapt 1, bframes 5:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:347 Avg QP:13.37 size: 23770 PSNR Mean Y:51.21 U:53.45 V:53.58 Avg:51.65 Global:49.71
x264 [info]: slice P:22314 Avg QP:15.36 size: 4844 PSNR Mean Y:48.18 U:50.74 V:50.96 Avg:48.85 Global:48.13
x264 [info]: slice B:7070 Avg QP:22.09 size: 533 PSNR Mean Y:47.86 U:51.19 V:51.51 Avg:48.66 Global:46.68
x264 [info]: consecutive B-frames: 65.1% 10.2% 5.2% 5.4% 9.2% 4.9%
x264 [info]: mb I I16..4: 32.9% 33.2% 33.9%
x264 [info]: mb P I16..4: 3.4% 5.5% 1.3% P16..4: 55.7% 8.5% 8.5% 0.0% 0.0% skip:17.0%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 24.0% 0.2% 0.2% direct: 0.5% skip:74.9% L0:41.9% L1:57.0% BI: 1.1%
x264 [info]: final ratefactor: 22.30
x264 [info]: 8x8 transform intra:50.6% inter:53.8%
x264 [info]: direct mvs spatial:98.1% temporal:1.9%
x264 [info]: ref P L0 72.4% 7.9% 5.2% 2.1% 1.9% 1.8% 1.7% 0.8% 0.9% 0.8% 0.8% 0.8% 0.7% 0.7% 0.8% 0.5%
x264 [info]: ref B L0 80.7% 7.8% 3.6% 1.5% 1.2% 1.0% 0.9% 0.6% 0.5% 0.4% 0.4% 0.4% 0.4% 0.4% 0.3%
x264 [info]: ref B L1 96.3% 3.7%
x264 [info]: SSIM Mean Y:0.9903629
x264 [info]: PSNR Mean Y:48.141 U:50.875 V:51.122 Avg:48.839 Global:47.752 kb/s:774.78
x264 [info]: encoded 29731 frames, 9.50 fps, 774.78 kb/s


b-adapt 2, bframes 5:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:346 Avg QP:13.17 size: 24704 PSNR Mean Y:51.34 U:53.55 V:53.70 Avg:51.76 Global:49.81
x264 [info]: slice P:18279 Avg QP:14.78 size: 5670 PSNR Mean Y:49.12 U:51.50 V:51.74 Avg:49.75 Global:48.44
x264 [info]: slice B:11106 Avg QP:20.40 size: 688 PSNR Mean Y:46.95 U:50.06 V:50.30 Avg:47.74 Global:47.15
x264 [info]: consecutive B-frames: 42.3% 9.6% 40.2% 3.8% 2.1% 2.0%
x264 [info]: mb I I16..4: 32.8% 32.5% 34.7%
x264 [info]: mb P I16..4: 3.6% 5.7% 1.7% P16..4: 55.4% 9.6% 9.4% 0.0% 0.0% skip:14.7%
x264 [info]: mb B I16..4: 0.5% 0.6% 0.1% B16..8: 23.8% 0.2% 0.2% direct: 1.1% skip:73.4% L0:51.1% L1:47.7% BI: 1.2%
x264 [info]: final ratefactor: 21.42
x264 [info]: 8x8 transform intra:49.3% inter:51.2%
x264 [info]: direct mvs spatial:99.8% temporal:0.2%
x264 [info]: ref P L0 68.1% 9.3% 5.9% 2.5% 2.2% 2.0% 2.0% 1.0% 1.0% 0.9% 0.9% 0.9% 0.9% 0.8% 0.9% 0.6%
x264 [info]: ref B L0 83.1% 7.6% 2.9% 1.2% 1.0% 0.8% 0.7% 0.5% 0.4% 0.3% 0.4% 0.4% 0.3% 0.3% 0.2%
x264 [info]: ref B L1 95.0% 5.0%
x264 [info]: SSIM Mean Y:0.9906032
x264 [info]: PSNR Mean Y:48.333 U:50.986 V:51.226 Avg:49.021 Global:47.922 kb/s:773.08
x264 [info]: encoded 29731 frames, 8.47 fps, 773.08 kb/s


b-adapt 1, bframes 6:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:347 Avg QP:13.35 size: 23794 PSNR Mean Y:51.22 U:53.48 V:53.59 Avg:51.66 Global:49.71
x264 [info]: slice P:22301 Avg QP:15.36 size: 4846 PSNR Mean Y:48.17 U:50.72 V:50.94 Avg:48.83 Global:48.13
x264 [info]: slice B:7083 Avg QP:22.06 size: 534 PSNR Mean Y:47.92 U:51.23 V:51.56 Avg:48.72 Global:46.68
x264 [info]: consecutive B-frames: 65.1% 10.2% 5.2% 5.4% 9.2% 3.9% 1.0%
x264 [info]: mb I I16..4: 32.8% 33.2% 34.0%
x264 [info]: mb P I16..4: 3.4% 5.5% 1.4% P16..4: 55.8% 8.5% 8.5% 0.0% 0.0% skip:17.0%
x264 [info]: mb B I16..4: 0.1% 0.1% 0.0% B16..8: 24.0% 0.2% 0.2% direct: 0.6% skip:74.8% L0:41.9% L1:57.1% BI: 1.1%
x264 [info]: final ratefactor: 22.30
x264 [info]: 8x8 transform intra:50.6% inter:53.8%
x264 [info]: direct mvs spatial:98.0% temporal:2.0%
x264 [info]: ref P L0 72.4% 7.9% 5.2% 2.1% 1.9% 1.8% 1.7% 0.8% 0.9% 0.8% 0.8% 0.8% 0.7% 0.7% 0.8% 0.5%
x264 [info]: ref B L0 80.6% 7.8% 3.7% 1.5% 1.2% 1.0% 0.9% 0.6% 0.5% 0.4% 0.4% 0.5% 0.4% 0.4% 0.3%
x264 [info]: ref B L1 96.2% 3.8%
x264 [info]: SSIM Mean Y:0.9903634
x264 [info]: PSNR Mean Y:48.143 U:50.870 V:51.119 Avg:48.841 Global:47.753 kb/s:774.93
x264 [info]: encoded 29731 frames, 9.64 fps, 774.93 kb/s


b-adapt 2, bframes 6:

x264 [info]: using cpu capabilities: MMX2 SSE2Fast
x264 [info]: slice I:346 Avg QP:13.16 size: 24736 PSNR Mean Y:51.35 U:53.55 V:53.71 Avg:51.76 Global:49.82
x264 [info]: slice P:18244 Avg QP:14.77 size: 5678 PSNR Mean Y:49.12 U:51.50 V:51.74 Avg:49.75 Global:48.45
x264 [info]: slice B:11141 Avg QP:20.39 size: 693 PSNR Mean Y:46.95 U:50.06 V:50.31 Avg:47.74 Global:47.14
x264 [info]: consecutive B-frames: 42.3% 9.5% 40.0% 3.4% 2.3% 1.0% 1.6%
x264 [info]: mb I I16..4: 32.8% 32.5% 34.8%
x264 [info]: mb P I16..4: 3.6% 5.7% 1.7% P16..4: 55.4% 9.6% 9.4% 0.0% 0.0% skip:14.6%
x264 [info]: mb B I16..4: 0.5% 0.6% 0.1% B16..8: 23.8% 0.2% 0.2% direct: 1.1% skip:73.4% L0:51.2% L1:47.5% BI: 1.2%
x264 [info]: final ratefactor: 21.41
x264 [info]: 8x8 transform intra:49.3% inter:51.3%
x264 [info]: direct mvs spatial:99.8% temporal:0.2%
x264 [info]: ref P L0 68.1% 9.4% 5.9% 2.5% 2.2% 2.0% 2.0% 1.0% 1.0% 0.9% 0.9% 0.9% 0.9% 0.8% 0.9% 0.6%
x264 [info]: ref B L0 83.4% 7.5% 2.7% 1.3% 1.0% 0.8% 0.7% 0.5% 0.4% 0.3% 0.4% 0.3% 0.3% 0.3% 0.2%
x264 [info]: ref B L1 94.2% 5.8%
x264 [info]: SSIM Mean Y:0.9906044
x264 [info]: PSNR Mean Y:48.334 U:50.986 V:51.227 Avg:49.022 Global:47.923 kb/s:773.29
x264 [info]: encoded 29731 frames, 8.47 fps, 773.29 kb/s


and for what everyone has already been discussing, the general trend analaysis (and a response):
With more bframes is the bitrate being decreased? And what of PSNR for the different values? You have to realize that B's are typically encoded with a higher quant than a P or I frame, so that could be the reason for the slightly lower SSIM. Plus, that difference is too little to be anything of note. If you're seeing 0.99 --> 0.98 decrease, we have huge issues.

since in my options i told x264 it could fiddle with the average bitrate as it encoded it was allowed to fluctuate as it was churning each time. (see --ratetol 100)
and the general trend is that with more b-frames the bitrate is dropping with having lower qps (higher quality) (see QP averages and final ratefactor)
b-adapt 2 is totally kicking b-adapt 1's butt in Avg QPs and still having a little lower average bitrate.

comparing the b-adapt 1 w/ 16 bframes against the b-adapt 2 even w/ only 2 bframes,
b-adapt 2 has significantly lower average QPs even with a drop in 14 max bframes.
the quality is much better even with taking the small average bitrate adjustment into account

[audyovydeo]

comparing the b-adapt 1 w/ 16 bframes against the b-adapt 2 even w/ only 2 bframes

can you post the speeds you got with these two settings ?
personally I consider speed as part of the "sacred encoding trilogy".


thanks
a/v

[kemuri-_9]

can you post the speeds you got with these two settings ?
personally I consider speed as part of the "sacred encoding trilogy".

thanks
a/v

the encoding fps rates are included in the postings
see the last x264 [info]: line for each block of output

[audyovydeo]

the encoding fps rates are included in the postings
see the last x264 [info]: line for each block of output

But nowhere did I see "1 w/ 16 bframes against the b-adapt 2 even w/ only 2 ", only other combinations.
Unless it was typos on your part.

cheers
a/v

[kemuri-_9]

But nowhere did I see "1 w/ 16 bframes against the b-adapt 2 even w/ only 2 ", only other combinations.
Unless it was typos on your part.

cheers
a/v

those posts are a bit far back now:
16 bframes: http://forum.doom9.org/showthread.php?p=1184868#post1184868
2 bframes: http://forum.doom9.org/showthread.php?p=1185229#post1185229

[Audionut]

Thanks kemuri-_9, I simply don't have the time to do a detailed test.

Good results. I'll have to find some time to run a few quick tests for myself