x264 development

[Manao]

By making the quantizer of bref lower than non-references bframes.

Moreover, bframes aren't only efficient because they allow to raise the quantizer : a clip with b & p at same quantizers will have a higher PSNR than a clip without bframes.

That allows bref, even if they are less quantized than 'normal' bframes, to raise yet again the quality.

[Tommy Carrot]

Thanks, that's what i thought too. :)

[sysKin]

Originally posted by Tommy Carrot
I'm curious, if a b-frame can be used as reference for other frames, the higher quantization of the reference b-frame wont hurt the overall quality? One of the advantage of the b-frame is that you can safely compress it more because it's not used as reference, hence it doesn't influence the quality of other frames. But this will change with pyramid b-frames. Could someone explain me how can x264 avoid this problem? Because AVC allows multiple references in every direction, any block has a choice, whether to motion compensate from closer (but possibly lower quality) picture, or from further away, but lower quant, pframe. So it's simply about extra choice :)

Radek

[Koti]

If you encode with the bframe pyramid on, it will not play in the latest ffdshow. Likely it will stall after a couple frames.

Hehe, Yep ffdshow stall - crash , but nero's avc decoder works with b frame pyramid.

[TheUnforgiven]

Great work
but could somebody add a status window or at least a simple debug output to the vfw version.
i hate to encode blindly!

[celtic_druid]

Someone want to test this? http://celticdruid.no-ip.com/test/libavcodec.bpyramid.7z

patched libavcodec for ffdshow. Works ok decoding bframe pyramid encodes here. Although I didn't test much.

[Yong]

Here's the info from mailing list:
r152 | pengvado | 2005-03-05 05:16:05 +0100 (Sat, 05 Mar 2005) | 2
lines
Changed paths:
M /trunk/common/common.h
M /trunk/encoder/encoder.c
M /trunk/encoder/ratecontrol.c
M /trunk/vfw/config.c

allow 16 refs (instead of 15)

--
This is the x264-devel mailing-list
I tried with mencoder with frameref=16, but mencoder still complaining frameref>16 and stop the encoding job...:(
EDIT: i use x264r153 and mplayer daily CVS build.

[Teegedeck]

Originally posted by celtic_druid
Someone want to test this? http://celticdruid.no-ip.com/test/libavcodec.bpyramid.7z

patched libavcodec for ffdshow. Works ok decoding bframe pyramid encodes here. Although I didn't test much.
It works flawlessly, even on a clip with max. 10 ref. frames and max-idr=500. Thanks a lot!

[akupenguin]

Originally posted by Yong
I tried with mencoder with frameref=16, but mencoder still complaining frameref>16 and stop the encoding job... I haven't patched mencoder to loosen that restriction, since 15 B-frames is overkill too. This is just to prevent a crash found by Spyder, who is writing his own frontend.

[Yong]

Originally posted by akupenguin
I haven't patched mencoder to loosen that restriction, since 15 B-frames is overkill too. This is just to prevent a crash found by Spyder, who is writing his own frontend.
I haven't test with b-frames=16,
but what i say is frameref=16:)

[Sharktooth]

Can someone please test my latest build (rev156) with the celtic druid's patch?

[mezzanine]

Originally posted by Sharktooth
Can someone please test my latest build (rev156) with the celtic druid's patch?

Just did, works fine.

[Sharktooth]

good ;)

[scorpdt]

Works fine on my end too.

[celtic_druid]

If you want 16 ref frames then you need to change ve_x264.c max ref value from 15 to 16 which I did in my last mencoder compile anyway.

About the VfW GUI. I shifted some stuff round. Anyone got any comments/advice on that? Just thought that with the two new bframes settings added it should have its own section.

[Chainmax]

I have a couple of questions that are probably extremely n00bish, so please bear with me. From what I read all this time, AVC used to have blocking issues on bframes and that was due to lack of deblicking on the decoding end. Will postprocessing be always necessary? I never play my Xvid encodes with postprocessing, and I'd like to do the same with future AVC encodes, will that be possible? Also, is there any chance that x264 can output .mp4 files since AVC in AVI is such a bad idea according to the FAQ?

[Doom9]

Also, is there any chance that x264 can output .mp4 files since AVC in AVI is such a bad idea according to the FAQ?Please have a look at the x264 sticky: http://forum.doom9.org/showthread.php?s=&threadid=89979

There you can find a quote from yours truly that applies here: x264 is encoding application agnostic. The output format is encoding application dependant, not codec dependant. VDub and every other VfW based tool will give you video in the AVI container. mencoder, can also give it to your RAW. ffmpeg can output to MP4, but that feature is kinda broken, and currently x264 is not integrated into ffmpeg. So, there is no application that can encode using the x264 codec and put the video data into the MP4 container. You'll need to either create a raw output and mux that into MP4 using a tool like mp4creator, or make an AVI, extract the AVC video using avi2raw, then mux the raw data into MP4. The first approach is used by MeGUI (http://forum.doom9.org/showthread.php?s=&threadid=87567), the 2nd by AutoAC (http://forum.doom9.org/showthread.php?s=&threadid=90732) and mp4muxer (http://forum.doom9.org/showthread.php?s=&threadid=89803). Now the choice is yours.

Note that even though mp4creator supports AVI input, it cannot handle the tricks used to get B-frames into AVI, so don't use mp4creator with AVI input.. always go via RAW. The tools mentioned above will do that automatically for you.

[Tommy Carrot]

Originally posted by Chainmax
I have a couple of questions that are probably extremely n00bish, so please bear with me. From what I read all this time, AVC used to have blocking issues on bframes and that was due to lack of deblicking on the decoding end. Will postprocessing be always necessary? I never play my Xvid encodes with postprocessing, and I'd like to do the same with future AVC encodes, will that be possible? Unfortunately the blocking in h.264 is much more visible and annoying than in ASP, so deblocking is necessary. The best you can do is to weaken the deblocking as far as you can while the image is still relatively clean, but completely disabling the loop-filter is not a good idea imo. Akupenguin hinted though that high profile may bring an improvement in this field.

[Doom9]

Unfortunately the blocking in h.264 is much more visible and annoying than in ASP, so deblocking is necessary. The best you can do is to weaken the deblocking as far as you can while the image is still relatively clean, but completely disabling the loop-filter is not a good idea imo. Should that even be possible? I thought the in-loop filter was a given for AVC. And to the best of my knowledge, neither Nero AVC nor VSSH actually has postprocessing in their decoder (I know for sure for Nero), but I figure the in-loop filter is still active because they don't have any issues with B-frames, even in the absence of dedicated postprocessing in the sense we know it from MPEG-4 ASP codecs.

And, I only have limited experience with x264 in that area, but when I disabled postprocessing in ffdshow, the movie I watched looked much more visually pleasing.. postprocessing introduced a lot of visible artifacts. So I guess I should watch that same movie using the Nero filters to check for a difference.. and it would probably also be interesting to see how the Nero filters do playing contents where ffdshow has/had those blocking issues.

[netchris]

Nero decoder should be more complete compatibility wise, though i dont have any problems decoding my nero-avc encodes with ffdshow.
Its true that even minimal ffdshow postprocessing only harms the picture. Besides postprocessing aint needed in avc (thanks to inloop magic).

[akupenguin]

Originally posted by Chainmax
From what I read all this time, AVC used to have blocking issues on bframes and that was due to lack of deblicking on the decoding end. Will postprocessing be always necessary?
The deblocking that was lacking in ffdshow was in-loop filtering, not postprocessing. It is not a decoder choice; any decoder that doesn't perform deblocking (when specified by the encoder) is not standards compliant.

At this time, there are no known postprocessors for H.264, and using a postprocessor designed for ASP will only reduce the video quality.

[hpn]

Just tested the ST's rev.157. The so called "B pyramid" is now officially available in the VFW under the "Use as a reference" option (correct me if I'm wrong). Max b-frames is limited to 5, hopefully temporarily until the bframes code gets tweaked in the future (more b-frames always sounds tempting). For now to enable "Use as a reference" is a bad idea cause ffdshow is still unable to play such encodes (hopefully also fixed soon). MPlayer playes such encodes fine, as long as you use less than 15 ref. frames. For 15 and 16 refs and "Use as a reference" option enabled, Mplayer crashes stating: "MPlayer interrupted by signal 11 in module: decode_video, - MPlayer crashed by bad usage of CPU/FPU/RAM." I have no idea if it's a bug in the encoder or Mplayer (same bug with the patched 156 yesterday). And finally I still wonder what was the reason for upping the Max ref. frames from 15 to only 16 (why not to 24 for example). Either 15 or 16 is overkill anyway, so I don't get it. I guess theoretically we don't have an upper limit for ref. frames, as long as the encoding time remains reasonably low and real time playback of the resulting encode can be achieved.

[akupenguin]

16 is the max allowed by the H.264 standard. It was previously limited to 15 because 16 crashed the encoder.

[hpn]

That explains everything, thank you, akupenguin. Just keep up the nice work.
(I wouldn't be surprised if the "Moving Picture Experts Group" guys are already secretely working on some next gen. standard)

[Doom9]

The deblocking that was lacking in ffdshow was in-loop filtering, not postprocessing. It is not a decoder choice; any decoder that doesn't perform deblocking (when specified by the encoder) is not standards compliant.Thanks for the confirmation. So, until recently ffdshow's AVC decoder was not specs compliant and hence the b-frame quality problems.

and using a postprocessor designed for ASP will only reduce the video quality.I can absolutely confirm that, so make sure you disabled postprocessing in ffdshow when watching x264, or other AVC content. I guess it would be nice if that were done automatically considering that having it turned on only harms the output.

[JimiK]

First, also from me a big thanks to all people involved in this project!
Should referring frames look better when B-Frames are also used as reference?
It does not seem to decrease filesize. At least in my (short) test the size increased always by a very slight amount when B-Frames were used as reference.
In the mplayer manpages also stood that it would introduce an additional delay of 2 frames (maybe of #max_bframes frames?). I couldn't check this with mp4, because VDubMod as my editing program can only open avi. With Avi there are #max_bframes as dummy frames insertet, no matter if B-Frames are used as reference or not.

[stephanV]

AFAIK b-refs should not necessarily introduce an additional delay when b-frames were already used.

How useful b-refs will be still has to be seen. For anime it seems to work, for real content it is not sure.

[Sagittaire]

VFWx264 rev 157 (celtic_druid)

With ffdshow or ateme dec
defaut + 16 refs: bug for player (MPC)
defaut + 15 refs: no bug
defaut + bframe ref: bug for player (MPC)

with Nero AVC dec:
defaut + 16 refs + bframe ref: no bug but very high CPU level for player (MPC)

[Sharktooth]

@sagittaire: What decoder you use?

[Sagittaire]

VFWx264 rev 157 (celtic_druid)

With ffdshow or ateme dec
defaut + 16 refs: bug for player (MPC)
defaut + 15 refs: no bug
defaut + bframe ref: bug for player (MPC)

with Nero AVC dec:
defaut + 16 refs + bframe ref: no bug but very high CPU level for player (MPC). Video fluidity is worst.

[bond]

Originally posted by stephanV
[B]AFAIK b-refs should not necessarily introduce an additional delay when b-frames were already usedi dunno in what way b-frames as references makes arbitrary frame orders necessary, but with .avi you get one additional delay caused by the reordering (as done by x264 atm), which .avi isnt really capable to handle

of course with better containers, like .mkv and .mp4, which are able to handle arbitrary frame orders explicitely, this shouldnt happen

[Tommy Carrot]

But even with avi, it shouldn't introduce more delay. If the delay is equal to the maximum consecutive b-frame number, all of them are already buffered, so the order of them shouldn't matter.

[bond]

(23:35:02) (@pengvado) bond: avi doesn't support arbitrary frame orders any more or less than it has always supported unpacked B-frames.
(23:35:34) (bond) yep, but in practice it seems to work!?
(23:35:40) (@pengvado) the only thing wrong with AVC-in-AVI is timestamps / A-V sync.
(23:35:59) (@pengvado) the codec has sufficient info to reorder the frames for output
(23:36:32) (bond) hm can you explain how this works now using arbitrary orders with x264 in .avi?
(23:36:34) (@pengvado) however, since the only things that AVI does are seking and A-V sync, losing half of that does make it "broken" for all I care
(23:38:09) (@pengvado) x264 creates an elementary stream and hands it to mencoder. mencoder adds AVI headers and writes it to a file. mplayer reads the file, throws away the AVI headers, and gives the elementary stream to libavcodec. libavcodec reorders frames based on H.264 headers, and outputs in the correct order with undefined timestamps.
(23:38:45) (bond) hm and how would the reordering work out in practice, eg if decoding in vfw
(23:39:00) (bond) i mean with that 1 frame in/1 frame out limit
(23:39:23) (@pengvado) the first frame is output 3 times, the last 2 frames are dropped, and you get a 80ms A-V desync.
(23:39:38) (bond) ah so simply a bigger delay?
(23:39:58) (@pengvado) yes

[Tommy Carrot]

Akupenguin obviously knows a lot more about this stuff than me, but i still don't get it why should a bigger delay be necessary?

The delay is caused by a buffer to reorder the frames, because a reference frame must be placed before the frames which are using it as reference. let's see an example with 3 bframes:I b b b P
1 2 3 4 5 ; frames in original order
You have to reorder it to:I P b b b
1 5 2 3 4 ; after the reorderingYou need to buffer 3 frames, because the P-frame moves 3 frames away from its original place.

With bref this will be:I b B b P ; B is the b-frame which is used as reference
1 2 3 4 5This is stored like this:I P B b b
1 5 3 2 4As you see, you still only need to buffer 3 frames, i don't see the reason of the additional delay.

[Sagittaire]

hmm ... x264 in mp4 or mkv container don't work very well with multiref + bref ...

Perhabs muxing process:
avi2raw.exe -v x264-447.avi clip.h264
mp4creator.exe -c clip.h264 -rate 25 clip.mp4

if I use remux ateme exe result is the same: video playback is not very good with multiref + bref

[bond]

Originally posted by Sagittaire
hmm ... x264 in mp4 or mkv container don't work very well with multiref + bref ...i get this too when using bref (not with mref alone). i think thats caused by ffdshow. try the nero decoder, wait for ffdshow to be officially updated by milan to the latest ffmpeg sources or use latest mplayer

[stephanV]

Tommy Carrot: thats what i thought too :)

for 2 cons b-frames with b-ref it might be necessary though... IIRC the frame sequence will always be like this then:

1 2 3 4
I b B P

1 4 3 2
I P B b

thus three frames delay needed...

[Sharktooth]

I had some spare time this evening so i added some CPU optimized builds back to the x264 daily builds thread.
If you have some request for a particular version (SSE3?) just ask, i have still a couple of free hours.

[Sagittaire]

x264 and variability:


|---------------|---------|---------|
| Variability | OPSNR | SSIM |
|---------------|---------|---------|
| 000% | 39.3871 | 64.05 |
| 010% | 39.6695 | 65.23 |
| 020% | 39.9377 | 66.31 |
| 030% | 40.1266 | 67.08 |
| 040% | 40.2793 | 67.70 |
| 050% | 40.3925 | 68.19 |
| 060% | 40.4644 | 68.50 |
| 070% | 40.5003 | 68.66 |
| 080% | 40.4851 | 68.63 |
| 090% | 40.4357 | 68.51 |
| 100% | 40.3381 | 68.13 |
|---------------|---------|---------|


Optimal setting for this source and this quant is 75%. Optimal variability is perhabs not constant (change perhabs with source and average quant level) ...

[Sharktooth]

Uhm, what source?

[virus]

Originally posted by Sagittaire
x264 and variability
This is quite consistent with my own tests. If you want to optimize for Average PSNR, something around 40-50% should be good. If instead you want to optimize for Overall PSNR, you need to have a more "constant quantizer" behaviour, and so high values of variability are needed (70-90%).

The actual default (60%) should instead be a good starting point to find the maximum visual quality... but since visual quality is totally subjective (and things change when you change source and/or bitrate), that 60% is really just an hint. If some consensus can be reached on a different default which "looks better" on a wide range of sources we may ask akupenguin to change it. Rememeber that akupenguin tests on anime, so I'd suggest to investigate natural material... ;)

virus

[Tommy Carrot]

Originally posted by stephanV
Tommy Carrot: thats what i thought too :)

for 2 cons b-frames with b-ref it might be necessary though... IIRC the frame sequence will always be like this then:

1 2 3 4
I b B P

1 4 3 2
I P B b

thus three frames delay needed... Why? No matter how i see this, 2 frames delay should be enough in this case. And i know this doesn't matter here, as this is a purely theoretical discussion but doesn't x264 only uses b-ref with 3 or more consecutive b-frames?

Anyway, the question is still: why should b-ref introduce more delay?

[DeathTheSheep]

16 is the max allowed by the H.264 standard. It was previously limited to 15 because 16 crashed the encoder.

Really?!! I used more than 30 reference frames with revision 144 (or 139, I forgot) and it worked perfectly. There was a small file decrease over using 15.

I never experianced a crash until I tried to use 1000 (just for a test), and only then after the encode was nearly half finished. I think its limited to the amont of ram available, but by god i'm not sure and I know nothing.

Strange, though, how it actually did work for me. And yes, pyramiding does crash ffdshow.

Sheers. I mean, Cheers, y'all. Baa

[Sirber]

16 has always been working for me :confused:

[Manao]

Really?!! I used more than 30 reference frames with revision 144 (or 139, I forgot) and it worked perfectly. There was a small file decrease over using 15.Lol, amazing considering that the norm doesn't allow more than 16 references, and that x264 didn't allow more than 15 references before rev 152.

Moreover, and for the nth time, you won't notice any improvements with more than 5 references, and in fact, more than 2 references is often plainly useless considering the slowdown it creates.

[akupenguin]

Originally posted by Sirber
16 has always been working for me :confused: As with many of the options, if you select an invalid value, x264 clips it into the valid range. So your encode actually only used 15 refs.

Anyway, I increased the max not because 16 makes any difference, but because the limatation to 15 was caused by a bug.

[DeathTheSheep]

I've just looked back on an older version of x264 Revision 94. It seems to have a better compression ratio than the current one, and its .dll is much larger. Why is this? Seriously, with q=26 and similar options in both, the older version compressed better. Is this just because I'm doing something wrong (as usual?)?
Also wierd.



Yeah, and that reference thing... makes sense now that you mention the automatic value truncation, but that didn't work in older ones, I don't think, because there actually was a crash with 1000 and the filesize was a bit less than when I had used 15. Very, very strange. Want a file sample? (If i had somewhere to upload)

And Moreover, and for the nth time, you won't notice any improvements with more than 5 references, and in fact, more than 2 references is often plainly useless considering the slowdown it creates.

Yes, I totally agree. I was just doing it for testing purposes. ;-)

[Sagittaire]

Originally posted by virus
This is quite consistent with my own tests. If you want to optimize for Average PSNR, something around 40-50% should be good. If instead you want to optimize for Overall PSNR, you need to have a more "constant quantizer" behaviour, and so high values of variability are needed (70-90%).

The actual default (60%) should instead be a good starting point to find the maximum visual quality... but since visual quality is totally subjective (and things change when you change source and/or bitrate), that 60% is really just an hint. If some consensus can be reached on a different default which "looks better" on a wide range of sources we may ask akupenguin to change it. Rememeber that akupenguin tests on anime, so I'd suggest to investigate natural material... ;)

virus

best visual quality is subjective but for [60%-90%] interval variability metric is in practice constant : 60% for default setting is a good value for metric too ...

@DeathTheSheep
q24 for previous version is not equivalent at q24 for current version. Probaly RD setting, bframe ratio, bframe sensivity or other ...

q2 size for DivX and q2 size for XviD are very different with exactly the same setting but DivX and XviD obtain near result with same bitrate and with very different average quant ...

[DeathTheSheep]

True. I see. So does the compression quality subtly increase after each revision? Or are the absolute only changes limited strictly to what is written in the log?

[akupenguin]

Encoding delay is equal to the max consecutive B-frames. Decoding delay is 1 when traditional B-frames are used, and 2 when B-pyramid is used.

Note that only 1 delay is needed in order to ensure that each decoded frame is available in time for output:I b b b P
1 2 3 4 5 ; input order

I P b b b
1 5 2 3 4 ; coded order
- 1 2 3 4 5 ; display order

With B-pyramid, you need 2 delay to make sure frame#2 is decoded in time:I b B b P
1 2 3 4 5 ; input order

I P B b b ; coded order
1 5 3 2 4
- - 1 2 3 4 5 ; display order

Either way, decoding delay is independent of max B-frames:I b b b B b b b P
1 2 3 4 5 6 7 8 9 ; input order

I P B b b b b b b ; coded order
1 9 5 2 3 4 6 7 8
- - 1 2 3 4 5 6 7 8 9 ; display order

But note that the current B-pyramid is not the only way to use B-refs. Not all strategies require extra reordering, and some require more than 2 decoding delay and/or extra encoding delay. The following (not used by x264) requires 3 delay:I b B b B b B b P
1 2 3 4 5 6 7 8 9 ; input order

I P B B b b B b b ; coded order
1 9 5 3 2 4 7 6 8
- - - 1 2 3 4 5 6 7 8 9 ; display order

Of course, when I speak of delay, it only means delay between when you give a coded frame to the codec and when it returns the result. If the demuxer provides correct timestamps, the player can always decode enough frames in advance that the user won't see any delay.