new x264 feature: mixed references

[bond]

thx to Alex_W x264 now has a new feature: mixed references

Allows each 8x8 or 16x8 partition to independently select a reference frame, as opposed to only one ref per macroblock.you can enable it in x264 via the --mixed-refs switch

are there already any findings how much this slows things down :D
is there a way to check how often different references per frame have been used?

[Kopernikus]

I found a slowdown of about 10 % with a PSNR Gain of about 0.03 dB

And I`m getting funky color artifacts. So be careful. It seems there is a bug.

edit: It was a compiler bug on my side, Sharktooths build works like a charm :D

[virus]

Congratulations to Alex for finally entering x264's changelog :)

Last time I saw results from a pre-release version of this patch, there was a 30% slowdown with up to 0.2-0.25 dB of PSNR gain at constant quantizer (that including the bitrate savings, using the approximate "rule" 1% bitrate = 0.05 dB). I don't know about the final version who made it into SVN. It's probably faster than that.

That only applies to content where multiple refs actually help. Very high motion scenes and scenes with lots of scenechanges usually don't benefit at all from multiple refs, so this patch improves little or nothing in those situations. However it is OK for most material. Probably useless for a trailer, certainly good for boosting the overall quality of a full-length movie ;)

[Sirber]

In which rev is that enabled?

[bob0r]

In which rev is that enabled?

------------------------------------------------------------------------
r318 | pengvado | 2005-10-08 08:49:29 +0200 (Sat, 08 Oct 2005) | 5 lines

new option: --mixed-refs
Allows each 8x8 or 16x8 partition to independently select a reference frame, as opposed to only one ref per macroblock.
patch mostly by Alex Wright (alexw0885 at hotmail dot com).


------------------------------------------------------------------------

[Suchy]

what about VfW version?
It works only in CLI?

[jellysandwich]

So in layman's terms, this feature increases quality at the expense of speed (encoding time)?

js

[Kopernikus]

So in layman's terms, this feature increases quality at the expense of speed (encoding time)?


Exactly!

[Caroliano]

Only encoding time is affected? The decoding complexity still the same?

[hpn]

A short test with 5 ref frames and 2 bframes:

without --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o b2_r5.mp4 a.avs
PSNR Global:43.264 8.00 fps, 700.13 kb/s

with --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r5_mixed.mp4 a.avs
PSNR Global:43.298 6.71 fps, 699.99 kb/s

About 0.034 PSNR gain with --mixed-refs and 16% slower encode. Decoding CPU usage is the same for both encodes (~40% )
As expected --mixed-refs makes no difference if encoding with only 1 max ref frames (the default value), so one should use --mixed-refs with at least -r 2, otherwise the resulting encodes are bit-identical.

I also squeezed 0.017 extra PSNR by upping the max ref. from 5 to 16 (but it's slow).

x264 -p 1 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r16_mixed.mp4 a.avs

PSNR Global:43.315 4.06 fps, 699.78 kb/s

[jellysandwich]

A short test with 5 ref frames and 2 bframes:

without --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o b2_r5.mp4 a.avs
PSNR Global:43.264 8.00 fps, 700.13 kb/s

with --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r5_mixed.mp4 a.avs
PSNR Global:43.298 6.71 fps, 699.99 kb/s

About 0.034 PSNR gain with --mixed-refs and 16% slower encode. Decoding CPU usage is the same for both encodes (~40% )
As expected --mixed-refs makes no difference if encoding with only 1 max ref frames (the default value), so one should use --mixed-refs with at least -r 2, otherwise the resulting encodes are bit-identical.

I also squeezed 0.017 extra PSNR by upping the max ref. from 5 to 16 (but it's slow).

x264 -p 1 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r16_mixed.mp4 a.avs

PSNR Global:43.315 4.06 fps, 699.78 kb/s

Newb q: How are PSNR values supposed to be interpreted? Ie, is a .034 PSNR increase signficant or just minor?

js

[celtic_druid]

what about VfW version?
It works only in CLI?
http://ffdshow.faireal.net/mirror/x264/X264VFW.exe

[hpn]

Newb q: How are PSNR values supposed to be interpreted? Ie, is a .034 PSNR increase signficant or just minor?
js
Let's find out by making an extra 710kbps encode:

--bitrate 700 -> PSNR 43.264 (resulting kb/s:700.13)
--bitrate 710 -> PSNR 43.314 (resulting kb/s:709.92)

so 9.79kbps more -> 0.050 PSNR gain

So to gain 0.034 PSNR without --mixed-refs, you'll need to add 6.65kbps (estimated) more bitrate to the original 700. It means that --mixed-refs contributes to almost 1% encoding efficiency in terms of quality. Not bad at all :). Of course this conclusion is valid for this particular 1 minute clip (1500 frames) and may be wrong in the general case.

[DarkZell666]

Question :
What source has been used for these test ?
Real-life movie or anime/cartoon ?

Another question (maybe off-topic):

Why is it when I download sharktooth's builds I get some wierd message
saying the executable file is corrupted ? (says: "use /NSRC to execute anyway"). I tried downloading the file several times over (and several builds over too, not just 318a) and I alwaygs get the same bl**dy message :/

Those on x264.nl (b0b0r's I suppose ?) work perfectly (at least, before the server went down :P), so I'm wondering if I haven't missed something :/

[celtic_druid]

Sounds like the downloads are incomplete.

[bob0r]

@celtic_druid

Source (x264 patch?) + standalone .dll for updated VFW, possible?

Thanks

(P.S. mirror05 is working (again), it was working, but dns was down)

[virus]

a quick test at constant quantizer to show the capabilities of mixed refs.
x264-r322, 800 frames, PAL DVD source, 560x304, mid-motion, quite rich texture detail.

--qp 22 -m 5 -r 3 -b 2 -8 -A all --me umh
PSNR Mean Y:41.246 U:45.522 V:46.297 Avg:42.304 Global:42.045
5.73 fps, 1320.83 kb/s

--qp 22 -m 5 -r 3 -b 2 -8 -A all --me umh --mixed-refs
PSNR Mean Y:41.253 U:45.522 V:46.296 Avg:42.310 Global:42.052
5.12 fps, 1299.14 kb/s

as expected, with --mixed-refs x264 basically kept the same output quality (differences <= 0.01 dB cannot absolutely be seen) while using a 1.6% lower bitrate. The price for that improvement was reasonably fair, roughly a 10% slowdown.
According to Alex, mixed-refs should perform even better when using R/D to select which reference to use (but R/D is slow, you know).

[AlexW]

Hi all, as virus said --mixed-refs seem to help more on some content than on others, though hopefully using RD to select references will help improve the gain a bit

[virus]

Hi all, as virus said --mixed-refs seem to help more on some content than on others, though hopefully using RD to select references will help improve the gain a bit
Man, this guy is really chatty :D
I cannot tell you "welcome on doom9's forums" since you already made one post eight months ago... :D
Anyway folks, this guy is Alex, regular x264 contributor (with a new kickass patch coming soon btw) and maybe future developer ;) Make sure you welcome him as he deserves.

[Kostarum Rex Persia]

Of course, he is more then welcome here. I wish him luck in compiling x264 codec.

[Sirber]

Went from 8FPS to 4FPS in my case :D

[Sirber]

Did a full encode on anime content, at 400kbps (inc. 32kbps audio), video is better than using 500kbps (inc 64kbps audio) when using the new multi-ref.

Excellent work and many thanks to x264 devs!!! :D

I'm trying now 268kbps for the video track to see how far I can go :D

[Sirber]

latest build (rev 328+) has faster code for that.

4FPS --> ~7FPS :D

[LoRd_MuldeR]

Can somebody explain (in words that a non codec programmer can understand ^^): What exactly is the advantage of "mixed" references?

[Kopernikus]

Every 16x16 Block of Pixels can be subdivided in smaller blocks, and each smaller block can have its own motion vector. With smaller partitions it is more likely to find an area that is similar, so that we only have to code the differences.
With mixed references, each smaller block can point to another reference frame. So the chance to find a very similar block is higher.

In other words: it helps compression. sometimes more sometimes less.

[Sharktooth]

Rising the number of reference frames will help maximize that "benefit" but will need more time for encoding.

[LoRd_MuldeR]

Okay, this means if I use a quantizer-based compresion with "Mixed Refs" I will get a smaller file at the same quality. And if I use a bitrate-based compression "Mixed Refs" will result in a better quality at the same bitrate. Correct?

In short:
"Mixed Refs" menas better results but slower encoding


BTW: What value for "Max Ref. frames" do you recommend for using "Mixed Refs" ???

[CeeJay.dk]

Exactly.
In general : Compression is always a tradeoff between compression gains and compression time.
It's important to find a good sweetspot between the two, that works well for you.

I don't have any recommendations for Max Ref. frames though , other than it should definatly be 2 or more, since 1 effectively disables Mixed Refs.
I'm guessing somewhere above 4 .. but how much above I have no idea.

Maybe someone else does ?

[CeeJay.dk]

I have a question of my own :
When AlexW mentioned that "using RD to select references will help improve the gain a bit" what was he talking about ?

Basicly , I don't know what RD is an abbreviation of.
I think's it must be some kind of technique to select better references , but what is it and how does it work ?

[Mnl]

RD is an abbrevation af Rate Distortion (Optimization). There is a description of the principle in Bond's sticky about MPEG-4 AVC (http://forum.doom9.org/showthread.php?t=96059). Also try doing a search for Rate Distortion.

[Kostarum Rex Persia]

Guys, when I try to use mixed references, with 3 reference frames and 2 B-frames, I get some strange artefacts in resulting video. I use FFDSHOW 20051013 Celtic Druid build.

[bond]

screenshot and exact settings plz

do you get the artefacts also with other decoders, eg nero?

[Kostarum Rex Persia]

I post sample very soon, probably tonight. Until now, I can say that I never try Nero H.264 Decoder, so I don't know about artefacts with nero decoder.

Also, tonight I will post exact settings and picture with some weird green artefacts.

[edit] Well, I accidentaly delete that file, in which weird green artefacts appear, so I can't post the picture. I was using rev. 333 then, and in rev. 333E, it seems that weird artefacts are gone.