new x264 feature: mixed references

[bond]

thx to Alex_W x264 now has a new feature: mixed references

Allows each 8x8 or 16x8 partition to independently select a reference frame, as opposed to only one ref per macroblock.

you can enable it in x264 via the --mixed-refs switch

are there already any findings how much this slows things down
is there a way to check how often different references per frame have been used?

[Kopernikus]

I found a slowdown of about 10 % with a PSNR Gain of about 0.03 dB

And I`m getting funky color artifacts. So be careful. It seems there is a bug.

edit: It was a compiler bug on my side, Sharktooths build works like a charm

[virus]

Congratulations to Alex for finally entering x264's changelog

Last time I saw results from a pre-release version of this patch, there was a 30% slowdown with up to 0.2-0.25 dB of PSNR gain at constant quantizer (that including the bitrate savings, using the approximate "rule" 1% bitrate = 0.05 dB). I don't know about the final version who made it into SVN. It's probably faster than that.

That only applies to content where multiple refs actually help. Very high motion scenes and scenes with lots of scenechanges usually don't benefit at all from multiple refs, so this patch improves little or nothing in those situations. However it is OK for most material. Probably useless for a trailer, certainly good for boosting the overall quality of a full-length movie

[Sirber]

In which rev is that enabled?

[bob0r]

Originally Posted by Sirber

In which rev is that enabled?

------------------------------------------------------------------------
r318 | pengvado | 2005-10-08 08:49:29 +0200 (Sat, 08 Oct 2005) | 5 lines

new option: --mixed-refs
Allows each 8x8 or 16x8 partition to independently select a reference frame, as opposed to only one ref per macroblock.
patch mostly by Alex Wright (alexw0885 at hotmail dot com).


------------------------------------------------------------------------

[Suchy]

what about VfW version?
It works only in CLI?

[jellysandwich]

So in layman's terms, this feature increases quality at the expense of speed (encoding time)?

js

[Kopernikus]

So in layman's terms, this feature increases quality at the expense of speed (encoding time)?

Exactly!

[Caroliano]

Only encoding time is affected? The decoding complexity still the same?

[hpn]

A short test with 5 ref frames and 2 bframes:

without --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o b2_r5.mp4 a.avs
PSNR Global:43.264 8.00 fps, 700.13 kb/s

with --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r5_mixed.mp4 a.avs
PSNR Global:43.298 6.71 fps, 699.99 kb/s

About 0.034 PSNR gain with --mixed-refs and 16% slower encode. Decoding CPU usage is the same for both encodes (~40% )
As expected --mixed-refs makes no difference if encoding with only 1 max ref frames (the default value), so one should use --mixed-refs with at least -r 2, otherwise the resulting encodes are bit-identical.

I also squeezed 0.017 extra PSNR by upping the max ref. from 5 to 16 (but it's slow).

x264 -p 1 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r16_mixed.mp4 a.avs

PSNR Global:43.315 4.06 fps, 699.78 kb/s

[jellysandwich]

Originally Posted by hpn

A short test with 5 ref frames and 2 bframes:

without --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --progress --frames 1500 -o b2_r5.mp4 a.avs
PSNR Global:43.264 8.00 fps, 700.13 kb/s

with --mixed-refs:
x264 -p 1 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 5 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r5_mixed.mp4 a.avs
PSNR Global:43.298 6.71 fps, 699.99 kb/s

About 0.034 PSNR gain with --mixed-refs and 16% slower encode. Decoding CPU usage is the same for both encodes (~40% )
As expected --mixed-refs makes no difference if encoding with only 1 max ref frames (the default value), so one should use --mixed-refs with at least -r 2, otherwise the resulting encodes are bit-identical.

I also squeezed 0.017 extra PSNR by upping the max ref. from 5 to 16 (but it's slow).

x264 -p 1 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o NUL a.avs
x264 -p 2 -B 700 -A all -r 16 -b 2 -w -8 --mixed-refs --progress --frames 1500 -o b2_r16_mixed.mp4 a.avs

PSNR Global:43.315 4.06 fps, 699.78 kb/s

Newb q: How are PSNR values supposed to be interpreted? Ie, is a .034 PSNR increase signficant or just minor?

js

[celtic_druid]

Originally Posted by Suchy

what about VfW version?
It works only in CLI?

http://ffdshow.faireal.net/mirror/x264/X264VFW.exe

[hpn]

Originally Posted by jellysandwich

Newb q: How are PSNR values supposed to be interpreted? Ie, is a .034 PSNR increase signficant or just minor?
js

Let's find out by making an extra 710kbps encode:

--bitrate 700 -> PSNR 43.264 (resulting kb/s:700.13)
--bitrate 710 -> PSNR 43.314 (resulting kb/s:709.92)

so 9.79kbps more -> 0.050 PSNR gain

So to gain 0.034 PSNR without --mixed-refs, you'll need to add 6.65kbps (estimated) more bitrate to the original 700. It means that --mixed-refs contributes to almost 1% encoding efficiency in terms of quality. Not bad at all . Of course this conclusion is valid for this particular 1 minute clip (1500 frames) and may be wrong in the general case.

[DarkZell666]

Question :
What source has been used for these test ?
Real-life movie or anime/cartoon ?

Another question (maybe off-topic):

Why is it when I download sharktooth's builds I get some wierd message
saying the executable file is corrupted ? (says: "use /NSRC to execute anyway"). I tried downloading the file several times over (and several builds over too, not just 318a) and I alwaygs get the same bl**dy message :/

Those on x264.nl (b0b0r's I suppose ?) work perfectly (at least, before the server went down :P), so I'm wondering if I haven't missed something :/

[celtic_druid]

Sounds like the downloads are incomplete.

[bob0r]

@celtic_druid

Source (x264 patch?) + standalone .dll for updated VFW, possible?

Thanks

(P.S. mirror05 is working (again), it was working, but dns was down)

[virus]

a quick test at constant quantizer to show the capabilities of mixed refs.
x264-r322, 800 frames, PAL DVD source, 560x304, mid-motion, quite rich texture detail.

--qp 22 -m 5 -r 3 -b 2 -8 -A all --me umh
PSNR Mean Y:41.246 U:45.522 V:46.297 Avg:42.304 Global:42.045
5.73 fps, 1320.83 kb/s

--qp 22 -m 5 -r 3 -b 2 -8 -A all --me umh --mixed-refs
PSNR Mean Y:41.253 U:45.522 V:46.296 Avg:42.310 Global:42.052
5.12 fps, 1299.14 kb/s

as expected, with --mixed-refs x264 basically kept the same output quality (differences <= 0.01 dB cannot absolutely be seen) while using a 1.6% lower bitrate. The price for that improvement was reasonably fair, roughly a 10% slowdown.
According to Alex, mixed-refs should perform even better when using R/D to select which reference to use (but R/D is slow, you know).

[AlexW]

Hi all, as virus said --mixed-refs seem to help more on some content than on others, though hopefully using RD to select references will help improve the gain a bit

[virus]

Originally Posted by AlexW

Hi all, as virus said --mixed-refs seem to help more on some content than on others, though hopefully using RD to select references will help improve the gain a bit

Man, this guy is really chatty
I cannot tell you "welcome on doom9's forums" since you already made one post eight months ago...
Anyway folks, this guy is Alex, regular x264 contributor (with a new kickass patch coming soon btw) and maybe future developer Make sure you welcome him as he deserves.

[Kostarum Rex Persia]

Of course, he is more then welcome here. I wish him luck in compiling x264 codec.