How to test for weighted P frame support on a device?

[Chengbin]

I got a beta firmware for the Archos 5 Android with weighted P frame support in the decoder and I'm asked to test it. My video that used to play with blocks now don't. I want to make sure it is implemented properly, as it took a second beta firmware to play my video. What are the "stress" test videos for testing weighted P support? Is there such thing?

Thanks.

[LoRd_MuldeR]

I'd say: Take a random video clip, e.g. an excerpt from a movie, cut it into several pieces and put those pieces together again with Dissolve() in order to force fades.

Then encode with something like "--preset slower" and maybe additionally "--b-pyramid normal". Weight-P is already enabled by default, so it doesn't need any special parameters.

You may also use the "Match Point" sample (http://forum.doom9.org/showpost.php?p=1318287&postcount=189) that was posted a while ago. It has a whole lot of fades...

[Chengbin]

I can't find the match point sample, so I'm just gonna do some dissolve

If you know exactly where the sample is, it would be nice as well, otherwise don't spend your time searching it for me.

[LoRd_MuldeR]

Maybe if you look again at my post, you'll see the link ;)

[Chengbin]

Maybe if you look again at my post, you'll see the link ;)

Ahhh, I see it. Unforunately it is too big.

I'm encoding my "dissolved" video with preset placebo and see how it plays.

[Chengbin]

Thank you LoRd_MuldeR for your help.

The video plays flawlessly. I can probably now say weighted P frames is properly implemented. I must also say it took 16 ref frames very well @ 800x448. Very impressive.

[LoRd_MuldeR]

Remember that testing can only show the presence of errors, it can never verify the absence of errors. So what you can say is that your test stream didn't trigger any apparent errors, but there still may be subtle errors. You can't verify that the stream was decoded 100% correct, unless you did compare it (pixel-wise) to the reference decoder. And even then it may still be possible that there are bugs in the decoder that just weren't triggered by your test stream, because they only happen in extremely rare situations. Anyway, you may conclude that the Archos 5 decoder is "good enough" now ;)

BTW: CoreAVC 2.0 states that they not only support Weight-P as implemented in x264 currently ("Multi-dupe"), but also some future implementation. So we may see new problems in the future...

[Chengbin]

BTW: CoreAVC 2.0 states that they not only support Weight-P as implemented in x264 currently ("Multi-dupe"), but also some future implementation. So we may see new problems in the future...

Are you saying that CoreAVC implied a new x264 feature coming that will supposedly break decoder compliancy again, and they "futureproofed" itself by adding support of this mystery feature?

EDIT: Never mind, I see it in the CoreAVC thread.

[LoRd_MuldeR]

Are you saying that CoreAVC implied a new x264 feature coming that will supposedly break decoder compliancy again, and they "futureproofed" itself by adding support of this mystery feature?

Nope. CoreAVC said that with version 2.0 they not only fully support x264's current implementation of Weighted P-Prediction, but also some future Weight-P improvement that is planned for x264. I have no idea whether those future improvement will break other decoders or not. But as x264 will certainly continue to produce valid H.264 streams, any proper decoder will have NO problems! However the fact that CoreAVC explicitly mentions the compatibility to those "future" improvements in their Changelog implies that we'll potentially see problems in the future, even with decoders that do support "current" Weight-P. I don't think that they have an "evil" plan to add a new feature to x264 that intentionally breaks all decoders, expect CoreAVC 2.0. That's too much conspiracy theory ^^

[lnatan25]

Speaking of conspiracy theories, perhaps they mention it to make it look more credible. CoreAVC has had so many bugs and problems, and usually takes months to fix, so this might make people think it's "future proof". ;)

[Dark Shikari]

The way that CoreAVC fixed the problem should be compatible with all possible methods of reference list ordering and such, so the problem should never occur again.

Of course, there could in theory be a mistake, and we never tested it on B-frames, since we don't have an encoder that does it on B-frames.

[Shevach]

My two coins into the discussion.

There are two non-free H264 testing suites for decoders verifications: HHI and Allegro. Each of these testing suites contains ton of streams including the streams with "stress" weighting cases.

It's not sufficient to check only weighting factors of 16 reference frames. It's needed to check that a decoder handles correctly negative offsets and wieghts and clipping of weighted predicted values are performed.