AVC vs. VP62: possible to get more quality at less bitrate?

[shae]

I'm experimenting with encoding for Flash playback. I thought AVC could get me better quality (or higher framerate) for less bits than VP62, but so far this isn't the case.

Encoding 320x240@12.5, VP6 at 340kbit vs AVC at 270kbit, VP6 blocks a bit more in motion and mosquitoes some in a few fadeins, but is much more detailed and less distorted. (If decode-time deblocking is on, AVC is much blurrier, too.)

Am I missing something important? The parameters I've used, with x264 core:59 r839 27c3071, are:

--bitrate 268 --pass 1 (and 2) --ref 4
--bframes 4 --b-pyramid --subme 6 --bime --partitions all
--8x8dct --me umh --b-rdo --mixed-refs --direct auto
--weightb --no-psnr --no-ssim --progress --level 1.3
--deblock 0:0

[Dark Shikari]

If you think AVC's deblocking is too strong, weaken it.

Of course, you're giving VP6 26% more bitrate, which is a bit unfair, but in general I would expect x264 to beat VP6 by a margin much larger than that, especially at such low bitrates. Can you give an example of AVC being "worse"? Perhaps pictures, streams, etc?

One thing I generally notice with VP6 streams is that the encoder fakes detail by taking advantage of the blockiness of VP6; you see "detail" but when you look closely you realize its nothing but a bunch of blocks.

[shae]

If you think AVC's deblocking is too strong, weaken it.Is there a way to control decode-time deblocking with encode-time parameters? Actually, in most cases (with some exceptions to the contrary), detail-wise AVC is okay if decoding is disabled on decoding. Though, it still veers more from the original in "shape".

Of course, you're giving VP6 26% more bitrate, which is a bit unfair, but in general I would expect x264 to beat VP6 by a margin much larger than that, On one hand unfair, on the other it should beat by a larger margin? :) If it couldn't better VP6 then there would be no point...

Can you give an example of AVC being "worse"? Perhaps pictures, streams, etc?I can't make too much of this content public at the moment, but I'll try to crop some frame parts to show.

One thing I generally notice with VP6 streams is that the encoder fakes detail by taking advantage of the blockiness of VP6Not in the cases where I find AVC lacking in comparison (that's with no deblocking on decode; otherwise it's even blurrier). But even if VP6 is "fake" sometimes, if it looks good, that's what counts.

Anyway, I'll try to post some images in a while.

[nm]

Is there a way to control decode-time deblocking with encode-time parameters?
Yes. In x264, deblocking strength can be changed with --deblock <alpha:beta> (which is 0:0 by default) and this applies to both encoding and decoding. Some people prefer less deblocking, so they use something like --deblock -2:-1. This is the right way to tune deblocking because no decoder that I know of allows disabling the deblocking without causing image corruption.

[shae]

Yes. In x264, deblocking strength can be changed with --deblock <alpha:beta> (which is 0:0 by default) and this applies to both encoding and decoding.I've tried negative values in a few encodings, but it didn't seem to change much, at least with decode deblocking. I should try even lower values.

no decoder that I know of allows disabling the deblocking without causing image corruption.What do you mean? I disabled it in ffdshow and it shows fine, besides blocking, which is to be expected. It looks better to me.

[Irakli]

On one hand unfair, on the other it should beat by a larger margin? :) If it couldn't better VP6 then there would be no point...

The only valid method of comparing encoders is choosing exactly same bitrate in both cases and then comparing resulting encodes visually.

Comparison of encoders using different bitrates will never be even close to objective.

[benwaggoner]

Of course, you're giving VP6 26% more bitrate, which is a bit unfair, but in general I would expect x264 to beat VP6 by a margin much larger than that, especially at such low bitrates.
You should also confirm that VP6 actually encoded at your requested bitrate. It can dramatically overshoot with aggressive settings - I've seen 20% and higher.

[shae]

The only valid method of comparing encoders is choosing exactly same bitrate in both cases and then comparing resulting encodes visually.My aim isn't to compare encoders but to get AVC at lower bitrate, higher quality, or a bit of both.

You should also confirm that VP6 actually encoded at your requested bitrate. It can dramatically overshoot with aggressive settings - I've seen 20% and higher.These figures are from the actual encode, of course.

I'm going to try another encode with different parameters.

[Inventive Software]

Do it at the same bitrate, then try reducing the bitrate for AVC. Double-check the file sizes too.

[nm]

I've tried negative values in a few encodings, but it didn't seem to change much, at least with decode deblocking. I should try even lower values.
Maybe, but post shots of the problematic areas first, so we can perhaps suggest better alternatives.

What do you mean? I disabled it in ffdshow and it shows fine, besides blocking, which is to be expected. It looks better to me.
When the loop filter is skipped in ff264, there is additional corruption that may even build up over time, depending on the skipping method (nonref, bidir or all). When deblocking is only skipped in non-referenced frames, errors don't build up, but there is probably not much visual difference either. To skip deblocking in all frames without causing corruption, the decoder should keep two copies of the decoded frames: one deblocked for referencing and other without deblocking for output.

So, if you want to disable the loop filter, do it at encode-time (--no-deblock).

[shae]

I tried the same 340kbit bitrate as VP, and with deblocking at -3:-3. It didn't drastically change it. The playtime-deblocking still makes it blurrier.

As for "detail" (I think I should make better distinction from now on between "detail" and "blurry"/"sharp"), some frames have more of it, like before, but a few less. I'm thinking now maybe due to keyframe placement or because that part fades in.

Another issue is a logo: red shapes against a black background. The red looks uneven, like black is seeping into it. VP does better. I tried encoding only that part at the bitrates I'm aiming for (like 250kbit), and it looked okay. I think not enough bits are given to this section somehow. Is there a way to see bitrate distribution?

BTW, what's more demanding, twice the resolution or twice the framerate (I'd guess resolution)?

[DarkZell666]

I tried the same 340kbit bitrate as VP, and with deblocking at -3:-3. It didn't drastically change it. The playtime-deblocking still makes it blurrier.

As for "detail" (I think I should make better distinction from now on between "detail" and "blurry"/"sharp"), some frames have more of it, like before, but a few less. I'm thinking now maybe due to keyframe placement or because that part fades in.

Another issue is a logo: red shapes against a black background. The red looks uneven, like black is seeping into it. VP does better. I tried encoding only that part at the bitrates I'm aiming for (like 250kbit), and it looked okay. I think not enough bits are given to this section somehow. Is there a way to see bitrate distribution?

BTW, what's more demanding, twice the resolution or twice the framerate (I'd guess resolution)?

I think you're talking about the deblocking settings inside the Post-Processing section of ffdshow. This should be kept off for h.264 playback since it's based on the frame quantizer, and h.264 quantizer scale uses larger numbers, so it post-processes much to strong on h.264 (and on h.264 only). Check post-processing is disabled before comparing any further :)

[shae]

No, it's the deblocking available as a checkbox (always/only when safe) at the bottom on the codecs page.

[Dark Shikari]

If you actually want us to care about your problem, I would suggest you stop describing the problems and start posting actual streams and images.

[shae]

If you actually want us to care about your problem, I would suggest you stop describing the problems and start posting actual streams and images.I know it's a problem without visuals, but I can't make the video public (at least for now). If the descriptions aren't useful I shall perhaps try to find a random video and see if I can reproduce the situation with it.

Either way thanks to all helpers. :)

[Dark Shikari]

I know it's a problem without visuals, but I can't make the video public (at least for now). If the descriptions aren't useful I shall perhaps try to find a random video and see if I can reproduce the situation with it.That'd work. I do understand if its a stream you can't publish, but its sort of useless reading words when I can't look at the actual stream ;)

[Shinigami-Sama]

I know it's a problem without visuals, but I can't make the video public (at least for now). If the descriptions aren't useful I shall perhaps try to find a random video and see if I can reproduce the situation with it.

Either way thanks to all helpers. :)

if you can't make it public why not just snip and send a stream in a PM to someone like dark here instead?

[shae]

if you can't make it public why not just snip and send a stream in a PM to someone like dark here instead?That's less public but still somewhat public. And why burden specifically Dark Shikari? :)

I think it's best if I tried making something that can be made public, so I'll try getting the pre-edit DV source to play with.

[DeathTheSheep]

Which version of the VP6 encoder are you using, and what settings specifically?

[e-Pawel]

I didn't get much into detail of your discussion but there's one thing I wanted to say about the comparison of x264 and vp62.

Today I made a small comparison test. The footage was a downsized (1280x528 -> 720x304) extract from "Simpsons the movie" and the test's result is that x264 delivers much better image quality. And it's faster too (encoding). Vp6 is no match for AVC but for ASP.

[foxyshadis]

Update your ffdshow!!! You're using a very old version that incorrectly adds deblocking as a post-processing option to AVC; the picture you get with post-processing off is the correct picture. ffdshow-tryouts fixed this as one of the very first bug fixes. Or just use any other decoder. You shouldn't be relying on the output of just one decoder if you're professionally employed to do this anyway; always check against at least two, preferably including the reference JM software, for completeness.