x264 Rev 667b quality issue

[desta]

I'm in the middle of a 2 pass encode at the moment, and just noticed something slightly odd.

I originally did the same encode with sharktooth's patched 665, and then decided to up the bitrate a bit. By the time I came to do the new encode, 667b was available, so I grabbed that and started again... same settings apart from bitrate, same source and same avs script.

A little while into the 2nd pass, I noticed that the results from the 1st pass in megui's log were only very slightly different from the previous encode at a lower bitrate.

I aborted the encode and compared the original completed encode (665) with what had done so far from the new encode (667b). Frame comparisons actually showed that the new encode, with a higher bitrate, actually came out looking worse!

I figured I must've just done something wrong, so I aborted the 2nd pass, closed and reopened megui, checked my settings, and started again. Got through the 1st pass, and again the same thing. I let it go for a little while into the 2nd pass, then aborted again and did another comparison. Same thing - the higher bitrate encode with 667b looked worse.

I decided to try 665 again. Same settings and bitrate as the newer encodes. This time after the first pass, I got this:


x264 665

--pass 1 --bitrate 2890 --stats "movie.stats" --bframes 3 --b-pyramid --direct auto --filter -3,-2 --subme 1 --analyse none --me dia --merange 24 --threads auto --thread-input --zones 181415,190060,q=36 --cqmfile "PRESTIGE_CQM.cfg" --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "movie.avs" --aq-strength 0.6 --aq-sensitivity 13


Log for job job1-1

avis [info]: 1024x432 @ 25.00 fps (190663 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE 3DNow!
x264 [info]: slice I:1828 Avg QP:18.77 size: 62485
x264 [info]: slice P:81110 Avg QP:21.03 size: 22344
x264 [info]: slice B:107725 Avg QP:22.54 size: 6714
x264 [info]: mb I I16..4: 48.4% 0.0% 51.6%
x264 [info]: mb P I16..4: 34.5% 0.0% 0.0% P16..4: 59.9% 0.0% 0.0% 0.0% 0.0% skip: 5.7%
x264 [info]: mb B I16..4: 3.5% 0.0% 0.0% B16..8: 24.9% 0.0% 0.0% direct:40.3% skip:31.3%
x264 [info]: final ratefactor: 22.83
x264 [info]: direct mvs spatial:99.5% temporal:0.5%
x264 [info]: kb/s:2779.5

encoded 190663 frames, 7.27 fps, 2779.65 kb/s


.. and here was the results from the 667b encode:


x264 667b

--pass 1 --bitrate 2890 --stats "movie.stats" --bframes 3 --b-pyramid --direct auto --filter -3,-2 --subme 1 --analyse none --me dia --merange 24 --threads auto --thread-input --zones 181415,190060,q=36 --cqmfile "PRESTIGE_CQM.cfg" --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "movie.avs" --aq-strength 0.6 --aq-sensitivity 13


avis [info]: 1024x432 @ 25.00 fps (190663 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE 3DNow!
x264 [info]: slice I:1835 Avg QP:19.20 size: 59846
x264 [info]: slice P:81106 Avg QP:21.44 size: 21570
x264 [info]: slice B:107722 Avg QP:22.96 size: 7334
x264 [info]: mb I I16..4: 39.6% 0.0% 60.4%
x264 [info]: mb P I16..4: 36.4% 0.0% 0.0% P16..4: 57.6% 0.0% 0.0% 0.0% 0.0% skip: 6.0%
x264 [info]: mb B I16..4: 8.6% 0.0% 0.0% B16..8: 25.1% 0.0% 0.0% direct:32.4% skip:33.9%
x264 [info]: final ratefactor: 23.25
x264 [info]: direct mvs spatial:99.6% temporal:0.4%
x264 [info]: kb/s:2779.0

encoded 190663 frames, 7.90 fps, 2779.17 kb/s


For what it's worth, the second pass commands were:


--pass 2 --bitrate 2890 --stats "movie.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -3,-2 --subme 6 --trellis 2 --analyse all --8x8dct --me umh --merange 24 --threads auto --thread-input --zones 181415,190060,q=36 --cqmfile "PRESTIGE_CQM.cfg" --progress --no-dct-decimate --no-psnr --no-ssim --output "movie.mkv" "movie.avs" --aq-strength 0.6 --aq-sensitivity 13

... though apart from the original lower bitrate encode, I aborted before the 2nd passes finished. In hindsight, I should've kept the screenshot comparisons to show here, but from both the screens and the logs, rev 665 actually seems to have done better than 667b - which seems only to have had a slight speed improvement.

I'm not actually complaining, but thought I'd obviously bring it to the attention of those that might need to know.

[fields_g]

The reason for the "b" in 667b is a reworking of the AQ patch and is described here (http://forum.doom9.org/showthread.php?t=128044).

I see you are using AQ and this change might be significant in your case. The original 667 build can be found here (http://mirror05.x264.nl/Cef/?dir=./old_builds). Give it a try.

Yes, the screenshots could help tremendously.

[desta]

Thanks for the link, interesting read.

Tbh, I thought most people were aware of the blocks that can be introduced by using AQ. I've seen it in many of my own encodes, and always ended up bypassing it by turning off b prediction.

I've still got quite a while to go on this current encode, but once it's done, I start another encode using 667b and then give some frame comparisons.


p.s. thanks neuron for changing the thread title.. I pressed submit before I remembered to make it more descriptive.

[Tack]

I think the whole point of AQ is to reduce blocks, not introduce them. I suppose with VBR it could end up doing this, where by lowering the quantizer used for low detail/darker areas, it ends up stealing needed bits from high detail areas. If the bitrate is low enough where this is visible, it seems to me AQ would be a bad idea.

I use AQ with CRF where there is no quality trade-off, only a bitrate tradeoff. In this context it makes much more sense, IMHO.

[desta]

I think the whole point of AQ is to reduce blocks, not introduce them.
Yes, and as I said in the cases where it's introduced them, I've simply turned b prediction to none, and the blocks are gone.

It does obviously have an even more significant impact on compressibility though.

[Tack]

I found that with the AQ modifications suggested by akupenguin in the thread fields_g linked to (which at least conceptually is clearly more correct), I'm able to (with CRF at least) use both brdo and bpred with AQ without seeing DCT blocks in the usual problem areas.

Mind you I've only done 3 or 4 encodes since then.

[desta]

Well that's a good thing, no doubt, and I'm not trying to push aside the latest tweaks put into the AQ patch, but from what I saw it did reduce overall quality.

Ignoring the log stats for a minute, just visually the difference was there. As an example (without having any screenshots to hand), from the VBR encodes I did, the 665 build actually held more grain in the background, whilst it's 667b counterpart seemed to gather the grain up, making it look chunkier and smeared.

Not the best description unfortunately, but the only one I could come up with.

[Tack]

Visually, my observations where rather different. However the AQ tweaks did cause bitrate to increase 10-15% at a given CRF. With 2-pass VBR then, it's possible that this is accounting for the quality difference you're seeing.

It would be interesting to compare your encode using CRF instead of 2-pass. If the quality decrease you're seeing goes away, it might indicate a rate control issue (as it interacts with AQ). Or perhaps simply too low a bitrate for what AQ wants to do.

Anyway, I'd say without clips or at the very least screenshots, this discussion remains entirely academic. :)

[desta]

Well, like I said, I'm not trying to put down any efforts put into tweaking the AQ patch. I'm only reporting my findings. As the newly tweaked patch increases bitrate, then it obviously stands to reason that quality will suffer compared to the previous AQ patch at the same bitrate.

Anyway, I'd say without clips or at the very least screenshots, this discussion remains entirely academic. :)
Well clearly, yeah. When this encode finishes (which isn't going to be too soon), I'll get on to doing some short comparison encodes. Then again, I'm sure it would be just as easy for someone else to do in the meantime - it's just a case of doing two same-setting VBR encodes, one with 665 and the other with 667b. Grainy sources (particularly fine grain) would illustrate it better.

[foxyshadis]

What you should see with the combination fixed is not only a general global quality degradation, but also b-frames should now not suffer from very heavy blocking on flat textures. You should search your original source, compared to the second, and see if you can find those frames. If not, perhaps AQ is set too high (which will also degrade overall quality), or maybe the skip measurement needs to be modified for AQ, but I'm not qualified to diagnose something like that.

[TheRyuu]

What you should see with the combination fixed is not only a general global quality degradation, but also b-frames should now not suffer from very heavy blocking on flat textures. You should search your original source, compared to the second, and see if you can find those frames. If not, perhaps AQ is set too high (which will also degrade overall quality), or maybe the skip measurement needs to be modified for AQ, but I'm not qualified to diagnose something like that.

Wait, your post confuses me. :confused:

So the little AQ patch will reduce overall quality at the same bitrate (is same bitrate key here?)
But wait again, I think I get it (sort of). AQ is now giving more bitrate to the effected areas (dark, whatnot) and taking it away from the rest?

I'm not really sure what the new AQ patch did really.
Something above moving the x264_adaptive_quant() before x264_mb_analyse_init()???

Is the new AQ patch really better? (opinion?)

:thanks:

BTW, is there like a x264 Wiki or something that has explanations of all the options? (like AQ, deadzones, and whatnot...)

[Tack]

So the little AQ patch will reduce overall quality at the same bitrate (is same bitrate key here?)
Although I haven't verified it, it seems intuitive to me that at a given bitrate, the global PSNR will be lower with AQ than without AQ. This says nothing about subjective, human-perceived quality, mind you. And yes, I would say "at the same bitrate" is key here.

But wait again, I think I get it (sort of). AQ is now giving more bitrate to the effected areas (dark, whatnot) and taking it away from the res?When there is a bitrate target, if AQ is lowering the quantizer used in low detail and dark areas, those bits necessarily must come from somewhere else.

I prefer CRF, so this isn't a problem for me, since in the CRF case there is no quality tradeoff, just an overall increase in bitrate.

I'm not really sure what the new AQ patch did really. I'm rather ignorant about the details, but my naive understanding is that AQ was working directly against RDO -- AQ was dropping the quantizers in certain areas and RDO was in effect trying to undo what it saw as an unnecessarily high bitrate, so ended up decimating the block.

Is the new AQ patch really better? (opinion?)Again, as I understand it, the patch before these (rather minor) changes was simply not correct and now it is. Certainly my own experience shows a significant improvement with the changes.

[Terranigma]

perhaps AQ is set too high (which will also degrade overall quality), or maybe the skip measurement needs to be modified for AQ, but I'm not qualified to diagnose something like that.
I can. A High AQ setting (along with aq-sensitivity) can definately reduce the overall quality by making the encode to look smudgy. I've ran across this problem myself a good while back, and the culprit was the aq setting (alongside aq-sensitivity). I was using an aq setting of 1.2 and aq-sensitivity of 10, and noticed a huge quality loss than using just aq-strength of 0.3 and leaving the sensitivity at it's default value of 15.