New Adaptive Quantization

[Dark Shikari]

I would be curios as to how the old AQ handles that riddick scene. The quality in backgrounds was also raised quite a lot with old one in all my encodes.
I'll try some tests on more difficult stuff, like Underworld. That riddick pic just doesn't have any high fine detail area (like the faces in underworld in contrast to the noisy dark backgrounds) to see how much that is affected.

EDIT:

Interesting, new aq shows some improvement in certain areas, but overall one would say it's worse, the castle in light lost detail heavily. After all, the strong detail and areas in focus/interest should keep high detail first of all (since that's what we're watching), then try to make the rest look acceptable.
New aq pic: http://img341.imageshack.us/img341/1432/newaqpicob4.th.png (http://img341.imageshack.us/my.php?image=newaqpicob4.png)
Old aq pic: http://img341.imageshack.us/img341/3017/oldaqpicjx5.th.png (http://img341.imageshack.us/my.php?image=oldaqpicjx5.png)
New aq pic with quants: http://img341.imageshack.us/img341/5276/newaqquantsus7.th.png (http://img341.imageshack.us/my.php?image=newaqquantsus7.png)
Old aq pic with quants: http://img341.imageshack.us/img341/3360/oldaqquantsrf7.th.png (http://img341.imageshack.us/my.php?image=oldaqquantsrf7.png)
The reason for that is probably because the new AQ has a somewhat different philosophy. Notice my sample, for example: I'm encoding at 2000kbps, a very high value for a letterboxed DVD-resolution video, but without AQ there is still a serious problem with blocking!

You're encoding at a low enough bitrate that there is normally a blocking problem to begin with; no AQ can possibly fix that, as there aren't enough bits in the scene to do so. Using my AQ on a high strength value is just going to decimate the most complex parts of the scene. Using my AQ on a very high strength value is certain to lower visual quality in at that bitrate.

[Sharktooth]

what about a non linear deblocking strength? using higher inloop deblocking on high quants will avoid blocks but will drop details, so the new AQ can be a sort of balance between foreground and background details...

[Dark Shikari]

what about a non linear deblocking strength? using higher inloop deblocking on high quants will avoid blocks but will drop details, so the new AQ can be a sort of balance between foreground and background details...Is that even possible? I thought the deblock filter had to be a single constant value throughout the video and was equivalent on playback and encoding?

[Sharktooth]

since deblock is triggered by QP, using AQ will surely change the deblocking...
the ateme encoder used to have inloop deblocking matrices too...

[Sergey A. Sablin]

Is that even possible? I thought the deblock filter had to be a single constant value throughout the video and was equivalent on playback and encoding?

filter offsets are written in each slice header, so one may adapt the values to average slice qp.

[MarcioAB]

Original post has been updated with new executable so you can test it out.

Well, at least for me the previous build was MUCH better.

[Dark Shikari]

Well, at least for me the previous build was MUCH better.Note that I did change the build in the meantime with something I thought wouldn't affect anything but I might have messed up. I was asked by someone on IRC to make the max/min QP adjustable, so I tied them to qpmax and qpmin, which by default were 51 and 10, what I was already using.

I assume I probably messed something up; I don't think a dumb typo would be beyond me. I'll check it later when I get back to my computer.

[Razorholt]

Well, at least for me the previous build was MUCH better.

Can you - or Dark Shikari - re-post the previous build for me to compare it with the recent one?

Cheers,
- Dan

[Dark Shikari]

Strange, I didn't mess anything up, my change worked fine... in fact the build I used for the images I posted is exactly the same as the one I'm currently using.

So if you want the old build you're probably out of luck, I don't have it anymore. :(

Also, looking at more Elecard tests of my build one thing that I notice it does is that it drastically cuts the number of skipped blocks, by using the bits from the most complex blocks to avoid skipping less complex blocks. This seems to be the core of its benefit.

[ChronoCross]

Strange, I didn't mess anything up, my change worked fine... in fact the build I used for the images I posted is exactly the same as the one I'm currently using.

So if you want the old build you're probably out of luck, I don't have it anymore. :(



We need to get you a patch svn repository.....lol

[Dark Shikari]

We need to get you a patch svn repository.....lolI actually have a friend who is running an SVN for me but I never get around to actually using it... :D

I do have my ungodly-long undo history in Notepad++ :)

Why in god's name doesn't the x264 SVN have branches? If it did I could just get access to a branch and develop whatever I wanted there.

[Daodan]

Old (row 1) vs new aq (row 2):

http://img504.imageshack.us/img504/7396/oldaq10eh8.th.jpg (http://img504.imageshack.us/my.php?image=oldaq10eh8.jpg) http://img504.imageshack.us/img504/7581/oldaq20cf2.th.jpg (http://img504.imageshack.us/my.php?image=oldaq20cf2.jpg) http://img504.imageshack.us/img504/3964/oldaq30yk3.th.jpg (http://img504.imageshack.us/my.php?image=oldaq30yk3.jpg) http://img504.imageshack.us/img504/7461/oldaq40ta0.th.jpg (http://img504.imageshack.us/my.php?image=oldaq40ta0.jpg) http://img504.imageshack.us/img504/1599/oldaq50ur7.th.jpg (http://img504.imageshack.us/my.php?image=oldaq50ur7.jpg) http://img504.imageshack.us/img504/4998/oldaq60yz3.th.jpg (http://img504.imageshack.us/my.php?image=oldaq60yz3.jpg) http://img504.imageshack.us/img504/7108/oldaq70gv7.th.jpg (http://img504.imageshack.us/my.php?image=oldaq70gv7.jpg)


http://img504.imageshack.us/img504/2305/newaq10dk5.th.jpg (http://img504.imageshack.us/my.php?image=newaq10dk5.jpg) http://img504.imageshack.us/img504/7480/newaq20lk5.th.jpg (http://img504.imageshack.us/my.php?image=newaq20lk5.jpg) http://img504.imageshack.us/img504/3215/newaq30gb6.th.jpg (http://img504.imageshack.us/my.php?image=newaq30gb6.jpg) http://img504.imageshack.us/img504/5999/newaq40kt9.th.jpg (http://img504.imageshack.us/my.php?image=newaq40kt9.jpg) http://img504.imageshack.us/img504/9776/newaq50ob1.th.jpg (http://img504.imageshack.us/my.php?image=newaq50ob1.jpg) http://img504.imageshack.us/img504/1940/newaq60xv4.th.jpg (http://img504.imageshack.us/my.php?image=newaq60xv4.jpg) http://img504.imageshack.us/img504/7273/newaq70cx8.th.jpg (http://img504.imageshack.us/my.php?image=newaq70cx8.jpg)

This is a more life-like encode, using reasonable bitrates (not too high, not too low)

What seems to happen is that bframes seem to get increased quality especially, while p and i stays quite same (and in very few cases it gets worse). Overall I'd say there's an improvement though. Any chance of a real build with it?

[Mutant_Fruit]

Why in god's name doesn't the x264 SVN have branches? If it did I could just get access to a branch and develop whatever I wanted there.

It does, it's called: http://code.google.com/hosting ;) Check the code out from the x264 svn, check it in there, bada bing bada bang. Not quite as nice as having a branch in the same SVN, but just as easy to use.

[imcold]

Why in god's name doesn't the x264 SVN have branches? If it did I could just get access to a branch and develop whatever I wanted there.
Making a svn repository on your machine, importing the current x264 version to it and working in this local repo could also be an option. Almost the same as branch.

[Dark Shikari]

This is a more life-like encode, using reasonable bitrates (not too high, not too low)

What seems to happen is that bframes seem to get increased quality especially, while p and i stays quite same (and in very few cases it gets worse). Overall I'd say there's an improvement though. Any chance of a real build with it?
That's some very impressive grain retention.

Interesting that B-frames get increased quality--I wonder if that's a good or a bad thing; should I try to save those bits for use elsewhere, or do the B-frames need the boost? Perhaps its because one needs a similar amount of bits regardless of frametype for the grain retention.

I can post a patch later today if someone wants to make a threaded build of it.

[Atak_Snajpera]

Daodan
What settings did you use in both examples?

[Daodan]

5500 kbps, aq 0.7, mean quantizer somewhere at 21.3. Unfortunately I'm not sure if I used or not turbo on first sample (on second I know I didn't), because I deleted the logs. So for now let's say the test it's not conclusive in any way and wait for some other people's tests untill I can make some new ones.
As for bframes being good quality, I'd say yes, gives a more stable look overall.

[Atak_Snajpera]

I would like to see it in CRF22.For instance old --aq-strength 0.5 --aq-sensitivity 5 vs new equivalent.

[CruNcher]

@Dark Shikari
im not sure if this is a non mod16 problem but look @ this hmm something is wrong ;)

http://s6.directupload.net/images/070920/temp/2b2M9vhq.png (http://s6.directupload.net/images/070920/2b2M9vhq.png)

settings: x264aq --pass 2 --bitrate 3000 --stats "full.stats" --level 4.1 --min-keyint 1 --ref 3 --nf --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-maxrate 25000 --me dia --merange 8 --threads auto --thread-input --progress --no-psnr --no-ssim --output "full.264" "full.avs" --aq-strength 1.0

[Dark Shikari]

@Dark Shikari
im not sure if this is a non mod16 problem but look @ this hmm something is wrong ;)

http://s6.directupload.net/images/070920/temp/2b2M9vhq.png (http://s6.directupload.net/images/070920/2b2M9vhq.png)
Ah crap, it probably is a non-mod16 problem of some sort... can you test the same footage in mod16 proportions (crop it) just to make sure that it is a non-mod-16 problem?

[CruNcher]

im gonna do it new with a shorter test sample and some different settings just to be sure, somehow i doub't it has todo with mod16 as Daodan his sample is also not mod16 ;)

[Dark Shikari]

im gonna do it new with a shorter test sample and some different settings just to be sure, somehow i doub't it has todo with mod16 as Daodan his sample is also not mod16 ;)
Can you upload the .h264 file so I can check the quants?

[CruNcher]

yep no problem, but also the mod16 shows the same problem resized to 1280x720 with this settings

1st pass:
x264aq --pass 1 --bitrate 3000 --stats "full1.stats" --level 4.1 --min-keyint 1 --nf --subme 1 --analyse none --vbv-maxrate 25000 --vbv-init 1.0 --vbv-bufsize 14745 --mvrange 511 --me dia --merange 8 --threads auto --thread-input --progress --no-psnr --no-ssim --output NUL "full.avs" --aq-strength 1.0

2nd pass:
x264aq --pass 2 --bitrate 3000 --stats "full1.stats" --level 4.1 --min-keyint 1 --nf --subme 1 --analyse none --vbv-maxrate 25000 --vbv-init 1.0 --vbv-bufsize 14745 --mvrange 511 --me dia --merange 8 --threads auto --thread-input --progress --no-psnr --no-ssim --output "full.264" "full.avs" --aq-strength 1.0

[Dark Shikari]

x264aq --pass 1 --bitrate 3000 --stats "full1.stats" --level 4.1 --min-keyint 1 --nf --subme 1 --analyse none --vbv-maxrate 25000 --vbv-init 1.0 --vbv-bufsize 14745 --mvrange 511 --me dia --merange 8 --threads auto --thread-input --progress --no-psnr --no-ssim --output NUL "full.avs" --aq-strength 1.0

2nd pass:
x264aq --pass 1 --bitrate 3000 --stats "full1.stats" --level 4.1 --min-keyint 1 --nf --subme 1 --analyse none --vbv-maxrate 25000 --vbv-init 1.0 --vbv-bufsize 14745 --mvrange 511 --me dia --merange 8 --threads auto --thread-input --progress --no-psnr --no-ssim --output "full.264" "full.avs" --aq-strength 1.0

First of all, you're only using one pass, because your second pass has --pass 1. Second, --subme 1 is atrocious, and lowers quality/bitrate over 20-25% over --subme 2 in my tests. Third, your overall settings are so low that I wouldn't be surprised at that quality at all...

[Daodan]

2nd pass:
x264aq --pass 1 --bitrate 3000 --stats "full1.stats" --level 4.1 --min-keyint 1 --nf --subme 1 --analyse none --vbv-maxrate 25000 --vbv-init 1.0 --vbv-bufsize 14745 --mvrange 511 --me dia --merange 8 --threads auto --thread-input --progress --no-psnr --no-ssim --output "full.264" "full.avs" --aq-strength 1.0


OMG, CABAC is still ON? Get rid of it quickly :D

Seriosusly though, those are horrible settings. Not much point in making tests for something that no one will ever use. Kinda like Dark Shikari's V for V encode :D

[CruNcher]

eh sorry the --pass 1 is a c&p error it is correct and 2pass was used but anyway Dark Shikari it should work regardless if you find this settings to low, else i would say it's bugged and im testing a very advanced masking @ the moment and it doesn't show this kind of problems even @ low settings ;)

@Daodan
Cabac is usefull especialy @ what im doing but anyway who are you that you say no one will ever use those settings, only because you don't that's really egoistic and wrong you can really get unbalanced in your encodings thinking like that, but yeah the more compression the better (wrong) Metrics wise you might be right but Metrics and Look & Feel are 2 completely different things (HVS) ,you still have to learn alot it seems, but you @ the right place for that ;)

[Dark Shikari]

You're using a bitrate of 3 Mb/s on something that huge and expect to be able to retain grain?

All you're going to end up with are extremely high quants... like what happened :)

You can't compress 1080p at 3Mb/s and retain grain without using FGM. --aq-strength 1.0 is absurd for that low a bitrate. What happens is that it has so few bits that its forced to steal bits from the foreground, resulting in the ugly artifacting you saw.

Remember, I was using 2 Mb/s for 720x480 letterboxed. You're using only 50% more bits for 1080p.

[CruNcher]

you can't compress 1080p at 3Mb/s and retain grain without using FGM.

not the full grain layer i agree here with you, but a lossy perceptable version of it, see the low bitrate thread :)

[Dark Shikari]

not the full grain layer i agree here with you, but a lossy perceptable version of it, see the low bitrate thread :)Also note with your encode that here's what's happened:

1. AQ tries to reduce the quants in the background area. This requires a whole lot of bits.
2. Ratecontrol raises the overall frame quant in order to try to get the bits necessary.
3. Repeat 1 and 2 a few times.
4. Overall frame quant is now very high, on the order of 35 or more.
5. AQ keeps the background area quants lower, and raises the high-detail area quants by even more... with a max of qp/2, which now maxes out at QP 51!
6. A bunch of quants in the foreground go really high as a result, and it looks like shit. The background, due to both low quants and your very low motion search, consists of nothing but I-blocks and skipped blocks.

Without enough bits to keep the whole frame's quant below 30 or so, AQ-strength at 1.0 will have atrocious results.

Also notice your encode has no B-frames.

[CruNcher]

ok i see so i have to finetune the --aq-strength value then to avoid the artifacting gonna do some tests with not so high settings those where just thought as a quick test but yeah i have to tune this into the complete workflow (this gonna take time).
And yes i know that i can't keep the complete grain noise/layer (wich btw is lossy allready Mpeg-2 ;) ) but my target is a complete visualy different then what you might think, for me it's important that a small part of the spacial noise/grain survives the quantization so the result doesn't look in the end like Digital Cinema (H.264 tendency with high ME settings) but more like the input source and i almost arrived this with another implementation, different settings tough (higher then those posted above also motion estimation wise).
Im gonna try the same with X264 later but i still didn't finished with this on the other implementation, but the results are allready wow for my visual understanding @ this low bitrate.
It would be no problem to keep alot of the grain @ 6 mbit allready tested that also with x264 but my target is the lowest possible (the rest scales to the higher bitrate automaticly) an how it looks in the end, and don't tell me you can predict how this is gonna look finetuned (also don't forget it's variable bitrate so bitrate changes and so also the grain layer does change to high qp (low bitrate) it would be less perceptable in high bitrate zones it will be more perceptable) ;)
And about b-frames i never really was a fan of them they can introduce more visual problems then fix, it got better for sure these days but still not optimal, and later settings for sure will use them but this here was just a fast test how the grain preservation would look like with your AQ, and now we found out that --aq-strength 1.0 was to much so i gonna try a lower setting.

Just to give you an impression qp of 21 is saving alot of details and also the grain layer very nice, see Quicktime trailer encodes (especialy the long evan almighty one) my avg qp of 9 mins with the other implementation is 28 (with the letterbox) @ the moment @ 1080p most likely would be something as 25 @ 720p (with reduced grain due to resizing) so that's not really far and i think a nice result for 3 mbit 1080p @ 42 dB :)

PS: The problem is from the other implementation im working alot now with im not used such many finetuneing options, you just set the AQ and it goes of and the end result looks fine @ any bitrate so yes it seems adaptive (or very low set but high enough to give a visual enhancement).
For me it feels like that in x264 partition distrobution,deadzones and adaptive quantization do still need alot of tuneing but another question is should they be tuned based on a metric or visualy ?

[lexor]

Results with some tweaks:

2000kbps, without AQ:

[pic1 snip]

2000kbps, with new AQ:

[pic2 snip]

The old AQ algorithm has similar appearance, but because it doesn't raise the quantizer of complex blocks, the frames with tons of AQ, like this one, get drastically larger than with my algorithm, hurting the rest of the video considerably. Interestingly enough, the old AQ algorithm on max strength doesn't lower the quantizers nearly as much as the new one does... a good or bad thing depending on how you look at it.

I don't know if any changes were made since that post, but the new AQ in those 2 pics does a terrible job at preserving detail in light areas copared to no AQ first shot. New AQ is much blurier in areas with fine detail, enough so to negate any gain in dark areas. (imho)

[Dark Shikari]

I don't know if any changes were made since that post, but the new AQ in those 2 pics does a terrible job at preserving detail in light areas copared to no AQ first shot. New AQ is much blurier in areas with fine detail, enough so to negate any gain in dark areas. (imho)Can you point out specifically, maybe with some quick MS paint circling, where the problems are? I can then correlate this with the quantizers and try to fix the problem.

[lexor]

http://img222.imageshack.us/img222/8900/4l65bv7pi2.th.png (http://img222.imageshack.us/my.php?image=4l65bv7pi2.png)
as you get closer to the hand the ridges disappear (as you can see it happens in other areas as well, but not as much). This problem is relatively small in this sample, due to overwhelming darkness of the shot, but in my experience the worst shots that I always spend time trying to correct are just like that but with more light in the centre and deep dark all around. It gets even more noticeable when you watch it full screen.

[Dark Shikari]

http://img222.imageshack.us/img222/8900/4l65bv7pi2.th.png (http://img222.imageshack.us/my.php?image=4l65bv7pi2.png)
as you get closer to the hand the ridges disappear (as you can see it happens in other areas as well, but not as much). This problem is relatively small in this sample, due to overwhelming darkness of the shot, but in my experience the worst shots that I always spend time trying to correct are just like that but with more light in the centre and deep dark all around. It gets even more noticeable when you watch it full screen.
Its the same in the original footage, and in the non-AQ encode, at least thats what my eyes tell me.

The quantizer in that area is a good bit higher though.

[lexor]

ok on the scale (it looks like some scaled creature in the background to me, I can't really tell what it is) right bellow the ball (top half of marked area) the 3 first ridges closest to the hand disappear in new AQ shot. That is, the black vertical lines that form the ridges disappear and what's left is a uniform light blue glow. If you have problem seeing it (monitor contrast and such may play a role), just save the two pics in new folder, open in Window default Picture and Fax viewer, and click forward button repeatedly so it cycles.

[Dark Shikari]

ok on the scale right bellow the ball (top half of marked area) the 2 first ridges closest to the hand disappear in new AQ shot. That is, the black vertical lines that form the ridges disappear and what's left is a uniform light blue glow. If you have problem seeing it (monitor contrast and such may play a role), just save the two pics in new folder, open in Window default Picture and Fax viewer, and click forward button repeatedly so it cycles.
Yeah, they're very very faint in the original though; its probably something that wouldn't be as noticable when playing back the video rather than freezeframing.

If you have a problem with that kind of artifact you can try setting --qpmax appropriately, or you can just use less powerful AQ.

I could also try to find a more intelligent method of picking which blocks to raise the QP on and by how much.

[CruNcher]

lexor i doub't you spot the difference in motion as from the position it looks like a high motion scene where a cut is following shortly :d

[lexor]

lexor i doub't you spot the difference in motion as from the position it looks like a high motion scene where a cut is following shortly :d

You'd think so, but... I still have nightmares about encoding a horror movie for a friend (back in Xvid days) and it had a long pan shot in a library room with light in the centre and darkness on peripheral. The general blurriness of Xvid would blend the thin vertical shadows that formed between books lined up on shelves into each other, horrible effect. This seems to bring it back and motion is likely to conceal more ridges and blur them into smooth light blue, if anything.

It is the greater natural sharpness of h264 (or at least x264 implementation) that brought me over. Give me blocks over blur any day. (it always puzzled me that doom9's codec comparisons always said that sharpness was the same, when every single xvid vs x264 shot screamed vast sharpness difference to my eyes)

[Dark Shikari]

lexor i doub't you spot the difference in motion as from the position it looks like a high motion scene where a cut is following shortly :d
Yeah, here's the full clips:

AQ (http://tjhsst.edu/~jgarrett/AQ.mkv)

No AQ (http://tjhsst.edu/~jgarrett/NoAQ.mkv)

[CruNcher]

It is the greater natural sharpness of h264 (or at least x264 implementation)
x264 as a H.264 implementation doesn't give the highest sharpest Look & Feel yet.
Mainconcept is a tad sharper but Ateme is WOW ;) i really can say that for me Ateme is the XviD of H.264 in Look & Feel without all the old ASP problems ;) it would be really strange if XviD 2.0 AVC would also Look & Feel like x264 does now.

[woah!]

can we see some of the Ateme results then please as i dont think their encoder is available to try is it.

i keep hearing about how good it is but never see any proof as such..

[CruNcher]

woah look in the low bitrate thread i posted some results their http://forum.doom9.org/showthread.php?t=129200&page=3


can we see some of the Ateme results then please as i dont think their encoder is available to try is it.


actually it is with Nero Recode 2/3 (Atemes consumer Encoder is working in their with High Profile support) :)

@Dark Shikari
i tested now with useing --aq-strength 0.1 and --subme 2 but the top line problem is still their hmm,
but yeah that padding is also visible without any AQ @ all but only very slightly (actually i didn't really realized it's there before), it seems the AQ amplifies (sharpens - makes it more perceptable) it.
visualy it seems that the higher --aq-strenght the more sharper everything get's to high and the result oversharpens and artifacting occours :) (this can indeed change the Digital Cinema Look & Feel of x264 to a much more sharper one if correctly used (due to the better quantization it seems), but this padding problem in the 1st macroblock colum is definately not in the source @ all.

[DeathTheSheep]

Which version of Ateme are you referring to above ("Ateme is WOW")? The one currently in Nero or the old betas?

[*.mp4 guy]

The old Betas, they are... much less restricted then what is in nero. Generally when someone says Ateme, I think it is safe to assume they are talking about Ateme' Beta 3 encoder cli, usually when people are talking about the version in recode, they say nero, in my experience.

[CruNcher]

The old Betas, they are... much less restricted then what is in nero. Generally when someone says Ateme, I think it is safe to assume they are talking about Ateme' Beta 3 encoder cli, usually when people are talking about the version in recode, they say nero, in my experience.

Yep but that has nothing todo with the Look & Feel both have the same only precission and end quality is affected but the core with all it's optimizations and tuneings is the same, else all my bug reports and improvement sugestions would have been for the trashbin ;)

[IgorC]

More and more people saying about their preferement to Ateme beta encoder.
Encoder has lower ssim and opsnr results (not much) but it's better visually.Blocking,banding free and higher level of details.
I suspect Ateme used more realistic and efficient metrics than MC's amd x264's ssim optimizations. cwssim?

[Dark Shikari]

More and more people saying about their preferement to Ateme beta encoder.
Encoder has lower ssim and opsnr results but it's visually.
Blocking,banding free and higher level of details.
I suspect Ateme used more realistic and efficient metrics than MC's amd x264's ssim optimizations. cwssim?
x264 is PSNR, not SSIM optimized in my experience--one can easily tweak the quantizers to raise SSIM more (my original AQ algorithm).

Anyways can we seriously get this thread back on topic?

[Fishman0919]

I noticed some loss of detail in the edge of the table. Noticed it a little when playing the clips... but much more so in the stills.

AQ = http://img217.imageshack.us/my.php?image=aqvt7.png

No AQ = http://img170.imageshack.us/my.php?image=noaqhb1.png

[Dark Shikari]

I noticed some loss of detail in the edge of the table. Noticed it a little when playing the clips... but much more so in the stills.

AQ = http://img217.imageshack.us/my.php?image=aqvt7.png

No AQ = http://img170.imageshack.us/my.php?image=noaqhb1.png
Ah, I see what's happening... that area doesn't actually have much detail, but it has a HUGE standard deviation due to the black/white contrast. So the QP is lowered a lot when it shouldn't be.

I'll see if there's a way to fix that, nice catch!

[Daodan]

I think that image has too strong aq in it. With dark movies (riddick could be considered as such) strong aq just takes too much bits from the few bright areas. Aq 0.5 should be a max in my opinion for those.
As to get back to my tests, they were accurate before. There is clear bframe quality boost which makes the image more stable (otherwise there's a slight flickering in background grain instead of constant movement). So at least for that movies, there's a clear improvement. As said before, wouldn't mind a real build with it:thanks:.
offtopic: my tests also show --me imh +subme 8 combination is a downgrade rather then upgrade in fine grain preservation. It gets slightly more ringy and inconsistent in the frame (so not equally distributed).