x264 brdo woes: bug or pathological case?

[Tack]

I was examining an x264 rip of my Harry Potter 4 DVD and discovered an area of especially egregious DCT blocking while otherwise using a fairly low crf value and high quality settings. I'm using x264 r661 with the AQ patch. Here are the settings (used with mencoder):
-x264encopts crf=20:frameref=10:partitions=all:bframes=4:threads=0:ssim:psnr:subq=7:mixed_refs:bime:brdo:nodct_decimate:weight_b:nofast_pskip:me=umh:me_range=24:cqm=/home/tack/.mplayer/prestige.matrix:b_pyramid:aq_strength=0.8:aq_sensitivity=17:8x8dct:qp_min=8:qp_max=49

The Prestige matrix can be found elsewhere on doom9.

The image below shows a frame demonstrating the problem spot using the above x264 settings. (I had to convert to jpeg to keep it within the attachment size limits, but the artifacts shown are not jpeg artifacts.)

http://img68.imageshack.us/img68/5877/brdosh0.jpg

Now compare with the one below, showing the same frame using the same settings except without brdo. Note that I did try several other settings, such as different CQMs (or no CQM at all), lower crf value, removing 8x8dct (which did mitigate the problem very slightly), no bframes (which unsurprisingly eliminated the problem altogether). brdo seems to be the culprit.

http://img409.imageshack.us/img409/2094/nobrdomh9.jpg

brdo does provide good bit savings and normally the quality is fairly good, so I think I've hit on one of those corner cases where brdo falls apart. Would this be considered a bug, or just an unfortunate reality of brdo?

[Sirber]

Best send on imageshack

[elguaxo]

This is a nice RDO thread: http://forum.doom9.org/showthread.php?p=1020459#post1020459

[Tack]

Best send on imageshack
Thanks, I've updated the post.

[Manao]

What does it give without the custom matrix ?

[Tack]

What does it give without the custom matrix ?
The default matrix doesn't change the situation much.

http://img442.imageshack.us/img442/3343/defaultcqmkd9.jpg

This particular frame isn't quite as bad, but the sequence overall still exhibits the blocking problem. (The worst-case frame has now just moved elsewhere in the sequence.)

[akupenguin]

See how many other options you can remove without affecting the problem.
Cut a small clip of the source including whatever the worst frame is after said tweaking of options, make sure the problem still occurs with just that clip, and send it to me.

[Tack]

See how many other options you can remove without affecting the problem.
Clearly this is an exercise I should have done sooner. Removing the AQ options made the blocks go away. So the blame does not lie with x264 here.

It's rather ironic that AQ should create the very artifacts I'm using it to prevent. What I don't understand is, based on my understanding of AQ, it would only raise per-macroblock quantizer based on a flatness threshold, never lower it. So I don't grok why AQ would interfere this way.

[akupenguin]

AQ lowers the QP. This makes any given mode take more bits and have higher quality. However, AQ does not change the lambda used to compare modes in RDO. Apparently RDO thinks the mode with reduced QP is taking too many bits compared to its relatively small improvement in quality, and decides to instead decimate the block (turn it into B-Skip). Solution: run x264_adaptive_quant() before x264_mb_analyse_init().

[Tack]

For completeness sake, here's a shot of the original settings (including brdo) without the AQ options.

http://img529.imageshack.us/img529/1808/noaqaz8.jpg

Now it's not as good as the case that includes AQ without brdo, but it also happens to use 30-35% less bits.

[Tack]

AQ lowers the QP.
Right, which is of course exactly what I meant. It's not even that difficult. :o

Solution: run x264_adaptive_quant() before x264_mb_analyse_init().
Interesting. I'll give this a try.

[bob0r]

"I'm using x264 r661 with the AQ patch"

What decoder are you using to make the screenshots?

x264 revision 663: limit vertical motion vectors to +/-512, since some decoders actually depend on that limit.

This fixes some major blocking issues on coreavc, maybe on other decoders too.

Before you continue to test, first try x264 revision 663 or newer and let us know if the problem remains.

Edit:
To below: Roger that

[akupenguin]

It's a DVD, and 2.4 DAR at that. It isn't even 512 pixels tall. The vertical mv limit thing only matters for HD.

[bob0r]

Okay, one of my encode masters has read this thread, and he said the following that might be the cause of all this + some added comments :)


that blocking issue he's getting is probably caused by him using 4 bframes and b-rdo
4th bframe in a chain will be complete mush
why the hell use 4 bframes with crf 20

I never exceed 3 on live action
more is only ok for anime or stuff with lots of static shots

at 0.8 you lose bits from light scenes majorly and overfeed blue/dark
so the 4th bframe in a chain will be like 5KB
next to no data
and instead of grain on the wall you end up with blocks because x264 has to pull bits from somewhere to feed blue/black with aq-strength so high
so it takes it from bframes leaving you blocks

yeah prestige/r663 aren't the cause of his woes
also dunno why he specified manual qp_min/qp_max values
the default 10/51 are fine
you will never reach 8 in an encode

well unless you're encoding like, pure black frames with 10mbps
It creates pixel crawl along those edges due to x264 being forced to use a row of 4x4 DCTs.

[akupenguin]

4th bframe in a chain will be complete mush
The P-frames and the middle B-frames are the ones that get worse, since they're farther from their references. The first and last B-frames are fine no matter how many consecutive B-frames you use.

and instead of grain on the wall you end up with blocks because x264 has to pull bits from somewhere to feed blue/black with aq-strength so high
so it takes it from bframes leaving you blocks
That would be a possibility in bitrate-based modes, but Tack used CRF. AQ in CRF doesn't pull bits from anywhere, it just increases the total bitrate. (as noted above: CRF20 w/ AQ was 30% higher bitrate than CRF20 w/o AQ.)
It's still an issue in that you could manually perform the bitrate adaption by reducing the value of CRF instead of turning on AQ, but it doesn't explain why the blocks appeared.

It creates pixel crawl along those edges due to x264 being forced to use a row of 4x4 DCTs.
Forces who what now? 4x4 vs 8x8 DCT is independent of bitrate/QP/AQ.

[Neo Fagin]

Forces who what now? 4x4 vs 8x8 DCT is independent of bitrate/QP/AQ.bob0r pasted the last line which was me talking to someone else in the same channel about a totally unrelated issue :P (wheither to crop/rescale sources with 2-3px of black borders along left/right edges)

[Tack]

Solution: run x264_adaptive_quant() before x264_mb_analyse_init().
Doing this appears to remove the effects of AQ altogether. That is, if I call x264_adaptive_quant() before x264_mb_analyse_init(), an encode with aq_strength=1.2:aq_sensitivity=5 has the exact same bitrate and SSIM as that with no AQ options. (I tested on one of my AQ torture tests where AQ definitely makes a difference.)

and instead of grain on the wall you end up with blocks because x264 has to pull bits from somewhere to feed blue/black with aq-strength so high so it takes it from bframes leaving you blocks
I'm a bit perplexed by this. I could understand this if I was using ABR, but with crf, why is it an issue?

[Tack]

(as noted above: CRF20 w/ AQ was 30% higher bitrate than CRF20 w/o AQ.)
Just to be clear, the 30% difference was comparing AQ+nobrdo with brdo. So brdo represents significant bit savings -- mind you, the difference is noticeable in that nobrdo retains more grain.

In this particular clip, all other things being equal, AQ uses only 4% more bits (1286kbit vs. 1336kbit). But I don't suppose this affects your point. :)

[akupenguin]

Doing this appears to remove the effects of AQ altogether. That is, if I call x264_adaptive_quant() before x264_mb_analyse_init(), an encode with aq_strength=1.2:aq_sensitivity=5 has the exact same bitrate and SSIM as that with no AQ options.
You probably left x264_ratecontrol_qp() as an input to x264_mb_analyse_init() ? ratecontrol's output needs to be modified by AQ.

[Tack]

You probably left x264_ratecontrol_qp() as an input to x264_mb_analyse_init() ? ratecontrol's output needs to be modified by AQ.
I did. This is my first look at x264 code, so I plead ignorance. :) All I did was move the call to x264_adaptive_quant above and left everything as is:

if( h->param.analyse.b_aq )
x264_adaptive_quant( h, &analysis );
/* init analysis */
x264_mb_analyse_init( h, &analysis, x264_ratecontrol_qp( h ) );


What ought the third parameter to x264_mb_analyse_init be?

[akupenguin]

h->mb.i_qp = x264_ratecontrol_qp( h );
if( h->param.analyse.b_aq )
x264_adaptive_quant( h, &analysis );
x264_mb_analyse_init( h, &analysis, h->mb.i_qp );
(The above is only theoretically correct, I haven't even tried to compile it.)
(It's not the simplest code to implement that behaviour, but it is the smallest change.)

Just to be clear, the 30% difference was comparing AQ+nobrdo with brdo. In this particular clip, all other things being equal, AQ uses only 4% more bits (1286kbit vs. 1336kbit).
There are 4 relevant bitrates: AQ+brdo, AQ+nobrdo, noAQ+brdo, noAQ+nobrdo. Which were those, and what were the other two?

[Tack]

(It's not the simplest code to implement that behaviour, but it is the smallest change.)Thanks, I will give it a try.

There are 4 relevant bitrates: AQ+brdo, AQ+nobrdo, noAQ+brdo, noAQ+nobrdo. Which were those, and what were the other two?
Well, I'm not sure where I got 4% from. I think I checked the bitrate from the wrong clip -- I try to keep the filenames descriptive but when you have 30 of them testing different options, it gets confusing. :)

I reran the encodes on this clip, taking more care:
AQ + brdo = 1349 kbit
AQ + nobrdo = 1694 kbit
noAQ + brdo = 1055 kbit
noAQ + nobrdo = 1128 kbit

One point of interest is that for the entire film (2h37), AQ+brdo was 1098kbit and AQ+nobrdo was 1465kbit.

[Tack]

(The above is only theoretically correct, I haven't even tried to compile it.)
It does seem to work in practice, as well.

AQ(modified as you suggested) + brdo = 1475kbit

http://img179.imageshack.us/img179/5986/aqtweaksqu0.jpg

I'll try reencoding the whole film with these new AQ tweaks and see what the overall bitrate ends up being. Thanks so much for all your help. :)

[Tack]

FYI, the reencode of the whole film with the AQ modifications ended up being 1224 kbit (as opposed to 1098 kbit with the stock AQ patch), but the problem areas have gone away.

Cheers!

[R3Z]

I did. This is my first look at x264 code, so I plead ignorance. :) All I did was move the call to x264_adaptive_quant above and left everything as is:

if( h->param.analyse.b_aq )
x264_adaptive_quant( h, &analysis );
/* init analysis */
x264_mb_analyse_init( h, &analysis, x264_ratecontrol_qp( h ) );


What ought the third parameter to x264_mb_analyse_init be?

Did you compile this as a binary ? MY next question is can i get a copy if so :)

[Cef]

I included this in my build (http://mirror05.x264.nl/Cef/), on Foxyshadis' request.

[R3Z]

I included this in my build (http://mirror05.x264.nl/Cef/), on Foxyshadis' request.

You absolute champion :) Thankyou so much :)

I made the changes to the code ok enough, but i couldnt compile without error :(

Again thanks mate :)

[Tack]

Did you compile this as a binary ? MY next question is can i get a copy if so :)I compiled it without any problems, and I'd be happy to provide a binary but I'm a Linux user, and I imagine you want a Windows build.

Looks like Cef has come to the rescue. :)

Cheers.

[Terranigma]

You absolute champion :) Thankyou so much :)

I agree with R3Z here. :D
Kudos to You. :)

[SpAwN_gUy]

I included this in my build (http://mirror05.x264.nl/Cef/), on Foxyshadis' request.can you update ".diff" and place it somewhere there? ;) ... and probably some other diffs.. it would be nice :) ...

[TheRyuu]

Thanks Cef.

On a side note, is it bad to use Trellis and AQ at the same time? (like trying to use Trellis and deadzone at the same time is impossible?)

I just encoded something with both. If I shouldn't use both please tell me. :)
I don't want to be screwing up my encodes.

Also, If I should use one, which one should I use.

Thanks
:thanks:

[Tack]

I found in my (albeit somewhat unmethodical) testing, trellis tended to introduce the very characteristics I was using AQ to prevent. So I avoid trellis.

[TheRyuu]

I found in my (albeit somewhat unmethodical) testing, trellis tended to introduce the very characteristics I was using AQ to prevent. So I avoid trellis.

So AQ would do almost the same thing as trellis only without the artifacts?

Just making sure since I was using them both (since at that time I really had no idea what trellis did). :p

[Manao]

So AQ would do almost the same thing as trellis only without the artifacts ?No, AQ and trellis have completely different purposes and effects.


Trellis will reduce the filesize while attempting to keep PSNR constant.
AQ will trade bits between low complexity areas and high complexity areas of the same frame ( improving the low complexity ones ), without caring about PSNR.

[TheRyuu]

No, AQ and trellis have completely different purposes and effects.


Trellis will reduce the filesize while attempting to keep PSNR constant.
AQ will trade bits between low complexity areas and high complexity areas of the same frame ( improving the low complexity ones ), without caring about PSNR.


Thanks. I guess I read something wrong when I was looking up the definition of trellis.

So is it ok to enable both of them at the same time?
Or would that be a bad idea.

I'm trying to figure out what to use on a cartoon source.
Both? One or the other? Or neither?

I think I'm just gonna use AQ for now.

[Tack]

There is no substitute for doing your own blind A/B tests. :)

[buzzqw]

i have done some test too... mainly on very dark movie

and this aq patch have deserved appreciation

from my drf analysis the quantitizer are distributed with less jump (and a less macroblock), also the bitrate hadn't go high (2 pass profile)


in next update of automkv i will include this build of x264 with a new 2 pass profile with aq enbled

good work!

thanks to all!

BHH

[elguaxo]

Thanks!

[SpAwN_gUy]

So is it ok to enable both of them at the same time?
Or would that be a bad idea.
mine tests show.. dead-zones=6 + aq = filesize ~700mb
trellis + aq = ~500Mb..
i guess i'm back to trellis
I'm trying to figure out what to use on a cartoon source.
Both? One or the other? Or neither?
me too (http://forum.doom9.org/showthread.php?p=1026282#post1026282) :)

I think I'm just gonna use AQ for now. i'm back to aq+trellis :)

[Tack]

i'm back to aq+trellis :)You might be better off without either. Certainly AQ without trellis results in a higher bitrate, but also in my experience a much better subjective quality. With trellis, I regularly observed overall muddiness and slight DCT blocking, which is precisely the kind of thing I was using AQ to prevent.

It'd be interesting then to compare a well-chosen clip with AQ without trellis, AQ with trellis, Trellis without AQ, and neither.

[ToS_Maverick]

i'd recommend this source:
http://forum.doom9.org/showthread.php?p=947775#post947775

i still got the file, if somebody is interested, i'd upload it again!

after you posted this thread i checked it with this sample, now i'm at subme 5, without trellis. but check yourself :D
the bitrate increase from sub6 + trellis to sub5 w/o trellis was about 10 %.

[Tack]

I'm interested in having a peek. I can't access the site referenced in the post you linked to though (xasonline.info). If you can upload it elsewhere, that'd be great.

[ToS_Maverick]

as promised, here's the link:
http://www.megaupload.com/?d=IBQ2DKD6

[Tack]

Thanks, I will experiment with this tonight.

It looks like even 9800kbit mpeg2 isn't enough for this content; I'm seeing quite a bit of ringing even in the source. (Although this particular clip averages 3600kbit.)

[Tack]

So I did four encodes (involving permutations of AQ and trellis) using the following base:
mencoder Black.Pearl.Sample.m2v -nosound -ovc x264 -x264encopts crf=20:frameref=10:partitions=all:bframes=4:threads=0:ssim:psnr:subq=7:mixed_refs:bime:nodct_decimate:weight_b:nofast_pskip:me=umh:me_range=24:cqm=/home/tack/.mplayer/prestige.matrix:brdo:b_pyramid:8x8dct -vf pullup,softskip,crop=704:352:8:62 -ofps 24000/1001 -o $1

When I enabed AQ, it was with aq_strength=0.8:aq_sensitivity=17 and when I enabled trellis, it was with trellis=2.

The objective results:

No AQ, no trellis: 1386kbit, global PSNR = 43.713, SSIM = 0.9738813
No AQ, trellis: 1287kbit, global PSNR = 43.914, SSIM = 0.9743550
AQ, no trellis: 1586kbit, global PSNR = 43.818, SSIM = 0.9748233
AQ, trellis: 1545kbit, global PSNR = 44.040, SSIM = 0.9754086


With a blind test, I was easily able to distinguish the source from the above encodes (which is typical, especially with very grainy films). I preferred 3 as it had the best grain retention and smeared less, with less DCT blocking in darker areas. I disliked 2 the most, as it was flat and smeared. I wasn't able to distinguish 1 and 4, but did rank them both below 3.

I'll continue to use AQ without trellis. Although I suppose to be fair, I should test the other combinations with a crf that yields the same (or close) bitrate as the AQ without trellis case.

[Manao]

Although I suppose to be fair, I should test the other combinations with a crf that yields the same (or close) bitrate as the AQ without trellis caseIndeed, because you prefered the largest encoding, and disliked the smallest one. That's not conclusive.

The only thing that can be concluded from your test is that PSNR sucks when it comes to psychovisual testing.

[Tack]

Indeed, because you prefered the largest encoding, and disliked the smallest one. That's not conclusive.
Yes, fair enough. For my own benefit I'll repeat these tests at as close a bitrate as I can manage.

Obviously anyone interested in the outcome should perform their own tests.

[RaynQuist]

I did some testing on the grain quality difference between 667 and 667b. I took the first 20070 frames of this DVD I'm working on. Only the first 2000ish frames are very grainy; the rest is relatively clean. This is typical of anime where only select parts have artificial grains added.

Script:
DGDecode_mpeg2source("C:\Documents and Settings\Ray Chen\Desktop\AQTest\VTS_01_1.d2v",info=3)

ColorMatrix(hints=true)

TDeint(full=false)

crop(8, 0, -4, 0)

LanczosResize(832,480,0,0,706,480)


Options:
--pass 2 --bitrate 937 --stats ".stats" --keyint 480 --min-keyint 12 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,1 --subme 7 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --merange 32 --threads auto --thread-input --progress --output "C:\Documents and Settings\Ray Chen\Desktop\AQTest\667b-aq-0.7.264" "C:\Documents and Settings\Ray Chen\Desktop\AQTest\script.avs"

Resulting numbers. The first bitrate number is the bitrate for just the grainy part. Overall bitrate is in ():
bitrate avgPSNR glbPSNR SSIM AvgIQP AvgPQP AvgBQP
667 -AQ-0.3 974(937.63) 48.666 48.285 0.9888645 15.19 16.51 17.99
667b-AQ-0.3 975(937.72) 48.667 48.286 0.9888940 15.30 16.66 18.09

667 -AQ-0.5 1086(937.59) 48.602 48.214 0.9887318 15.26 16.80 18.16
667b-AQ-0.5 1097(937.81) 48.592 48.202 0.9887604 15.47 17.08 18.38

667 -AQ-0.7 1264(937.97) 48.513 48.111 0.9885490 15.37 17.21 18.41
667b-AQ-0.7 1283(938.01) 48.493 48.086 0.9885743 15.65 17.54 18.69
Total bitrate achieved is very constant across all encodes. High AQ strength moves more bits to the grain. 667b didn't move much more bits to the grainy section than 667 as I had theorized. Instead 667b mostly moved bits from I and P frames to B frames. PSNR decreased and average quantizers increased going from 667 to 667b as expected. SSIM seems to get a tiny increase however.

Quantizers and Bit Distribution (from ffdshow)
667 AQ 0.7
http://img46.imageshack.us/img46/5919/rcqpbeforecroppedqh0.png
667b AQ 0.7
http://img510.imageshack.us/img510/7501/rcqpaftercroppedtu9.png
Here you can see how much 667 skips its B blocks. I and P frames are fine for 667 and get AQ'ed consistently across the entire frame. 667b fixes all the b-skip and as you can see in the bitrate graph that b frames are actually consuming bits. Be careful that the two bitrate graphs are not directly comarable since ffdshow scales the graph based on the max bitrate.

Usually I do aq-strength 0.7 on my encodes. With 667b I thought I may need to tone down the strength due to all the b blocks that are now not getting skipped. This is not the case as 667b did not take much more bits from the rest of the film to encode the grains. What we have here is a tradeoff. 667 gives you finer grains that get b-skipped, while 667b gives you lower quality grains but its temporal randomness is more consistent.

Academically I think 667b is more "correct". Subjectively I would not be able to tell the difference between under normal playback conditions. Although the grain quality in this particular source is quite poor (which I did not know until started zooming in in my comparisons), and my eyes aren't that good at pick up things (hence why I'm encoding at low bitrate).

Did increasing AQ strength decrease the quality of the rest of the film? C'mon, it's anime. All the lines are beautifully preserved. To me all 6 encodes achieved transparency for all the non-grainy parts.

[Tack]

I did some testing on the grain quality difference between 667 and 667b.Very interesting assessment.

Academically I think 667b is more "correct". Subjectively I would not be able to tell the difference between under normal playback conditions.At least until you hit a sequence that causes 667 to crumble, such as the one that originally prompted me to start this thread. 667b solved the problem very nicely with an acceptable increase in bitrate (IMHO, as always).

Cheers!

[SpAwN_gUy]

/EVERYTHING DELETED/
yes i'm with stupid... i guess i've made something wrong.. and my builds of cef's sources with external pthread.dll (i can't manage it to compile as internal lib) show good results..

sorry no testings for now.. i'll be back (i guess) next week or so..

[Sharktooth]

Cef's 667b sources already include AQ...

[SpAwN_gUy]

Cef's 667b sources already include AQ...Well... i DO know that.. but i can't build... 'kay this is not the proper thread..

i just asked him to update .diff for aq-patch ... 'cause on your site it is very old and does not include latest changes from somwhere here...

[plane]

It's a DVD, and 2.4 DAR at that. It isn't even 512 pixels tall. The vertical mv limit thing only matters for HD.

I have some stupid questions to ask. I don't know what is vertical motion vectors, is that related to the vertical resolution? I usually use x264 to encode my PAL DV clips, which is 576 pixels tall? It seems this change is good to use but will there be any potential negative impact by limiting the vertical motion vectors to +/-512?

[RaynQuist]

That only matters if you have something that moves more than 512 pixels up or down the screen from one frame to the next, which is unlikely to happen especially for SD sources. Usually for SD sources you should limit your search range to well under 512, like 16~32. It won't hurt compressibility to have more than that, it's just diminishing returns.

[Dark Shikari]

That only matters if you have something that moves more than 512 pixels up or down the screen from one frame to the next, which is unlikely to happen especially for SD sources. Usually for SD sources you should limit your search range to well under 512, like 16~32. It won't hurt compressibility to have more than that, it's just diminishing returns.
And isn't it basically impossible for the encoder to find such motion anyways, especially with the default motion search radius of... 16?

[Manao]

This has been explained over and over again :
- merange sets the search radius around the best candidate. ie, with a merange of 16, if a neighbouring macroblock has a motion vector of (x,y), you can reach from (x-16,y-16) to (x+16,y+16). It adds up.
- mvrange limits the actual vector's length

Furthermore, there's no diminishing returns for merange over 16, except with esa which you shouldn't use anyway. By that, I mean that except for esa, you can use as big a merange as you want, it doesn't actually matter for neither speed nor quality. So it's best to leave it untouched.

[Dark Shikari]

By that, I mean that except for esa, you can use as big a merange as you want, it doesn't actually matter for neither speed nor quality.
Yeah, right. :rolleyes:

--subme 7, --trellis 2, --me umh, --ref 16, --no-fast-pskip and other standard settings on 40,000 frames of a DVD:

--merange 64: 2.45 FPS
--merange 32: 3.5 FPS
--merange 16: 4.26 FPS

[Terranigma]

Shikari, have you seen neuron2's in-depth .pdf document (http://www.neuron2.net/library/avc/overview_x264_v8_5.pdf) about motion estimation (2.2)?

Doesn't get more thorough than that.

[Dark Shikari]

Shikari, have you seen neuron2's in-depth .pdf document (http://www.neuron2.net/library/avc/overview_x264_v8_5.pdf) about motion estimation (2.2)?

Doesn't get more thorough than that.
I just read it and most of it I already knew from looking at the code, but it makes sense.

I know exactly why high --merange slows it down though; there are plenty of for loops with merange as the primary maximum variable, so doubling merange results in those for loops running more times.

[foxyshadis]

Those for loops have break;s though, for when the ME reaches what it believes is the global minimum. Your content must confuse the ME. =p

I just tested a few scenes, one cake, one moderate, one hard. 1: Speed dropped (15->13->10), SSIM went up minutely, no effect on bitrate or PSNR. 2: Steady speed drop (13->11->9), no effect on SSIM, PSNR, or bitrate. 3: Again, steady but not horrible speed drop (8->6.5->5), no effect on SSIM, bitrate dropped marginally, PSNR dropped marginally.

Overall impression: Worthless. :p

Oops, that was tested on the SSIM-SAD optimized; however, testing with the original shows virtually identical behavior. (And better SSIM/bitrate tradeoff than the optimized, again, though it's a totally different type of source, anime/cg this time.)

Command line:
--crf 22.0 --keyint 400 --ref 6 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -1,0 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --output "V:\video\DVDRIP\Rozen Maiden 1\cossette-fast.mkv" "V:\video\DVDRIP\Rozen Maiden 1\cossette-fast.avs" --aq-strength 0.3

[Manao]

I wasn't aware that umh ended with an exhaustive hex search. So indeed umh slows down with merange, but the quality doesn't need anything bigger that 16.

hex & dia aren't affected by big merange however, because though there's a loop based on merange, there's only an exit to that loop.

[Dark Shikari]

Command line:
--crf 22.0 --keyint 400 --ref 6 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -1,0 --subme 6 --trellis 2 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --threads auto --thread-input --progress --no-dct-decimate --output "V:\video\DVDRIP\Rozen Maiden 1\cossette-fast.mkv" "V:\video\DVDRIP\Rozen Maiden 1\cossette-fast.avs" --aq-strength 0.3
Rozen Maiden ftw :cool: