VAQ 2.0 Alpha Testing

[Dark Shikari]

VAQ 1 is official (http://git.videolan.org/?p=x264.git;a=commit;h=dc4f40ce74c996a1e15021b82ab71ebcb8652d0b), so as I promised, I am releasing VAQ 2.0 for initial alpha testing. Thanks to Alex W for the basic concept behind VAQ 2.0!

How to use (change from normal VAQ):

--aq-metric presents four options for the AQ metric to use. Each is progressively slower than the last; option 0 is equivalent to VAQ1.

Since the VAQ committed to official x264 has 1.0 strength as default (equivalent to 0.5 with unofficial AQ, since all values were doubled for simplicity's sake), 1.0 is default here also.

Try playing with the different AQ metrics and strengths and testing the results.

Download patched build (based on r777) (http://www.mediafire.com/?yyzxvzcz2tm).

[IgorC]

--aq-mode?
Should it be by default fot this time of testing?


Is r777 with VAQ 2.0 multhitreaded? It uses only one core of my e2160 while r798.modified (VAQ 0.48) use 2 cores.

x264_AQ2.exe --threads 3 --pass 3 --progress --stats "x264_stat.log" --qcomp 0.75 --bframes 3 --no-b-adapt --bime --weightb --subme 7 --keyint 300 --ref 8 --trellis 2 --mixed-refs --8x8dct --partitions all --b-rdo --direct auto --b-pyramid --bitrate 2500 --no-fast-pskip --me umh --merange 16 --aq-metric 3 --deblock -1:-1 -o 2xVAQ2_3_Balance2_777.h264 1.avs

[LoRd_MuldeR]

--aq-mode?

--aq-mode <integer> How AQ distributes bits
- 0: Disabled
- 1: Avoid moving bits between frames
- 2: Move bits between frame

I'd guess that "--aq-mode 2" is the default, while "--aq-mode 1" equals "aq-sensitivity 0" in VAQ v0.48

[IgorC]

I know. The question is what to do with it. Leave it or test it?

[LoRd_MuldeR]

I know. The question is what to do with it. Leave it or test it?
Well, unless I have missed something, "--aq-mode 1" (aka "aq-sensitivity 0") is not recommended and "--aq-mode 0" will turn off all AQ.
So I'd go with "--aq-mode 2" unless there is a good to do otherwise...

[Dark Shikari]

Well, unless I have missed something, "--aq-mode 1" (aka "aq-sensitivity 0") is not recommended and "--aq-mode 0" will turn off all AQ.
So I'd go with "--aq-mode 2" unless there is a good to do otherwise...Correct

[MythCreator]

@Dark Shikari

Could you release the diff file of VAQ 2.0? Thanks a lot~

[Dark Shikari]

@Dark Shikari

Could you release the diff file of VAQ 2.0? Thanks a lot~The code is extremely ugly and the diff won't work on any recent revision of x264. Anyone who publicly distributes builds of this with any intent other than to experiment is crazy.

diff (http://pastebin.com/f5034219f)

[Selur]

two side questions:
1. What in the value range allowed for --aq-strength?
2. Does --aq-strength 0.0 also disable adaptive quantization?

Cu Selur

Ps.: there's a 'new-line' missing before '--nr' (after metric options) when calling --long-help ;)

[Dark Shikari]

two side questions:
1. What in the value range allowed for --aq-strength?
2. Does --aq-strength 0.0 also disable adaptive quantization?
AQ strength = 0 to infinity (though values above 2.0 may get stupid).

AQ strength = 0 = disabled.

Exactly the same as the AQ in the official x264.

[Selur]

Thanks for the Info. :)

Cu Selur

[MasterNobody]

For those who are interested here (http://stashbox.org/97383/x264_vaq2_alpha.diff) is diff which can be applied on the current git version (r808). Also I fix bug in x264_autosense_aq (incorrect use of h->mb.i_mb_x and h->mb.i_mb_y instead of mb_x and mb_y)

[bob0r]

Warning: ALPHA BUILD, ONLY USE FOR TESTING VAQ2!

x264.808.vaq2.modified.exe (http://files.x264.nl/VAQ2/x264.808.vaq2.modified.exe) (gcc 3.4.6 fprofiled)

General thread:
http://forum.doom9.org/showthread.php?t=130364

x264_vaq2_alpha.01.r808.diff
http://forum.doom9.org/showthread.php?p=1120023#post1120023
x264.gaussian.cplxblur.01.diff
Dark Shikari: - gaussian cplxblur: gives a tiny improvement in 2pass ratecontrol
x264_me-prepass_DeathTheSheep.01.diff
http://forum.doom9.org/showthread.php?p=1093523
x264_2pass_vbv.6.diff
http://thread.gmane.org/gmane.comp.video.x264.devel/3093/focus=3550
x264_hrd_pulldown.04_interlace.diff
- HRD and pulldown for HD compatibility, updated patch for interlacing
http://forum.doom9.org/showthread.php?p=1047919#post1047919

Link to x264 patches collected: http://files.x264.nl/x264_patches/

Warning: ALPHA BUILD, ONLY USE FOR TESTING VAQ2!

[Umamio]

Here is a collection of tests I did on a couple of 4 second football clips. They are slightly worthless for comparison purposes because they are mostly different framesizes and different bitrates and CQ / 2pass / 3pass whatever.

The only thing they have in common is they all use VAQ2 (using patched build based on r777)! Unless specified.

So if you want a collection of slightly useless (for comparative purposes) encodes of football on a rainy imperfect pitch you can get them here (http://www.mediafire.com/?t1mgyltzkti).

Better comparisons in here (http://forum.doom9.org/showthread.php?p=1120492#post1120492)

[DeathTheSheep]

Excellent job, MasterNobody. Thou art truly the master o'er bugs, o BugMaster. :)

I will certainly give this a test soon. How is the speed on BugMaster's mod? Does it do the same thing as the original on 777?

[DeathTheSheep]

The answer appears to be yes on both fronts.

I made a test on my quintessential anime clip.
0.46 yields 0.9741681 SSIM 4537KB
2.0a yields 0.9710652 SSIM 4557KB (--aq-strength 2 --aq-metric 3)
2.0a yields 0.9724646 SSIM 4546KB (--aq-strength .3 --aq-metric 3)

Baseline profile, defaults, no fast pskip.

[Razorholt]

The answer appears to be yes on both fronts.

I made a test on my quintessential anime clip.
0.46 yields 0.9741681 SSIM 4537KB
2.0a yields 0.9710652 SSIM 4557KB (--aq-strength 2 --aq-metric 3)

Ah. Baseline profile, defaults, no fast pskip,

What's your --aq settings with 0.46?

[DeathTheSheep]

Updated post.

Settings with 0.46 were strength 1.1, sens 24.5. Of course with better settings and keyframe tweaks (since DS found problems near keyframes in my past 46vs47 tests), SSIM can go up to .9752XXX easily with 0.46. With qcomp active for anime, even more still.

[edit] But by goodness that's not to say 0.46 is better than 2.0!! Just that for this clip, for these settings, at this bitrate, for this material, SSIM is higher.

[Dark Shikari]

The answer appears to be yes on both fronts.

I made a test on my quintessential anime clip.
0.46 yields 0.9741681 SSIM 4537KB
2.0a yields 0.9710652 SSIM 4557KB (--aq-strength 2 --aq-metric 3)
2.0a yields 0.9724646 SSIM 4546KB (--aq-strength .3 --aq-metric 3)

Baseline profile, defaults, no fast pskip.Please stop poisoning my development threads with your totally useless comparisons that:

1) Do not include no-AQ numbers.
2) Only include values for one metric, and with two wildly varying strengths with nothing sane in between.

:rolleyes:

P.S. SSIM is not that useful. SSIM tells me that AQ mode 1 is better than 2, which it isn't.

[Razorholt]

@DS:

1.- Is there any minimum birates required for VAQ2 to be efficient?
2.- How can we compare different settings if SSIM is irrelevant?
3.- What is the --aq mode/metric/strength default in VAQ2?


Thanks a lot DS for your great job!

- Dan

[Dark Shikari]

@DS:

1.- Is there any minimum birates required for VAQ2 to be efficient?I don't really know yet.
2.- How can we compare different settings if SSIM is irrelevant?SSIM isn't necessarily irrelevant, but its not always useful. The best judge is always your eyes.
3.- What is the --aq mode/metric/strength default in VAQ2?Mode=2, Metric=3, Strength=1.

[DeathTheSheep]

DS: You could've just asked for the non-AQ number, some strengths in between, and/or some PSNR's. I'd be only too happy to oblige. It somehow gives off a nicer impression than calling something poison. :)

[Dark Shikari]

DS: You could've just asked for the non-AQ number, some strengths in between, and/or some PSNR's. I'd be only too happy to oblige. It somehow gives off a nicer impression than calling something poison. :)Okay, so I was a little harsh :p It just seems like you intentionally ignored the most obvious settings, and chose ones way far to the sides. :)

[ToS_Maverick]

i did a short test @CRF 22 and got these results:

VAQ1 0.9716895
VAQ2 0.9727707

VAQ2 was slightly better visually too. at CRF 20 there is only a very slight difference, which i think is also true for CRFs below 20

[Encoder888]

I apologize for my dumb questions in advance, as I'm not as experienced with AQ as you are:

So, in VAQ2, the sensitivity option is eliminated, right? Since mode should almost always be set on 2, in order to even have the AQ on, then all you can really adjust is the strength and the metric right? So, I take it the strength works the same way as before, the higher, the better the quality. And the metric? From what I got from your posts, you can only adjust it from 0 - 6, and so, is it the higher the metric, the better the quality? What exactly does the metric do? Is it like sensitivity in the previous VAQ, or was that mode...? Thanks for reading my stupid question :)

[Dark Shikari]

I apologize for my dumb questions in advance, as I'm not as experienced with AQ as you are:

So, in VAQ2, the sensitivity option is eliminated, right? Since mode should almost always be set on 2, in order to even have the AQ on, then all you can really adjust is the strength and the metric right? So, I take it the strength works the same way as before, the higher, the better the quality. And the metric? From what I got from your posts, you can only adjust it from 0 - 6, and so, is it the higher the metric, the better the quality? What exactly does the metric do? Is it like sensitivity in the previous VAQ, or was that mode...? Thanks for reading my stupid question :)The metric is the method of calculating which blocks get what quantizer. Its a mathematical formula.

[cogman]

So VAQ uses metricies to attempt to give better results correct? I thought part of the purpose of VAQ 1 was to eliminate the need for metricies (cqm). So is the some major fundamental difference between the terms?

Also, given that differing metricies give different results, is the quality difference going to be somewhat of a constant (IE Matrix one gives better quality then matrix two, but with a slower encode)? Or is this going to be a guess and check sort of solution to picking which matrix to encode with.

Just curious.

[Dark Shikari]

So VAQ uses metricies to attempt to give better results correct? I thought part of the purpose of VAQ 1 was to eliminate the need for metricies (cqm). So is the some major fundamental difference between the terms?A matrix has nothing to do with a metric. They're about as related as a beaver and a muon.

[cogman]

A matrix has nothing to do with a metric. They're about as related as a beaver and a muon.

lol, ok, I guess I was just mixing up terms. Thanks for the responce.

Though, is there going to be a fairly standard quality gain/loss with the use of different metrics? or will it just depend video to video?

[Umamio]

Source:
Football Clip - 720x576 (16:9) - 25fps interlaced - Huffyuv
Smartbobbed & Resized - 688x384 - 50fps - Huffyuv [ Download Here (79.3MB) (http://www.mediafire.com/?jjfzmzrex4j) ]
Duration: 6s (331 frames)

x264 build:
x264.808.vaq2.modified.exe (gcc 3.4.6 fprofiled)
[ http://files.x264.nl/VAQ2/x264.808.vaq2.modified.exe ] [ http://forum.doom9.org/showthread.php?p=1120033#post1120033 ]

x264 args:
--pass 2 --bitrate 1284* --stats ".stats" --ref 3 --bframes 16 --b-pyramid --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 12 --aq-strength (variable:[0.5,1,1.5,2,2.5]) --aq-metric (variable:[0,1,2,3])

*1284k/50frames == 642k/25frames

Encodes:

Metrics 0(VAQ1.0),1,2,3 * Strengths 0.5,1.0,1.5,2.0,2.5
+ VAQ off (Strength 0, Metric 0)
[ Download Here (22.85MB, 21 files zipped) (http://www.mediafire.com/?izzzwxjxdma) ]


Single Frame Comparison (VAQ off Vs. Metric 3 Strength 1 Vs. Metric 3 Strength 1.5 Vs. Metric 3 Strength 2 Vs. Source):

http://thumbnails.freeimagehost.eu/291/8fd4c82908457.gif (http://www.freeimagehost.eu/image/8fd4c82908457)
(click to enlarge)

I only have PSNR and SSIM numbers for the metric 3 encodes and with VAQ off, but you can see from the files that there is a clear visual improvement without looking at the values (I haven't seen the logs yet)

For this specific type of high... grass source at least the optimal Strength seems to be between 1.3 and 2.2, I'm still staring at these trying to compare the metrics but there might not be enough fine detail in my source to make an accurate judgement using this material.

Conclusion: VAQ loves grass sports and I love VAQ.

Edit:


Some PSNRSSIMnumberstuff:
VAQ Off:
SSIM Mean Y:0.9087551
PSNR Mean Y:33.601 U:40.486 V:41.777 Avg:34.978 Global:34.571 kb/s:1377.74

Metric 3 - Strength 0.5
SSIM Mean Y:0.9157095
PSNR Mean Y:33.513 U:40.607 V:41.806 Avg:34.904 Global:34.598 kb/s:1387.76

Metric 3 - Strength 1.0
SSIM Mean Y:0.9233749
PSNR Mean Y:33.339 U:40.752 V:41.876 Avg:34.751 Global:34.409 kb/s:1385.98

Metric 3 - Strength 1.5
SSIM Mean Y:0.9265114
PSNR Mean Y:32.728 U:40.651 V:41.664 Avg:34.173 Global:33.816 kb/s:1369.50

Metric 0 - Strength 1.5
SSIM Mean Y:0.9256282
xPSNR Mean Y:33.066 U:40.626 V:41.715 Avg:34.488 Global:34.161 kb/s:1379.88

Metric 3 - Strength 2.0
SSIM Mean Y:0.9226995
PSNR Mean Y:31.642 U:40.306 V:41.231 Avg:33.132 Global:32.750 kb/s:1357.14

Metric 0 - Strength 2.0
SSIM Mean Y:0.9247225
PSNR Mean Y:32.291 U:40.411 V:41.386 Avg:33.749 Global:33.420 kb/s:1366.41

Metric 3 - Strength 2.5
SSIM Mean Y:0.9108505
PSNR Mean Y:30.138 U:39.825 V:40.674 Avg:31.679 Global:31.292 kb/s:1347.17

Disclaimer: I mostly don't know what I am talking about or what I am doing, so I advise you to not trust any of this and do try things out for yourself.

[ToS_Maverick]

you shouldn't forget, VAQ uses qcomp of 1.0 while the standard x264 setting is 0.6.

the bottom line is, with qcomp 0.6, fast motion (like football) doesn't get as much bitrate as i would need. please try 1.0 without AQ, that would be interesting. maybe that's a reason why AQ looks so much better.

[Terranigma]

you shouldn't forget, VAQ uses qcomp of 1.0 while the standard x264 setting is 0.6.

the bottom line is, with qcomp 0.6, fast motion (like football) doesn't get as much bitrate as i would need. please try 1.0 without AQ, that would be interesting. maybe that's a reason why AQ looks so much better.

You sure you know what you're talking about? According to x264 --longhelp, qcomp's default is now 1.0.
(Well, from the build I grabbed.)

[Dark Shikari]

You sure you know what you're talking about? According to x264 --longhelp, qcomp's default is now 1.0.
(Well, from the build I grabbed.)That's because AQ is on by default ;)

[Umamio]

you shouldn't forget, VAQ uses qcomp of 1.0 while the standard x264 setting is 0.6.

Ah you're right, I forgot about qcomp getting changed but in this clip it doesn't seem to make very much difference at all when I set it to 1. QAV off, Qcomp-1, Download (1.11MB) (http://www.mediafire.com/?c2lmg3bxhdj)
8275

I'm interested in seeing the best that I can get out of x264 for football without using QAV to make it a better (but less fair) comparison, do you have any suggestions for other parameters I could change?

[ToS_Maverick]

That's because AQ is on by default ;)

well does it revert back to 0.6 if you switch it off?

[Dark Shikari]

well does it revert back to 0.6 if you switch it off?It did in the patch I gave to pengvado... you can check pretty easily by looking at the header of the output H.264 file to see if he kept it that way.

[Tack]

Conclusion: VAQ loves grass sports and I love VAQ.
The first thing I noticed was how much worse the BBC logo was in the bottom (VAQ) image.

No doubt though that the grass is much better. I suppose one could make a fairly solid argument that VAQ is causing the bits to be spent in the right places.

[Umamio]

The first thing I noticed was how much worse the BBC logo was in the bottom (VAQ) image.

No doubt though that the grass is much better. I suppose one could make a fairly solid argument that VAQ is causing the bits to be spent in the right places.

If you look at the logo in any of the encodes (http://www.mediafire.com/?izzzwxjxdma) with strengths < 1.5 the logo and score suffer a lot less with not much of a loss in grass detail.

I only tested in strength steps of 0.5 so it's likely that I missed the VAQ strength "sweet spot" for this particular video but that's for another day. This post was really just to illustrate the variations of the different strengths and metrics. It's likely that the quality could be improved with a little more bitrate and more focussed small-step strength testing but the differences may not be as noticeable if the algorithm has lots of bits to spare.

If you encode something at a stupidly low bitrate with a stupidly high VAQ and compare it to the same thing encoded at the same stupidly low bitrate with vaq turned off you can really see the types of detail that VAQ focusses its bits on.

Actually, I will do that for you (http://www.mediafire.com/?wtyacgyxjcy). But make sure you look at the regular stuff before you look at these extreme/stupid examples or it might affect your judgement. :)

P.S: I could be wrong about everything I just said.

[Dark Shikari]

Edit:


Some PSNRSSIMnumberstuff:
VAQ Off:
SSIM Mean Y:0.9087551
PSNR Mean Y:33.601 U:40.486 V:41.777 Avg:34.978 Global:34.571 kb/s:1377.74

Metric 3 - Strength 0.5
SSIM Mean Y:0.9157095
PSNR Mean Y:33.513 U:40.607 V:41.806 Avg:34.904 Global:34.598 kb/s:1387.76

Metric 3 - Strength 1.0
SSIM Mean Y:0.9233749
PSNR Mean Y:33.339 U:40.752 V:41.876 Avg:34.751 Global:34.409 kb/s:1385.98

Metric 3 - Strength 1.5
SSIM Mean Y:0.9265114
PSNR Mean Y:32.728 U:40.651 V:41.664 Avg:34.173 Global:33.816 kb/s:1369.50

Metric 0 - Strength 1.5
SSIM Mean Y:0.9256282
xPSNR Mean Y:33.066 U:40.626 V:41.715 Avg:34.488 Global:34.161 kb/s:1379.88

Metric 3 - Strength 2.0
SSIM Mean Y:0.9226995
PSNR Mean Y:31.642 U:40.306 V:41.231 Avg:33.132 Global:32.750 kb/s:1357.14

Metric 0 - Strength 2.0
SSIM Mean Y:0.9247225
PSNR Mean Y:32.291 U:40.411 V:41.386 Avg:33.749 Global:33.420 kb/s:1366.41

Metric 3 - Strength 2.5
SSIM Mean Y:0.9108505
PSNR Mean Y:30.138 U:39.825 V:40.674 Avg:31.679 Global:31.292 kb/s:1347.17Interesting that the optimal strength for SSIM is so high...

[DeathTheSheep]

A really interesting thing happened to me with this.

While using CQP of 30 for the original and modeling the strength around it to produce similar sized files (since there's no sensitivity any more for 1-pass bitrate fine-tuning!!), 0.46 is much better (see my post on the first page of this thread). Only strength 2 and 0.3 were close enough to the original filesize, but even with a larger filesize, strength 1.0's ssim was pretty low compared to the AQ 0.46. ALL aq tested then were better than the non-AQ SSIM.

BUT. Here's the interesting part. In 2-pass vbr, there is MUCH LESS DIFFERENCE between the various forms of AQ for the sample at target bitrate 400kbps (size ~4155KB). See the results below, all AQ using mode 3 for now:
NO AQ: 0.9702865, PSNR: 37.043-o36.709
NO AQ(qcomp1): 0.9672502, PSNR: 36.844-o36.397
st 0.5: 0.9692864, PSNR: 36.553-o36.010
st 1.0: 0.9693076, PSNR: 35.846-o35.166
st 1.5: 0.9660272, PSNR: 34.648-o33.833
st 2.0: 0.9576726, PSNR: 32.911-o31.958
0.46, st 1.0, sens 16: 0.9705006, PSNR: 36.159-o34.722.264

As you can see, the SSIM hardly rises at all (in fact it falls for every AQ strength other than 0.46).

I don't usually use 2-pass mode, preferring a target quality rather than an exact filesize. This is why I haven't done much testing in this area.

I do think (like another member has pointed out), that either sensitivity needs to be re-implemented (preferably, since I get very different results using sens 16 and sens 25 at the same filesize for aq 0.46), or allow decimal QP/CRF to be specified.

[edit]AH, forgot to mention. Clip is same one as I used/gave you before.
Commandline was: x264 test.avs -o out.264 --aq-strength {x} --aq-metric 3 --no-fast-pskip --no-cabac -A all --thread-input --me umh --pass 2 --bitrate 400 {--qcomp 1} --keyint 1500

[Umamio]

Interesting that the optimal strength for SSIM is so high...

My theory is that because over 80% of the pixels in this clip make up grass, and because the detail in the grass is so faint, even if I destroyed all the foreground stuff with massive strength the SSIM would still be relatively high so the SSIM penalty for smashing foreground detail is lower than it would be for something more normal, like a film or TV series.

Maybe.

[Dark Shikari]

since I get very different results using sens 16 and sens 25 at the same filesize for aq 0.46)If this is the case, its a bug. I will begin reading your posts again if you can pinpoint the source of this bug.

I'm serious here; I noticed this a while back, and it really shouldn't be the case. If you can find me the reason in the code why varying sensitivity can potentially change the quantizer distribution significantly in twopass mode, that would be quite useful...

[woah!]

VAQ48 = SSIM Mean Y:0.9810858 8.17 fps, 5415.96 kb/s

VAQ 0 = SSIM Mean Y:0.9811255 8.47 fps, 5440.41 kb/s

VAQ 1 = SSIM Mean Y:0.9812286 8.21 fps, 5427.13 kb/s

VAQ 2 = SSIM Mean Y:0.9815481 7.89 fps, 5432.41 kb/s

VAQ 3 = SSIM Mean Y:0.9815457 7.10 fps, 5431.28 kb/s

VAQ 4 = SSIM Mean Y:0.9815457 7.20 fps, 5431.28 kb/s



all using strength 1.2

i used a clip from Day after Tomorrow bluray 1080p not resized or anything at a bitrate of 5550 2 pass. all other settings are what i always use for my hd-dvd compatible authoring. the scene is 4 mins in where they are coring for samples and the crack in the ice sheet starts to open up...

[Umamio]

VAQ 3 = SSIM Mean Y:0.9815457 7.10 fps, 5431.28 kb/s

VAQ 4 = SSIM Mean Y:0.9815457 7.20 fps, 5431.28 kb/s

all using strength 1.2


I thought there were only 4 metrics: 0, 1, 2, 3
I suspect your metric 3 and metric 4 files may be identical!
What SSIM do you get with --aq-strength 0 --qcomp 1?


0.46, st 1.0, sens 16: 0.9705006, PSNR: 36.159-o34.722.264


Can I get the 0.46 build you're using to test it for funky behaviour with my non-anime footballisms?

[DeathTheSheep]

DS: Wait, so you don't read my posts?? And yet you yell at me anyway? :( (ps: post edited with more details like you wanted before).

Anway. The sensitivity "bug" is probably not confined to a line in the algo itself, but rather to the nature of the algo and ratecontrol. It likely stems from the fact that when the AQ lowers/raises everything's quant at a different threshold while ratecontrol is given the same bitrate to work with, it makes decisions differently. I doubt it's limited to the algorithm alone--if the bug is confined to it, I certainly wouldn't know where to look. I hardly even use 2pass, much less know how x264's rate control works.

But the problem thickens. Try using constant quantizer. CQ X -sens Y at bitrate B is slightly different than CQ (X-lambda) -sens (y+-lambda') at same bitrate B.

One thing to do with it is the keyframes, perhaps. I know boosting the ipratio at CQ X can effectively raise SSIM when sens Y is adjusted to maintain B, but the same keyframe boost trick with CQ x-lambda doesn't raise SSIM at all, in practice. Perhaps this is due to the fact that x264 simply lowers keyframe quantizers (or raises B-frame quantizers) by an integer value of factor 1.4 times the p frame quant. If you specify QP 30, the I frames could be at 27, whereas if you specify QP 15, I would be 12, etc, all at different bitrates, all with different sizes and SSIMs of their own, even after the total bitrate is re-leveled with a new sensitivity.

Maybe the same argument applies to B-framez. Maybe we can test the QP hypothesis at least by forcing all ratios to 1.0 then comparing the two. QP30 -sens 25, QP29 -sens 21 give me the same sizes with .46 (after a bit of adjustment), so go figure.

PS: According to the rounding, 0.46 is by definition "funky behavior." I am a fan of funky music though. Go figure! One such build can be found here (http://forum.doom9.org/showpost.php?p=1120419&postcount=916). A bit dated, though.

[Umamio]

I am a fan of funky music though. Go figure!

Play that funky mUUuusic white boooooooy

:thanks:

[woah!]

source is The Rock bluray AVC 1080p original file bitrate 31858 kbps.


VAQ48 = SSIM Mean Y:0.9568357 4.06 fps, 16119.15 kb/s

VAQ 0 = SSIM Mean Y:0.9566338 4.09 fps, 15905.28 kb/s

VAQ 1 = SSIM Mean Y:0.9563507 4.10 fps, 15795.36 kb/s

VAQ 2 = SSIM Mean Y:0.9569956 3.96 fps, 16092.06 kb/s

VAQ 3 = SSIM Mean Y:0.9571898 3.83 fps, 16151.72 kb/s


all default VAQ2 settings, just metric changed - CRF26

result screenshots in a pack ... http://www.smalllinks.com/46G

[IgorC]

Seems like VAQ 2 uses more advanced algorithms than based on SSIM purely. I just wanted to mention (W)CWSSIM. It's more reliable metric than simple SSIM. Are some kind of these metrics used in new VAQ?

Here is already well known paper about WCWSSIM http://www.ece.northwestern.edu/~pappas/papers/brooks_hvei06.pdf

[Avenger007]

I tested 3 non-anime clips containing short distance (>30% of frame area) to closeups of face and body with a fair amount of skin shown. Thus, I'm testing how well VAQ handles little facial and skin details given that skin tone has a gradient and usually results in blurring of details and blocking on large skin areas.

Clip 1
This clip has 3 camera angle changes of the same people.

Settings:
avis : 352x288 @ 25.00 fps (865 frames)

--pass 2 --bitrate 500 --ref 16 --mixed-refs --no-fast-pskip --bframes 8 --b-pyramid --b-rdo --bime --weightb --direct auto --subme 7 --trellis 2 --partitions all --8x8dct --me umh --merange 32 --threads auto --thread-input --sar 1:1 --progress --no-psnr --sar 12:11 --aq-metric [0, 1, 3] --aq-strength [0.75, 1, 1.25]

strength 0
SSIM 0.9855646

strength 0.75
metric 0: SSIM 0.9861316
metric 3: SSIM 0.9864114

strength 1
metric 0: SSIM 0.9861450 4.2% improvement over VAQ off
metric 1: SSIM 0.9863353
metric 3: SSIM 0.9864244 6.3% improvement over VAQ off

strength 1.25
metric 0: SSIM 0.9861327
metric 3: SSIM 0.9862227


Clip 2
This clip has only one camera angle with minimal background movement.

Settings:
avis [info]: 512x384 @ 29.97 fps (327 frames)

Same as Clip 1 except:
--bitrate 800 --keyint 300 --min-keyint 30 --sar 1:1

strength 0
SSIM 0.9786412

strength 0.75
metric 0: SSIM 0.9799664
metric 3: SSIM 0.9803084

strength 1
metric 0: SSIM 0.9802144 8.0% improvement over VAQ off
metric 3: SSIM 0.9803256 8.6% improvement over VAQ off

strength 1.25
metric 0: SSIM 0.9799985
metric 3: SSIM 0.9801740


Clip 3
This clip also has only one camera angle but shows a closeup of face and body with camera moving up and down and zooming in and out slowly.

Settings:
avis [info]: 512x384 @ 29.97 fps (2461 frames)
Same as Clip 2.

strength 0
SSIM 0.9850764

strength 0.75
metric 0: SSIM 0.9853343 1.8% improvement over VAQ off
metric 1: SSIM 0.9855772
metric 3: SSIM 0.9856469 4.0% improvement over VAQ off

strength 1
metric 0: SSIM 0.9852713 1.3% improvement over VAQ off
metric 1: SSIM 0.9854063
metric 3: SSIM 0.9855809 3.5% improvement over VAQ off

strength 1.25
metric 0: SSIM 0.9851264
metric 3: SSIM 0.9851633


Observations (My eyes only :eek:):
For strength 1, metric 3 gives better quality than metric 0.

For strength 1.25, metric 3 may give better quality than metric 0.
However, I noticed in some frames that very small areas of skin between skin-background borders seem to extend slightly beyond the border and cover the background, but these artifacts are hard to spot. Maybe this sort of "bleeding" of color caused the lower SSIM. Thus, metric 0 gives better quality in these clips for strength 1.25.
Strength 1.25 kept more detail (less blurring) on textured parts of skin and face than strength 1.

Lower strength values (0.75) generally don't seem to help much.
In clip 3, the outfit worn is latex (I think) and causes lots of sharp borders between skin and clothes. It also has "curves of brightness", i.e. creases and folds are brightly lit as opposed to gradient color of flat areas. It seems like aliasing artifacts are occurring, so lower strength results in higher SSIM.
It's hard to tell which metric was better at which strength for clip 3 since each combination improved some parts while making other parts look worse.


Well that took a while to type, I hope it's helpful and not too long :)

P.S. I used TemporalDegrain on the source to remove noise and improve compressibility resulting in pre-rendered clips which were used for encoding.

[I]Disclaimer:
Since I don't know how VAQ works, I don't know what to look for and what to overlook.

[burfadel]

Just a VAQ related question, does VAQ organise the distribution of bits as is, or is it a reallocation of bits from another process?