VAQ 2.0 Alpha Testing

[Dark Shikari]

Originally Posted by Razorholt

@DS:

1.- Is there any minimum birates required for VAQ2 to be efficient?

I don't really know yet.

Originally Posted by Razorholt

2.- How can we compare different settings if SSIM is irrelevant?

SSIM isn't necessarily irrelevant, but its not always useful. The best judge is always your eyes.

Originally Posted by Razorholt

3.- What is the --aq mode/metric/strength default in VAQ2?

Mode=2, Metric=3, Strength=1.

[DeathTheSheep]

DS: You could've just asked for the non-AQ number, some strengths in between, and/or some PSNR's. I'd be only too happy to oblige. It somehow gives off a nicer impression than calling something poison.

[Dark Shikari]

Originally Posted by DeathTheSheep

DS: You could've just asked for the non-AQ number, some strengths in between, and/or some PSNR's. I'd be only too happy to oblige. It somehow gives off a nicer impression than calling something poison.

Okay, so I was a little harsh It just seems like you intentionally ignored the most obvious settings, and chose ones way far to the sides.

[ToS_Maverick]

i did a short test @CRF 22 and got these results:

VAQ1 0.9716895
VAQ2 0.9727707

VAQ2 was slightly better visually too. at CRF 20 there is only a very slight difference, which i think is also true for CRFs below 20

[Encoder888]

I apologize for my dumb questions in advance, as I'm not as experienced with AQ as you are:

So, in VAQ2, the sensitivity option is eliminated, right? Since mode should almost always be set on 2, in order to even have the AQ on, then all you can really adjust is the strength and the metric right? So, I take it the strength works the same way as before, the higher, the better the quality. And the metric? From what I got from your posts, you can only adjust it from 0 - 6, and so, is it the higher the metric, the better the quality? What exactly does the metric do? Is it like sensitivity in the previous VAQ, or was that mode...? Thanks for reading my stupid question

[Dark Shikari]

Originally Posted by Encoder888

I apologize for my dumb questions in advance, as I'm not as experienced with AQ as you are:

So, in VAQ2, the sensitivity option is eliminated, right? Since mode should almost always be set on 2, in order to even have the AQ on, then all you can really adjust is the strength and the metric right? So, I take it the strength works the same way as before, the higher, the better the quality. And the metric? From what I got from your posts, you can only adjust it from 0 - 6, and so, is it the higher the metric, the better the quality? What exactly does the metric do? Is it like sensitivity in the previous VAQ, or was that mode...? Thanks for reading my stupid question

The metric is the method of calculating which blocks get what quantizer. Its a mathematical formula.

[cogman]

So VAQ uses metricies to attempt to give better results correct? I thought part of the purpose of VAQ 1 was to eliminate the need for metricies (cqm). So is the some major fundamental difference between the terms?

Also, given that differing metricies give different results, is the quality difference going to be somewhat of a constant (IE Matrix one gives better quality then matrix two, but with a slower encode)? Or is this going to be a guess and check sort of solution to picking which matrix to encode with.

Just curious.

[Dark Shikari]

Originally Posted by cogman

So VAQ uses metricies to attempt to give better results correct? I thought part of the purpose of VAQ 1 was to eliminate the need for metricies (cqm). So is the some major fundamental difference between the terms?

A matrix has nothing to do with a metric. They're about as related as a beaver and a muon.

[cogman]

Originally Posted by Dark Shikari

A matrix has nothing to do with a metric. They're about as related as a beaver and a muon.

lol, ok, I guess I was just mixing up terms. Thanks for the responce.

Though, is there going to be a fairly standard quality gain/loss with the use of different metrics? or will it just depend video to video?

[Umamio]

Source:
Football Clip - 720x576 (16:9) - 25fps interlaced - Huffyuv
Smartbobbed & Resized - 688x384 - 50fps - Huffyuv [ Download Here (79.3MB) ]
Duration: 6s (331 frames)

x264 build:
x264.808.vaq2.modified.exe (gcc 3.4.6 fprofiled)
[ http://files.x264.nl/VAQ2/x264.808.vaq2.modified.exe ] [ http://forum.doom9.org/showthread.ph...33#post1120033 ]

x264 args:
--pass 2 --bitrate 1284* --stats ".stats" --ref 3 --bframes 16 --b-pyramid --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 12 --aq-strength (variable:[0.5,1,1.5,2,2.5]) --aq-metric (variable:[0,1,2,3])

*1284k/50frames == 642k/25frames

Encodes:

Metrics 0(VAQ1.0),1,2,3 * Strengths 0.5,1.0,1.5,2.0,2.5
+ VAQ off (Strength 0, Metric 0)
[ Download Here (22.85MB, 21 files zipped) ]


Single Frame Comparison (VAQ off Vs. Metric 3 Strength 1 Vs. Metric 3 Strength 1.5 Vs. Metric 3 Strength 2 Vs. Source):


(click to enlarge)

I only have PSNR and SSIM numbers for the metric 3 encodes and with VAQ off, but you can see from the files that there is a clear visual improvement without looking at the values (I haven't seen the logs yet)

For this specific type of high... grass source at least the optimal Strength seems to be between 1.3 and 2.2, I'm still staring at these trying to compare the metrics but there might not be enough fine detail in my source to make an accurate judgement using this material.

Conclusion: VAQ loves grass sports and I love VAQ.

Edit:


Some PSNRSSIMnumberstuff:
VAQ Off:
SSIM Mean Y:0.9087551
PSNR Mean Y:33.601 U:40.486 V:41.777 Avg:34.978 Global:34.571 kb/s:1377.74

Metric 3 - Strength 0.5
SSIM Mean Y:0.9157095
PSNR Mean Y:33.513 U:40.607 V:41.806 Avg:34.904 Global:34.598 kb/s:1387.76

Metric 3 - Strength 1.0
SSIM Mean Y:0.9233749
PSNR Mean Y:33.339 U:40.752 V:41.876 Avg:34.751 Global:34.409 kb/s:1385.98

Metric 3 - Strength 1.5
SSIM Mean Y:0.9265114
PSNR Mean Y:32.728 U:40.651 V:41.664 Avg:34.173 Global:33.816 kb/s:1369.50

Metric 0 - Strength 1.5
SSIM Mean Y:0.9256282
xPSNR Mean Y:33.066 U:40.626 V:41.715 Avg:34.488 Global:34.161 kb/s:1379.88

Metric 3 - Strength 2.0
SSIM Mean Y:0.9226995
PSNR Mean Y:31.642 U:40.306 V:41.231 Avg:33.132 Global:32.750 kb/s:1357.14

Metric 0 - Strength 2.0
SSIM Mean Y:0.9247225
PSNR Mean Y:32.291 U:40.411 V:41.386 Avg:33.749 Global:33.420 kb/s:1366.41

Metric 3 - Strength 2.5
SSIM Mean Y:0.9108505
PSNR Mean Y:30.138 U:39.825 V:40.674 Avg:31.679 Global:31.292 kb/s:1347.17

Disclaimer: I mostly don't know what I am talking about or what I am doing, so I advise you to not trust any of this and do try things out for yourself.

[ToS_Maverick]

you shouldn't forget, VAQ uses qcomp of 1.0 while the standard x264 setting is 0.6.

the bottom line is, with qcomp 0.6, fast motion (like football) doesn't get as much bitrate as i would need. please try 1.0 without AQ, that would be interesting. maybe that's a reason why AQ looks so much better.

[Terranigma]

Originally Posted by ToS_Maverick

you shouldn't forget, VAQ uses qcomp of 1.0 while the standard x264 setting is 0.6.

the bottom line is, with qcomp 0.6, fast motion (like football) doesn't get as much bitrate as i would need. please try 1.0 without AQ, that would be interesting. maybe that's a reason why AQ looks so much better.

You sure you know what you're talking about? According to x264 --longhelp, qcomp's default is now 1.0.
(Well, from the build I grabbed.)

[Dark Shikari]

Originally Posted by Terranigma

You sure you know what you're talking about? According to x264 --longhelp, qcomp's default is now 1.0.
(Well, from the build I grabbed.)

That's because AQ is on by default

[Umamio]

you shouldn't forget, VAQ uses qcomp of 1.0 while the standard x264 setting is 0.6.

Ah you're right, I forgot about qcomp getting changed but in this clip it doesn't seem to make very much difference at all when I set it to 1. QAV off, Qcomp-1, Download (1.11MB)
VAQoff-Qcomp1000.jpg

I'm interested in seeing the best that I can get out of x264 for football without using QAV to make it a better (but less fair) comparison, do you have any suggestions for other parameters I could change?

[ToS_Maverick]

Originally Posted by Dark Shikari

That's because AQ is on by default

well does it revert back to 0.6 if you switch it off?

[Dark Shikari]

Originally Posted by ToS_Maverick

well does it revert back to 0.6 if you switch it off?

It did in the patch I gave to pengvado... you can check pretty easily by looking at the header of the output H.264 file to see if he kept it that way.

[Tack]

Originally Posted by Umamio

Conclusion: VAQ loves grass sports and I love VAQ.

The first thing I noticed was how much worse the BBC logo was in the bottom (VAQ) image.

No doubt though that the grass is much better. I suppose one could make a fairly solid argument that VAQ is causing the bits to be spent in the right places.

[Umamio]

Originally Posted by Tack

The first thing I noticed was how much worse the BBC logo was in the bottom (VAQ) image.

No doubt though that the grass is much better. I suppose one could make a fairly solid argument that VAQ is causing the bits to be spent in the right places.

If you look at the logo in any of the encodes with strengths < 1.5 the logo and score suffer a lot less with not much of a loss in grass detail.

I only tested in strength steps of 0.5 so it's likely that I missed the VAQ strength "sweet spot" for this particular video but that's for another day. This post was really just to illustrate the variations of the different strengths and metrics. It's likely that the quality could be improved with a little more bitrate and more focussed small-step strength testing but the differences may not be as noticeable if the algorithm has lots of bits to spare.

If you encode something at a stupidly low bitrate with a stupidly high VAQ and compare it to the same thing encoded at the same stupidly low bitrate with vaq turned off you can really see the types of detail that VAQ focusses its bits on.

Actually, I will do that for you. But make sure you look at the regular stuff before you look at these extreme/stupid examples or it might affect your judgement.

P.S: I could be wrong about everything I just said.

[Dark Shikari]

Originally Posted by Umamio

Edit:


Some PSNRSSIMnumberstuff:
VAQ Off:
SSIM Mean Y:0.9087551
PSNR Mean Y:33.601 U:40.486 V:41.777 Avg:34.978 Global:34.571 kb/s:1377.74

Metric 3 - Strength 0.5
SSIM Mean Y:0.9157095
PSNR Mean Y:33.513 U:40.607 V:41.806 Avg:34.904 Global:34.598 kb/s:1387.76

Metric 3 - Strength 1.0
SSIM Mean Y:0.9233749
PSNR Mean Y:33.339 U:40.752 V:41.876 Avg:34.751 Global:34.409 kb/s:1385.98

Metric 3 - Strength 1.5
SSIM Mean Y:0.9265114
PSNR Mean Y:32.728 U:40.651 V:41.664 Avg:34.173 Global:33.816 kb/s:1369.50

Metric 0 - Strength 1.5
SSIM Mean Y:0.9256282
xPSNR Mean Y:33.066 U:40.626 V:41.715 Avg:34.488 Global:34.161 kb/s:1379.88

Metric 3 - Strength 2.0
SSIM Mean Y:0.9226995
PSNR Mean Y:31.642 U:40.306 V:41.231 Avg:33.132 Global:32.750 kb/s:1357.14

Metric 0 - Strength 2.0
SSIM Mean Y:0.9247225
PSNR Mean Y:32.291 U:40.411 V:41.386 Avg:33.749 Global:33.420 kb/s:1366.41

Metric 3 - Strength 2.5
SSIM Mean Y:0.9108505
PSNR Mean Y:30.138 U:39.825 V:40.674 Avg:31.679 Global:31.292 kb/s:1347.17

Interesting that the optimal strength for SSIM is so high...

[DeathTheSheep]

A really interesting thing happened to me with this.

While using CQP of 30 for the original and modeling the strength around it to produce similar sized files (since there's no sensitivity any more for 1-pass bitrate fine-tuning!!), 0.46 is much better (see my post on the first page of this thread). Only strength 2 and 0.3 were close enough to the original filesize, but even with a larger filesize, strength 1.0's ssim was pretty low compared to the AQ 0.46. ALL aq tested then were better than the non-AQ SSIM.

BUT. Here's the interesting part. In 2-pass vbr, there is MUCH LESS DIFFERENCE between the various forms of AQ for the sample at target bitrate 400kbps (size ~4155KB). See the results below, all AQ using mode 3 for now:
NO AQ: 0.9702865, PSNR: 37.043-o36.709
NO AQ(qcomp1): 0.9672502, PSNR: 36.844-o36.397
st 0.5: 0.9692864, PSNR: 36.553-o36.010
st 1.0: 0.9693076, PSNR: 35.846-o35.166
st 1.5: 0.9660272, PSNR: 34.648-o33.833
st 2.0: 0.9576726, PSNR: 32.911-o31.958
0.46, st 1.0, sens 16: 0.9705006, PSNR: 36.159-o34.722.264

As you can see, the SSIM hardly rises at all (in fact it falls for every AQ strength other than 0.46).

I don't usually use 2-pass mode, preferring a target quality rather than an exact filesize. This is why I haven't done much testing in this area.

I do think (like another member has pointed out), that either sensitivity needs to be re-implemented (preferably, since I get very different results using sens 16 and sens 25 at the same filesize for aq 0.46), or allow decimal QP/CRF to be specified.

[edit]AH, forgot to mention. Clip is same one as I used/gave you before.
Commandline was: x264 test.avs -o out.264 --aq-strength {x} --aq-metric 3 --no-fast-pskip --no-cabac -A all --thread-input --me umh --pass 2 --bitrate 400 {--qcomp 1} --keyint 1500