VAQ 2.0 Alpha Testing

[nicko]

Except that git now uses "metric 4" :rolleyes:
Some "small" differences still existed. From Komisar666 words:

GIT: *satd += h->pixf.satd[pix]( zero, 0, h->fenc->plane[i]+offset, stride ) - sad/2;
BM: *satd += h->pixf.satd[pix]( zero, 0, h->fenc->plane[i]+offset, stride );

delta CRF (GIT, metric4) ~1.2... ;)

As I told, imho, it is not obviously equivalent to compensate bitrate by CRF+metric or sensitivity.

Yes, it would be the best solution to find this hole in codec direct. :)

I hope Sharktooth is right and somebody can solve this task finaly without metrics.

[CruNcher]

@nicko
Could you ask Komisar666 if he could additionaly compile a libx264.dll of his build and make it available @ the side of the cli encoder for Avidemux usage, that would be really great :)

Update 906 (CRF)


--aq-metric 4 CRF = 20.34 SSIM = 0.9812044 PSNR = 43.567 BR = 2187.61 FPS = 34.57
--git (906) CRF = 21.47 SSIM = 0.9810663 PSNR = 43.610 BR = 2189.67 FPS = 33.61


(Bitrate)


--aq-metric 4 BRT = 1860 BRH = 1816.13 FCRF = 20.88 SSIM = 0.9767143 PSNR = 42.354 FPS = 36.08
--git (906) BRT = 1860 BRH = 1829.05 FCRF = 22.01 SSIM = 0.9766777 PSNR = 42.410 FPS = 34.77


Visualy it makes really no difference (comparing the overal PSNR for git vs the overal SSIM for komissars build) tough the speedup is the most visible here i gonna try some very low bitrate stuff next to see where it visualy starts to make a difference (if @ all).

[nicko]

@nicko
Could you ask Komisar666 if he could additionaly compile a libx264.dll of his build and make it available @ the side of the cli encoder for Avidemux usage, that would be really great :)

CruNcher
OK
I'll transmit your request to komisar666

By the way
906 Cli is ready
http://komisar.gin.by/


Update 906 (CRF)


--aq-metric 4 CRF = 20.34 SSIM = 0.9812044 PSNR = 43.567 BR = 2187.61 FPS = 34.57
--git (906) CRF = 21.47 SSIM = 0.9810663 PSNR = 43.610 BR = 2189.67 FPS = 33.61


(Bitrate)


--aq-metric 4 BRT = 1860 BRH = 1816.13 FCRF = 20.88 SSIM = 0.9767143 PSNR = 42.354 FPS = 36.08
--git (906) BRT = 1860 BRH = 1829.05 FCRF = 22.01 SSIM = 0.9766777 PSNR = 42.410 FPS = 34.77


Visualy it makes really no difference (comparing the overal PSNR for git vs the overal SSIM for komissars build) tough the speedup is the most visible here i gonna try some very low bitrate stuff next to see where it visualy starts to make a difference (if @ all).

Thanks for tests!

Yes, better to look it at ~250-500 kbps (CRF~24-26), than result is more visible. :)

[CruNcher]

At low bitrate it really shows it's performance visualy it's still hard to track the difference i would say you have less b-frame distortion problems (better stability in motion) in some scenes then with the git version


--aq-metric 4 CRF = 31.21 SSIM = 0.9447644 PSNR = 37.496 BR = 524.25 FPS = 44.71
--git (906) CRF = 32.44 SSIM = 0.9441511 PSNR = 37.505 BR = 524.50 FPS = 43.23

[nicko]

At low bitrate it really shows it's performance visualy it's still hard to track the difference i would say you have less b-frame distortion problems (better stability in motion) in some scenes then with the git version


--aq-metric 4 CRF = 31.21 SSIM = 0.9447644 PSNR = 37.496 BR = 524.25 FPS = 44.71
--git (906) CRF = 32.44 SSIM = 0.9441511 PSNR = 37.505 BR = 524.50 FPS = 43.23

OK
Thanks!
delta CRF (GIT, metric4) ~1.23

Actually for such tests is important codec setup,
I have some very extremal setup-version with --aq-strength~1.6-2 (Dark Shikari thanks for VAQ2, now together with metric 4 this magic is possible without artefacts for very high aq-strength -thanks BM) to made a relatively good quality rips with 240-600kbps for 704x528, which is very sensitive for such changes...

[Razorholt]

@nicko: how about VAQ2+PsyRDO0.5 ? :D

Thanks,
- Dan

[nicko]

@nicko: how about VAQ2+PsyRDO0.5 ? :D

Thanks,
- Dan
It is question to komisar666, I hope he will be taken by it.

Vfw-VAQ2+PsyRDO-0.5 version should be complited by BugMaster

[nicko]

@nicko
Could you ask Komisar666 if he could additionaly compile a libx264.dll of his build and make it available @ the side of the cli encoder for Avidemux usage, that would be really great :)



Unfortunately, it seems practically is impossible, after long discussions and attempts several experts were stated that such operation by correct way is possible only for GIT, therefore developers Avidemux if they interested to work with new advanced codecs version are obliged to adapt new x264 builds for Avidemux themselves or to adapt Avidemux for new x264 builds. :(


Now good news.

BugMaster uploaded a new stable 912vfw builds with new variant of metric 4 (by definition, a bit different from the versions in komisar666 build), Psy RDO-0.5 and many other features ....

x264vfw_13_912bm_VAQ2mod_PsyRDO_14208.exe

and standard build
x264vfw_13_912bm_14208.exe

http://sourceforge.net/project/showfiles.php?group_id=213809

[nicko]

All who wants to take part in testing of a full set of the metrics (0-12, all together 13) from BugMaster can take complete-metric builds based on b912 here:

VFW: http://stashbox.org/159397/x264vfw_13_912bm_VAQ2test_PsyRDO_14208_more.exe
CLI: http://stashbox.org/159398/x264_test_more.exe


Parameters by definition
- aq-mode=2 "aq-sensitivity can be adjusted by hand"
- aq-metric=0 "it is a GIT (or bm_M6_4ver0c)"
- aq-sensitivity=logf(5000)
- qcomp=1
- psy-rd=0 "for tests better to swich off"

To swich off VAQ for comparison, one can use aq-mode=0.

PS
Be care, --help in CLI is not updated!

[nicko]

BM fixed important bug in lossless CRF (and others modes, 2pass...) broked in r901.

r913vfw

x264vfw_13_913bm_VAQ2mod_PsyRDO_14259.exe
x264vfw_13_913bm_14259.exe

http://sourceforge.net/project/showfiles.php?group_id=213809

PS
For metric-test-bulds not important (axcept lossless mode, CRF=0,...).

[DarkZell666]

BM fixed important bug in lossless CRF.
b913vfw

x264vfw_13_913bm_VAQ2mod_PsyRDO_14259.exe
x264vfw_13_913bm_14259.exe

http://sourceforge.net/project/showfiles.php?group_id=213809

PS
For metric-test-bulds not important (axcept lossless CRF).

Is it me .... or aren't "lossless" and "CRF" actually supposed to be mutually exclusive ? ...

[Dark Shikari]

Is it me .... or aren't "lossless" and "CRF" actually supposed to be mutually exclusive ? ...CRF==0 is the same as QP==0 is the same as lossless.

[CruNcher]

@nicko
is it correct that in the new AQ test build the CRF range changed dramaticly (less precise then before) so most of the times float values like xx.xx have no effect anymore compared to xx.x ?.

[nicko]

@nicko
is it correct that in the new AQ test build the CRF range changed dramaticly (less precise then before) so most of the times float values like xx.xx have no effect anymore compared to xx.x ?.

Now I can not tell you - I can begin the tests only today evening.

But earlier at me such happened only when I worked with old computers (Intel-P4, ....).

Do you use now the same computer as before?

[elguaxo]

BM fixed important bug in lossless CRF (and others modes, 2pass...) broked in r901.

What was the problem exactly? I have two lossless encodes that I did with r901 and I'm wondering if I should them again. Thanks!

[Dark Shikari]

What was the problem exactly? I have two lossless encodes that I did with r901 and I'm wondering if I should them again. Thanks!Image corruption in any i16x16 macroblock due to a slight coding error of mine.

[elguaxo]

Image corruption in any i16x16 macroblock due to a slight coding error of mine.

thanks for the quick answer! :)

[nicko]

What was the problem exactly? I have two lossless encodes that I did with r901 and I'm wondering if I should them again. Thanks!
The speech went about serious distortions of a picture.

[CruNcher]

@nicko



912 (revision before the AQ changes ) --aq-metric 4

--aq-mode 0 CRF = 23.33 SSIM = 0.9796536 PSNR = 43.497 BR = 1848.94 FPS = 32.15
--aq-mode 0 CRF = 23.36 SSIM = 0.9795736 PSNR = 43.476 BR = 1840.99 FPS = 32.50

--aq-mode 1 CRF = 22.66 SSIM = 0.9802948 PSNR = 43.269 BR = 1850.19 FPS = 30.35
--aq-mode 1 CRF = 22.7 SSIM = 0.9802133 PSNR = 43.248 BR = 1840.99 FPS = 30.19

--aq-mode 2 CRF = 23.35 SSIM = 0.9799445 PSNR = 43.110 BR = 1848.04 FPS = 30.72
--aq-mode 2 CRF = 23.38 SSIM = 0.9798735 PSNR = 43.092 BR = 1839.74 FPS = 30.73

--aq-mode 3 CRF = 21.86 SSIM = 0.9799475 PSNR = 43.097 BR = 1849.66 FPS = 30.25
--aq-mode 3 CRF = 21.9 SSIM = 0.9798579 PSNR = 43.074 BR = 1839.68 FPS = 30.26


912 (new AQ test) --aq-metric 4

--aq-mode 0 CRF = 25 SSIM = 0.9798004 PSNR = 43.672 BR = 1849.40 FPS = 31.70

--aq-mode 1 CRF = 25.46 SSIM = 0.9800692 PSNR = 43.388 BR = 1839.20 FPS = 29.94

--aq-mode 2 CRF = 23.5 SSIM = 0.9802944 PSNR = 43.319 BR = 1845.20 FPS = 30.48
--aq-mode 2 CRF = 23.53 SSIM = 0.9802486 PSNR = 43.306 BR = 1838.80 FPS = 30.24

--aq-mode 3 CRF = 25.55 SSIM = 0.9803369 PSNR = 43.341 BR = 1849.31 FPS = 29.87
--aq-mode 3 CRF = 25.6 SSIM = 0.9802577 PSNR = 43.319 BR = 1839.99 FPS = 29.83

[nicko]

@nicko



912 (revision before the AQ changes ) --aq-metric 4

--aq-mode 0 CRF = 23.33 SSIM = 0.9796536 PSNR = 43.497 BR = 1848.94 FPS = 32.15
--aq-mode 0 CRF = 23.36 SSIM = 0.9795736 PSNR = 43.476 BR = 1840.99 FPS = 32.50

--aq-mode 1 CRF = 22.66 SSIM = 0.9802948 PSNR = 43.269 BR = 1850.19 FPS = 30.35
--aq-mode 1 CRF = 22.7 SSIM = 0.9802133 PSNR = 43.248 BR = 1840.99 FPS = 30.19

--aq-mode 2 CRF = 23.35 SSIM = 0.9799445 PSNR = 43.110 BR = 1848.04 FPS = 30.72
--aq-mode 2 CRF = 23.38 SSIM = 0.9798735 PSNR = 43.092 BR = 1839.74 FPS = 30.73

--aq-mode 3 CRF = 21.86 SSIM = 0.9799475 PSNR = 43.097 BR = 1849.66 FPS = 30.25
--aq-mode 3 CRF = 21.9 SSIM = 0.9798579 PSNR = 43.074 BR = 1839.68 FPS = 30.26


912 (new AQ test) --aq-metric 4

--aq-mode 0 CRF = 25 SSIM = 0.9798004 PSNR = 43.672 BR = 1849.40 FPS = 31.70

--aq-mode 1 CRF = 25.46 SSIM = 0.9800692 PSNR = 43.388 BR = 1839.20 FPS = 29.94

--aq-mode 2 CRF = 23.5 SSIM = 0.9802944 PSNR = 43.319 BR = 1845.20 FPS = 30.48
--aq-mode 2 CRF = 23.53 SSIM = 0.9802486 PSNR = 43.306 BR = 1838.80 FPS = 30.24

--aq-mode 3 CRF = 25.55 SSIM = 0.9803369 PSNR = 43.341 BR = 1849.31 FPS = 29.87
--aq-mode 3 CRF = 25.6 SSIM = 0.9802577 PSNR = 43.319 BR = 1839.99 FPS = 29.83



Many thanks!

I did not get which version is it, can you give a references on this builds?

Imho, here valid comparison only in aq-mode 2, as for other definition parameters can be different vor this versions of codec.

And in 2-d version of metric 4 I see a good changes!

[nicko]

Now I finished my metrics tests.

I used 3 builds:
1. r889
2. r812-test, through numbers I take from this version (M0-M12)
3.r813

Bitrate was adjusted with the help of aq-sensitivity, imho, the most correct way which is not affects a temporal part of codec.


r889-metr4-(M7)-q25-sensitivity 8.3
x264vfw [info]: SSIM Mean Y:0.9542146
x264vfw [info]: PSNR Mean Y:35.724 U:41.165 V:41.565 Avg:36.911 Global:36.667 kb/s:585.59


r913-metr4-(M6)-q25-sensitivity 8.3024
x264vfw [info]: SSIM Mean Y:0.9541604
x264vfw [info]: PSNR Mean Y:35.692 U:41.165 V:41.540 Avg:36.882 Global:36.643 kb/s:585.97


r912test-metr6-(M6)-q25-sensitivity 8.3024
x264vfw [info]: SSIM Mean Y:0.9541604
x264vfw [info]: PSNR Mean Y:35.692 U:41.165 V:41.540 Avg:36.882 Global:36.643 kb/s:585.97


r912test-metr6-(M6)-q25-sensitivity8.3
x264vfw [info]: SSIM Mean Y:0.9541724
x264vfw [info]: PSNR Mean Y:35.576 U:40.820 V:41.312 Avg:36.740 Global:36.523 kb/s:582.30


r912test-metr0-(M0)-q25-sensitivity 8.295
x264vfw [info]: SSIM Mean Y:0.9541020
x264vfw [info]: PSNR Mean Y:35.706 U:41.169 V:41.523 Avg:36.893 Global:36.647 kb/s:609.71

r912test-metr4-(M4)-q25-sensitivity 8.295
x264vfw [info]: SSIM Mean Y:0.9540974
x264vfw [info]: PSNR Mean Y:35.705 U:41.186 V:41.555 Avg:36.895 Global:36.651 kb/s:609.12


r912test-metr3-(M3)-q25-sensitivity7.581
x264vfw [info]: SSIM Mean Y:0.9529578
x264vfw [info]: PSNR Mean Y:35.803 U:41.986 V:42.312 Avg:37.069 Global:36.716 kb/s:590.45

r912test-metr3-(M3)-q25-sensitivity7.564
x264vfw [info]: SSIM Mean Y:0.9524461
x264vfw [info]: PSNR Mean Y:35.692 U:41.870 V:42.182 Avg:36.957 Global:36.631 kb/s:581.62


r912test-metr3-(M3)-q25-sensitivity7.568
x264vfw [info]: SSIM Mean Y:0.9526719
x264vfw [info]: PSNR Mean Y:35.775 U:41.954 V:42.267 Avg:37.040 Global:36.690 kb/s:587.11


r912test-metr8-(M8-)q25-sensitivity8.294
x264vfw [info]: SSIM Mean Y:0.9541336
x264vfw [info]: PSNR Mean Y:35.692 U:41.182 V:41.578 Avg:36.886 Global:36.643 kb/s:600.32

r912test-metr12-(M12)-q25-sensitivity7.568
x264vfw [info]: SSIM Mean Y:0.9526719
x264vfw [info]: PSNR Mean Y:35.775 U:41.954 V:42.267 Avg:37.040 Global:36.690 kb/s:587.11


r912test-metr11-(M11)-q25-sensitivity7.568
x264vfw [info]: SSIM Mean Y:0.9527516
x264vfw [info]: PSNR Mean Y:35.759 U:41.938 V:42.261 Avg:37.024 Global:36.678 kb/s:589.43


r912test-metr10-(M10)-q25-sensitivity7.6
x264vfw [info]: SSIM Mean Y:0.9511864
x264vfw [info]: PSNR Mean Y:35.647 U:41.985 V:42.305 Avg:36.930 Global:36.562 kb/s:585.51

r912test-metr5-(M5)-q25-sensitivity8.295
x264vfw [info]: SSIM Mean Y:0.9540974
x264vfw [info]: PSNR Mean Y:35.705 U:41.186 V:41.555 Avg:36.895 Global:36.651 kb/s:609.12


r912test-metr7-(M7)-q25-sensitivity8.3
x264vfw [info]: SSIM Mean Y:0.9541724
x264vfw [info]: PSNR Mean Y:35.576 U:40.820 V:41.312 Avg:36.740 Global:36.523 kb/s:582.30

r912test-metr7-(M7)-q25-sensitivity8.3024
x264vfw [info]: SSIM Mean Y:0.9541604
x264vfw [info]: PSNR Mean Y:35.692 U:41.165 V:41.540 Avg:36.882 Global:36.643 kb/s:585.97

r912test-metr9(M9)-q25-sensitivity8.297
x264vfw [info]: SSIM Mean Y:0.9541933
x264vfw [info]: PSNR Mean Y:35.728 U:41.164 V:41.548 Avg:36.914 Global:36.667 kb/s:601.92

Now only winners:

r889-metr4-(M7)-q25-sensitivity 8.3
x264vfw [info]: SSIM Mean Y:0,9542146
x264vfw [info]: PSNR Mean Y:35.724 U:41.165 V:41.565 Avg:36.911 Global:36.667 kb/s:585,59


r913-metr4-(M6)-q25-sensitivity 8.3024
x264vfw [info]: SSIM Mean Y:0.9541604
x264vfw [info]: PSNR Mean Y:35.692 U:41.165 V:41.540 Avg:36.882 Global:36.643 kb/s:585.97

r912test-metr6-(M6)-q25-sensitivity8.3.avi
x264vfw [info]: SSIM Mean Y:0.9541724
x264vfw [info]: PSNR Mean Y:35.576 U:40.820 V:41.312 Avg:36.740 Global:36.523 kb/s:582.30


r912test-metr6-(M6)-q25-sensitivity 8.3024.avi
x264vfw [info]: SSIM Mean Y:0.9541604
x264vfw [info]: PSNR Mean Y:35.692 U:41.165 V:41.540 Avg:36.882 Global:36.643 kb/s:585.97


r912test-metr7-(M7)-q25-sensitivity8.3024.avi
x264vfw [info]: SSIM Mean Y:0.9541604
x264vfw [info]: PSNR Mean Y:35.692 U:41.165 V:41.540 Avg:36.882 Global:36.643 kb/s:585.97

r912test-metr7-(M7)-q25-sensitivity8.3.avi
x264vfw [info]: SSIM Mean Y:0.9541724
x264vfw [info]: PSNR Mean Y:35.576 U:40.820 V:41.312 Avg:36.740 Global:36.523 kb/s:582.30


M6 и М7 in r912-test and r913 are equal (as predicted by BM for aq-mode 2)

For some reason M7 from r889 a tiny bit ahead than M6 and М7 in new revision, but I think it is only changes inside codec itself.

So for my test winner is metric by definiton 4 (or M6 from r912test ) from r913.

[komisar]

Some tests with various "aq-metric" and visual comparsion:
http://komisar.gin.by/mtest/

P.S. In my last x264 builds present only GIT, 4ver0 and 4ver1 metrics...

[Sharktooth]

using PSNR and SSIM for comparing AQ metrics is useless. at that point you would better use PSNR or SSSIM in place of the AQ metric...
AQ is all about visuals... comparing using metrics you're destroying the AQ original intent.

[komisar]

I understand... But you may see "visual comparsion"...

[Warpman]

I understand... But you may see "visual comparsion"...

you hide the sceenshots very well ;)

[Sharktooth]

i mean... dont rely on PSNR or SSIM... higher values do not mean better IQ.
you need to look at comparative pictures or clips.

[komisar]

Pay attention to "avgQ" column in tables and "CMPLX"
avgQ -- average (I+P+B) quant for all frames;
CMPLX -- average bits for one macroblock encoding;
I upload all testing material here:
http://komisar.gin.by/test/

[Sharktooth]

that doesnt express visual quality.

[komisar]

But you can see difference (or make you own tests)... Quality low because CRF=26 used...
You may compare frame redistribute...
(sorry, i not have sufficient HDD space for all test result)

[nm]

that doesnt express visual quality.
He does have easily comparable screenshots there. Click the links at the top of the page (for example: http://komisar.gin.by/mtest/03_2pass.html).
And the full clips are also available in the test directory.

[Sharktooth]

they're well hidden... :)

[komisar]

:-) Sorry, I add red description above links to visual comparsion...

[Sharktooth]

i would change it in "right below there are the links for a visual comparison" though.

[komisar]

Ok. Changed...

[Sharktooth]

uhm... "...there are links..." maybe it's better... english is not my native language
btw, i would also add a blind test visual comparison instead of choosing places by metrics.

[komisar]

You have taunted me? English is not my native language. Who wants, he will find what it needs...

[Sharktooth]

no, i probably made a mistake in the phrase i suggested you, so actually i taunted myself...

[Quark.Fusion]

From 2-pass screenshots I vote for this:
http://komisar.gin.by/mtest/03_2pass/mtst_2pass_3_11_0000.png
But it may be borrowed bits from other frames

And for same method from CRF-BR:
http://komisar.gin.by/mtest/05_crf1/mtst_crf1_3_11_0000.png
CRF-BR is also better for this frame

[komisar]

Ok. It is important not language expression, and the result.

[komisar]

From 2-pass screenshots I vote for this:
But it may be borrowed bits from other frames
And for same method from CRF-BR:
CRF-BR is also better for this frame
Did you forget to specify metric used in the screenshots. (This is 11, or 4ver1)

aq-mode 3 by BugMaster is 3: Hybrid mode of moving bitsAlso metrics, for which the "Final AQ sensitivity" smaller (AQS column in my tables), better save bitrate.

[CruNcher]

From 2-pass screenshots I vote for this:
http://komisar.gin.by/mtest/03_2pass/mtst_2pass_3_11_0000.png
But it may be borrowed bits from other frames

And for same method from CRF-BR:
http://komisar.gin.by/mtest/05_crf1/mtst_crf1_3_11_0000.png
CRF-BR is also better for this frame

yeah and that's the problem how does the difference look in Motion, and then i think for most mini differences there it wouldn't be much different Visualy but then you can go look @ the Speed/Bitrate and then it gets quite clear wich seem to be very efficient :) my tendency here also goes to 4ver1

[nicko]

Sory for delay but from this test http://komisar.gin.by/mtest/03_2pass.html

4ver0 looks a litle bit better (see a soft details on a dark background) even than 4ver1.

[komisar]

New tests for new version of VAQ modification by BugMaster named AutoVAQ.
Main tests place (http://komisar.gin.by/avaq/index.html) (with visual comparsion)
Builds named kAVAQ from main page (http://komisar.gin.by/)

[Dark Shikari]

New tests for new version of VAQ modification by BugMaster named AutoVAQ.
Main tests place (http://komisar.gin.by/avaq/index.html) (with visual comparsion)
Builds named kAVAQ from main page (http://komisar.gin.by/)Use a different thread; don't bump one that's half a year old.