AutoVAQ patch testing

[juGGaKNot]

x264 is running as a normal user-space process. Hence x264 itself may crash, but it cannot cause a BSOD. Something unrelated to x264 is seriously wrong with your system.

Check the stability of your system with Memtest86+ and Prim95. Keep an eye on the CPU temperature...

Crashes for me too, not the pc, just x264 on the 2'nd pass.

x264.1153kAVAQ.generic.x32.exe crashes on 2'nd pass

x264_AutoVAQ.02.diff is this the same as in this 1148 build or new ?

[menlvd]

Crashes for me too, not the pc, just x264 on the 2'nd pass.
same crash
x264.nl - passed
komisar - fail
techhouse - fail
all builds x64/x86
what can I do, to help solve this problem?

[burfadel]

Might be something to do with an patch incompatibility, since the vanilla build works and the others don't...

[desta]

Crashes for me too, not the pc, just x264 on the 2'nd pass.
same crash
x264.nl - passed
komisar - fail
techhouse - fail
all builds x64/x86
what can I do, to help solve this problem?


fwiw, same thing here. All 1153 x86 builds crash as the 2nd pass starts (though I havent tried the stable builds). Going back to 1148 fixes it for me.

[menlvd]

Might be something to do with an patch incompatibility, since the vanilla build works and the others don't...

yes... no-patch build's work fine!

[kemuri-_9]

the main differences between everyone's r1148 patched builds and the r1153 patched builds is that
r1148 used the v11 hrd pulldown/interlace patch while r1153 is using the v12.

so this should be looked into to see if it is indeed the problem.

[imk]

Indeed. My ICC builds apparently crash on the second pass, too. The patches I am using:

x264_hrd_pulldown.12_interlace.diff
x264_icc.diff
x264_win_zone_parse_fix_05.diff

Using the AutoVAQ patch or not seems to make no difference in getting x264 to crash on the second pass or not.

[kemuri-_9]

Using the AutoVAQ patch or not seems to make no difference in getting x264 to crash on the second pass or not.

the x264_hrd_pulldown.12_interlace.diff patch is causing the crash, wait for a v13 or revert back to v11 for the time being.

[Chengbin]

OK, here is my test. This is CRF 30 on a episode of Prison Break from a BD source.

IMO the VAQ one looked better for some frames, but some frames suffered from loss of detail.

Non VAQ

http://www.megaupload.com/?d=ANUUVRAJ

VAQ

http://www.megaupload.com/?d=ZAP78XOJ

[Razorholt]

Can you share your settings?

Thanks,
- Dan

[Chengbin]

--keyint 250 --bframes 4 --qpmin 10 --qpmax 51 --no-psnr --no-fast-pskip --mixed-refs --trellis 2 --ref 6 --filter 0,0 --subme 9 --direct auto --me umh --no-ssim --level 4.1 --merange 32 --qcomp 0.8 --weightb --b-adapt 2 --b-pyramid --partitions p8x8,b8x8,i4x4,i8x8 --threads 2 --aq-mode 1 --aq-strength 1 --psy-rd 1.0:0.4

[LoRd_MuldeR]

OK, here is my test. This is CRF 30 on a episode of Prison Break from a BD source.

IMO the VAQ one looked better for some frames, but some frames suffered from loss of detail.

Non VAQ

http://www.megaupload.com/?d=ANUUVRAJ

VAQ

http://www.megaupload.com/?d=ZAP78XOJ

If your test produced two files of different size, which most likely did happen in CRF mode, then your results are completely useless :scared:

For example: If one file looks worse, but also is smaller at the same time, how do you decide whether the quality loss is worth the bitrate saving or not?

Well, you can't. Hence you must compare files of identical size! Either find the CRF's that give identical size or use 2-Pass mode...

(BTW: You can remove "--aq-mode 1" and "--aq-strength 1", these are the default values)

[juGGaKNot]

the x264_hrd_pulldown.12_interlace.diff patch is causing the crash, wait for a v13 or revert back to v11 for the time being.

So autoVAQ 0.2 is used in 1148 also ? this is the only reason i updated to 1153.

[LoRd_MuldeR]

So autoVAQ 0.2 is used in 1148 also ? this is the only reason i updated to 1153.

AutoVAQ is not used in any x264 revision yet. It depends on whether the AutoVAQ patch was included in the individual build or not ;)

Skystrife's r1153 builds with and without AutoVAQ patch can be found here:
http://forum.doom9.org/showpost.php?p=1289106&postcount=1879

[juGGaKNot]

AutoVAQ is not used in any x264 revision yet. It depends on whether the AutoVAQ patch was included in the individual build or not ;)

Skystrife's r1153 builds with and without AutoVAQ patch can be found here:
http://forum.doom9.org/showpost.php?p=1289106&postcount=1879

k thnx.

[CruNcher]

wow it reduces i-frame pulsing on short gops a lot (the pulsing is not that heavy anymore) :)
though it's also slower for that by around 1 fps

[kemuri-_9]

though it's also slower for that by around 1 fps

1 fps less from what originally? 50? 25? 1.5?

[CruNcher]

31.x vs 30 but the visual difference @ the same even a little less bitrate is definitely there the change is less and the pulsing less visible it's much nicer to the eyes that way :)

[Dark Shikari]

31.x vs 30 but the visual difference @ the same even a little less bitrate is defiantly there the change is less and the pulsing less visible it's much nicer to the eyes that way :)If you can't visually separate the effects of AQ and VBV ratecontrol, then don't test them at the same time.

[CruNcher]

No vbv active here just crf with a short gop (even not that short almost 4 seconds)

here 1 second gop result

http://mirror05.x264.nl/CruNcher/force.php?file=./parkrun-25-oldvaq.mp4
http://mirror05.x264.nl/CruNcher/force.php?file=./parkrun-25-newvaq.mp4

pretty obvious the difference is much lower for that bitrate :)

[jefrey]

very nice patch :)
but what is the range from aq that is in use for the whole encode? ? 0--> 1.5?
and would it be possible (maybe in the near future or in the next patch :D ) to set the auto aq range manually, for example from 0.5 --> 1.3, instead of the whole AQ range? that would be great, i think so :D

[desta]

Just did some small tests using skystrifes 1158M builds (with and without AutoVAQ). First I did CRF encodes, which turned out a bit surprising (as far as bitrate is concerned). Commandline for both builds was:
--crf 19 --level 3.1 --ref 8 --mixed-refs --bframes 3 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock -2:-1 --subme 9 --trellis 2 --psy-rd 0.9:0.6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --output "output" "input"

And here were the results....

VAQ
slice I:64 Avg QP:15.13 size: 47393 PSNR Mean Y:48.95 U:51.80 V:52.52 Avg:49.72 Global:48.52
slice P:1166 Avg QP:17.28 size: 20599 PSNR Mean Y:47.69 U:50.97 V:52.07 Avg:48.50 Global:45.63
slice B:2070 Avg QP:19.42 size: 5546 PSNR Mean Y:44.56 U:48.00 V:49.57 Avg:45.42 Global:44.67
consecutive B-frames: 3.3% 10.3% 71.9% 14.5%
mb I I16..4: 22.8% 61.8% 15.4%
mb P I16..4: 4.6% 17.7% 2.5% P16..4: 29.2% 19.3% 21.3% 0.0% 0.0% skip: 5.4%
mb B I16..4: 0.3% 3.9% 0.2% B16..8: 31.0% 1.6% 2.7% direct: 8.4% skip:51.8% L0:31.2% L1:39.8% BI:29.0%
8x8 transform intra:73.3% inter:28.3%
direct mvs spatial:100.0% temporal:0.0%
coded y,uvDC,uvAC intra:98.4% 96.9% 93.6% inter:32.4% 41.9% 13.4%
ref P L0 46.1% 30.5% 6.6% 5.9% 2.7% 3.6% 2.1% 2.4%
ref B L0 57.9% 27.9% 4.8% 3.9% 2.1% 2.3% 1.1%
ref B L1 89.8% 10.2%
SSIM Mean Y:0.9840857
PSNR Mean Y:45.751 U:49.124 V:50.509 Avg:46.587 Global:45.039 kb/s:2335.29
encoded 3300 frames, 1.96 fps, 2335.47 kb/s
Final statistics
Constant Quality Mode: Quality 19 computed...
Video Bitrate Obtained (approximate): 2338 kbit/s

AutoVAQ
slice I:64 Avg QP:17.55 size: 34304 PSNR Mean Y:47.86 U:50.49 V:51.44 Avg:48.61 Global:47.64
slice P:1166 Avg QP:19.66 size: 10877 PSNR Mean Y:46.71 U:49.89 V:51.22 Avg:47.50 Global:44.99
slice B:2070 Avg QP:22.00 size: 2733 PSNR Mean Y:43.84 U:47.44 V:49.00 Avg:44.72 Global:44.19
consecutive B-frames: 3.3% 10.3% 71.9% 14.5%
mb I I16..4: 10.2% 78.7% 11.1%
mb P I16..4: 0.6% 5.8% 0.7% P16..4: 48.1% 17.5% 18.3% 0.0% 0.0% skip: 8.9%
mb B I16..4: 0.0% 0.4% 0.0% B16..8: 35.4% 1.0% 1.5% direct: 5.5% skip:56.0% L0:33.9% L1:48.1% BI:17.9%
8x8 transform intra:81.0% inter:45.0%
direct mvs spatial:99.9% temporal:0.1%
coded y,uvDC,uvAC intra:90.2% 95.6% 80.0% inter:24.4% 38.6% 5.1%
ref P L0 45.1% 28.8% 7.7% 6.4% 3.0% 4.1% 2.4% 2.5%
ref B L0 59.5% 25.8% 5.1% 3.8% 2.2% 2.3% 1.3%
ref B L1 91.1% 8.9%
SSIM Mean Y:0.9811661
PSNR Mean Y:44.931 U:48.361 V:49.830 Avg:45.775 Global:44.508 kb/s:1244.54
encoded 3300 frames, 2.14 fps, 1244.73 kb/s
Final statistics
Constant Quality Mode: Quality 19 computed...
Video Bitrate Obtained (approximate): 1247 kbit/s


I figured I'd use an average between the two as a bitrate for a 2-pass encode. The commandline this time was:
--pass 1 --bitrate 1790 --stats ".stats" --level 3.1 --bframes 3 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock -2:-1 --psy-rd 0.9:0 --partitions none --vbv-bufsize 14000 --vbv-maxrate 17500 --threads auto --thread-input --progress --output NUL "input"

--pass 2 --bitrate 1790 --stats ".stats" --level 3.1 --ref 8 --mixed-refs --bframes 3 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock -2:-1 --subme 9 --trellis 2 --psy-rd 0.9:0.6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 14000 --vbv-maxrate 17500 --me umh --threads auto --thread-input --progress --output "output" "input"

This results this time looked like this...

VAQ
1st Pass
slice I:64 Avg QP:15.63 size: 36994 PSNR Mean Y:48.08 U:50.25 V:51.10 Avg:48.72 Global:47.40
slice P:1163 Avg QP:17.10 size: 13731 PSNR Mean Y:46.73 U:49.58 V:50.74 Avg:47.45 Global:44.58
slice B:2073 Avg QP:19.34 size: 5483 PSNR Mean Y:44.16 U:47.45 V:48.80 Avg:44.96 Global:44.05
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 19.5% 0.0% 80.5%
mb P I16..4: 7.9% 0.0% 0.0% P16..4: 85.6% 0.0% 0.0% 0.0% 0.0% skip: 6.6%
mb B I16..4: 14.0% 0.0% 0.0% B16..8: 33.4% 0.0% 0.0% direct:24.9% skip:27.6% L0:24.1% L1:48.1% BI:27.8%
final ratefactor: 18.63
direct mvs spatial:100.0% temporal:0.0%
coded y,uvDC,uvAC intra:73.3% 94.0% 80.7% inter:32.7% 49.5% 10.9%
SSIM Mean Y:0.9821088
PSNR Mean Y:45.144 U:48.255 V:49.528 Avg:45.913 Global:44.278 kb/s:1800.20
encoded 3300 frames, 11.28 fps, 1800.23 kb/s

2nd Pass
slice I:64 Avg QP:16.10 size: 42037 PSNR Mean Y:48.42 U:51.21 V:51.99 Avg:49.18 Global:48.00
slice P:1163 Avg QP:18.12 size: 16140 PSNR Mean Y:47.24 U:50.40 V:51.61 Avg:48.03 Global:45.44
slice B:2073 Avg QP:20.56 size: 3866 PSNR Mean Y:44.15 U:47.68 V:49.27 Avg:45.02 Global:44.47
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 17.2% 69.3% 13.5%
mb P I16..4: 1.2% 14.2% 1.2% P16..4: 35.4% 20.7% 21.1% 0.0% 0.0% skip: 6.2%
mb B I16..4: 0.0% 1.2% 0.1% B16..8: 34.1% 1.3% 2.0% direct: 7.5% skip:53.7% L0:34.2% L1:43.1% BI:22.7%
8x8 transform intra:82.5% inter:31.8%
direct mvs spatial:97.2% temporal:2.8%
coded y,uvDC,uvAC intra:96.6% 96.1% 90.1% inter:29.2% 40.9% 10.7%
ref P L0 44.7% 29.7% 7.2% 6.3% 3.0% 4.0% 2.4% 2.6%
ref B L0 57.4% 26.9% 5.2% 4.2% 2.3% 2.6% 1.4%
ref B L1 90.4% 9.6%
SSIM Mean Y:0.9827564
PSNR Mean Y:45.318 U:48.708 V:50.144 Avg:46.160 Global:44.836 kb/s:1786.41
encoded 3300 frames, 2.01 fps, 1786.43 kb/s
Final statistics
Video Bitrate Desired: 1790 kbit/s
Video Bitrate Obtained (approximate): 1789 kbit/s


AutoVAQ
1st Pass
slice I:64 Avg QP:15.07 size: 39908 PSNR Mean Y:48.83 U:50.78 V:51.61 Avg:49.42 Global:48.40
slice P:1163 Avg QP:16.83 size: 13979 PSNR Mean Y:47.30 U:49.79 V:51.00 Avg:47.96 Global:45.42
slice B:2073 Avg QP:19.31 size: 5314 PSNR Mean Y:44.72 U:47.67 V:49.07 Avg:45.47 Global:44.84
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 20.4% 0.0% 79.6%
mb P I16..4: 6.8% 0.0% 0.0% P16..4: 86.6% 0.0% 0.0% 0.0% 0.0% skip: 6.7%
mb B I16..4: 11.8% 0.0% 0.0% B16..8: 35.5% 0.0% 0.0% direct:27.9% skip:24.8% L0:23.3% L1:45.2% BI:31.6%
final ratefactor: 15.97
direct mvs spatial:100.0% temporal:0.0%
coded y,uvDC,uvAC intra:61.4% 93.5% 73.5% inter:34.5% 53.9% 9.9%
SSIM Mean Y:0.9837163
PSNR Mean Y:45.710 U:48.477 V:49.797 Avg:46.420 Global:45.083 kb/s:1807.79
encoded 3300 frames, 10.60 fps, 1807.82 kb/s

2nd Pass
slice I:64 Avg QP:16.04 size: 41120 PSNR Mean Y:48.67 U:51.28 V:52.10 Avg:49.41 Global:48.44
slice P:1163 Avg QP:17.97 size: 15823 PSNR Mean Y:47.58 U:50.53 V:51.77 Avg:48.33 Global:45.99
slice B:2073 Avg QP:20.54 size: 4079 PSNR Mean Y:44.47 U:47.87 V:49.45 Avg:45.32 Global:44.93
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 12.2% 75.5% 12.3%
mb P I16..4: 0.8% 9.8% 1.3% P16..4: 39.4% 20.4% 22.4% 0.0% 0.0% skip: 6.0%
mb B I16..4: 0.0% 0.9% 0.1% B16..8: 34.9% 1.3% 2.2% direct: 7.8% skip:52.8% L0:33.0% L1:41.0% BI:25.9%
8x8 transform intra:81.1% inter:34.1%
direct mvs spatial:97.2% temporal:2.8%
coded y,uvDC,uvAC intra:94.9% 97.6% 90.3% inter:30.8% 44.0% 10.6%
ref P L0 43.3% 30.0% 7.3% 6.7% 3.1% 4.3% 2.5% 2.8%
ref B L0 57.5% 27.7% 4.9% 4.1% 2.1% 2.4% 1.3%
ref B L1 91.0% 9.0%
SSIM Mean Y:0.9832697
PSNR Mean Y:45.645 U:48.870 V:50.319 Avg:46.462 Global:45.324 kb/s:1787.20
encoded 3300 frames, 2.14 fps, 1787.22 kb/s
Final statistics
Video Bitrate Desired: 1790 kbit/s
Video Bitrate Obtained (approximate): 1789 kbit/s

To a certain degree you can ignore the encoded @ fps as I was doing other stuff on my pc, but I have found that AutoVAQ is usually faster on both passes than standard VAQ builds.

As for visual quality, with the CRF encodes I did much prefer the output from the standard VAQ, but that's hardly surprising when it's almost double the bitrate of the AutoVAQ!

As far as bitrate encodes goes, it was kinda evenly tied. Some frames looked better with AutoVAQ, while others looked worse. If I had to lean one way, I'd probably say the the standard VAQ looked a bit better overall, but that's simply because it seemed to handle grain better. That's purely on one source though, so I'm not saying it would be the case always.

Obviously as it stands if/when AutoVAQ does get committed, people will need to find a new CRF value to suit their needs.

:)

[burfadel]

AutoVAQ is optimised to output files that are a similar size to what you would receive without using and VAQ at all! I saw that comment in the patch code. The tuning can be fairly easily modified by the looks of it to reflect a similar output size to what you have with CRF+VAQ (and of course the extra bitrate used for quality difference used optimisations in bitrate mode). With autovaq, maybe half way between with and without standard VAQ would be an idea? You could even set it up to have different optimisations, such that you can have '--autovaq low' (current tunings), '--autovaq normal' (default, not needed to be added) - this is the halfway tuning method I suggested, and '--autovaq high', to reflect the bitrate you'd expect with using standard VAQ. Just an idea.

I think with autovaq you also need auto psy-rd/psy-trellis. No reason why these can't be auto either from what I can tell, and may help is the situations desta pointed out!

[Dark Shikari]

I think with autovaq you also need auto psy-rd/psy-trellis. No reason why these can't be auto either from what I can tell, and may help is the situations desta pointed out!"Auto" psy-rd is completely nonsensical. The entire point of "auto" VAQ (which is really a misnomer, it should be "Variance-Hadamard AQ") is to adjust bit distribution throughout the frame. Psy-rd is solely a per-macroblock decision, so it makes no sense to "adjust it per frame."

The name "auto" is a terrible one and has almost no relation to the real algorithm. I intend to excise it from the patch if/when I commit it.

[burfadel]

ah ok! still, how does the weighting of low/normal/high sound?

[desta]

I know AQ in general was usually advised against when dealing with anime, but with AutoVAQ would it be safe to assume it could be left on at default? I mean if it's adjusting its strength as required per frame, then wouldn't it only be utilized if deemed necessary?

[Sharktooth]

@desta: http://forum.doom9.org/showthread.php?p=1290782#post1290782

[desta]

Gotcha. Apologies. :)

[Dark Shikari]

I know AQ in general was usually advised against when dealing with anime, but with AutoVAQ would it be safe to assume it could be left on at default? I mean if it's adjusting its strength as required per frame, then wouldn't it only be utilized if deemed necessary?AQ isn't "bad" with anime at all. At worst, it'll give pretty neutral results, and at best it'll improve background detail and reduce banding.

You seem to, again, misunderstand how "AutoVAQ" works. It is not an artificially intelligent program that can magically figure out how much AQ a frame needs to look best to you. All it does is change the quantizer distribution somewhat based on framewide variance.

[desta]

You seem to, again, misunderstand how "AutoVAQ" works.
Again? It was my first question on the subject. If my question seemed a bit uneducated, it was based purely on reading this...

"The main purpose of this patch was to make algorithm automaticaly choose close to optimal strength of AQ (on per frame base) and by this increase quality of encoding (without need of manually finding optimal strength for every source)."

As it happens I use 'VAQ' on anime anyway - I merely asked as I'd seen it advised against before, and of course it's off by default in megui's anime profiles.

[juGGaKNot]

AQ isn't "bad" with anime at all. At worst, it'll give pretty neutral results, and at best it'll improve background detail and reduce banding.


Yes but at the cost of bitrate from my tests.

And also very dark areas suffer bad with it on.

[sho_t]

I intend to excise it from the patch if/when I commit it.

When do you think that this patch is committed?
Do you think that it is necessary for AutoVAQ to be improved more?
Or just needs more tests?

[burfadel]

I personally think the quality weightings for CRF should be changed to reflect VAQ, not non-VAQ. Since VAQ has been around a little while, people have optimised the CRF value they typically use. To change this again would be silly, and since there is typically a bitrate saving, having the quality for a particular CRF higher than before is not necessarily a bad thing. This hold especially true since the required bitrate drops between AutoVAQ and normal VAQ.

So:
Same CRF + higher quality + slightly lower bitrate = very few complaints
Same CRF + lower quality due to weightings compared between VAQ and non-VAQ + much lower bitrate = lots of complaints
!!

[rack04]

Here are some results:

Both samples were encoded using --crf 18 for the 1st Pass and the bitrate obtained from the 1st Pass was used in the 2nd Pass.

--aq-mode 1 (Default) (http://www.mediafire.com/file/ymdyyomnnmy/sample-aq-mode 1.mkv)
File size : 14.8 MiB
Bit rate : 9 828 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9950061
x264 [info]: PSNR Mean Y:56.510 U:59.188 V:59.676 Avg:57.034 Global:51.288 kb/s:10263.95

2nd Pass:
x264 [info]: SSIM Mean Y:0.9959064
x264 [info]: PSNR Mean Y:57.234 U:59.983 V:60.452 Avg:57.817 Global:52.749 kb/s:10069.79

--aq-mode 2 (AutoVAQ) (http://www.mediafire.com/file/om2egdqkfqz/sample-aq-mode 2.mkv)
File size : 8.86 MiB
Bit rate : 5 290 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9935745
x264 [info]: PSNR Mean Y:55.438 U:58.289 V:58.908 Avg:56.010 Global:50.181 kb/s:5512.29

2nd Pass:
x264 [info]: SSIM Mean Y:0.9944984
x264 [info]: PSNR Mean Y:56.076 U:58.903 V:59.581 Avg:56.650 Global:51.324 kb/s:5426.56

The best way that I know to compare the encodes is to use the following AviSynth script:

source1=DirectShowSource("[PATH\]sample-aq-mode 1.mkv", fps=23.976, audio=false, convertfps=true)
source2=DirectShowSource("[PATH\]sample-aq-mode 2.mkv", fps=23.976, audio=false, convertfps=true)
left=source1.script1()
right=source2.script2()
width=left.width()/2
height=left.height()
left=crop(left,0,0,width,height).addborders(0,0,2,0,$0000FF).crop(2,0,-0,-0)
right=crop(right,width,0,width,height).addborders(2,0,0,0,$0000FF).crop(0,0,-2,-0)
StackHorizontal(left.subtitle("aq-mode 1"),right.subtitle("aq-mode 2"))
function script1(clip c) {
c
#----- ENTER CODE OF SCRIPT ONE HERE -----
#----- END OF CODE OF SCRIPT ONE-----
}
function script2(clip c) {
c
#----- ENTER CODE OF SCRIPT TWO HERE -----
#----- END OF CODE OF SCRIPT TWO-----
}

[rack04]

Here are some results:

Both samples were encoded using --bitrate 10000 for the 1st Pass and the 2nd Pass.

--aq-mode 1 (Default) (http://www.mediafire.com/file/no2d5mvh0mn/sample_10000_aq-mode 1.mkv)

1st Pass:
x264 [info]: SSIM Mean Y:0.9952087
x264 [info]: PSNR Mean Y:56.955 U:59.735 V:60.278 Avg:57.498 Global:51.345 kb/s:12298.52

2nd Pass:
x264 [info]: SSIM Mean Y:0.9958063
x264 [info]: PSNR Mean Y:57.084 U:59.831 V:60.287 Avg:57.666 Global:52.680 kb/s:9664.95

--aq-mode 2 (AutoVAQ) (http://www.mediafire.com/file/2emx0ommi2v/sample_10000_aq-mode 2.mkv)

1st Pass:
x264 [info]: SSIM Mean Y:0.9953888
x264 [info]: PSNR Mean Y:57.097 U:59.850 V:60.387 Avg:57.627 Global:51.881 kb/s:11483.60

2nd Pass:
x264 [info]: SSIM Mean Y:0.9960353
x264 [info]: PSNR Mean Y:57.534 U:60.593 V:61.208 Avg:58.152 Global:53.226 kb/s:10010.41

[rack04]

Here is my last test.

My aim was to vary the --crf for aq-mode 2 to obtain similar file size as aq-mode 1.

--aq-mode 1 (Default) @ --crf 18 (http://www.mediafire.com/file/ymdyyomnnmy/sample-aq-mode 1.mkv)

File size : 14.8 MiB
Bit rate : 9 828 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9950061
x264 [info]: PSNR Mean Y:56.510 U:59.188 V:59.676 Avg:57.034 Global:51.288 kb/s:10263.95

2nd Pass:
x264 [info]: SSIM Mean Y:0.9959064
x264 [info]: PSNR Mean Y:57.234 U:59.983 V:60.452 Avg:57.817 Global:52.749 kb/s:10069.79

--aq-mode 2 (AutoVAQ) @ --crf 14 (http://www.mediafire.com/file/dmoz4m2g1am/sample_2 Pass_CRF14_aq-mode 2.mkv)

File size : 14.8 MiB
Bit rate : 9 874 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9953311
x264 [info]: PSNR Mean Y:56.883 U:59.608 V:60.150 Avg:57.401 Global:51.979 kb/s:10144.42

2nd Pass:
x264 [info]: SSIM Mean Y:0.9960669
x264 [info]: PSNR Mean Y:57.552 U:60.608 V:61.229 Avg:58.172 Global:53.309 kb/s:10104.36

[Keiyakusha]

When I read 1st post of this thread, I thought --aq-mode 2 doesn't exist. Am I miss something?

Patch does not add or remove any x264 options but replace the current offical VAQ implementation.

[rack04]

When I read 1st post of this thread, I thought --aq-mode 2 doesn't exist. Am I miss something?

http://forum.doom9.org/showthread.php?p=1299330#post1299330

[MasterNobody]

When I read 1st post of this thread, I thought --aq-mode 2 doesn't exist. Am I miss something?
Starting from x264_AutoVAQ.03.diff (http://forum.doom9.org/showthread.php?p=1299510#post1299510) version it was made as addition (--aq-mode 2 for activating).

[Keiyakusha]

rack04, MasterNobody
Thanks :)

[Manaka]

Starting from x264_AutoVAQ.03.diff (http://forum.doom9.org/showthread.php?p=1299510#post1299510) version it was made as addition (--aq-mode 2 for activating).

It will adjust aq strength automatically right? Or I need to set --aq-strength? :confused:

[juGGaKNot]

It will adjust aq strength automatically right? Or I need to set --aq-strength? :confused:

I'm still using the "optimal" 1.0

[MasterNobody]

I make some test of different AQ modes and here is results: test_AQ.rar (http://stashbox.org/571675/test_AQ.rar)
And here is patch and script used for testing: test_AQ_script_and_patch.rar (http://stashbox.org/571744/test_AQ_script_and_patch.rar)
Used acronyms: auto=AutoVAQ (as in 0.2 and 0.3 version), std=standard VAQ, base=no AQ (its size is used as reference for other modes), pow*=additional experimental modes

As conclusion: this test shows that AutoVAQ is not always good and sometimes better results can be made by other modes (if manually find --aq-strength for encoded sample), but also shows that AutoVAQ and pow2-mode in most situations have pick near --aq-strength 1.0 (and other modes doesn't have such pick stability).

[komisar]

MasterNobody, and my second attempt (http://komisar.gin.by/CRF2Size.html) for help to gather CRF-value to match bitrate...

[Sharc]

MasterNobody, and my second attempt (http://komisar.gin.by/CRF2Size.html) for help to gather CRF-value to match bitrate...
Interesting. Can we replace 'NeedBitrate' by 'NeedFilesize' and 'GotBitrate' by 'GotFilesize'? I would assume that the algorithm should still work based on filesize, right?
Did you experience problems with convergence?

[komisar]

Sharc, current "second attempt" work with bitrate (not filesize). This is a simplest way to analyse x264-log-file. Convert you FileSize to Bitrate:

FileSize = MovieLenInSeconds * Bitrate * 1000 / 8
Bitrate = FileSize * 8 / (MovieLenInSeconds * 1000)

(FileSize in bytes)

I add Bitrate2Bytes(aBitrate, aMovieLenInSeconds), Bytes2Bitrate(aFileSize, aMovieLenInSeconds) for you help to xUtils.py

[Sharc]

Thank you, komisar.

[LoRd_MuldeR]

New AQ algorithm option
"Auto-variance" uses log(var)^2 instead of log(var) and attempts to adapt strength per-frame.
Generates significantly better SSIM; on by default with --tune ssim.
Whether it generates visually better quality is still up for debate.
Available as --aq-mode 2.

It's been quiet around AutoVAQ lately ;)

I wonder, is there still any research/development going on into this direction or is the committed AutoVAQ kind of "final" ???

The x264 developers seem to have some objections against current AutoVAQ, as they didn't make the "new" AQ method the new default.

So is there any chance that AutoVAQ will be improved further and finally become the default or will it remain the way it is?

[MasterNobody]

My researches are currently stopped after testing --aq-mode 3 patch (http://stashbox.org/594973/AQ.diff) and AQ2mod patch (http://stashbox.org/643517/x264_AQ2mod.diff) (basically the same idea but different options for tuning). Their purpose are improving fades (a specially when using MBTree) by creating bias for dark/flat frames (similar to what --aq-mode 1 have) in addition to standard --aq-mode 2 QP distribution. After that I am out of ideas for AQ. As for "why --aq-mode 2 is not default" bother Dark Shikari or pengvado.

P.S. Next time I would look at AQ probably after committing Weight-P (I hope we will see this sometime. I prefer it would be earlier then later).

[Dark Shikari]

The primary problem I have with all these approaches is that the lambda used for a given macroblock depends on the other blocks in the frame. A lot of x264 (particularly MB-tree) makes the assumption that a two blocks with identical complexity will have identical quant offsets regardless of where they are. It also seems conceptually wrong to me.