AutoVAQ patch testing

[MasterNobody]

I think it is the time for testing my experimental AutoVAQ patch (http://stashbox.org/502291/x264_AutoVAQ.02.diff) (updated 03 version (http://stashbox.org/549826/x264_AutoVAQ.03.diff) which doesn't overwrite standard VAQ and add "--aq-mode 2" option for activating AutoVAQ) in wider audience than users of #x264dev IRC-channel and small Russian-speaking group of people. The main purpose of this patch was to make algorithm automaticaly choose close to optimal strength of AQ (on per frame base) and by this increase quality of encoding (without need of manually finding optimal strength for every source).
Patch does not add or remove any x264 options but replace the current offical VAQ implementation. The algorithm is tweaked to "--aq-strength 1.0" (which is default) and changing of this parameter is not recommended.
For testing you may use:
- my reference x264 CLI build (http://stashbox.org/515099/x264_AutoVAQ.exe) based on x264 revision 1148 (non fprofiled, without MP4 support, no other patches)
- Komisar's kAVAQ builds (http://komisar.gin.by/)
- LoRd_MuldeR's libx264 binaries for Avidemux (http://forum.doom9.org/showthread.php?p=1284949#post1284949)
- or build x264 yourself with AutoVAQ patch (http://stashbox.org/502291/x264_AutoVAQ.02.diff)
For comparison you may need clean x264 CLI build (http://x264.nl/).

Here is comparison of current offical VAQ (StdVAQ) and my AutoVAQ builds on two rather different samples (for which optimal strength of StdVAQ differ considerably):

Parkrun encodes (http://www.mediafire.com/file/vdeevfhywjw/parkrun.zip) of Parkrun source (ftp://ftp.ldv.e-technik.tu-muenchen.de/pub/test_sequences/720p/720p50_parkrun_ter.yuv) with base options:
--fps 25 --threads 3 --8x8dct --b-adapt 2 --bframes 4 --b-pyramid --weightb --ref 3 --mixed-refs --me umh --subme 9 --trellis 2 --no-fast-pskip
without Psy options (--psy-rd 0.0:0.0):
StdVAQ --crf 28.00 --aq-strength 1.0 SSIM Y: 0.8383909 APSNR: 28.364 OPSNR: 28.001 kb/s: 1568.94
StdVAQ --crf 26.15 --aq-strength 1.5 SSIM Y: 0.8486719 APSNR: 28.151 OPSNR: 27.764 kb/s: 1568.91
StdVAQ --crf 24.65 --aq-strength 2.0 SSIM Y: 0.8514804 APSNR: 27.749 OPSNR: 27.292 kb/s: 1571.41
AutoVAQ --crf 30.23 --aq-strength 1.0 SSIM Y: 0.8586000 APSNR: 28.077 OPSNR: 27.724 kb/s: 1567.00
with Psy options (--psy-rd 1.0:0.3):
StdVAQ --crf 29.33 --aq-strength 1.0 SSIM Y: 0.8316043 APSNR: 27.694 OPSNR: 27.318 kb/s: 1570.51
StdVAQ --crf 27.56 --aq-strength 1.5 SSIM Y: 0.8395926 APSNR: 27.439 OPSNR: 27.039 kb/s: 1571.78
StdVAQ --crf 26.18 --aq-strength 2.0 SSIM Y: 0.8397329 APSNR: 26.980 OPSNR: 26.509 kb/s: 1573.75
AutoVAQ --crf 31.58 --aq-strength 1.0 SSIM Y: 0.8500254 APSNR: 27.386 OPSNR: 27.021 kb/s: 1568.49

Azumanga encodes (http://www.mediafire.com/file/drya23yrz0y/azumanga.zip) of Azumanga source (http://mirror05.x264.nl/Dark/LosslessAzumanga.mkv) with base options:
--frames 1156 --fps 25 --threads 3 --8x8dct --b-adapt 2 --bframes 4 --b-pyramid --weightb --ref 3 --mixed-refs --me umh --subme 9 --trellis 2 --no-fast-pskip
without Psy options (--psy-rd 0.0:0.0):
StdVAQ --crf 29.55 --aq-strength 0.5 SSIM Y: 0.9921343 APSNR: 41.893 OPSNR: 41.173 kb/s: 297.54
StdVAQ --crf 28.00 --aq-strength 1.0 SSIM Y: 0.9922015 APSNR: 41.575 OPSNR: 40.809 kb/s: 297.78
StdVAQ --crf 26.63 --aq-strength 1.5 SSIM Y: 0.9919121 APSNR: 41.084 OPSNR: 40.249 kb/s: 297.91
StdVAQ --crf 25.50 --aq-strength 2.0 SSIM Y: 0.9911979 APSNR: 40.387 OPSNR: 39.479 kb/s: 298.04
AutoVAQ --crf 26.78 --aq-strength 1.0 SSIM Y: 0.9921364 APSNR: 41.837 OPSNR: 40.989 kb/s: 297.46
with Psy options (--psy-rd 1.0:0.3):
StdVAQ --crf 31.13 --aq-strength 0.5 SSIM Y: 0.9903588 APSNR: 41.438 OPSNR: 40.598 kb/s: 297.76
StdVAQ --crf 29.67 --aq-strength 1.0 SSIM Y: 0.9901783 APSNR: 40.989 OPSNR: 40.099 kb/s: 298.11
StdVAQ --crf 28.55 --aq-strength 1.5 SSIM Y: 0.9893354 APSNR: 40.279 OPSNR: 39.318 kb/s: 298.30
StdVAQ --crf 27.60 --aq-strength 2.0 SSIM Y: 0.9880089 APSNR: 39.444 OPSNR: 38.410 kb/s: 298.44
AutoVAQ --crf 28.35 --aq-strength 1.0 SSIM Y: 0.9903360 APSNR: 41.409 OPSNR: 40.418 kb/s: 297.76

I wouldn't make any conclusions from these tests. Make your own comparison (on other sources) and write yours results and conclusions.

[LoRd_MuldeR]

Hi. I noticed that with the AutoVAQ patch I get significant smaller files in CRF mode. Like ~15% smaller at CRF=22.

Of course I can lower the CRF value by one or two in order to compensate. But is this intended?

[MasterNobody]

Hi. I noticed that with the AutoVAQ patch I get significant smaller files in CRF mode. Like ~15% with CRF=22.

Of course I can lower the CRF value by one or two in order to compensate. But is this intended?
Yes, this is intended that it changes CRF/size ratio and you need probably find new CRF which give good quality (and it can not only decrease size but also increase it on other sources). Look at my comparison. For Parkrun sample it increases size on the same CRF and for Azumanga sample it decreases size on the same CRF.

[Dark Shikari]

When I eventually commit this, it will (for the fourth time) rebalance CRF.

[LoRd_MuldeR]

Okay. Thanks for the info.

(BTW: May it be possible to apply this patch in a way that doesn't need GCC 4.x and the "force_align_arg_pointer" attribute to prevent crash? Tried using "x264_stack_align()", but it only accepts one param)

[LoRd_MuldeR]

http://img12.imageshack.us/img12/3037/autovaqdefault.png

http://img14.imageshack.us/img14/6080/autovaqenabled.png

Encoding settings: cabac=1 / ref=8 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=9 / psy_rd=1.0:0.2 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-3 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=5 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc=2pass / bitrate=1500 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00

I find the Non-AutoVAQ version more pleasing with this source :o

[Dark Shikari]

Single-frame comparisons are completely useless for this patch for the following reasons:

1. The patch inherently reverts AQ's effect on redistribution of bits. This may reduce the quality of some frames and increase the quality of others.

2. This patch changes AQ on a per-frame basis. In theory, some frames could get worse.

[LoRd_MuldeR]

Somehow I need to compare and it's really hard to compare two encodes in full playback speed. So I step through the interleaved video frame by frame. How do you do it?

Also I can find many frames like that in my encode where the Non-AutoVAQ version looks more pleasing. But I hardly see any frames that actually look better with AutoVAQ.

[Dark Shikari]

Somehow I need to compare and it's really hard to compare two encodes in full playback speed. So I step through the interleaved video frame by frame. How do you do it?

Also I can find many frames like that in my encode where the Non-AutoVAQ version looks more pleasing. But I hardly see any frames that actually look better with AutoVAQ.Pick a whole bunch of different frames.

Also, adjust the AQ strength. If it is overall worse, try adjusting the AQ strength until you can make it look better. If you can't, then it's worse, if you can, then its default is bad.

[Dark Shikari]

Specifically, there are three things about this patch that have to be tested separately:

1. Is its default AQ value good?

2. Is the ratio between AQ values in different frames good? That is, is the "auto" part of AutoVAQ good?

3. Is the distribution of quantizers in a single frame, ignoring the AQ strength of that frame, good? This patch is not really VAQ, as it uses a different metric. I'd call it more like "Hadamard Variance Adaptive Quantization", or HVAQ. Or VHAQ.

[MasterNobody]

Also I can find many frames like that in my encode where the Non-AutoVAQ version looks more pleasing. But I hardly see any frames that actually look better with AutoVAQ.
I would recommend to compare CRF-encodes with close bitrates (because of problem with few first frames in 2-pass encoding and high quant for first I-frame) or at least at long samples (with more than three I-frames). Also it would be good if you provide the source sample if you think that at it AutoVAQ gives considerably worse results than StdVAQ. How long is this sample?

[LoRd_MuldeR]

Pick a whole bunch of different frames.

Well, I scanned through the entire video and I easily spotted worse looking frames, but no better looking ones.

Also, adjust the AQ strength. If it is overall worse, try adjusting the AQ strength until you can make it look better. If you can't, then it's worse, if you can, then its default is bad.

Will try to do some more tests with different AQ strengths...

[LoRd_MuldeR]

I would recommend to compare CRF-encodes with close bitrates (because of problem with few first frames in 2-pass encoding and high quant for first I-frame) or at least at long samples (with more than three I-frames). Also it would be good if you provide the source sample if you think that at it AutoVAQ gives considerably worse results than StdVAQ. How long is this sample?

From all that I learned using CRF for comparison is a bad idea. Especially because the AutoVAQ version would be ~15% smaller in size. Using two 2-Pass encodes of the same (average) bitrate is highly recommended. Also my test sequence was a 5000 frame clip (taken from an even longer source). Long enough I guess. And the screenshot above was not taken from the very first part of the sequence.

[MasterNobody]

From all that I learned using CRF for comparison is a bad idea.I am not talking about comparison on the same CRF but on the CRFs that gives close bitrate.
Also my test sequence was a 5000 frame clip (taken from an even longer source). Long enough I guess.Yes, this is long enough.

[kemuri-_9]

hmm...

i would gander that if the new algo permanently replaces the current algo, there's probably gonna be (yet another) rush of complaints,
with some of them being 'I want the old version back because of xxxx!'

I also didn't notice anything in the patch code to prevent the aq-strength from being altered,
this would be currently due to the testing state of the patch to allow modifications for further testing,
but it has already been stated to be optimized for aq-strength=1.0,
so will the final version prevent such changes or at least warn when the value is changed?

I would say the ride would be less bumpy if it was instituted as a new aq-mode rather than completely replacing the current algorithm...
even if the majority consensus says the new version is better, chances are there will be some that claim the old version is better and would like to keep using it.
(On this point, what is the consensus as of yet?)


more on the testing sides of things when i find the time....
How long should a test sample be for it to be long enough to properly test the new algorithm? (and what should we expect to occur with short samples?)

and as a person who generally does 2pass bitrate encodes, what changes should i expect to occur (ssim/psnr/visual changes, etc.)? (as the bitrate won't change like it is in CRF)

[Sharktooth]

if the old method is objectively visually better than the new one i dont see any reason why the new method should replace the old one.
just a couple of point of SSIM score do not justify ANY visual detail loss.
on the other hand, if the new method is visually consistently better than the old one, go on...

[burfadel]

If the lower quality is attributed to a lower bitrate of ~15 percent (may vary) for a given CRF, it sounds reasonable to say that for a set bitrate the quality would be noticeably improved. It would also be reasonable to suggest to patch x264 to increase the bitrate used in CRF mode since you now have some spare, to the point where the image quality is actually improved (and still probably saving some bitrate)! It doesn't matter if the SSIM is reasonably higher in this case, the main point would be the increased quality at a given CRF whilst still saving a few percentage points of quality for most encodes. Complaints only occur if people have to change the CRF they use from say, 24 to 22 to make up the shortfall in quality.

To summarise, regardless of bitrate saving it would be more important to adjust CRF mode so people can use the same settings in CRF mode and have better quality, without using more bitrate!

[audyovydeo]

If the lower quality is attributed to a lower bitrate of ~15 percent (may vary) for a given CRF, it sounds reasonable to say that for a set bitrate the quality would be noticeably improved. It would also be reasonable to suggest to patch x264 to increase the bitrate used in CRF mode since you now have some spare, to the point where the image quality is actually improved (and still probably saving some bitrate)! It doesn't matter if the SSIM is reasonably higher in this case, the main point would be the increased quality at a given CRF whilst still saving a few percentage points of quality for most encodes. Complaints only occur if people have to change the CRF they use from say, 24 to 22 to make up the shortfall in quality.

To summarise, regardless of bitrate saving it would be more important to adjust CRF mode so people can use the same settings in CRF mode and have better quality, without using more bitrate!


Improvements are always welcome.
On the other hand, given the time-consuming process of empirically finding a new CRF value range by each user, the less CRF changes, the better.
As things stand, it would seems wise (as in : user-friendly) to add this new patch as an option, as has been suggested in the earlier post

cheers
audyovydeo

[nurbs]

Well, there is already an --aq-mode option, so I don't see how making this optional can be a problem. Looks like an interesting change. I'll have time to test this at the weekend.

[Dark Shikari]

Well, there is already an --aq-mode option, so I don't see how making this optional can be a problem. Looks like an interesting change. I'll have time to test this at the weekend.Yes, if there are any reasons to keep the old algorithm, I will make both available as --aq-modes.

[Sharktooth]

any news?

[Sagekilla]

I'll try testing this out, because if I can get good visual quality but at lower bitrates (i.e. use a higher CRF using autoVAQ than I did for regular VAQ) then I'd love to see this patch in public builds as an option.

[burfadel]

The AutoVAQ does seem to be much more bitrate friendly! Any loss visually can be compensated with algorithm adjustments, whilst still being automatic based around AQ 1! ... Even with compensation I could envisage a more efficient bitrate usage.

The reason for this statement is setting the CRF with normal AQ to get the same filesize for the same clip with AutoVAQ, using the exact same settings:
--keyint 450 --ref 5 --mixed-refs --bframes 6 --b-adapt 2 --b-pyramid --weightb --direct auto --subme 9 --trellis 2 --psy-rd 1:0.5 --partitions all --8x8dct --me umh --threads 6 --progress --thread-input --no-psnr --no-ssim --nr 500

The AutoVAQ clip had quite noticeably better quality!

Just a thought.

[CruNcher]

@Dark
any motion comparison of it on tuwabo :) ehh thouho dammit thouo ahh you know what i mean :D
also don't you see a chance of getting a visual improvement by finally go some adaptive deadzone approach in the encoder ?

[juGGaKNot]

WOW this is the best i can get out of aq ( i had it on 0 because it looks bad in general with psy rd )

settings :

start "encode" /b /low /max /wait "%myfiles%\x264v.exe" --pass 1 --bitrate %btrate% --stats "%mypath%\@temp\%mymovie%.stats" --level 4.1 --ref 4 --mixed-refs --bframes 4 --b-adapt 2 --weightb --direct auto --deblock -2:-1 --subme 9 --trellis 2 --psy-rd 0:0 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 24000 --vbv-maxrate 24000 --me umh --merange 24 --threads auto --aq-strength 1.0 --sar 1:1 --progress --no-psnr --no-ssim --output NUL "%mypath%\@temp\%mymovie%.avs"
start "encode" /b /low /max /wait "%myfiles%\x264v.exe" --pass 2 --bitrate %btrate% --stats "%mypath%\@temp\%mymovie%.stats" --level 4.1 --ref 4 --mixed-refs --bframes 4 --b-adapt 2 --weightb --direct auto --deblock -2:-1 --subme 9 --trellis 2 --psy-rd 0:0 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.1 --pbratio 1.1 --vbv-bufsize 24000 --vbv-maxrate 24000 --me umh --merange 24 --threads auto --aq-strength 1.0 --sar 1:1 --progress --no-psnr --no-ssim --output "%mypath%\@temp\%mymovie%.264" "%mypath%\@temp\%mymovie%.avs"

Bitrate 4500


avis [info]: 1024x576 @ 30.00 fps (1800 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 4.1
x264 [info]: slice I:24 Avg QP:18.00 size: 36535
x264 [info]: slice P:705 Avg QP:18.28 size: 20664
x264 [info]: slice B:1071 Avg QP:19.03 size: 12306
x264 [info]: consecutive B-frames: 5.6% 31.6% 37.5% 14.0% 11.3%
x264 [info]: mb I I16..4: 16.6% 63.2% 20.3%
x264 [info]: mb P I16..4: 12.3% 36.0% 6.2% P16..4: 25.8% 9.8% 4.3% 0.0% 0
.0% skip: 5.6%
x264 [info]: mb B I16..4: 4.4% 11.0% 2.1% B16..8: 31.2% 3.1% 2.7% direct:
12.1% skip:33.5% L0:43.8% L1:43.5% BI:12.7%
x264 [info]: final ratefactor: 16.70
x264 [info]: 8x8 transform intra:64.9% inter:73.0%
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: coded y,uvDC,uvAC intra:64.3% 58.9% 11.3% inter:34.2% 30.5% 1.4%
x264 [info]: ref P L0 77.8% 11.3% 6.9% 4.0%
x264 [info]: ref B L0 81.2% 12.1% 6.6%
x264 [info]: kb/s:3816.6

encoded 1800 frames, 2.30 fps, 3816.81 kb/s
avis [info]: 1024x576 @ 30.00 fps (1800 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 4.1
x264 [info]: slice I:24 Avg QP:18.19 size: 34036
x264 [info]: slice P:705 Avg QP:18.20 size: 23144
x264 [info]: slice B:1071 Avg QP:19.08 size: 14918
x264 [info]: consecutive B-frames: 5.6% 31.6% 37.5% 14.0% 11.3%
x264 [info]: mb I I16..4: 15.4% 68.5% 16.1%
x264 [info]: mb P I16..4: 13.2% 36.5% 5.3% P16..4: 24.0% 10.3% 4.2% 0.0% 0
.0% skip: 6.5%
x264 [info]: mb B I16..4: 4.6% 11.1% 2.0% B16..8: 30.2% 3.7% 2.9% direct:
13.1% skip:32.5% L0:42.1% L1:41.4% BI:16.5%
x264 [info]: 8x8 transform intra:65.4% inter:77.5%
x264 [info]: direct mvs spatial:95.7% temporal:4.3%
x264 [info]: coded y,uvDC,uvAC intra:63.3% 59.2% 10.8% inter:37.1% 30.2% 1.4%
x264 [info]: ref P L0 77.0% 11.9% 7.0% 4.1%
x264 [info]: ref B L0 81.5% 12.2% 6.3%
x264 [info]: kb/s:4414.8

encoded 1800 frames, 2.37 fps, 4415.05 kb/s

Encoding started at 05:50 PM and finished at 06:16 PM


26 minutes, with psy-rd it took 3 hours

Maybe aq is not the problem when it comes to my source and psy rd is.

Without aq :

avis [info]: 1024x576 @ 30.00 fps (1800 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 4.1
x264 [info]: slice I:24 Avg QP:18.71 size: 36688
x264 [info]: slice P:705 Avg QP:18.81 size: 20726
x264 [info]: slice B:1071 Avg QP:19.45 size: 12345
x264 [info]: consecutive B-frames: 5.6% 31.6% 37.5% 14.0% 11.3%
x264 [info]: mb I I16..4: 21.4% 59.1% 19.5%
x264 [info]: mb P I16..4: 15.2% 33.4% 5.6% P16..4: 26.0% 9.1% 4.3% 0.0% 0
.0% skip: 6.5%
x264 [info]: mb B I16..4: 4.8% 9.7% 2.0% B16..8: 30.4% 3.0% 2.7% direct:
11.4% skip:36.1% L0:43.8% L1:43.5% BI:12.7%
x264 [info]: final ratefactor: 17.54
x264 [info]: 8x8 transform intra:60.7% inter:73.2%
x264 [info]: direct mvs spatial:99.9% temporal:0.1%
x264 [info]: coded y,uvDC,uvAC intra:58.3% 54.1% 10.5% inter:32.4% 29.4% 1.7%
x264 [info]: ref P L0 77.9% 11.3% 6.9% 3.9%
x264 [info]: ref B L0 81.7% 11.9% 6.4%
x264 [info]: kb/s:3828.5

encoded 1800 frames, 1.97 fps, 3828.74 kb/s
avis [info]: 1024x576 @ 30.00 fps (1800 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Slow
x264 [info]: profile High, level 4.1
x264 [info]: slice I:24 Avg QP:18.75 size: 34385
x264 [info]: slice P:705 Avg QP:18.88 size: 23076
x264 [info]: slice B:1071 Avg QP:19.50 size: 14955
x264 [info]: consecutive B-frames: 5.6% 31.6% 37.5% 14.0% 11.3%
x264 [info]: mb I I16..4: 18.7% 66.9% 14.4%
x264 [info]: mb P I16..4: 16.2% 33.8% 4.6% P16..4: 24.0% 9.8% 3.9% 0.0% 0
.0% skip: 7.6%
x264 [info]: mb B I16..4: 4.9% 9.8% 1.8% B16..8: 29.7% 3.5% 2.9% direct:
12.3% skip:35.1% L0:42.3% L1:41.4% BI:16.3%
x264 [info]: 8x8 transform intra:61.3% inter:78.5%
x264 [info]: direct mvs spatial:95.8% temporal:4.2%
x264 [info]: coded y,uvDC,uvAC intra:57.8% 53.5% 10.1% inter:35.2% 29.0% 1.7%
x264 [info]: ref P L0 76.9% 12.0% 7.0% 4.1%
x264 [info]: ref B L0 81.7% 12.0% 6.3%
x264 [info]: kb/s:4414.7

encoded 1800 frames, 2.22 fps, 4414.87 kb/s

When will this be out of "testing".

[nm]

26 minutes, with psy-rd it took 3 hours

Maybe aq is not the problem when it comes to my source and psy rd is.
There is something seriously wrong with psy-rd taking that long.

[Dark Shikari]

Psy-rd should have no significant speed impact.

[juGGaKNot]

There is something seriously wrong with psy-rd taking that long.

Will test more tomorrow, it was 2:1

quality really sucked

Psy-rd should have no significant speed impact.

will see, just saying that aq is not a problem anymore ..

[poisondeathray]

What is the performance hit for using AutoVAQ? If it analyzes on a frame basis, it must have some overhead?

[juGGaKNot]

What is the performance hit for using AutoVAQ? If it analyzes on a frame basis, it must have some overhead?

Its faster by 0.15 FPS on my source, tested with low priority, see my logs.

Also the final ratefactor is 16.70, without autovac its 17.54

[LoRd_MuldeR]

What is the performance hit for using AutoVAQ? If it analyzes on a frame basis, it must have some overhead?

I can't detect a noteworthy speed difference between normal VAQ, AutoVAQ and AQ disabled. This is with both, Psy-RDO and Psy-Trellis, enabled.

[poisondeathray]

Have you done any more testing on that 1st clip Lord Mulder? (the one with the bad results)?

I'll try to do some testing this weekend

[LoRd_MuldeR]

Have you done any more testing on that 1st clip Lord Mulder? (the one with the bad results)?

No, not really. I can only say that AutoVAQ creates significant smaller files at CRF 22 with most of my sources (and bigger files only with a few sources).

This doesn't say anything about the quality of AutoVAQ, but it's an observation worth mentioning anyway...

[Chengbin]

Anybody have opinion on the AutoVAQ patch in terms of quality? Any pictures?

[LoRd_MuldeR]

Anybody have opinion on the AutoVAQ patch in terms of quality? Any pictures?

I did a first test and posted my results here:
http://forum.doom9.org/showpost.php?p=1285122&postcount=6

[MasterNobody]

Chengbin
There are encoded samples (not screenshots) in my first post by which you can compare quality yourself. But if you don't trust me, or your usual sources significantly differ from my test samples than you probably need to test it yourself on usual for you sources (and only after this you could understand would it give better quality FOR YOU or not).

[Fr4nz]

I did a first test and posted my results here:
http://forum.doom9.org/showpost.php?p=1285122&postcount=6

I don't see differences...maybe clips in "action" reveal more?

EDIT: Well, there are indeed some differences, and precisely on the dark background behind the singer...without autovaq the background seems to be more "noisy", which is better at the eyes IMO (so yes, I agree with you Mulder :D).

Anyway, as I stated previously, it would be interesting to see these clips in action and judge which one is the winner.

[LoRd_MuldeR]

Anyway, as I stated previously, it would be interesting to see these clips in action and judge which one is the winner.

In the first post you find clips that you can download an see "in action". I will also upload my test clip...

[komisar]

Some visual comparsion found here: http://komisar.gin.by/avaq/index.html
All result for actions and logs named "All test results in archive: tests.7z (58M)"

[juGGaKNot]

Some visual comparsion found here: http://komisar.gin.by/avaq/index.html
All result for actions and logs named "All test results in archive: tests.7z (58M)"

x264.1153kAVAQ.generic.x32.exe crashes on 2'nd pass

x264_AutoVAQ.02.diff is this the same as in this 1148 build or new ?

[vucloutr]

i got a BSOD + instant reboot on 2nd pass with skystrife's x264.1153M.AutoVAQ.x64.exe.

[Chengbin]

I am by no means have a eagle's eye (I'm actually quite blind compared to you guys). But is it possible to get a lower image quality when you use autovaq?

I did my own test, and the file is about 20% smaller at the same crf value. I can't see a difference.

I'm gonna do a test now with a ridiculously high crf value (30) on autovaq and normal x264 for both encodes so I can spot the quality differences easier.

Why is x264 so much smaller with autovaq? (794KB vs 1MB)

[LoRd_MuldeR]

i got a BSOD + instant reboot on 2nd pass with skystrife's x264.1153M.AutoVAQ.x64.exe.

x264 is running as a normal user-space process. Hence x264 itself may crash, but it cannot cause a BSOD. Something unrelated to x264 is seriously wrong with your system.

Check the stability of your system with Memtest86+ and Prim95. Keep an eye on the CPU temperature...

[LoRd_MuldeR]

I am by no means have a eagle's eye (I'm actually quite blind compared to you guys). But is it possible to get a lower image quality when you use autovaq?

I did my own test, and the file is about 20% smaller at the same crf value. I can't see a difference.

I'm gonna do a test now with a ridiculously high crf value (30) on autovaq and normal x264 for both encodes so I can spot the quality differences easier.

Why is x264 so much smaller with autovaq? (794KB vs 1MB)

First of all CRF doesn't deliver perfect constant quality, it only delivers "similar" quality. Furthermore AutoVAQ changes the meaning of the CRF values!

So there is only one way to do a proper comparison: Do two 2-Pass encodes at the same bitrate, one with AutoVAQ and one with the "normal" AQ algorithm.

Only files of the same size can be compared visually! Comparing files of different sizes doesn't lead anywhere...

[kumi]

Are comparisons of 2 equally-sized CRF encodes also valid?

[Dark Shikari]

Are comparisons of 2 equally-sized CRF encodes also valid?Yes, that's probably even better.

[LoRd_MuldeR]

Here is the sample I used for my first post:
* http://www.rapidspread.com/file.jsp?id=gzhurhg70n
* http://www.rapidspread.com/file.jsp?id=f9wzkp5iu2
* http://www.rapidspread.com/file.jsp?id=bp0inkxhj0
* http://www.rapidspread.com/file.jsp?id=4yduqmzzde

[burfadel]

From a comment line in the AutoVAQ patch:

constants chosen to result in approximately the same overall bitrate as without AQ.


That means that autovaq is optimised to produce file sizes similar to what is possible WITHOUT VAQ enabled at all. When VAQ was introduced it actually increased the file size for a given CRF.

Autovaq code can be adjusted to represent a final output size closer to that of the current VAQ (with higher quality supposedly). I think VAQ should be adjust to represented roughly a half way point between with and without VAQ, which should give a better picture whilst still resulting in a smaller file size (in CRF mode).

I think the next great step would be auto psy-rd/psy-trellis, the more complex the scene is the more it benefits from a higher value for each (to a certain extent). An autoPSY (ok, maybe not the best name for it!) would be helpful so adjustments for each encoding wouldn't be necessary, in addition to being very beneficial inside the same encode. A bright, complex detail wise scene could use --psy-rd 1.0:0.6 say as a base, and a flat scene, of which anime is very flat, can almost be turned completely off. --psy-rd function can be used as a basis, and it could be turned off (so its fixed as the values state) with a --noauto-psyrd command (for example).

[vucloutr]

x264 is running as a normal user-space process. Hence x264 itself may crash, but it cannot cause a BSOD. Something unrelated to x264 is seriously wrong with your system.

Check the stability of your system with Memtest86+ and Prim95. Keep an eye on the CPU temperature...
ok, i did not knew that (my sys worked without problems so far).
so i set all back to defaults and the 2nd pass went fine.

[LoRd_MuldeR]

ok, i did not knew that (my sys worked without problems so far).

x264 puts heavy load on your CPU. And that for a very long time. Hence an unstable system can crash easily, even if you didn't encounter problems before.

Still this isn't a problem with x264 itself, but a problem with your hardware...

[juGGaKNot]

x264 is running as a normal user-space process. Hence x264 itself may crash, but it cannot cause a BSOD. Something unrelated to x264 is seriously wrong with your system.

Check the stability of your system with Memtest86+ and Prim95. Keep an eye on the CPU temperature...

Crashes for me too, not the pc, just x264 on the 2'nd pass.

x264.1153kAVAQ.generic.x32.exe crashes on 2'nd pass

x264_AutoVAQ.02.diff is this the same as in this 1148 build or new ?

[menlvd]

Crashes for me too, not the pc, just x264 on the 2'nd pass.
same crash
x264.nl - passed
komisar - fail
techhouse - fail
all builds x64/x86
what can I do, to help solve this problem?

[burfadel]

Might be something to do with an patch incompatibility, since the vanilla build works and the others don't...

[desta]

Crashes for me too, not the pc, just x264 on the 2'nd pass.
same crash
x264.nl - passed
komisar - fail
techhouse - fail
all builds x64/x86
what can I do, to help solve this problem?


fwiw, same thing here. All 1153 x86 builds crash as the 2nd pass starts (though I havent tried the stable builds). Going back to 1148 fixes it for me.

[menlvd]

Might be something to do with an patch incompatibility, since the vanilla build works and the others don't...

yes... no-patch build's work fine!

[kemuri-_9]

the main differences between everyone's r1148 patched builds and the r1153 patched builds is that
r1148 used the v11 hrd pulldown/interlace patch while r1153 is using the v12.

so this should be looked into to see if it is indeed the problem.

[imk]

Indeed. My ICC builds apparently crash on the second pass, too. The patches I am using:

x264_hrd_pulldown.12_interlace.diff
x264_icc.diff
x264_win_zone_parse_fix_05.diff

Using the AutoVAQ patch or not seems to make no difference in getting x264 to crash on the second pass or not.

[kemuri-_9]

Using the AutoVAQ patch or not seems to make no difference in getting x264 to crash on the second pass or not.

the x264_hrd_pulldown.12_interlace.diff patch is causing the crash, wait for a v13 or revert back to v11 for the time being.

[Chengbin]

OK, here is my test. This is CRF 30 on a episode of Prison Break from a BD source.

IMO the VAQ one looked better for some frames, but some frames suffered from loss of detail.

Non VAQ

http://www.megaupload.com/?d=ANUUVRAJ

VAQ

http://www.megaupload.com/?d=ZAP78XOJ

[Razorholt]

Can you share your settings?

Thanks,
- Dan

[Chengbin]

--keyint 250 --bframes 4 --qpmin 10 --qpmax 51 --no-psnr --no-fast-pskip --mixed-refs --trellis 2 --ref 6 --filter 0,0 --subme 9 --direct auto --me umh --no-ssim --level 4.1 --merange 32 --qcomp 0.8 --weightb --b-adapt 2 --b-pyramid --partitions p8x8,b8x8,i4x4,i8x8 --threads 2 --aq-mode 1 --aq-strength 1 --psy-rd 1.0:0.4

[LoRd_MuldeR]

OK, here is my test. This is CRF 30 on a episode of Prison Break from a BD source.

IMO the VAQ one looked better for some frames, but some frames suffered from loss of detail.

Non VAQ

http://www.megaupload.com/?d=ANUUVRAJ

VAQ

http://www.megaupload.com/?d=ZAP78XOJ

If your test produced two files of different size, which most likely did happen in CRF mode, then your results are completely useless :scared:

For example: If one file looks worse, but also is smaller at the same time, how do you decide whether the quality loss is worth the bitrate saving or not?

Well, you can't. Hence you must compare files of identical size! Either find the CRF's that give identical size or use 2-Pass mode...

(BTW: You can remove "--aq-mode 1" and "--aq-strength 1", these are the default values)

[juGGaKNot]

the x264_hrd_pulldown.12_interlace.diff patch is causing the crash, wait for a v13 or revert back to v11 for the time being.

So autoVAQ 0.2 is used in 1148 also ? this is the only reason i updated to 1153.

[LoRd_MuldeR]

So autoVAQ 0.2 is used in 1148 also ? this is the only reason i updated to 1153.

AutoVAQ is not used in any x264 revision yet. It depends on whether the AutoVAQ patch was included in the individual build or not ;)

Skystrife's r1153 builds with and without AutoVAQ patch can be found here:
http://forum.doom9.org/showpost.php?p=1289106&postcount=1879

[juGGaKNot]

AutoVAQ is not used in any x264 revision yet. It depends on whether the AutoVAQ patch was included in the individual build or not ;)

Skystrife's r1153 builds with and without AutoVAQ patch can be found here:
http://forum.doom9.org/showpost.php?p=1289106&postcount=1879

k thnx.

[CruNcher]

wow it reduces i-frame pulsing on short gops a lot (the pulsing is not that heavy anymore) :)
though it's also slower for that by around 1 fps

[kemuri-_9]

though it's also slower for that by around 1 fps

1 fps less from what originally? 50? 25? 1.5?

[CruNcher]

31.x vs 30 but the visual difference @ the same even a little less bitrate is definitely there the change is less and the pulsing less visible it's much nicer to the eyes that way :)

[Dark Shikari]

31.x vs 30 but the visual difference @ the same even a little less bitrate is defiantly there the change is less and the pulsing less visible it's much nicer to the eyes that way :)If you can't visually separate the effects of AQ and VBV ratecontrol, then don't test them at the same time.

[CruNcher]

No vbv active here just crf with a short gop (even not that short almost 4 seconds)

here 1 second gop result

http://mirror05.x264.nl/CruNcher/force.php?file=./parkrun-25-oldvaq.mp4
http://mirror05.x264.nl/CruNcher/force.php?file=./parkrun-25-newvaq.mp4

pretty obvious the difference is much lower for that bitrate :)

[jefrey]

very nice patch :)
but what is the range from aq that is in use for the whole encode? ? 0--> 1.5?
and would it be possible (maybe in the near future or in the next patch :D ) to set the auto aq range manually, for example from 0.5 --> 1.3, instead of the whole AQ range? that would be great, i think so :D

[desta]

Just did some small tests using skystrifes 1158M builds (with and without AutoVAQ). First I did CRF encodes, which turned out a bit surprising (as far as bitrate is concerned). Commandline for both builds was:
--crf 19 --level 3.1 --ref 8 --mixed-refs --bframes 3 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock -2:-1 --subme 9 --trellis 2 --psy-rd 0.9:0.6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --output "output" "input"

And here were the results....

VAQ
slice I:64 Avg QP:15.13 size: 47393 PSNR Mean Y:48.95 U:51.80 V:52.52 Avg:49.72 Global:48.52
slice P:1166 Avg QP:17.28 size: 20599 PSNR Mean Y:47.69 U:50.97 V:52.07 Avg:48.50 Global:45.63
slice B:2070 Avg QP:19.42 size: 5546 PSNR Mean Y:44.56 U:48.00 V:49.57 Avg:45.42 Global:44.67
consecutive B-frames: 3.3% 10.3% 71.9% 14.5%
mb I I16..4: 22.8% 61.8% 15.4%
mb P I16..4: 4.6% 17.7% 2.5% P16..4: 29.2% 19.3% 21.3% 0.0% 0.0% skip: 5.4%
mb B I16..4: 0.3% 3.9% 0.2% B16..8: 31.0% 1.6% 2.7% direct: 8.4% skip:51.8% L0:31.2% L1:39.8% BI:29.0%
8x8 transform intra:73.3% inter:28.3%
direct mvs spatial:100.0% temporal:0.0%
coded y,uvDC,uvAC intra:98.4% 96.9% 93.6% inter:32.4% 41.9% 13.4%
ref P L0 46.1% 30.5% 6.6% 5.9% 2.7% 3.6% 2.1% 2.4%
ref B L0 57.9% 27.9% 4.8% 3.9% 2.1% 2.3% 1.1%
ref B L1 89.8% 10.2%
SSIM Mean Y:0.9840857
PSNR Mean Y:45.751 U:49.124 V:50.509 Avg:46.587 Global:45.039 kb/s:2335.29
encoded 3300 frames, 1.96 fps, 2335.47 kb/s
Final statistics
Constant Quality Mode: Quality 19 computed...
Video Bitrate Obtained (approximate): 2338 kbit/s

AutoVAQ
slice I:64 Avg QP:17.55 size: 34304 PSNR Mean Y:47.86 U:50.49 V:51.44 Avg:48.61 Global:47.64
slice P:1166 Avg QP:19.66 size: 10877 PSNR Mean Y:46.71 U:49.89 V:51.22 Avg:47.50 Global:44.99
slice B:2070 Avg QP:22.00 size: 2733 PSNR Mean Y:43.84 U:47.44 V:49.00 Avg:44.72 Global:44.19
consecutive B-frames: 3.3% 10.3% 71.9% 14.5%
mb I I16..4: 10.2% 78.7% 11.1%
mb P I16..4: 0.6% 5.8% 0.7% P16..4: 48.1% 17.5% 18.3% 0.0% 0.0% skip: 8.9%
mb B I16..4: 0.0% 0.4% 0.0% B16..8: 35.4% 1.0% 1.5% direct: 5.5% skip:56.0% L0:33.9% L1:48.1% BI:17.9%
8x8 transform intra:81.0% inter:45.0%
direct mvs spatial:99.9% temporal:0.1%
coded y,uvDC,uvAC intra:90.2% 95.6% 80.0% inter:24.4% 38.6% 5.1%
ref P L0 45.1% 28.8% 7.7% 6.4% 3.0% 4.1% 2.4% 2.5%
ref B L0 59.5% 25.8% 5.1% 3.8% 2.2% 2.3% 1.3%
ref B L1 91.1% 8.9%
SSIM Mean Y:0.9811661
PSNR Mean Y:44.931 U:48.361 V:49.830 Avg:45.775 Global:44.508 kb/s:1244.54
encoded 3300 frames, 2.14 fps, 1244.73 kb/s
Final statistics
Constant Quality Mode: Quality 19 computed...
Video Bitrate Obtained (approximate): 1247 kbit/s


I figured I'd use an average between the two as a bitrate for a 2-pass encode. The commandline this time was:
--pass 1 --bitrate 1790 --stats ".stats" --level 3.1 --bframes 3 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock -2:-1 --psy-rd 0.9:0 --partitions none --vbv-bufsize 14000 --vbv-maxrate 17500 --threads auto --thread-input --progress --output NUL "input"

--pass 2 --bitrate 1790 --stats ".stats" --level 3.1 --ref 8 --mixed-refs --bframes 3 --b-adapt 2 --b-pyramid --weightb --direct auto --deblock -2:-1 --subme 9 --trellis 2 --psy-rd 0.9:0.6 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 14000 --vbv-maxrate 17500 --me umh --threads auto --thread-input --progress --output "output" "input"

This results this time looked like this...

VAQ
1st Pass
slice I:64 Avg QP:15.63 size: 36994 PSNR Mean Y:48.08 U:50.25 V:51.10 Avg:48.72 Global:47.40
slice P:1163 Avg QP:17.10 size: 13731 PSNR Mean Y:46.73 U:49.58 V:50.74 Avg:47.45 Global:44.58
slice B:2073 Avg QP:19.34 size: 5483 PSNR Mean Y:44.16 U:47.45 V:48.80 Avg:44.96 Global:44.05
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 19.5% 0.0% 80.5%
mb P I16..4: 7.9% 0.0% 0.0% P16..4: 85.6% 0.0% 0.0% 0.0% 0.0% skip: 6.6%
mb B I16..4: 14.0% 0.0% 0.0% B16..8: 33.4% 0.0% 0.0% direct:24.9% skip:27.6% L0:24.1% L1:48.1% BI:27.8%
final ratefactor: 18.63
direct mvs spatial:100.0% temporal:0.0%
coded y,uvDC,uvAC intra:73.3% 94.0% 80.7% inter:32.7% 49.5% 10.9%
SSIM Mean Y:0.9821088
PSNR Mean Y:45.144 U:48.255 V:49.528 Avg:45.913 Global:44.278 kb/s:1800.20
encoded 3300 frames, 11.28 fps, 1800.23 kb/s

2nd Pass
slice I:64 Avg QP:16.10 size: 42037 PSNR Mean Y:48.42 U:51.21 V:51.99 Avg:49.18 Global:48.00
slice P:1163 Avg QP:18.12 size: 16140 PSNR Mean Y:47.24 U:50.40 V:51.61 Avg:48.03 Global:45.44
slice B:2073 Avg QP:20.56 size: 3866 PSNR Mean Y:44.15 U:47.68 V:49.27 Avg:45.02 Global:44.47
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 17.2% 69.3% 13.5%
mb P I16..4: 1.2% 14.2% 1.2% P16..4: 35.4% 20.7% 21.1% 0.0% 0.0% skip: 6.2%
mb B I16..4: 0.0% 1.2% 0.1% B16..8: 34.1% 1.3% 2.0% direct: 7.5% skip:53.7% L0:34.2% L1:43.1% BI:22.7%
8x8 transform intra:82.5% inter:31.8%
direct mvs spatial:97.2% temporal:2.8%
coded y,uvDC,uvAC intra:96.6% 96.1% 90.1% inter:29.2% 40.9% 10.7%
ref P L0 44.7% 29.7% 7.2% 6.3% 3.0% 4.0% 2.4% 2.6%
ref B L0 57.4% 26.9% 5.2% 4.2% 2.3% 2.6% 1.4%
ref B L1 90.4% 9.6%
SSIM Mean Y:0.9827564
PSNR Mean Y:45.318 U:48.708 V:50.144 Avg:46.160 Global:44.836 kb/s:1786.41
encoded 3300 frames, 2.01 fps, 1786.43 kb/s
Final statistics
Video Bitrate Desired: 1790 kbit/s
Video Bitrate Obtained (approximate): 1789 kbit/s


AutoVAQ
1st Pass
slice I:64 Avg QP:15.07 size: 39908 PSNR Mean Y:48.83 U:50.78 V:51.61 Avg:49.42 Global:48.40
slice P:1163 Avg QP:16.83 size: 13979 PSNR Mean Y:47.30 U:49.79 V:51.00 Avg:47.96 Global:45.42
slice B:2073 Avg QP:19.31 size: 5314 PSNR Mean Y:44.72 U:47.67 V:49.07 Avg:45.47 Global:44.84
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 20.4% 0.0% 79.6%
mb P I16..4: 6.8% 0.0% 0.0% P16..4: 86.6% 0.0% 0.0% 0.0% 0.0% skip: 6.7%
mb B I16..4: 11.8% 0.0% 0.0% B16..8: 35.5% 0.0% 0.0% direct:27.9% skip:24.8% L0:23.3% L1:45.2% BI:31.6%
final ratefactor: 15.97
direct mvs spatial:100.0% temporal:0.0%
coded y,uvDC,uvAC intra:61.4% 93.5% 73.5% inter:34.5% 53.9% 9.9%
SSIM Mean Y:0.9837163
PSNR Mean Y:45.710 U:48.477 V:49.797 Avg:46.420 Global:45.083 kb/s:1807.79
encoded 3300 frames, 10.60 fps, 1807.82 kb/s

2nd Pass
slice I:64 Avg QP:16.04 size: 41120 PSNR Mean Y:48.67 U:51.28 V:52.10 Avg:49.41 Global:48.44
slice P:1163 Avg QP:17.97 size: 15823 PSNR Mean Y:47.58 U:50.53 V:51.77 Avg:48.33 Global:45.99
slice B:2073 Avg QP:20.54 size: 4079 PSNR Mean Y:44.47 U:47.87 V:49.45 Avg:45.32 Global:44.93
consecutive B-frames: 3.3% 9.6% 72.3% 14.7%
mb I I16..4: 12.2% 75.5% 12.3%
mb P I16..4: 0.8% 9.8% 1.3% P16..4: 39.4% 20.4% 22.4% 0.0% 0.0% skip: 6.0%
mb B I16..4: 0.0% 0.9% 0.1% B16..8: 34.9% 1.3% 2.2% direct: 7.8% skip:52.8% L0:33.0% L1:41.0% BI:25.9%
8x8 transform intra:81.1% inter:34.1%
direct mvs spatial:97.2% temporal:2.8%
coded y,uvDC,uvAC intra:94.9% 97.6% 90.3% inter:30.8% 44.0% 10.6%
ref P L0 43.3% 30.0% 7.3% 6.7% 3.1% 4.3% 2.5% 2.8%
ref B L0 57.5% 27.7% 4.9% 4.1% 2.1% 2.4% 1.3%
ref B L1 91.0% 9.0%
SSIM Mean Y:0.9832697
PSNR Mean Y:45.645 U:48.870 V:50.319 Avg:46.462 Global:45.324 kb/s:1787.20
encoded 3300 frames, 2.14 fps, 1787.22 kb/s
Final statistics
Video Bitrate Desired: 1790 kbit/s
Video Bitrate Obtained (approximate): 1789 kbit/s

To a certain degree you can ignore the encoded @ fps as I was doing other stuff on my pc, but I have found that AutoVAQ is usually faster on both passes than standard VAQ builds.

As for visual quality, with the CRF encodes I did much prefer the output from the standard VAQ, but that's hardly surprising when it's almost double the bitrate of the AutoVAQ!

As far as bitrate encodes goes, it was kinda evenly tied. Some frames looked better with AutoVAQ, while others looked worse. If I had to lean one way, I'd probably say the the standard VAQ looked a bit better overall, but that's simply because it seemed to handle grain better. That's purely on one source though, so I'm not saying it would be the case always.

Obviously as it stands if/when AutoVAQ does get committed, people will need to find a new CRF value to suit their needs.

:)

[burfadel]

AutoVAQ is optimised to output files that are a similar size to what you would receive without using and VAQ at all! I saw that comment in the patch code. The tuning can be fairly easily modified by the looks of it to reflect a similar output size to what you have with CRF+VAQ (and of course the extra bitrate used for quality difference used optimisations in bitrate mode). With autovaq, maybe half way between with and without standard VAQ would be an idea? You could even set it up to have different optimisations, such that you can have '--autovaq low' (current tunings), '--autovaq normal' (default, not needed to be added) - this is the halfway tuning method I suggested, and '--autovaq high', to reflect the bitrate you'd expect with using standard VAQ. Just an idea.

I think with autovaq you also need auto psy-rd/psy-trellis. No reason why these can't be auto either from what I can tell, and may help is the situations desta pointed out!

[Dark Shikari]

I think with autovaq you also need auto psy-rd/psy-trellis. No reason why these can't be auto either from what I can tell, and may help is the situations desta pointed out!"Auto" psy-rd is completely nonsensical. The entire point of "auto" VAQ (which is really a misnomer, it should be "Variance-Hadamard AQ") is to adjust bit distribution throughout the frame. Psy-rd is solely a per-macroblock decision, so it makes no sense to "adjust it per frame."

The name "auto" is a terrible one and has almost no relation to the real algorithm. I intend to excise it from the patch if/when I commit it.

[burfadel]

ah ok! still, how does the weighting of low/normal/high sound?

[desta]

I know AQ in general was usually advised against when dealing with anime, but with AutoVAQ would it be safe to assume it could be left on at default? I mean if it's adjusting its strength as required per frame, then wouldn't it only be utilized if deemed necessary?

[Sharktooth]

@desta: http://forum.doom9.org/showthread.php?p=1290782#post1290782

[desta]

Gotcha. Apologies. :)

[Dark Shikari]

I know AQ in general was usually advised against when dealing with anime, but with AutoVAQ would it be safe to assume it could be left on at default? I mean if it's adjusting its strength as required per frame, then wouldn't it only be utilized if deemed necessary?AQ isn't "bad" with anime at all. At worst, it'll give pretty neutral results, and at best it'll improve background detail and reduce banding.

You seem to, again, misunderstand how "AutoVAQ" works. It is not an artificially intelligent program that can magically figure out how much AQ a frame needs to look best to you. All it does is change the quantizer distribution somewhat based on framewide variance.

[desta]

You seem to, again, misunderstand how "AutoVAQ" works.
Again? It was my first question on the subject. If my question seemed a bit uneducated, it was based purely on reading this...

"The main purpose of this patch was to make algorithm automaticaly choose close to optimal strength of AQ (on per frame base) and by this increase quality of encoding (without need of manually finding optimal strength for every source)."

As it happens I use 'VAQ' on anime anyway - I merely asked as I'd seen it advised against before, and of course it's off by default in megui's anime profiles.

[juGGaKNot]

AQ isn't "bad" with anime at all. At worst, it'll give pretty neutral results, and at best it'll improve background detail and reduce banding.


Yes but at the cost of bitrate from my tests.

And also very dark areas suffer bad with it on.

[sho_t]

I intend to excise it from the patch if/when I commit it.

When do you think that this patch is committed?
Do you think that it is necessary for AutoVAQ to be improved more?
Or just needs more tests?

[burfadel]

I personally think the quality weightings for CRF should be changed to reflect VAQ, not non-VAQ. Since VAQ has been around a little while, people have optimised the CRF value they typically use. To change this again would be silly, and since there is typically a bitrate saving, having the quality for a particular CRF higher than before is not necessarily a bad thing. This hold especially true since the required bitrate drops between AutoVAQ and normal VAQ.

So:
Same CRF + higher quality + slightly lower bitrate = very few complaints
Same CRF + lower quality due to weightings compared between VAQ and non-VAQ + much lower bitrate = lots of complaints
!!

[rack04]

Here are some results:

Both samples were encoded using --crf 18 for the 1st Pass and the bitrate obtained from the 1st Pass was used in the 2nd Pass.

--aq-mode 1 (Default) (http://www.mediafire.com/file/ymdyyomnnmy/sample-aq-mode 1.mkv)
File size : 14.8 MiB
Bit rate : 9 828 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9950061
x264 [info]: PSNR Mean Y:56.510 U:59.188 V:59.676 Avg:57.034 Global:51.288 kb/s:10263.95

2nd Pass:
x264 [info]: SSIM Mean Y:0.9959064
x264 [info]: PSNR Mean Y:57.234 U:59.983 V:60.452 Avg:57.817 Global:52.749 kb/s:10069.79

--aq-mode 2 (AutoVAQ) (http://www.mediafire.com/file/om2egdqkfqz/sample-aq-mode 2.mkv)
File size : 8.86 MiB
Bit rate : 5 290 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9935745
x264 [info]: PSNR Mean Y:55.438 U:58.289 V:58.908 Avg:56.010 Global:50.181 kb/s:5512.29

2nd Pass:
x264 [info]: SSIM Mean Y:0.9944984
x264 [info]: PSNR Mean Y:56.076 U:58.903 V:59.581 Avg:56.650 Global:51.324 kb/s:5426.56

The best way that I know to compare the encodes is to use the following AviSynth script:

source1=DirectShowSource("[PATH\]sample-aq-mode 1.mkv", fps=23.976, audio=false, convertfps=true)
source2=DirectShowSource("[PATH\]sample-aq-mode 2.mkv", fps=23.976, audio=false, convertfps=true)
left=source1.script1()
right=source2.script2()
width=left.width()/2
height=left.height()
left=crop(left,0,0,width,height).addborders(0,0,2,0,$0000FF).crop(2,0,-0,-0)
right=crop(right,width,0,width,height).addborders(2,0,0,0,$0000FF).crop(0,0,-2,-0)
StackHorizontal(left.subtitle("aq-mode 1"),right.subtitle("aq-mode 2"))
function script1(clip c) {
c
#----- ENTER CODE OF SCRIPT ONE HERE -----
#----- END OF CODE OF SCRIPT ONE-----
}
function script2(clip c) {
c
#----- ENTER CODE OF SCRIPT TWO HERE -----
#----- END OF CODE OF SCRIPT TWO-----
}

[rack04]

Here are some results:

Both samples were encoded using --bitrate 10000 for the 1st Pass and the 2nd Pass.

--aq-mode 1 (Default) (http://www.mediafire.com/file/no2d5mvh0mn/sample_10000_aq-mode 1.mkv)

1st Pass:
x264 [info]: SSIM Mean Y:0.9952087
x264 [info]: PSNR Mean Y:56.955 U:59.735 V:60.278 Avg:57.498 Global:51.345 kb/s:12298.52

2nd Pass:
x264 [info]: SSIM Mean Y:0.9958063
x264 [info]: PSNR Mean Y:57.084 U:59.831 V:60.287 Avg:57.666 Global:52.680 kb/s:9664.95

--aq-mode 2 (AutoVAQ) (http://www.mediafire.com/file/2emx0ommi2v/sample_10000_aq-mode 2.mkv)

1st Pass:
x264 [info]: SSIM Mean Y:0.9953888
x264 [info]: PSNR Mean Y:57.097 U:59.850 V:60.387 Avg:57.627 Global:51.881 kb/s:11483.60

2nd Pass:
x264 [info]: SSIM Mean Y:0.9960353
x264 [info]: PSNR Mean Y:57.534 U:60.593 V:61.208 Avg:58.152 Global:53.226 kb/s:10010.41

[rack04]

Here is my last test.

My aim was to vary the --crf for aq-mode 2 to obtain similar file size as aq-mode 1.

--aq-mode 1 (Default) @ --crf 18 (http://www.mediafire.com/file/ymdyyomnnmy/sample-aq-mode 1.mkv)

File size : 14.8 MiB
Bit rate : 9 828 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9950061
x264 [info]: PSNR Mean Y:56.510 U:59.188 V:59.676 Avg:57.034 Global:51.288 kb/s:10263.95

2nd Pass:
x264 [info]: SSIM Mean Y:0.9959064
x264 [info]: PSNR Mean Y:57.234 U:59.983 V:60.452 Avg:57.817 Global:52.749 kb/s:10069.79

--aq-mode 2 (AutoVAQ) @ --crf 14 (http://www.mediafire.com/file/dmoz4m2g1am/sample_2 Pass_CRF14_aq-mode 2.mkv)

File size : 14.8 MiB
Bit rate : 9 874 Kbps
Writing library : x264 core 67 r1171M 2c7cb4c

1st Pass:
x264 [info]: SSIM Mean Y:0.9953311
x264 [info]: PSNR Mean Y:56.883 U:59.608 V:60.150 Avg:57.401 Global:51.979 kb/s:10144.42

2nd Pass:
x264 [info]: SSIM Mean Y:0.9960669
x264 [info]: PSNR Mean Y:57.552 U:60.608 V:61.229 Avg:58.172 Global:53.309 kb/s:10104.36

[Keiyakusha]

When I read 1st post of this thread, I thought --aq-mode 2 doesn't exist. Am I miss something?

Patch does not add or remove any x264 options but replace the current offical VAQ implementation.

[rack04]

When I read 1st post of this thread, I thought --aq-mode 2 doesn't exist. Am I miss something?

http://forum.doom9.org/showthread.php?p=1299330#post1299330

[MasterNobody]

When I read 1st post of this thread, I thought --aq-mode 2 doesn't exist. Am I miss something?
Starting from x264_AutoVAQ.03.diff (http://forum.doom9.org/showthread.php?p=1299510#post1299510) version it was made as addition (--aq-mode 2 for activating).

[Keiyakusha]

rack04, MasterNobody
Thanks :)

[Manaka]

Starting from x264_AutoVAQ.03.diff (http://forum.doom9.org/showthread.php?p=1299510#post1299510) version it was made as addition (--aq-mode 2 for activating).

It will adjust aq strength automatically right? Or I need to set --aq-strength? :confused:

[juGGaKNot]

It will adjust aq strength automatically right? Or I need to set --aq-strength? :confused:

I'm still using the "optimal" 1.0

[MasterNobody]

I make some test of different AQ modes and here is results: test_AQ.rar (http://stashbox.org/571675/test_AQ.rar)
And here is patch and script used for testing: test_AQ_script_and_patch.rar (http://stashbox.org/571744/test_AQ_script_and_patch.rar)
Used acronyms: auto=AutoVAQ (as in 0.2 and 0.3 version), std=standard VAQ, base=no AQ (its size is used as reference for other modes), pow*=additional experimental modes

As conclusion: this test shows that AutoVAQ is not always good and sometimes better results can be made by other modes (if manually find --aq-strength for encoded sample), but also shows that AutoVAQ and pow2-mode in most situations have pick near --aq-strength 1.0 (and other modes doesn't have such pick stability).

[komisar]

MasterNobody, and my second attempt (http://komisar.gin.by/CRF2Size.html) for help to gather CRF-value to match bitrate...

[Sharc]

MasterNobody, and my second attempt (http://komisar.gin.by/CRF2Size.html) for help to gather CRF-value to match bitrate...
Interesting. Can we replace 'NeedBitrate' by 'NeedFilesize' and 'GotBitrate' by 'GotFilesize'? I would assume that the algorithm should still work based on filesize, right?
Did you experience problems with convergence?

[komisar]

Sharc, current "second attempt" work with bitrate (not filesize). This is a simplest way to analyse x264-log-file. Convert you FileSize to Bitrate:

FileSize = MovieLenInSeconds * Bitrate * 1000 / 8
Bitrate = FileSize * 8 / (MovieLenInSeconds * 1000)

(FileSize in bytes)

I add Bitrate2Bytes(aBitrate, aMovieLenInSeconds), Bytes2Bitrate(aFileSize, aMovieLenInSeconds) for you help to xUtils.py

[Sharc]

Thank you, komisar.

[LoRd_MuldeR]

New AQ algorithm option
"Auto-variance" uses log(var)^2 instead of log(var) and attempts to adapt strength per-frame.
Generates significantly better SSIM; on by default with --tune ssim.
Whether it generates visually better quality is still up for debate.
Available as --aq-mode 2.

It's been quiet around AutoVAQ lately ;)

I wonder, is there still any research/development going on into this direction or is the committed AutoVAQ kind of "final" ???

The x264 developers seem to have some objections against current AutoVAQ, as they didn't make the "new" AQ method the new default.

So is there any chance that AutoVAQ will be improved further and finally become the default or will it remain the way it is?

[MasterNobody]

My researches are currently stopped after testing --aq-mode 3 patch (http://stashbox.org/594973/AQ.diff) and AQ2mod patch (http://stashbox.org/643517/x264_AQ2mod.diff) (basically the same idea but different options for tuning). Their purpose are improving fades (a specially when using MBTree) by creating bias for dark/flat frames (similar to what --aq-mode 1 have) in addition to standard --aq-mode 2 QP distribution. After that I am out of ideas for AQ. As for "why --aq-mode 2 is not default" bother Dark Shikari or pengvado.

P.S. Next time I would look at AQ probably after committing Weight-P (I hope we will see this sometime. I prefer it would be earlier then later).

[Dark Shikari]

The primary problem I have with all these approaches is that the lambda used for a given macroblock depends on the other blocks in the frame. A lot of x264 (particularly MB-tree) makes the assumption that a two blocks with identical complexity will have identical quant offsets regardless of where they are. It also seems conceptually wrong to me.

[MasterNobody]

New try to create AQ mode which supposed to improve visual quality. This time I announce "--aq-mode 4", which is the result of hybridization of "--aq-mode 2" and "--aq-mode 3" (here "--aq-mode 3" refers to this patch: x264_experimental_AQ.diff (http://stashbox.org/714384/x264_experimental_AQ.diff)). By adding parts of "--aq-mode 2" new AQ this time is not so aggressive to sharp edges as "--aq-mode 3" (which due this sometimes have torn edges).

x264 CLI: x264_AQ_experiments_v4.zip (http://stashbox.org/751923/x264_AQ_experiments_v4.zip)
Patch: x264_AQ_experiments_v4.diff (http://stashbox.org/751926/x264_AQ_experiments_v4.diff)

For simplification of testing different AQs this build and patch contains all my previous version of AQ (so possible not only "--aq-mode 4" but also "--aq-mode 2 --aq-strength 1.0:1.0" and "--aq-mode 3").

P.S. I know that D_S want to keep as less different AQ modes as possible (and other options also) but here all this modes included only for testing.

[Audionut]

Patch fails miserably on the latest revision.

[MasterNobody]

Patch fails miserably on the latest revision.

You certainly are doing something wrong. It can't fail on the unpatched latest revision (r1376 3feaec2) because diff was maid using it and especially for it.

[Audionut]

Indeed. Using google browser with stashbox added crap to the diff file.

[burfadel]

In terms of --aq-mode 4, lets just say it would have been nice to be the original mode 2! It seems to work much better preserving detail that the normal mode 2. Faces in a distance were a a bit blurred in aq-more 2 whereas in aq-mode 4 they were fine :). It just looks better and closer to the original, without being harsh to the filesize, and thats just what you want!

Since aq-mode 2 is listed as experimental, if this aq-mode 4 is finalised tuning wise, I don't see any reason why it can't replace the current -aq-mode 2! Actually with one encode I was much impressed of it over aq-mode 1 and 2 that I'd suggest it be a default when finalised! Problem with objective testing though is it might not always be the case that the results are bette, but just thought I'd share my experience thus far.

[juGGaKNot]

Patched builds ?

[MuLTiTaSK]

komisar build: x264.1376kMod.core2.x86_64.aq4 (http://komisar.gin.by/test/x264.1376kMod.core2.x86_64.aq4.exe)

[burfadel]

komisar build: x264.1376kMod.core2.x86_64.aq4 (http://komisar.gin.by/test/x264.1376kMod.core2.x86_64.aq4.exe)

Fails to encode. The Masternobody's works fine!

That using --aq-mode 4

It says x264 fails to load avisynth, I'm using the 2.6.0 test version.

[MuLTiTaSK]

@burfadel

i have AviSynth 2.5.8.5 installed and the build runs fine
Source: J:\AutoVAQ\test.avs
Output: J:\AutoVAQ\test.mkv
Preset: Medium
Tuning: None
Profile: High
Params: --aq-mode 4

Analyzing source file:
J:\AutoVAQ\test.avs: 1280x528, 24000/1001 fps, 7188 frames

Resolution: 1280 x 528
Frame No. : 7188
Frame Rate: 24000/1001

x264 0.80.1376kMod 3feaec2
built by Komisar on Dec 29 2009, gcc: 4.4.2 (x86_64.core2.Komisar)
Revision: 1376

Commandline for x264:
"I:\Simple x264 Launcher\pipebuf.exe" "I:\Simple x264 Launcher\avs2yuv.exe" J:\AutoVAQ\test.avs - : "I:\Simple x264 Launcher\x264_x64.exe" --crf 18.0 --aq-mode 4 --output J:\AutoVAQ\test.mkv --frames 7188 --stdin y4m - : 4

yuv4mpeg: 1280x528@24000/1001fps, 0:0
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.0:0.0 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 sliced_threads=0 nr=0 decimate=1 mbaff=0 constrained_intra=0 bframes=3 b_pyramid=0 b_adapt=1 b_bias=0 direct=1 wpredb=1 wpredp=2 keyint=250 keyint_min=25 scenecut=40 rc_lookahead=40 rc=crf mbtree=1 crf=18.0000 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=4:1.00
x264 [info]: profile High, level 3.1
J:\AutoVAQ\test.avs: 1280x528, 24000/1001 fps, 7188 frames

x264 [info]: frame I:79 Avg QP:15.84 size: 77946
x264 [info]: frame P:3568 Avg QP:19.50 size: 24911
x264 [info]: frame B:3541 Avg QP:20.86 size: 8200
x264 [info]: consecutive B-frames: 21.0% 33.6% 12.4% 33.0%
x264 [info]: mb I I16..4: 7.3% 74.8% 17.9%
x264 [info]: mb P I16..4: 1.2% 10.6% 2.0% P16..4: 41.8% 21.4% 12.0% 0.0% 0.0% skip:11.1%
x264 [info]: mb B I16..4: 0.1% 1.2% 0.3% B16..8: 49.1% 1.2% 2.0% direct: 4.7% skip:41.6% L0:40.7% L1:51.4% BI: 7.9%
x264 [info]: 8x8 transform intra:76.9% inter:69.7%
x264 [info]: coded y,uvDC,uvAC intra: 84.3% 83.4% 54.0% inter: 32.5% 32.9% 2.9%
x264 [info]: i16 v,h,dc,p: 24% 15% 6% 55%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 14% 19% 7% 8% 9% 8% 9% 9%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 16% 13% 7% 10% 10% 9% 8% 7%
x264 [info]: Weighted P-Frames: Y:15.7%
x264 [info]: ref P L0: 54.0% 17.1% 17.5% 9.4% 1.9%
x264 [info]: ref B L0: 82.5% 17.5%
x264 [info]: kb/s:3310.96
encoded 7188 frames, 29.22 fps, 3310.96 kb/s

[burfadel]

hmmm, well it doesn't work here, even with no command line options except for the input and output files!. Quite literally, just changing the x264 executable, masternobody's works and komisars doesn't :S

Komisar's build results in:
avs [error]: failed to load avisynth
x264 [error]: could not open input file (filename)

There must be a difference somewhere between the two builds :)

I do appreciate the different builds that Komisar etc make, I just thought I'd point out the problem which looks like its case specific! I reinstalled Avisynth v2.5.8 instead of using the v2.6.0 test version, but the same thing still occurred.

[burfadel]

Ok, just tested the recently posted rack04's AQ v4 patched x264, as found here:

http://forum.doom9.org/showpost.php?p=1362661&postcount=2878

And that works perfectly fine too! its only the Komisar build that doesn't...

[komisar]

MuLTiTaSK, burfadel my build compiled with "-msse4 -mcx16 -msahf"... sorry... work fine on Core-i7. And my build IS x86_64.

[burfadel]

AHHH! OOPS :D

I completely missed the correct labelling you had, I just saw x86, not x86_64 :)

I had renamed it for ease of use and so I could use it with staxrip.

I also realise the correct naming is x86-32 and x86-64, but I think we've been conditioned such that x86 is 32 bit and x64 is 64 bit (AMD64 not IA64).

I apologise for my error :)

[MuLTiTaSK]

@komisar

that explains why it works fine for me;):thanks:

[moviefan]

Actually, how can I download the patch from stashbox.org? I don't see any download link...

[sneaker_ger]

Click on the filename. I actually had to reconfigure my browser to make it work. .If you are on Windows try Internet Explorer. Site developers these days....

[moviefan]

Ha, thanks a lot! This is really weird. I surf with Firefox and Chromium and none of them worked. But I didn't try IE... Who develops a website that only works properly with IE??? :devil:

[komisar]

x264_AQ_experiments_v4.diff (http://komisar.gin.by/test/x264_AQ_experiments_v4.diff)

[moviefan]

Would you assume that the new --aq-mode 4 helps with CG content in rather flat areas (or very slight gradient areas)?

[burfadel]

I think the patch works very well, I've encoded many more files all with great success over the standard aq-mode 2. I haven't encoded any animation/CG yet, and probably won't for a couple of days. I really hope that this gets committed (probably replacing aq-mode 2) and be the default. At least you know by that comment that it is an improvement! I'm guessing there are some refinements due on it yet?

[burfadel]

Looking forward to trying this patch with rev 1391 :)

[julius666]

rev 1391, built by me :):
http://uploading.com/files/e7f77d8e/x264.aq4.exe/

Toolchain:
gcc-core-4.4.2 (built by Komisar)

Patches used:
x264_AQ_experiments_v4.diff

Optimalisation:
--extra-cflags="-march=core2"

[moviefan]

Does --aq-strength take one or two parameters? The x264 help only describes one, but if I recall correctly, there was an AQ mod that had a second one. If so, what does the second parameter do?

[julius666]

Does --aq-strength take one or two parameters? The x264 help only describes one, but if I recall correctly, there was an AQ mod that had a second one. If so, what does the second parameter do?

http://forum.doom9.org/showpost.php?p=1357555&postcount=101 :search:

[burfadel]

I highly recommend using aq-mode 4 rather than changing the strength parameters of aq-mode 2 :) I don't mean to be of annoyance going on about how impressed I am with aq-mode 4, but I guess at least it shows I have found it very good!

[moviefan]

http://forum.doom9.org/showpost.php?p=1357555&postcount=101 :search:

Yes, I read it, but my question was rather, whether there is a second parameter for --aq-strength in combination with --aq-mode 4 too and if so, what is its meaning?

[julius666]

Yes, I read it, but my question was rather, whether there is a second parameter for --aq-strength in combination with --aq-mode 4 too and if so, what is its meaning?

Watch the quotation marks! :rolleyes:

so possible not only "--aq-mode 4" but also "--aq-mode 2 --aq-strength 1.0:1.0" and "--aq-mode 3"

"--aq-mode 2 --aq-strength 1.0:1.0" -> it works only with aq-mode 2. Aq-mode 3 and 4 has no second parameter.

And it's as burfadel said, use aq-mode 4, it's better IMHO.

[burfadel]

Since this tool is now available (thanks), such that x64 builds can be used in place of 32 bit x264, even from Staxrip etc using 32 avisynth, I was wondering whether anyone has a x64 version of AQ4 using rev1391 x264.
http://forum.doom9.org/showthread.php?p=1363912#post1363912

Thanks!
btw, just make sure you name it x264_64.exe and not x264_x64.exe ;)

Edit: Just found it actually runs slower on my Core 2 Q9400 by several fps. It could be that different processor types (Core i7, Phenom II's etc) have a different experience, or it could be simply because I'm using multithread avisynth?... I am using Julius666 build of x264 (32 bit, AQ4) for the 32 bit build and the one from Komisar's test folder for the x64.

[Chengbin]

Hi.

I tried out this patch and the first thing I notice is that the speed is a lot slower. Is AQ mode 4 that slow? I get about 8fps before on AQ mode 2, x64 x264. Now I only get 6.5fps.

[MasterNobody]

Chengbin
First of all with which build? If with mine than that is most probably due the fact that I don't fprofile it. And yes AQ4 can be slower then AQ2 (because it is not optimized and use slow powf function and not lookup table as AQ2 for log2f) but I still expect not so big difference (so it is probably due the use of not fprofiled build or not specifically optimized for your cpu as your previous build). Also did they both have the same bitrate (because speed of encode can depend from the used bitrate).
moviefan
Yes, --aq-strength can receive two params. And yes, this doesn't described in help. Second param have effect only for --aq-mode 2 (all others simply ignore it).

[Chengbin]

I used julius666's build. I'm encoding both at CRF 20. I have a Q6600.

[burfadel]

If Aq4 hasn't got those features now, I'm guessing they will be added later once the proof of concept is proven valid? It certainly seems valid at the moment!

[julius666]

Hi.

I tried out this patch and the first thing I notice is that the speed is a lot slower. Is AQ mode 4 that slow? I get about 8fps before on AQ mode 2, x64 x264. Now I only get 6.5fps.

It's most probably because the faster x264 build you used was fprofiled, and the one built by me wasn't.
If you don't believ me, try out aq-mode 2 with my build too.

[nurbs]

Maybe a stupid question, but is it normal that encodes with aq-mode 1 and mode 4 come out about the same size (low single digit percentage difference). It's the first time I tried the patch and so I'm not sure if that's expected or if I got a miscompiled build from the "Current Patches" thread.

[MasterNobody]

nurbs
I don't guarantee anything, but AQ3 and AQ4 was tuned to have close bitrate (can't say how much in percents but close) to AQ1 at the same CRF in contrast to AQ2 which usually resulted in a lot different bitrate.

[nurbs]

Thanks for the answer. I expected something radically different because of aq mode 2.

[aBra]

Just tested x264.aq4.exe aq-mode 4 and must say, that I'm impressed.

http://comparescreenshots.slicx.com/comparison/31518

x264.aq4.exe --preset veryslow --crf 28 --bframes 5 --rc-lookahead 100 --ref 4 --psy-rd 0.5:0.0 --aq-mode 4 --aq-strength 0.8 --b-pyramid normal --no-fast-pskip

Compared with the x264.nl build:
Filesize +1,5%, Speed -4,3%

[Terranigma]

julius666 build crashes for me, even if i use preset ultrafast alone without any additional parameters. MasterNobody's build (#101 (http://forum.doom9.org/showpost.php?p=1357555&postcount=101)) works just fine for me. MasterNobody, do you mind posting an updated build of revision 1391?

[sneaker_ger]

Try this (http://forum.doom9.org/showpost.php?p=1364251&postcount=2888) build.

[Terranigma]

Try this (http://forum.doom9.org/showpost.php?p=1364251&postcount=2888) build.

Nope, still doesn't work. There has to be something different on how masternobody's build was compiled vs. vfr_maniac's and julius666.

I've noticed this a while back as well with techouse builds that i stopped using them over like 100 revisions ago. I'm on an ancient machine atm (P4/XP), so all this newer stuff used to compiled is probably throwing errors

[burfadel]

x264 Rev 1400 with aq patch 4 anyone? :) thanks!

[burfadel]

32 bit thanks :) for me, the 64 bit workaround is actually slower!

[jpsdr]

Hello.
Yes, the aq patch 4 + nal_hrd patch would be a very great thing, if someone can do it.
Thanks !

[MuLTiTaSK]

patches.used
x264_intel_support.diff (http://komisar.gin.by/x.patch/last.used/x264_intel_support.diff)
x264_NAL_HRD.10b.diff (http://komisar.gin.by/x.patch/last.used/x264_NAL_HRD.10b.diff)
x264_thread_pool_v2.5.diff (http://komisar.gin.by/x.patch/last.used/x264_thread_pool_v2.5.diff)
x264_fix_float_point_exception.r1360.diff (http://komisar.gin.by/x.patch/last.used/x264_fix_float_point_exception.r1360.diff)
x264_log_file_k08.r1400.diff (http://komisar.gin.by/x.patch/last.used/x264_log_file_k08.r1400.diff)
k.86.x264_restore_console_title.1391.diff (http://komisar.gin.by/x.patch/last.used/k.86.x264_restore_console_title.1391.diff)
k.76.k_Cosmetic.03.r1360.diff (http://komisar.gin.by/x.patch/last.used/k.76.k_Cosmetic.03.r1360.diff)
x264_AQ_experiments_v4.diff (http://komisar.gin.by/x.patch/last.used/x264_AQ_experiments_v4.diff)

x264.1400kMod.generic.x86 (http://komisar.gin.by/old/1400/x264.1400kMod.generic.x86.exe)
x264.1400kMod.generic.x86_64 (http://komisar.gin.by/old/1400/x264.1400kMod.generic.x86_64.exe)
x264.1400kMod.core2.x86 (http://komisar.gin.by/old/1400/x264.1400kMod.core2.x86.exe)
x264.1400kMod.core2.x86_64 (http://komisar.gin.by/old/1400/x264.1400kMod.core2.x86_64.exe)

[menlvd]

maybe it's bug in r1400. tested w/wo aq4 and clear build (no patches) same output.
http://s001.radikal.ru/i196/1001/20/d3f54753a3f9t.jpg (http://radikal.ru/F/s001.radikal.ru/i196/1001/20/d3f54753a3f9.png.html)
Using old r1376 w/wo aq4 - normal encode out

[komisar]

menlvd, whose build you use?

[menlvd]

menlvd, whose build you use?

1 test x264.1400kMod.core2.x86_64
2 test clear build x264.1400.core2.x86_64.exe
using aq4 for kMod and aq1 for clear - same out :devil:

[komisar]

menlvd, my build recompiled now... bugs found...

[menlvd]

menlvd, my build recompiled now... bugs found...
thanks! waiting update on http://komisar.gin.by/

Upd: new one work well...

[Terranigma]

Upd: new one work well...
Yeah, it's working for me as well, so so far it seems that only komisar and masternobody builds perfect compiles (excluding the svn builds by bob0r).

[MuLTiTaSK]

32 bit thanks :) for me, the 64 bit workaround is actually slower!

try with vfw4x264.exe instead it's faster for me and others it also doesnt corrupt MPEG2 streams loaded with MPEG2Source("") like avs4x264.exe does;)

[Terranigma]

Sorry for being annoying, but could someone patch the newest build (r1416*) with the new AQ Experimental? I don't care being x86 or x64 :) (http://forum.doom9.org/showthread.php?p=1369567#post1369567)

I'd like to see this as well. :thanks:

[julius666]

Here you are :)

http://uploading.com/files/c834mfd1/x264.exe/

Toolchain:
gcc-core-4.4.3 (built by Komisar)

Patches used:
x264_AQ_experiments_v4.diff

Optimization:
--extra-cflags="-march=core2"
fprofiled VIDS="C:/git/station2_720p.yuv"

Platform: x86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: no
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

[Terranigma]

Optimization:
--extra-cflags="-march=core2"
fprofiled VIDS="C:/git/station2_720p.yuv"

Platform: x86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: no
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

Thanks, but your builds crashes for me. Perhaps you can compile another build without the optimizations i have highlighted?

[julius666]

Thanks, but your builds crashes for me. Perhaps you can compile another build without the optimizations i have highlighted?

Of course, a build withouth -march=core2 (and not fprofiled):
http://uploading.com/files/7d1c63a3/x264.exe/

[jpsdr]

Hello.
Thanks already for all the people who are making compiles.
Can someone nevertheless do the avq patch + nal_hrd beta 1.22 with 14xx (last one) ?

Thanks.

[Terranigma]

Of course, a build withouth -march=core2 (and not fprofiled):
http://uploading.com/files/7d1c63a3/x264.exe/

OK, it works perfect now, so it could be just one of the patches that causes it to crash on my system. If you noticed by now, there has been some additional changes (now rev. 1419), so if you wouldn't mind compiling with your normal configuration (with march=core2/fprofiled) + 3 Additional builds would be great. :o
1. One without the march=core2 patch (and not fprofiled).
2. One without the march=core2 patch (and fprofiled).
3 And one with the march=core2 patch (and not profiled).
?

Trying to figure out which one of the 2 patches (possibly both) crashes my system, which i'm suspecting is more than likely the "march=core2" patch.

:thanks: In advance.

[masterkivat]

Hello guys.
I learned last night how to compile using this guide (http://doom10.org/index.php?topic=26.msg1230#msg1230) and with some help from the user @julius666.
I'd like to know if someone could run some tests to see if my version was compiled without any kind of error.
Here we go:

http://www.megaupload.com/?d=QLDYZSQ2

Toolchain:
gcc-core-4.4.3 (built by Komisar)

Patches used:
x264_AQ_experiments_v4.diff

Optimization:
--extra-cflags="-march=core2" && fprofiled VIDS="C:/home/ICE_352x288_30_orig_02.yuv"

Platform: x86
System: MINGW
asm: yes
avs input: yes
lavf input: no
ffms input: no
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

Just a observation: since I'm still learning to do it with care, I didn't add lavf neither ffms suport. If someone could teach me, I'll appreciate.

[julius666]

OK, it works perfect now, so it could be just one of the patches that causes it to crash on my system. If you noticed by now, there has been some additional changes (now rev. 1419), so if you wouldn't mind compiling with your normal configuration (with march=core2/fprofiled) + 3 Additional builds would be great. :o
1. One without the march=core2 patch (and not fprofiled).
2. One without the march=core2 patch (and fprofiled).
3 And one with the march=core2 patch (and not profiled).
?

Trying to figure out which one of the 2 patches (possibly both) crashes my system, which i'm suspecting is more than likely the "march=core2" patch.

:thanks: In advance.

of course :)
as far as I know fprofiled mode shouldn't cause any trouble, but I could be wrong.
Anyway, I made two fprofiled builds, if neither of them works then I'll make the not fprofiled ones.

Oh, and a question: what CPU do you have?

rev1419:

Toolchain:
gcc-core-4.4.3 (built by Komisar)

Patches used:
x264_AQ_experiments_v4.diff

Platform: x86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: no
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

1. Optimization:
--extra-cflags="-march=core2"
fprofiled VIDS="C:/git/station2_720p.yuv"

http://uploading.com/files/9533a5fd/x264.aq4.core2.fporf.exe/

2. Optimization:
fprofiled VIDS="C:/git/station2_720p.yuv"

http://uploading.com/files/854ma677/x264.aq4.gen.fprof.exe/

(for some reason the one without core2 optimization was compiled without lavf input, because of that it is just 1.1 MB)

[Terranigma]

a question: what CPU do you have?


Link (http://forum.doom9.org/showthread.php?p=1364469#post1364469)


rev1419:

Toolchain:
gcc-core-4.4.3 (built by Komisar)

Patches used:
x264_AQ_experiments_v4.diff

Platform: x86
System: MINGW
asm: yes
avs input: yes
lavf input: yes
ffms input: no
mp4 output: yes
pthread: yes
debug: no
gprof: no
PIC: no
shared: no
visualize: no

1. Optimization:
--extra-cflags="-march=core2"
fprofiled VIDS="C:/git/station2_720p.yuv"

http://uploading.com/files/9533a5fd/x264.aq4.core2.fporf.exe/

2. Optimization:
fprofiled VIDS="C:/git/station2_720p.yuv"

http://uploading.com/files/854ma677/x264.aq4.gen.fprof.exe/

(for some reason the one without core2 optimization was compiled without lavf input, because of that it is just 1.1 MB)


Thanks, the first one crashed, but the second one works perfectly. The first one is obviously crashing because i'm not on a dual-core system. :)

[kemuri-_9]

Thanks, the first one crashed, but the second one works perfectly. The first one is obviously crashing because i'm not on a dual-core system. :)

and here's another reason people need to drop the -march=core2 compilation placebo.

[julius666]

and here's another reason people need to drop the -march=core2 compilation placebo.

It gives me 5-6% speedup on my core2.

[aBra]

It gives me 5-6% speedup on my core2.
No speedup on my core2 at all.

[nurbs]

Is the new --aq-mode 2 the same as --aq-mode 4 in the patched builds?

[komisar]

nurbs, yes

[nurbs]

Cool. No waiting for patched builds for me anymore. :thanks: