AutoVAQ patch testing

[MasterNobody]

New try to create AQ mode which supposed to improve visual quality. This time I announce "--aq-mode 4", which is the result of hybridization of "--aq-mode 2" and "--aq-mode 3" (here "--aq-mode 3" refers to this patch: x264_experimental_AQ.diff (http://stashbox.org/714384/x264_experimental_AQ.diff)). By adding parts of "--aq-mode 2" new AQ this time is not so aggressive to sharp edges as "--aq-mode 3" (which due this sometimes have torn edges).

x264 CLI: x264_AQ_experiments_v4.zip (http://stashbox.org/751923/x264_AQ_experiments_v4.zip)
Patch: x264_AQ_experiments_v4.diff (http://stashbox.org/751926/x264_AQ_experiments_v4.diff)

For simplification of testing different AQs this build and patch contains all my previous version of AQ (so possible not only "--aq-mode 4" but also "--aq-mode 2 --aq-strength 1.0:1.0" and "--aq-mode 3").

P.S. I know that D_S want to keep as less different AQ modes as possible (and other options also) but here all this modes included only for testing.

[Audionut]

Patch fails miserably on the latest revision.

[MasterNobody]

Patch fails miserably on the latest revision.

You certainly are doing something wrong. It can't fail on the unpatched latest revision (r1376 3feaec2) because diff was maid using it and especially for it.

[Audionut]

Indeed. Using google browser with stashbox added crap to the diff file.

[burfadel]

In terms of --aq-mode 4, lets just say it would have been nice to be the original mode 2! It seems to work much better preserving detail that the normal mode 2. Faces in a distance were a a bit blurred in aq-more 2 whereas in aq-mode 4 they were fine :). It just looks better and closer to the original, without being harsh to the filesize, and thats just what you want!

Since aq-mode 2 is listed as experimental, if this aq-mode 4 is finalised tuning wise, I don't see any reason why it can't replace the current -aq-mode 2! Actually with one encode I was much impressed of it over aq-mode 1 and 2 that I'd suggest it be a default when finalised! Problem with objective testing though is it might not always be the case that the results are bette, but just thought I'd share my experience thus far.

[juGGaKNot]

Patched builds ?

[MuLTiTaSK]

komisar build: x264.1376kMod.core2.x86_64.aq4 (http://komisar.gin.by/test/x264.1376kMod.core2.x86_64.aq4.exe)

[burfadel]

komisar build: x264.1376kMod.core2.x86_64.aq4 (http://komisar.gin.by/test/x264.1376kMod.core2.x86_64.aq4.exe)

Fails to encode. The Masternobody's works fine!

That using --aq-mode 4

It says x264 fails to load avisynth, I'm using the 2.6.0 test version.

[MuLTiTaSK]

@burfadel

i have AviSynth 2.5.8.5 installed and the build runs fine
Source: J:\AutoVAQ\test.avs
Output: J:\AutoVAQ\test.mkv
Preset: Medium
Tuning: None
Profile: High
Params: --aq-mode 4

Analyzing source file:
J:\AutoVAQ\test.avs: 1280x528, 24000/1001 fps, 7188 frames

Resolution: 1280 x 528
Frame No. : 7188
Frame Rate: 24000/1001

x264 0.80.1376kMod 3feaec2
built by Komisar on Dec 29 2009, gcc: 4.4.2 (x86_64.core2.Komisar)
Revision: 1376

Commandline for x264:
"I:\Simple x264 Launcher\pipebuf.exe" "I:\Simple x264 Launcher\avs2yuv.exe" J:\AutoVAQ\test.avs - : "I:\Simple x264 Launcher\x264_x64.exe" --crf 18.0 --aq-mode 4 --output J:\AutoVAQ\test.mkv --frames 7188 --stdin y4m - : 4

yuv4mpeg: 1280x528@24000/1001fps, 0:0
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
x264 [info]: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.0:0.0 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 sliced_threads=0 nr=0 decimate=1 mbaff=0 constrained_intra=0 bframes=3 b_pyramid=0 b_adapt=1 b_bias=0 direct=1 wpredb=1 wpredp=2 keyint=250 keyint_min=25 scenecut=40 rc_lookahead=40 rc=crf mbtree=1 crf=18.0000 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=4:1.00
x264 [info]: profile High, level 3.1
J:\AutoVAQ\test.avs: 1280x528, 24000/1001 fps, 7188 frames

x264 [info]: frame I:79 Avg QP:15.84 size: 77946
x264 [info]: frame P:3568 Avg QP:19.50 size: 24911
x264 [info]: frame B:3541 Avg QP:20.86 size: 8200
x264 [info]: consecutive B-frames: 21.0% 33.6% 12.4% 33.0%
x264 [info]: mb I I16..4: 7.3% 74.8% 17.9%
x264 [info]: mb P I16..4: 1.2% 10.6% 2.0% P16..4: 41.8% 21.4% 12.0% 0.0% 0.0% skip:11.1%
x264 [info]: mb B I16..4: 0.1% 1.2% 0.3% B16..8: 49.1% 1.2% 2.0% direct: 4.7% skip:41.6% L0:40.7% L1:51.4% BI: 7.9%
x264 [info]: 8x8 transform intra:76.9% inter:69.7%
x264 [info]: coded y,uvDC,uvAC intra: 84.3% 83.4% 54.0% inter: 32.5% 32.9% 2.9%
x264 [info]: i16 v,h,dc,p: 24% 15% 6% 55%
x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 14% 19% 7% 8% 9% 8% 9% 9%
x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 16% 13% 7% 10% 10% 9% 8% 7%
x264 [info]: Weighted P-Frames: Y:15.7%
x264 [info]: ref P L0: 54.0% 17.1% 17.5% 9.4% 1.9%
x264 [info]: ref B L0: 82.5% 17.5%
x264 [info]: kb/s:3310.96
encoded 7188 frames, 29.22 fps, 3310.96 kb/s

[burfadel]

hmmm, well it doesn't work here, even with no command line options except for the input and output files!. Quite literally, just changing the x264 executable, masternobody's works and komisars doesn't :S

Komisar's build results in:
avs [error]: failed to load avisynth
x264 [error]: could not open input file (filename)

There must be a difference somewhere between the two builds :)

I do appreciate the different builds that Komisar etc make, I just thought I'd point out the problem which looks like its case specific! I reinstalled Avisynth v2.5.8 instead of using the v2.6.0 test version, but the same thing still occurred.

[burfadel]

Ok, just tested the recently posted rack04's AQ v4 patched x264, as found here:

http://forum.doom9.org/showpost.php?p=1362661&postcount=2878

And that works perfectly fine too! its only the Komisar build that doesn't...

[komisar]

MuLTiTaSK, burfadel my build compiled with "-msse4 -mcx16 -msahf"... sorry... work fine on Core-i7. And my build IS x86_64.

[burfadel]

AHHH! OOPS :D

I completely missed the correct labelling you had, I just saw x86, not x86_64 :)

I had renamed it for ease of use and so I could use it with staxrip.

I also realise the correct naming is x86-32 and x86-64, but I think we've been conditioned such that x86 is 32 bit and x64 is 64 bit (AMD64 not IA64).

I apologise for my error :)

[MuLTiTaSK]

@komisar

that explains why it works fine for me;):thanks:

[moviefan]

Actually, how can I download the patch from stashbox.org? I don't see any download link...

[sneaker_ger]

Click on the filename. I actually had to reconfigure my browser to make it work. .If you are on Windows try Internet Explorer. Site developers these days....

[moviefan]

Ha, thanks a lot! This is really weird. I surf with Firefox and Chromium and none of them worked. But I didn't try IE... Who develops a website that only works properly with IE??? :devil:

[komisar]

x264_AQ_experiments_v4.diff (http://komisar.gin.by/test/x264_AQ_experiments_v4.diff)

[moviefan]

Would you assume that the new --aq-mode 4 helps with CG content in rather flat areas (or very slight gradient areas)?

[burfadel]

I think the patch works very well, I've encoded many more files all with great success over the standard aq-mode 2. I haven't encoded any animation/CG yet, and probably won't for a couple of days. I really hope that this gets committed (probably replacing aq-mode 2) and be the default. At least you know by that comment that it is an improvement! I'm guessing there are some refinements due on it yet?

[burfadel]

Looking forward to trying this patch with rev 1391 :)

[julius666]

rev 1391, built by me :):
http://uploading.com/files/e7f77d8e/x264.aq4.exe/

Toolchain:
gcc-core-4.4.2 (built by Komisar)

Patches used:
x264_AQ_experiments_v4.diff

Optimalisation:
--extra-cflags="-march=core2"

[moviefan]

Does --aq-strength take one or two parameters? The x264 help only describes one, but if I recall correctly, there was an AQ mod that had a second one. If so, what does the second parameter do?

[julius666]

Does --aq-strength take one or two parameters? The x264 help only describes one, but if I recall correctly, there was an AQ mod that had a second one. If so, what does the second parameter do?

http://forum.doom9.org/showpost.php?p=1357555&postcount=101 :search:

[burfadel]

I highly recommend using aq-mode 4 rather than changing the strength parameters of aq-mode 2 :) I don't mean to be of annoyance going on about how impressed I am with aq-mode 4, but I guess at least it shows I have found it very good!

[moviefan]

http://forum.doom9.org/showpost.php?p=1357555&postcount=101 :search:

Yes, I read it, but my question was rather, whether there is a second parameter for --aq-strength in combination with --aq-mode 4 too and if so, what is its meaning?

[julius666]

Yes, I read it, but my question was rather, whether there is a second parameter for --aq-strength in combination with --aq-mode 4 too and if so, what is its meaning?

Watch the quotation marks! :rolleyes:

so possible not only "--aq-mode 4" but also "--aq-mode 2 --aq-strength 1.0:1.0" and "--aq-mode 3"

"--aq-mode 2 --aq-strength 1.0:1.0" -> it works only with aq-mode 2. Aq-mode 3 and 4 has no second parameter.

And it's as burfadel said, use aq-mode 4, it's better IMHO.

[burfadel]

Since this tool is now available (thanks), such that x64 builds can be used in place of 32 bit x264, even from Staxrip etc using 32 avisynth, I was wondering whether anyone has a x64 version of AQ4 using rev1391 x264.
http://forum.doom9.org/showthread.php?p=1363912#post1363912

Thanks!
btw, just make sure you name it x264_64.exe and not x264_x64.exe ;)

Edit: Just found it actually runs slower on my Core 2 Q9400 by several fps. It could be that different processor types (Core i7, Phenom II's etc) have a different experience, or it could be simply because I'm using multithread avisynth?... I am using Julius666 build of x264 (32 bit, AQ4) for the 32 bit build and the one from Komisar's test folder for the x64.

[Chengbin]

Hi.

I tried out this patch and the first thing I notice is that the speed is a lot slower. Is AQ mode 4 that slow? I get about 8fps before on AQ mode 2, x64 x264. Now I only get 6.5fps.

[MasterNobody]

Chengbin
First of all with which build? If with mine than that is most probably due the fact that I don't fprofile it. And yes AQ4 can be slower then AQ2 (because it is not optimized and use slow powf function and not lookup table as AQ2 for log2f) but I still expect not so big difference (so it is probably due the use of not fprofiled build or not specifically optimized for your cpu as your previous build). Also did they both have the same bitrate (because speed of encode can depend from the used bitrate).
moviefan
Yes, --aq-strength can receive two params. And yes, this doesn't described in help. Second param have effect only for --aq-mode 2 (all others simply ignore it).

[Chengbin]

I used julius666's build. I'm encoding both at CRF 20. I have a Q6600.

[burfadel]

If Aq4 hasn't got those features now, I'm guessing they will be added later once the proof of concept is proven valid? It certainly seems valid at the moment!

[julius666]

Hi.

I tried out this patch and the first thing I notice is that the speed is a lot slower. Is AQ mode 4 that slow? I get about 8fps before on AQ mode 2, x64 x264. Now I only get 6.5fps.

It's most probably because the faster x264 build you used was fprofiled, and the one built by me wasn't.
If you don't believ me, try out aq-mode 2 with my build too.

[nurbs]

Maybe a stupid question, but is it normal that encodes with aq-mode 1 and mode 4 come out about the same size (low single digit percentage difference). It's the first time I tried the patch and so I'm not sure if that's expected or if I got a miscompiled build from the "Current Patches" thread.

[MasterNobody]

nurbs
I don't guarantee anything, but AQ3 and AQ4 was tuned to have close bitrate (can't say how much in percents but close) to AQ1 at the same CRF in contrast to AQ2 which usually resulted in a lot different bitrate.

[nurbs]

Thanks for the answer. I expected something radically different because of aq mode 2.

[aBra]

Just tested x264.aq4.exe aq-mode 4 and must say, that I'm impressed.

http://comparescreenshots.slicx.com/comparison/31518

x264.aq4.exe --preset veryslow --crf 28 --bframes 5 --rc-lookahead 100 --ref 4 --psy-rd 0.5:0.0 --aq-mode 4 --aq-strength 0.8 --b-pyramid normal --no-fast-pskip

Compared with the x264.nl build:
Filesize +1,5%, Speed -4,3%

[Terranigma]

julius666 build crashes for me, even if i use preset ultrafast alone without any additional parameters. MasterNobody's build (#101 (http://forum.doom9.org/showpost.php?p=1357555&postcount=101)) works just fine for me. MasterNobody, do you mind posting an updated build of revision 1391?

[sneaker_ger]

Try this (http://forum.doom9.org/showpost.php?p=1364251&postcount=2888) build.

[Terranigma]

Try this (http://forum.doom9.org/showpost.php?p=1364251&postcount=2888) build.

Nope, still doesn't work. There has to be something different on how masternobody's build was compiled vs. vfr_maniac's and julius666.

I've noticed this a while back as well with techouse builds that i stopped using them over like 100 revisions ago. I'm on an ancient machine atm (P4/XP), so all this newer stuff used to compiled is probably throwing errors

[burfadel]

x264 Rev 1400 with aq patch 4 anyone? :) thanks!

[burfadel]

32 bit thanks :) for me, the 64 bit workaround is actually slower!

[jpsdr]

Hello.
Yes, the aq patch 4 + nal_hrd patch would be a very great thing, if someone can do it.
Thanks !

[MuLTiTaSK]

patches.used
x264_intel_support.diff (http://komisar.gin.by/x.patch/last.used/x264_intel_support.diff)
x264_NAL_HRD.10b.diff (http://komisar.gin.by/x.patch/last.used/x264_NAL_HRD.10b.diff)
x264_thread_pool_v2.5.diff (http://komisar.gin.by/x.patch/last.used/x264_thread_pool_v2.5.diff)
x264_fix_float_point_exception.r1360.diff (http://komisar.gin.by/x.patch/last.used/x264_fix_float_point_exception.r1360.diff)
x264_log_file_k08.r1400.diff (http://komisar.gin.by/x.patch/last.used/x264_log_file_k08.r1400.diff)
k.86.x264_restore_console_title.1391.diff (http://komisar.gin.by/x.patch/last.used/k.86.x264_restore_console_title.1391.diff)
k.76.k_Cosmetic.03.r1360.diff (http://komisar.gin.by/x.patch/last.used/k.76.k_Cosmetic.03.r1360.diff)
x264_AQ_experiments_v4.diff (http://komisar.gin.by/x.patch/last.used/x264_AQ_experiments_v4.diff)

x264.1400kMod.generic.x86 (http://komisar.gin.by/old/1400/x264.1400kMod.generic.x86.exe)
x264.1400kMod.generic.x86_64 (http://komisar.gin.by/old/1400/x264.1400kMod.generic.x86_64.exe)
x264.1400kMod.core2.x86 (http://komisar.gin.by/old/1400/x264.1400kMod.core2.x86.exe)
x264.1400kMod.core2.x86_64 (http://komisar.gin.by/old/1400/x264.1400kMod.core2.x86_64.exe)

[menlvd]

maybe it's bug in r1400. tested w/wo aq4 and clear build (no patches) same output.
http://s001.radikal.ru/i196/1001/20/d3f54753a3f9t.jpg (http://radikal.ru/F/s001.radikal.ru/i196/1001/20/d3f54753a3f9.png.html)
Using old r1376 w/wo aq4 - normal encode out

[komisar]

menlvd, whose build you use?

[menlvd]

menlvd, whose build you use?

1 test x264.1400kMod.core2.x86_64
2 test clear build x264.1400.core2.x86_64.exe
using aq4 for kMod and aq1 for clear - same out :devil:

[komisar]

menlvd, my build recompiled now... bugs found...

[menlvd]

menlvd, my build recompiled now... bugs found...
thanks! waiting update on http://komisar.gin.by/

Upd: new one work well...

[Terranigma]

Upd: new one work well...
Yeah, it's working for me as well, so so far it seems that only komisar and masternobody builds perfect compiles (excluding the svn builds by bob0r).