PDA

View Full Version : Variance AQ Megathread (AQ v0.48 update--defaults changed)


Pages : 1 [2] 3 4

DeathTheSheep
18th January 2008, 01:18
Da...kaz...? :eek:

Dark Shikari
18th January 2008, 01:22
Da...kaz...? :eek:x264 is not only used by hobbyists, you know... :)

bob0r
18th January 2008, 01:27
I find these results stunning:

No AQ:
x264.exe --pass 2 --bitrate 3096 --threads auto --deblock 0:0 --bframes 3 --b-pyramid --bime --weightb --b-rdo --me umh --ref 5 --mixed-refs --subme 7 --trellis 1 --analyse all --8x8dct --no-fast-pskip --progress --fps=25 --output x264aq.mkv 720p50_parkrun_ter.yuv 1280x720
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 Cache64
x264 [info]: slice I:3 Avg QP:25.33 size:131303 PSNR Mean Y:41.13 U:42.91 V:45.56 Avg:41.78 Global:38.10
x264 [info]: slice P:233 Avg QP:34.52 size: 29549 PSNR Mean Y:29.02 U:36.95 V:39.19 Avg:30.50 Global:30.22
x264 [info]: slice B:268 Avg QP:35.70 size: 2964 PSNR Mean Y:29.10 U:37.13 V:39.35 Avg:30.56 Global:29.96
x264 [info]: mb I I16..4: 27.9% 66.2% 6.0%
x264 [info]: mb P I16..4: 0.3% 0.8% 0.3% P16..4: 46.4% 12.1% 9.8% 1.0% 0.7% skip:28.7%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 17.4% 0.1% 0.4% direct: 3.1% skip:79.0%
x264 [info]: 8x8 transform intra:60.9% inter:56.9%
x264 [info]: ref P 80.2% 9.6% 6.0% 2.2% 2.0%
x264 [info]: ref B 87.2% 5.4% 3.9% 2.1% 1.3%
x264 [info]: SSIM Mean Y:0.8698542
x264 [info]: PSNR Mean Y:29.138 U:37.082 V:39.312 Avg:30.597 Global:30.099 kb/s:3203.68
encoded 504 frames, 9.37 fps, 3202.74 kb/s

AQ:
x264.exe --pass 2 --bitrate 3096 --threads auto --aq-strength 1.0 --deblock 0:0 --bframes 3 --b-pyramid --bime --weightb --b-rdo --me umh --ref 5 --mixed-refs --subme 7 --trellis 1 --analyse all --8x8dct --no-fast-pskip --progress --fps=25 --output x264aq.mkv 720p50_parkrun_ter.yuv 1280x720
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 SSSE3 Cache64
x264 [info]: slice I:3 Avg QP:24.67 size:138018 PSNR Mean Y:40.74 U:42.93 V:45.56 Avg:41.47 Global:38.03
x264 [info]: slice P:233 Avg QP:33.93 size: 30242 PSNR Mean Y:28.66 U:37.06 V:39.26 Avg:30.16 Global:29.88
x264 [info]: slice B:268 Avg QP:35.23 size: 2405 PSNR Mean Y:28.75 U:37.24 V:39.42 Avg:30.23 Global:29.63
x264 [info]: mb I I16..4: 26.9% 66.5% 6.5%
x264 [info]: mb P I16..4: 0.2% 0.9% 0.3% P16..4: 50.3% 15.7% 11.6% 0.7% 0.3% skip:20.1%
x264 [info]: mb B I16..4: 0.0% 0.0% 0.0% B16..8: 20.5% 0.1% 0.4% direct: 1.4% skip:77.6%
x264 [info]: 8x8 transform intra:66.3% inter:56.9%
x264 [info]: ref P 81.7% 8.8% 5.5% 2.1% 1.9%
x264 [info]: ref B 84.3% 7.3% 4.6% 2.3% 1.5%
x264 [info]: SSIM Mean Y:0.9032475
x264 [info]: PSNR Mean Y:28.781 U:37.189 V:39.386 Avg:30.266 Global:29.768 kb/s:3216.25
encoded 504 frames, 9.47 fps, 3215.30 kb/s

x264noaq3mbit.0.43.mkv (http://files.x264.nl/AQ/x264noaq3mbit.0.43.mkv) = 7.70 MB (8,078,599 bytes)
x264aq3mbit.0.43.mkv (http://files.x264.nl/AQ/x264aq3mbit.0.43.mkv) = 7.73 MB (8,110,271 bytes)

Screenshots may show local difference, in some parts the NOAQ is slightly better (the bits have to come from somewhere), but just look at these samples in motion!

Edit:
Here is the x264.exe with the patch: x264.720.dark.aq.rdrc.0.431.exe (http://files.x264.nl/AQ/x264.720.dark.aq.rdrc.0.431.exe) (pthreads/mp4 = yes, not made with make fprofiled)

CruNcher
18th January 2008, 01:53
yep that's even the same quality as i saw last time from Atemes Encoder :) it's a big step for X264.
i wonder if Mainconcepts Encoder can handle that now last time i saw it it couldn't :)


Reduced I-Frame Closed Gop Pulseing
http://mirror05.x264.nl/CruNcher/parkrun-newaq-reduced-iframe-pulseing.mkv

Still Realtime Encoded ABR 1Pass :)

Don_Genaro
18th January 2008, 04:13
I have done several encodings with Dark´s patch and comparint them to some without it. I have noticed a sustancial reduction in the size of the final stream.

Using CRF 21,22,23,24 the resulting files (Using the Patch) end with just 84% of the size of the files that don´t use the patched encoder and with no noticebly loss of quality (maybe even a gain in quality). Very good! :cool:

Sagekilla
18th January 2008, 04:36
I'll be testing the new AQ on a notoriously dark movie (Chronicles of Riddick anyone? I'll follow up on Pitch Black, which is even worse) Will post results as soon as my encoding finishes, which should be sometime around 5 PM tomorrow, for at least the AQ'd version. The non-AQ may not finish by then.

Edit: A cursory glance of the video using aq strength of 1 shows the new algo dealing with dark details very well. I'm viewing it before PC scale correction to exaggerate the blocking and I see a lot of detail in areas where normally the detail would be decimated.

Razorholt
18th January 2008, 04:45
Here is the x264.exe with the patch: x264.720.dark.aq.rdrc.0.431.exe (http://x264.nl/x264.720.dark.aq.rdrc.0.431.exe) (pthreads/mp4 = yes, not made with make fprofiled)

What's the difference between that build and the one on the OP?

Dark Shikari
18th January 2008, 04:48
What's the difference between that build and the one on the OP?Faster, statically built... minor differences though.

Razorholt
18th January 2008, 04:52
Thanks Darky for the previous answer. Also:

1. Since I'm using your GrainOptimizer, do you recommend using P4x4 with the new AQ?

2. How about intra? Have you fixed the bug or should I stick to Trellis?


Thanks,
- Dan

Dark Shikari
18th January 2008, 04:54
Thanks Darky for the previous answer. Also:

1. Since I'm using your GrainOptimizer, do you recommend using P4x4 with the new AQ?GrainOptimizer shouldn't get a specific benefit from p4x4 now, since the later versions have the blocks try to flip all at once when possible.
2. How about intra? Have you fixed the bug or should I stick to Trellis?


Thanks,
- DanThe bug was fixed a while ago--I removed the lambda-based AQ. Deadzone/trellis are both fine.

Sagekilla
18th January 2008, 04:58
Just thought about something.. I'd like to request a new feature for your AQ: sensitivity offset. It would only apply to automatic mode for obvious reasons. Would be nice to force AQ to be slightly more (or less) sensitive sometimes while using the same strength.

Edit: CoR is averaging 1400 kbps and looks great with your new AQ.

Reuf Toc
18th January 2008, 05:56
I've made a test on an animation clip mixing totaly flat parts and finely textured parts and I must say that the result is bluffing. AQ reduce considerably blocking and banding while retain a lot more details. Here are some pictures to illustrate my point :

Frame 120 :

AQ (http://reuf.toc.free.fr/AQ/120_AQ.png)
no AQ (http://reuf.toc.free.fr/AQ/120_no_AQ.png)

Frame 350 :

AQ (http://reuf.toc.free.fr/AQ/350_AQ.png)
no AQ (http://reuf.toc.free.fr/AQ/350_no_AQ.png)


x264 command line :

"C:\Program Files\megui\tools\x264\x264.exe" --pass 2 --bitrate 3000 --stats "E:\factory.stats" --level 4.1 --ref 3 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -1,-1 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --sar 1:1 --progress --no-dct-decimate --output "E:\factory AQ 3M.mkv" "E:\factory.avs" --aq-strength 1.0

Log from the clip with AQ :

x264 [info]: slice I:7 Avg QP:24.43 size: 37380 PSNR Mean Y:42.64 U:44.68 V:45.30 Avg:43.23 Global:42.14
x264 [info]: slice P:986 Avg QP:26.31 size: 15043 PSNR Mean Y:40.72 U:43.05 V:43.30 Avg:41.34 Global:40.33
x264 [info]: slice B:347 Avg QP:27.21 size: 5232 PSNR Mean Y:40.16 U:42.82 V:42.35 Avg:40.76 Global:39.68

x264 [info]: mb I I16..4: 33.3% 50.7% 16.0%
x264 [info]: mb P I16..4: 14.2% 13.8% 2.0% P16..4: 40.4% 11.1% 3.4% 0.0% 0.0% skip:15.1%
x264 [info]: mb B I16..4: 0.6% 0.9% 0.2% B16..8: 36.9% 1.1% 2.4% direct: 0.6% skip:57.2%

x264 [info]: 8x8 transform intra:46.3% inter:65.3%
x264 [info]: direct mvs spatial:82.7% temporal:17.3%
x264 [info]: ref P 88.1% 8.0% 3.9%
x264 [info]: ref B 86.4% 8.8% 4.8%

x264 [info]: SSIM Mean Y:0.9667820
x264 [info]: PSNR Mean Y:40.589 U:43.001 V:43.068 Avg:41.199 Global:40.158 kb/s:3025.50

Log from the clip without AQ :

x264 [info]: slice I:7 Avg QP:25.57 size: 41877 PSNR Mean Y:43.93 U:45.58 V:46.10 Avg:44.44 Global:43.37
x264 [info]: slice P:986 Avg QP:27.68 size: 14984 PSNR Mean Y:41.56 U:43.74 V:43.95 Avg:42.14 Global:41.04
x264 [info]: slice B:347 Avg QP:28.50 size: 5379 PSNR Mean Y:40.77 U:43.42 V:42.95 Avg:41.37 Global:40.26

x264 [info]: mb I I16..4: 53.7% 29.4% 16.9%
x264 [info]: mb P I16..4: 25.3% 6.3% 2.0% P16..4: 31.7% 11.1% 3.7% 0.0% 0.0% skip:19.8%
x264 [info]: mb B I16..4: 0.8% 0.6% 0.2% B16..8: 32.1% 1.3% 2.8% direct: 0.6% skip:61.5%

x264 [info]: 8x8 transform intra:19.2% inter:63.5%
x264 [info]: direct mvs spatial:82.7% temporal:17.3%
x264 [info]: ref P 88.8% 7.5% 3.7%
x264 [info]: ref B 86.0% 9.0% 5.0%

x264 [info]: SSIM Mean Y:0.9659700
x264 [info]: PSNR Mean Y:41.370 U:43.665 V:43.700 Avg:41.953 Global:40.830 kb/s:3029.81


Here are the link for the two clip (16,1 MB) :

With AQ (http://reuf.toc.free.fr/AQ/factory_AQ.mkv)
Without AQ (http://reuf.toc.free.fr/AQ/factory_no_AQ.mkv)

Mirror on sendspace and mediafire if my FTP is too slow (33,3MB) :

Sendspace (http://www.sendspace.com/file/29ewp5)
Mediafire (http://www.mediafire.com/?499bz1j3w23)

Dark Shikari
18th January 2008, 06:39
And finally, the last bug... interlacing is fixed!

After a whole lot of CABAC debugging, narrowing down the bug, and finally a very good bug-spot by akupenguin, we found and fixed a bug in x264's handling of the CABAC encoding of mb_qp_delta with interlaced video.

Build and patch updated.

Sharktooth
18th January 2008, 14:34
@Reuf Toc: results are quite impressive. however, even if the second pic has less blocking, some details are missing (right side of the pic. look at details behind the colored smoke) but i guess the pic looks globally better.

@Dark Shikari: it seems you made CQMs a thing of the past. good job :)

CruNcher
18th January 2008, 14:47
Im looking forward to see Sagekillas Dark Encoding results because their it showed flaws to me with my test encode, tough it's still better then without any AQ at all, but not as good as the Old AQ in such specific situations. To enhance Dark Scenes a little more with this AQ it seems you have to lower --aq-strength from the 1.0 setting down this gives then a boost in Dark Sequences but it seemes to be not really visible @ all so 1.0 seems also for such scenes the most efficient if you don't wan't todo segmented Encoding with different AQ strengths ;)

@Dark
Did you tried your New AQ on your Vendeta Sample ?

foxyshadis
18th January 2008, 15:37
Hey, you didn't use 721. ;_; Like Chainmax, I'd still like to test this with prepass SATD ESA, but concentrate on what you think will make the best improvement.

I wonder how your elephant's dream running scene would look at this point.

bob0r
18th January 2008, 15:38
source:
ftp://ftp.ldv.e-technik.tu-muenchen.de/pub/test_sequences/720p/720p50_parkrun_ter.yuv

commandline:
aq:
start /belownormal /b /w x264.exe --pass 1 --bitrate 5096/3096 --threads auto --aq-strength 1.0 --thread-input --deblock 0:0 --bframes 3 --me dia --ref 1 --subme 1 --no-dct-decimate --partitions none --progress --fps=25 --output NUL 720p50_parkrun_ter.yuv 1280x720
start /belownormal /b /w x264.exe --pass 2 --bitrate 5096/3096 --threads auto --aq-strength 1.0 --thread-input --deblock 0:0 --bframes 3 --b-pyramid --bime --weightb --b-rdo --me umh --ref 5 --mixed-refs --subme 7 --trellis 1 --analyse all --8x8dct --no-fast-pskip --progress --fps=25 --output x264aq.mkv 720p50_parkrun_ter.yuv 1280x720

no aq:
start /belownormal /b /w x264.exe --pass 1 --bitrate 5096/3096 --threads auto --thread-input --deblock 0:0 --bframes 3 --me dia --ref 1 --subme 1 --no-dct-decimate --partitions none --progress --fps=25 --output NUL 720p50_parkrun_ter.yuv 1280x720
start /belownormal /b /w x264.exe --pass 2 --bitrate 5096/3096 --threads auto --thread-input --deblock 0:0 --bframes 3 --b-pyramid --bime --weightb --b-rdo --me umh --ref 5 --mixed-refs --subme 7 --trellis 1 --analyse all --8x8dct --no-fast-pskip --progress --fps=25 --output x264aq.mkv 720p50_parkrun_ter.yuv 1280x720

Change bitrate.

3mbit:
x264noaq3mbit.0.44.mkv (http://files.x264.nl/AQ/x264noaq3mbit.0.44.mkv)
x264aq3mbit.0.44.mkv (http://files.x264.nl/AQ/x264aq3mbit.0.44.mkv)

5mbit:
x264noaq5mbit.0.44.mkv (http://files.x264.nl/AQ/x264noaq5mbit.0.44.mkv)
x264aq5mbit.0.44.mkv (http://files.x264.nl/AQ/x264aq5mbit.0.44.mkv)

.exe:
x264.721.dark.aq.rdrc.0.44.exe (http://files.x264.nl/AQ/x264.721.dark.aq.rdrc.0.44.exe) (pthreads/mp4 = yes, not made with make fprofiled)

CruNcher
18th January 2008, 15:45
Hey, you didn't use 721. ;_; Like Chainmax, I'd still like to test this with prepass SATD ESA, but concentrate on what you think will make the best improvement.

I wonder how your elephant's dream running scene would look at this point.

the scene with the fine green moving plasma gradient in the background is much more problematic (visual blockfest) (low bitrate) for alot encoders then the running scene itself :D

Dark Shikari
18th January 2008, 16:41
Im looking forward to see Sagekillas Dark Encoding results because their it showed flaws to me with my test encode, tough it's still better then without any AQ at all, but not as good as the Old AQ in such specific situations. To enhance Dark Scenes a little more with this AQ it seems you have to lower --aq-strength from the 1.0 setting down this gives then a boost in Dark Sequences but it seemes to be not really visible @ all so 1.0 seems also for such scenes the most efficient if you don't wan't todo segmented Encoding with different AQ strengths ;)
The issue here is likely one of the automatic sensitivity; because the adaptive quantization is unwilling to move bits between frames, it chooses too low a sensitivity on such frames.

Wait, you're saying LOWERING the AQ strength gives a boost in dark sequences? That seems nonsensical.

tetsuo55
18th January 2008, 17:16
Is there any reason why i should not be using this (besides the slower encoding time)

Dark Shikari
18th January 2008, 17:18
Is there any reason why i should not be using this (besides the slower encoding time)The encoding time difference is extremely minimal at this point.

Reasons you shouldn't be using it:

1) Your source doesn't seem to benefit from it (unlikely, but some cartoons at low bitrates seemed to suffer in my early tests).

2) uh, I can't think of anything else... :p

Some people seem to be pressing to make this an x264 default, even though its not even in SVN yet :)

DeathTheSheep
18th January 2008, 17:19
Like Chainmax, I'd still like to test this with prepass SATD ESA...


Woah, add me to the group. I'm stuck with 681.

tetsuo55
18th January 2008, 17:35
The encoding time difference is extremely minimal at this point.

Reasons you shouldn't be using it:

1) Your source doesn't seem to benefit from it (unlikely, but some cartoons at low bitrates seemed to suffer in my early tests).

2) uh, I can't think of anything else... :p

Some people seem to be pressing to make this an x264 default, even though its not even in SVN yet :)

~Default~ ~default~ ~default~

CruNcher
18th January 2008, 17:44
Dark why does your build creates every run (small) different results? isn't your AQ deterministic?
And yes exactly that's whats happening with the scene you also have lowering the AQ-strength improves it (bitrate increase) higher SSIM, highering it lowers SSIM (bitrate decrease).

Dark Shikari
18th January 2008, 17:46
Dark why does your build creates every run (small) different results? isn't your AQ deterministic?It should be 100% deterministic. Is it deterministic without AQ enabled?

ToS_Maverick
18th January 2008, 17:50
nope it isn't, with our without AQ

Dark Shikari
18th January 2008, 17:52
nope it isn't, with our without AQSounds like its not my problem then :p

CruNcher
18th January 2008, 17:57
nope it isn't, with our without AQ

Strange for me it's only non deterministic with AQ enabled without it i get every run the same results with it every run small differences.
--aq-strength 0 (allways the same results) without --aq-strength on the commandline same results every run as with --aq-strength 0 and with parkrun starting with --aq-strength 0.4 allways different results each run.

1st run = SSIM Mean Y:0.8447759
2nd run = SSIM Mean Y:0.8439500
3rd run = SSIM Mean Y:0.8447161
4rd run = SSIM Mean Y:0.8446099

Settings = x264-aq 720p50parkrun.yuv 1280x720 --bitrate 3000 --level 4.1 --
min-keyint 1 --keyint 25 --bframes 0 --ref 1 --weightb --subme 1 --8x8dct --qpmi
n 15 --trellis 0 --nf --me dia --threads auto --partitions all --no-fast-pskip -
-no-dct-decimate --aq-strength 0.4 --progress --sar 1:1 -o parkrun-0.4.mkv

bob0r
18th January 2008, 18:06
And with 1 thread?

CruNcher
18th January 2008, 18:17
With 1 thread it's ok @ 0.4 allways SSIM Mean Y:0.8448698 @ 12.40 fps
With 2 threads @ 0.4 SSIM changes every run @ 22.62 fps

Seems not threadsafe Dark ;)

Sagekilla
18th January 2008, 19:02
Sorry guys, results on Chronicles of Riddick won't be up until tomorrow at the earliest now. I ended up having to standby my computer so I could sleep (Had finals today) so I'm only 20% through the first encode.

bob0r
18th January 2008, 19:03
its thread safe, just not non deterministic thread safe.
with or without aq, makes no difference.
pengvado is aware of this issue, and one days hopes to find the cure :)

Snowknight26
18th January 2008, 19:08
In the .diff, it says the --aq-strength can go to 1.4 for strong AQ, but in the 1st post you say 1.0. :x

Dark Shikari
18th January 2008, 19:09
In the .diff, it says the --aq-strength can go to 1.4 for strong AQ, but in the 1st post you say 1.0. :xI didn't say what it can go to, I just gave examples.

There is technically no limit on how high AQ can go, but after about 2.0-3.0 it will really start to backfire. Perhaps in the help I should say 1.0 is a good medium.

Sagekilla
18th January 2008, 19:09
In the .diff, it says the --aq-strength can go to 1.4 for strong AQ, but in the 1st post you say 1.0. :x

1.0 is the recommended setting to use to start with. It gives, on average, very good results. If you read the help, it says 0.7 is "medium" and 1.4 is "stronger."

chipzoller
18th January 2008, 19:10
DS, in your original post you mentioned it may not be good for Anime/Cartoon and that you wouldn't use it there. Do you still maintain this, even at AQ's present state?
Is --aq-strength 1.0 a safe-to-use general setting for anime/cartoon if using AQ is advisable?

Snowknight26
18th January 2008, 19:11
I didn't say what it can go to, I just gave examples.

Add --aq-strength X to the commandline, where X is a value between 0.0 and 1.0.

Sorry, seemed a little misleading. :p

Dark Shikari
18th January 2008, 19:22
DS, in your original post you mentioned it may not be good for Anime/Cartoon and that you wouldn't use it there. Do you still maintain this, even at AQ's present state?
Is --aq-strength 1.0 a safe-to-use general setting for anime/cartoon if using AQ is advisable?Try it... I'm not sure at this point. I haven't done much testing.

If you do testing, post the results.

Razorholt
18th January 2008, 19:28
This new AQ seems to darken the picture, am I right?

Dark Shikari
18th January 2008, 19:29
This new AQ seems to darken the picture, am I right?AQ has absolutely no affect on image brightness. If it does so, its because your levels are off.

Sagekilla
18th January 2008, 19:30
Here's a frame I grabbed from my AQ. (That's still encoding) AQ did a pretty good job at preventing blocking from occurring on the jacket, as you can see. Unfortunately, I'll have to run the encode a third time to possibly increase the strength again. In the second image, you can see some blocking on the light shining in.

http://img.photobucket.com/albums/v621/Sagekilla/Riddick.png

http://img.photobucket.com/albums/v621/Sagekilla/riddick2.png

DeathTheSheep
18th January 2008, 22:07
its thread safe, just not non deterministic thread safe.
with or without aq, makes no difference.
pengvado is aware of this issue, and one days hopes to find the cure :)

I have a stopgap cure: don't use threads auto. Use threads=3 if on dual core and threads=6 on quad.

salehin
18th January 2008, 22:08
Sagekilla, is it possible for you to share your avs script, encode cmd, & logs. thanks :)

Sagekilla
18th January 2008, 22:15
Sagekilla, is it possible for you to share your avs script, encode cmd, & logs. thanks :)

Yes, I'll post my script and settings for now. I have to wait for my encoding to finish before I can post my logs however.


AVS Script
SetMTMode(2,2)
MPEG2Source("source.d2v")
Crop(0,62,0,-66).Spline36Resize(848,352)

# Some light processing, helps x264 compress better.
o = last
b1vec = o.MVanalyse(pel=2,overlap=2,idx=1,delta=1,isb=true)
f1vec = o.MVanalyse(pel=2,overlap=2,idx=1,delta=1,isb=false)
return(o.MVDegrain1(b1vec,f1vec,thSAD=350,idx=2))

x264 Settings
AQ On: x264 --keyint 1000 --crf 18 --ref 4 --mixed-refs --no-fast-pskip --bframes 16 --bime --weightb --b-pyramid --b-rdo --8x8dct --subme 7 --me umh --trellis 1 --aq-strength 1 --threads auto --thread-input --progress --output "videoAQ.264" --pass 1 --stats "stats_AQ.log" "source.avs"

AQ Off: x264 --keyint 1000 --crf 18 --ref 4 --mixed-refs --no-fast-pskip --bframes 16 --bime --weightb --b-pyramid --b-rdo --8x8dct --subme 7 --me umh --trellis 1 --aq-strength 0 --threads auto --thread-input --progress --output "video_NoAQ.264" --pass 1 --stats "stats_NoAQ.log" "source.avs"

Note, I used a long keyint because I don't mind the long latency to decode and I don't seek much to begin with. Change this to something a little more sane (250) if you wish.

bob0r
18th January 2008, 22:41
My bad on the nondeterministic issue.
This is only the case with Dark's AQ.

I guess the fixes since 713 also solved the threads issue with different bitrates.

Still as Dark explained AQ + something within x264 code can trigger something nondeterministic.
Guess they have to figure that one out :)


So if any of you got some nondeterministic issue, please report a reproducable method and pengvado will look into it!

CruNcher
18th January 2008, 22:45
Yes, I'll post my script and settings for now. I have to wait for my encoding to finish before I can post my logs however.


AVS Script
SetMTMode(2,2)
MPEG2Source("source.d2v")
Crop(0,62,0,-66).Spline36Resize(848,352)

# Some light processing, helps x264 compress better.
o = last
b1vec = o.MVanalyse(pel=2,overlap=2,idx=1,delta=1,isb=true)
f1vec = o.MVanalyse(pel=2,overlap=2,idx=1,delta=1,isb=false)
return(o.MVDegrain1(b1vec,f1vec,thSAD=350,idx=2))

x264 Settings
AQ Off: x264 --keyint 1000 --crf 18 --ref 4 --mixed-refs --no-fast-pskip --bframes 16 --bime --weightb --b-pyramid --b-rdo --8x8dct --subme 7 --me umh --trellis 1 --aq-strength 1 --threads auto --thread-input --progress --output "videoAQ.264" --pass 1 --stats "stats_AQ.log" "source.avs"

AQ On: x264 --keyint 1000 --crf 18 --ref 4 --mixed-refs --no-fast-pskip --bframes 16 --bime --weightb --b-pyramid --b-rdo --8x8dct --subme 7 --me umh --trellis 1 --aq-strength 0 --threads auto --thread-input --progress --output "video_NoAQ.264" --pass 1 --stats "stats_NoAQ.log" "source.avs"

Note, I used a long keyint because I don't mind the long latency to decode and I don't seek much to begin with. Change this to something a little more sane (250) if you wish.

Ehhh you mean AQ ON/AQ OFF? and don't you think those settings are a little insane ;)

foxyshadis
18th January 2008, 22:50
Woah, add me to the group. I'm stuck with 681.

Sorry, meant to type DTS.

I get some non-determinism, but it's only in the least-significant two digits of the SSIM. Not something I'm going to worry about.

chipzoller
18th January 2008, 22:53
When using CRF mode, is there a threshold one should not go under while using AQ, i.e. what's the lowest CRF that can be used with AQ while still producing helpful results?

bob0r
18th January 2008, 22:54
....

I get some non-determinism, but it's only in the least-significant two digits of the SSIM. Not something I'm going to worry about.

With AQ or None AQ?
If you get it with revision 721 SVN, please report a way to reproduce.

Dark Shikari
18th January 2008, 22:56
When using CRF mode, is there a threshold one should not go under while using AQ, i.e. what's the lowest CRF that can be used with AQ while still producing helpful results?I don't think there's any. AQ should work well at any bitrate--the only exception to this rule is if the bitrate is so low that the qp_deltas become a considerable portion of the bit cost of the video.

Sagekilla
18th January 2008, 23:06
Ehhh you mean AQ ON/AQ OFF? and don't you think those settings are a little insane ;)

D'oh, yeah it should be AQ on/off. Fixed that. Those settings are perfectly sane for me :) 8 fps on my dual core Opty 170. Besides, that's partially the reason why I can get good quality @ 1400 kbps on CoR.

jmnk
18th January 2008, 23:38
Ehhh you mean AQ ON/AQ OFF? and don't you think those settings are a little insane ;)
@CruNcher: could you elaborate on why these settings are 'little insane'?

burfadel
18th January 2008, 23:46
Just a slightly irrelevent question, but its something that may be asked or have confusion about when your patch build is updated! Afterall, depending on the actual change, using the same settings from 720 to 721 may result in inaccurate comparisons?...

On the revision log (from www.x264.nl) for rev 721, its says:

- change the meaning of --ref: it now selects DPB size (including B-frames), rather than L0 size (which B-frames are added to)

Does that mean if we normally select 5 reference frames and set b frames to 16, that --ref should now be set to 21?...

How would this affect the use of Megui/Staxrip etc?

Dark Shikari
18th January 2008, 23:54
Just a slightly irrelevent question, but its something that may be asked or have confusion about when your patch build is updated! Afterall, depending on the actual change, using the same settings from 720 to 721 may result in inaccurate comparisons?...

On the revision log (from www.x264.nl) for rev 721, its says:

- change the meaning of --ref: it now selects DPB size (including B-frames), rather than L0 size (which B-frames are added to)

Does that mean if we normally select 5 reference frames and set b frames to 16, that --ref should now be set to 21?...

How would this affect the use of Megui/Staxrip etc?No, that's not what the patch means.

Previously, DPB size required was equal to (--ref + [0,1,2]), with 0 in the case of no b-frames, 1 in the case of b-frames, and 2 in the case of b-pyramid. Now, I think its equal to (--ref + 1), no matter what. Akupenguin, feel free to correct me if I'm not exactly correct.

Sagekilla
19th January 2008, 00:00
@CruNcher: could you elaborate on why these settings are 'little insane'?

It's insane because I enabled every setting short of esa using 16 reference threads, and unlimited merange. Some of these settings can easily be turned off without increasing the bitrate massively (think 1500 kbps instead of 1400 kbps) while providing a sizeable speed boost. But, I prefer the extra quality. ESA I don't use because the speed vs quality tradeoff isn't worth it yet.

Dark Shikari
19th January 2008, 00:05
It's insane because I enabled every setting short of esa using 16 reference threads, and unlimited merange. Some of these settings can easily be turned off without increasing the bitrate massively (think 1500 kbps instead of 1400 kbps) while providing a sizeable speed boost. But, I prefer the extra quality. ESA I don't use because the speed vs quality tradeoff isn't worth it yet.I'd say you could probably improve that commandline by adding a few more refs--I've found that 4 is generally not where the the curve begins to top out, even for live-action.

burfadel
19th January 2008, 00:05
No, that's not what the patch means.

Previously, DPB size required was equal to (--ref + [0,1,2]), with 0 in the case of no b-frames, 1 in the case of b-frames, and 2 in the case of b-pyramid. Now, I think its equal to (--ref + 1), no matter what. Akupenguin, feel free to correct me if I'm not exactly correct.

How does that affect b-pyramid then, if its always +1 instead of +2?

Sagekilla
19th January 2008, 00:11
I'd say you could probably improve that commandline by adding a few more refs--I've found that 4 is generally not where the the curve begins to top out, even for live-action.

I probably could yes, especially if you merged the fast ref search diff to this build. I use very slow preprocessing though (to remove grain, noise, etc), which manages to cut down on the bitrate needed by a huge margin compared to increasing refs. I actually was using 6 refs back when your fast ref build was being actively developed, but since the AQ on this build is vastly better than the older AQ I'd rather use this one. All a matter of tradeoffs, no?

On a side note, CoR is coming out very nicely.. 154k of 192k frames encoded and the file is a mere 1.09 GB big. Looking at the end file being around 1.36 GB, possibly more. Throw on an extra 440 MB for the audio track though.

CruNcher
19th January 2008, 00:19
I i finished my 3 longcut encodes (CRF 25) :)

No AQ

x264 [info]: SSIM Mean Y:0.9849869
x264 [info]: PSNR Mean Y:46.721 U:47.332 V:51.035 Avg:47.291 Global:46.857 kb/s:
4125.27

encoded 6842 frames, 10.16 fps, 4132.79 kb/s

AQ ON --aq-strength 1.0

x264 [info]: SSIM Mean Y:0.9843622
x264 [info]: PSNR Mean Y:45.762 U:46.754 V:50.071 Avg:46.394 Global:46.023 kb/s:
4228.15

encoded 6842 frames, 9.72 fps, 4235.46 kb/s

Old Aq --aq-strength 0.9

x264 [info]: SSIM Mean Y:0.9858071
x264 [info]: PSNR Mean Y:46.987 U:47.763 V:51.368 Avg:47.590 Global:47.106 kb/s:
6594.71

encoded 6842 frames, 9.64 fps, 6602.20 kb/s

Sagekilla
19th January 2008, 00:23
Can we some screen caps of that clip, CruNcher?

CruNcher
19th January 2008, 00:25
First i have todo a subjective test of all 3 to determine the problematic areas for this the 3rd encode has to finish first

foxyshadis
19th January 2008, 00:45
With AQ or None AQ?
If you get it with revision 721 SVN, please report a way to reproduce.

With AQ, I didn't bother to test multiple identical SVN encodes.

How does that affect b-pyramid then, if its always +1 instead of +2?

It now uses the same number of references for all frames, eliminating the need to trade off b or b-pyramid for extra references.

Cruncher, pleeeease link instead of inlining the images this time. ;)

akupenguin
19th January 2008, 04:18
Now, I think its equal to (--ref + 1), no matter what. Akupenguin, feel free to correct me if I'm not exactly correct.
DPB is equal to --ref.
Unless --ref is less than the minimum (1 for P only, 2 for B-frames, 3 for pyramid), in which case --ref selects the number of L0 refs used for P-frames (like it did before) and DPB is equal to said minimum.

CruNcher
19th January 2008, 04:55
Ok im finished and the result is the same as what i told you before from the visual side the difference in Bright Scenes between NoAQ and NewAQ is almost 0 (if their is a enhancement i couldn't see it ,most probably the bitrate for those scenes is allready perfectly chosen by the encoder), but the Black Scenes with very defined gradients (and especialy when noise was left) did improved with the NewAQ compared to noAQ a little, see bellow. And as you allready see from the Size difference the OldAQ gave the best visual results also in the Dark Scenes (but this is still unbalanced and not really useable that way most probably this behaveiour can be tweaked alot).

Im gonna post pictures from every Scene this testcut contains and the problematic scenes (that doesn't get enhanced enough by either the NewAQ or NoAQ).

The Testcut is buildup from 6 scenes 5 of them are dark 1 is very bright and detailed (the Dark scenes are also detailed and when wrong encoded band extremely and most of the visible problems are in the ROI then, so a very bad situation)

Scene 1 = http://s6.directupload.net/images/080119/xjbzbe2m.png
Scene 2 = http://s6.directupload.net/images/080119/hc5re8nm.png
Scene 3 = http://s6.directupload.net/images/080119/hc5re8nm.png
Scene 4 = http://s5.directupload.net/images/080119/bfs48jze.png
Scene 5 = http://s2.directupload.net/images/080119/8bmbnscd.png
Scene 6 = http://s2.directupload.net/images/080119/lxw5ntsx.png

Problematic Scenes (areas)

Scene 2
No AQ = http://s4.directupload.net/images/080119/b3r6jc5i.png
New AQ = http://s3.directupload.net/images/080119/j4933af4.png
Old AQ = http://s6.directupload.net/images/080119/ojc2ha8s.png

Scene 3
No AQ = http://s5.directupload.net/images/080119/ackxyl3o.png
New AQ = http://s3.directupload.net/images/080119/gfl2dcw9.png
Old AQ = http://s1.directupload.net/images/080119/dggqn965.png

Scene 5-1
No AQ = http://s3.directupload.net/images/080119/oxuq5zo3.png
New AQ = http://s1.directupload.net/images/080119/zm3m6gj3.png
Old AQ = http://s2.directupload.net/images/080119/afa45iww.png

Scene 5-2
No AQ = http://s6.directupload.net/images/080119/mvz5ahzn.png
New AQ = http://s1.directupload.net/images/080119/ebgix8it.png
Old AQ = http://s3.directupload.net/images/080119/qdocncsk.png

Scene 6
No AQ = http://s1.directupload.net/images/080119/e29e7rgw.png
New AQ = http://s2.directupload.net/images/080119/tnenotri.png
Old AQ = http://s5.directupload.net/images/080119/yno3ves7.png

All in all the NewAQ is very good compared to noAQ at all (especialy in those Dark Scenes they look better in Motion alot then with NoAQ this way) and this just for 100 kbps more bitrate usage (think about the OldAQ result and the bitrate difference yada yada think about 100 kbps more *g*), everyone should invest that for this big HVS Improvement :)
and it definately should go into SVN :D

Final Size
No AQ = 140 mb
New AQ = 144 mb
Old AQ = 224 mb

Tough in a ABR Encode @ 3 Mbit Visualy the New AQ fails in the Problematic ROI Scenes and the old AQ is supirior in those and shows no visible sign of any Visual degredation to any of the other scenes.

Sagekilla
19th January 2008, 05:28
Nice results there CruNcher. Very bothersome that we keep getting closer to that "great" flat area performance yet are still quite far from it.

Dark Shikari, you said increasing aq-strength affects how high and low a qp can raise for a frame. Is this all that it affects in the new build or do higher values give a greater tendency to allocate more bits? Because in my testing, it seemed like going from 0.5 to 1, and then to 2 had little effect on overall quality of flat areas.

Dark Shikari
19th January 2008, 05:31
Nice results there CruNcher. Very bothersome that we keep getting closer to that "great" flat area performance yet are still quite far from it.

Dark Shikari, you said increasing aq-strength affects how high and low a qp can raise for a frame. Is this all that it affects in the new build or do higher values give a greater tendency to allocate more bits? Because in my testing, it seemed like going from 0.5 to 1, and then to 2 had little effect on overall quality of flat areas.One of the problems with the existing AQ is that it makes a compromise--that it won't move bits considerably between frames. As a result, it avoids screwing up ratecontrol--but that also means that in extremely flat frames, it isn't willing to lower the framewide quantizer to compensate.

DeathTheSheep
19th January 2008, 05:36
Which is a darn shame if you use CQ. :)

Dark Shikari
19th January 2008, 05:41
Which is a darn shame if you use CQ. :)Actually, one interesting thought: AQ currently limits the amount it raises and lowers QPs. If I make an option to customize or remove that limitation on the commandline, then what you can do is use a static (non-automatic) sensitivity--and use AQ for ratecontrol with CQ as a base :devil:

Sagekilla
19th January 2008, 05:47
Any chance of a patch coming out to try this out? *wink wink* :)

Dark Shikari
19th January 2008, 05:51
Line 293, ratecontrol.c clips the QPs to (-5 * AQ strength, 5 * AQ strength). Comment that out, or change it to whatever you want.

That's all you need to do.

I *strongly* suggest you don't put the strength too high when doing this, and use a sensitivity between 15 and 30 or something like that--you'll have to fool around to find the best values.

CruNcher
19th January 2008, 06:00
this is so funny i did now a Encode @ 3 Mbit with the Old Aq and geez it looks stunning in those scenes even better then the CRF 25 New Aq encode with 4 mbit so the Old one is really thought for ABR or 2 Pass but not CRF :) those scenes look now as if they used the 6 mbit :D and the degredation on the other scenes is visualy not visible :) This is what i wanted (balanced distribution) :D

SSIM everything lowered as expected but the Visual Quality is Perfect :) no ROI problems anymore in any of those Problematic Scenes i posted above and the Final size is 104 Mb :)

x264 [info]: SSIM Mean Y:0.9807726
x264 [info]: PSNR Mean Y:44.941 U:46.045 V:49.483 Avg:45.600 Global:45.188 kb/s:
3043.12

encoded 6842 frames, 9.49 fps, 3050.65 kb/s

Line 293, ratecontrol.c clips the QPs to (-5 * AQ strength, 5 * AQ strength). Comment that out, or change it to whatever you want.

That's all you need to do.

I *strongly* suggest you don't put the strength too high when doing this, and use a sensitivity between 15 and 30 or something like that--you'll have to fool around to find the best values.

gonna try that and do the 3 mbit again, i think this should improve it like the Old AQ also for ABR :)

TheRyuu
19th January 2008, 06:46
I've built it with the newest rev. (r721) with this patch, mp4 output, pthreads, and avis input.

http://www.fileducky.com/NWvOdpST/

I've tested it to the point were it built and it didn't error.
Just thought I'd share this build.

Dark Shikari
19th January 2008, 07:47
Here are some things that I want people to feel free to experiment with my AQ (since the function itself is relatively easy to modify):

1. I find that the cap I've placed on the AQ adjustment seems like a bad solution to the issue of raising/lowering quantizers too much. Can anyone here come up with a better solution?

2. As mentioned earlier, I tried to avoid the problem of AQ redistributing bits with the automatic sensitivity. However, this doesn't bode well for very flat scenes, it seems. Is there a way to avoid screwing up ratecontrol while biasing the AQ a bit in the favor of flat scenes getting lower quantizers? This would most likely take the form of some sort of bias in the summing of the x264_aq_autosense() function.

3. Would the AQ formula benefit from any change in scaling method?

Etc.

CruNcher
19th January 2008, 14:07
Dark I tried now to max it out @ CRF 25 like the oldAQ does i did it with --aq-sensitivity 40 without comenting out that line now the dark small test scene gets the same enhancement as the OldAQ did @ --ag-strength 0.9 --aq-sensitivity 15 (even more bitwise but not visualy have to find the perfect visual max out point now) :) so the key for the new AQ seems to be as i thought at first really to be in the --aq-sensitivity :D

This is the max i get now with the new AQ
x264 [info]: SSIM Mean Y:0.9918510
x264 [info]: PSNR Mean Y:49.527 U:50.649 V:52.874 Avg:50.108 Global:49.794 kb/s:
8130.14

encoded 701 frames, 9.09 fps, 8136.86 kb/s

This is the max i got with the OldAQ @ --aq-strength 0.9 --aq-sensitivity 30
x264 [info]: SSIM Mean Y:0.9916371
x264 [info]: PSNR Mean Y:49.032 U:50.341 V:52.570 Avg:49.657 Global:49.356 kb/s:
7589.89

encoded 701 frames, 9.13 fps, 7596.56 kb/s

Btw isn't this now the same behaveiour like a VBR Ratecontroll ? (you get the best possible output, without any limitations) i think such a mode is really missing and this seems a cheap way to force this by useing AQ (Atemes Encoder has this mode and seems X264 now too) :D (and it seems to work perfectly) and by maxing it out it seems to work perfectly then in ABR for lower bitrates too (best possible visual result for the bitrate target) :) i think this is indeed a big big step. I think this is even better then CRF because now you can decide with the tools of H.264 how much compression and complexity in the end you wan't (like a Stream Complexity Ratecontrol) and how much Speed loss you wan't to invest for that (both sides Encoding/Decoding or based on Compression Efficiency) if it really works out as i think (this is what i allways dreamed of for a mode for EDP) :) as base for this CRF 19 or 21 should be used i think.

If my ideas work out i think there are 2 new Modes born for X264 in this moment :D
Variable Bitrate Mode (useing CRF or QP X as start)
Stream Complexity Mode (Encoding/Decoding Speed Mode)


It looks like this now

OldAQ
CRF/ABR Visually max out (CRF doesn't max out as good as with New AQ)

New AQ
CRF Visually maxes out (better then OldAQ)
ABR didn't found yet how to max it out Visually

gonna try that aproach now on ParkRun with the Old AQ maxed out :)

DeathTheSheep
19th January 2008, 16:33
Actually, one interesting thought: AQ currently limits the amount it raises and lowers QPs. If I make an option to customize or remove that limitation on the commandline, then what you can do is use a static (non-automatic) sensitivity--and use AQ for ratecontrol with CQ as a base :devil:

Wow, I was actually just thinking about doing something like that... sheesh. Well, I'm encoding with it now. Strength is at 0.7, and I'm trying both 15 and 30 sensitivity.

CruNcher
19th January 2008, 16:55
This is really a hard balance act definately the OldAQ can't improve Parkrun in any way, now i tried to balance the New AQ between Parkrun and my testcut and got better results then before in the ROI scenes (a little SSIM decrease in Parkrun, but still visualy better then No AQ) but still it's nowhere near the OldAQ results on my testcut (ABR).

DeathTheSheep
19th January 2008, 16:56
Okay, some results.
0.9741502 SSIM for original code, q29 AQ07. (4248KB)
0.9737121 SSIM for RC mod, q28 AQ07. (4258KB)

Visually, undecided.
Of course, I'll try other settings... but at least as far as sensitivity goes, 15 might just be too low.

Dark Shikari
19th January 2008, 17:06
Okay, some results.
0.9741502 SSIM for original code, q29 AQ07. (4248KB)
0.9737121 SSIM for RC mod, q28 AQ07. (4258KB)

Visually, undecided.
Of course, I'll try other settings... but at least as far as sensitivity goes, 15 might just be too low.It is too low. That sets the "center variance" at about 25000. 25 will set it at around 200,000.

DeathTheSheep
19th January 2008, 17:13
I'm using sensitivity 30 now (the "other extreme"), and things might be looking up.

[edit] Filesize is ridiculously high, I'm going to have to up my qp by at least 2 notches...

chipzoller
19th January 2008, 17:34
I'm finding this AQ function very handy. Do you know if it is currently used in any other encoders? And this may be a stupid question as I'm not an expert on the inner-workings of x264 and, indeed, H.264 in general, but does using AQ in any way tweak or perhaps "break" the H.264 spec to your knowledge? Is this a profile-limited tool, or can it be used in the creation of any stream?

CruNcher
19th January 2008, 17:38
I think i found a good balance :) have to test it on the longcut first :D

Sharktooth
19th January 2008, 17:39
@chipzoller: no. it doesnt break anything. other encoders use some kind of AQ too.

DeathTheSheep
19th January 2008, 17:45
Q31: 0.9750365 (RC, strength 1.0, size 4293KB)
Q29: 0.9732632 (std strength 1.0, size 4129KB)
Q29: 0.9741502 (std strength 0.6, size 4248KB)

Since it's quants we're dealing with here, it's hard to settle on a single, uniform bitrate. The three points here seem almost linear in bitrate/SSIM scaling, so I really can't say. (Needless to say, it takes forever to take derivatives of each one's SSIM vs bitrate curve for comparison).

Should I post the resultant clips?

Dark Shikari
19th January 2008, 17:48
Q31: 0.9750365 (RC, strength 1.0, size 4293KB)
Q29: 0.9732632 (std strength 1.0, size 4129KB)
Q29: 0.9741502 (std strength 0.6, size 4248KB)

Since it's quants we're dealing with here, it's hard to settle on a single, uniform bitrate. The three points here seem almost linear in bitrate/SSIM scaling, so I really can't say. (Needless to say, it takes forever to take derivatives of each one's SSIM vs bitrate curve for comparison).

Should I post the resultant clips?
Compare visual quality, not SSIM.

Eliminating blocking in very flat areas, for example, doesn't generally help SSIM.

DeathTheSheep
19th January 2008, 18:19
That's true, but at exactly the same bitrate (<1KB difference out of >4000), the large SSIM increase and PSNR slight increase isn't easy to count out.

As for blocking, it's a non-issue for me anyway at Q31...or is it?
And now for a complete 180 degree turnaround, I'm going to check the visual quality of the clips in dark scenes, if you'll excuse me. :p

CruNcher
19th January 2008, 19:17
I give up it seems impossible that way (changeing settings) to achive good results on the problematic ROI stuff in this testcut @ 3 Mbit with ABR and the New AQ (with CRF it's no problem or ABR and OldAQ)

DeathTheSheep
19th January 2008, 20:01
Okay, here's a test pack. The settings (and SSIM) can be inferred from the filenames. One of the files is without AQ, and one of them uses standard .431 at strength 0.7 with auto threshold, I believe. All AQ strengths are 1.0 unless specified otherwise in the filename with st_. As you can tell, I tried to achieve roughly 4201KB for each file; however, 9749861_RC_q29_s21 was blatantly oversized (lowering the sensitivity even one point makes it severely undersized).

http://gabe.ej.am/samples/

CruNcher
19th January 2008, 21:44
Okay, here's a test pack. The settings (and SSIM) can be inferred from the filenames. One of the files is without AQ, and one of them uses standard .431 at strength 0.7 with auto threshold, I believe. All AQ strengths are 1.0 unless specified otherwise in the filename with st_. As you can tell, I tried to achieve roughly 4201KB for each file; however, 9749861_RC_q29_s21 was blatantly oversized (lowering the sensitivity even one point makes it severely undersized).

http://gabe.ej.am/samples/

Wussa what for fast pictures (not used to this speed) very hard to realize any difference @ all, but they definitely all behind my visual understanding of quality, especialy the ringing and extreme blurienes would annoy me to death (it allready does in my stuff) and especialy with H.264 (blurrienes of X264 can for sure be enhanced tough). Im only used @ this problem from past codecs and ASP (except blurrienes) since the better partitioning, Qpel and Inloop deblocker should prevent such stuff from happening in H.264 (was the source that bad?). I would say no real visual improvement there visible that might come from the New AQ.

DeathTheSheep
19th January 2008, 22:14
Yes, the source wasn't exactly par excellence, if you catch my drift. I do see markedly better quality (detail preservation) in the q29 (and to some extent the q30) samples as opposed to both original and AR-std.

Also keep in mind this is baseline QVGA resolution (320x240), not even VGA, much less HD, not to mention q30+ equivalent bitrate, which might account for some "blurriness." :D

[edit] CruNcher: for your viewing pleasure, a max setting DivX6.8Pro and CruNcherEDP1.4.5 have been added to the site. Even at max settings, ASP (at least without tons of filtering) looks like sheer buttocks compared to baseline AVC at this bitrate/resolution.

CruNcher
19th January 2008, 22:48
(detail preservation) - ehh what details if i might allowed to ask, i see only a bunch of colors flipping around in ultra speed ;)

*hides far far away deep inside of looney tunes and hanna babera land from the wrath of the anime community that might come over him like a Dragonball @ the sunset dawn, for this statement*

DeathTheSheep
19th January 2008, 22:53
NAPA: Vegeta, what does the SSIM say about its quality level?

VEGETA: It's over nine thousaaaand!!

NAPA: What 9000?!! There's no WAY that can be right!

You don't watch much anime, do you, CruNcher? Take a look at how your EDP with max settings (except GMC) does on the clip, for instance... tsk, tsk. If you don't know the rubric, maybe you shouldn't judge. ;)

Dark Shikari
20th January 2008, 00:02
My hunch was right.

A constant AQ sensitivity with unlimited (no clipping) AQ, in a sense using AQ to hack at ratecontrol, is IMPRESSIVE, to say the least. And I mean astounding.

Before (this thread's automatic-sensing AQ):

http://i1.tinypic.com/80neaz9.png

After:

http://i16.tinypic.com/6l8ke14.png

Obviously the framesize is drastically different, but that's the point; this scene needed more bits. The effect of this is simply astounding; it may make this AQ even better, though it cannot work with CRF if you want any hope of a bitrate close to that you would get without AQ.

DeathTheSheep
20th January 2008, 00:04
You just took out that one line and used sensitivity X, right? That's what I did (see previous posts).

If so, what sensitivity precisely did you use?

CruNcher
20th January 2008, 00:06
Indeed i don't, but don't forget your sample uses inloop deblocking and ASP isn't haveing that feature it can make only use of outside Postprocessing, but you know all that. Think about the same AVC Baseline sample but with the sharpness of ASP, im sure it's doable someone just has to find the right way to achive that and nope you won't with only turning off deblocking unfortunately it's not that easy ;). Btw i watched anime back when i was a kid but todays stuff is ehh different i loved http://www.imdb.com/title/tt0185110/ , now you know my depest secret ;)

Yes Dark as i said it's like a VBR then it gives the bitrates to those scenes that need it the most, but i have problems to reproduce the same effect in ABR with the OldAQ it works in both RC modes, tough the OldAQ doesn't improve ParkRun the way the new does. :)

Dark Shikari
20th January 2008, 00:07
You just took out that one line and used sensitivity X, right? That's what I did (see previous posts).

If so, what sensitivity precisely did you use?Sensitivity was 30, which seemed like a good medium. And yes, I just removed that line.

DeathTheSheep
20th January 2008, 00:11
Hmm, I use 25. 30 seems rather excessive to me. Besides, as you can see from my previous post (or gabe.ej.am/samples/), there's many ways to get at the same filesize via QP, with SSIM and visual quality to match. I find qX without ratecontrol is approximately equal to -qX --aq-sensitivity ~25 OR -q[X+1] --aq-sensitivity ~30 for a good range of balanced sources.

Dark Shikari
20th January 2008, 00:20
Also, it was requested that I post this updated, working version of me-prepass (supposedly, I haven't tested it) on doom9, so I will post it here.

ME-prepass patch, working with 721, supposedly. (http://pastebin.com/f387ef1d)

Reuf Toc
20th January 2008, 00:30
The result of your hack is really awesome :eek:

Here is a gif of your pictures with contrast enhanced :

http://img255.imageshack.us/my.php?image=sanstitre1rl1.gif

DeathTheSheep
20th January 2008, 00:34
May I hope you have the updated, working satd floating around there as well? If so, here's your request... :devil:

Dark Shikari
20th January 2008, 00:38
Working SATD doesn't exist at the moment, peng will have to updated the patch.

Also, apparently that Prepass patch is broken and won't compile. Blegh. At least I did make it.

Also, major major major major update. Read the new instructions, and be shocked at the quality improvement.

DeathTheSheep
20th January 2008, 00:43
"major major major major update" = disable auto threshold (in 2pass)? Hmm... ;)

Dark Shikari
20th January 2008, 00:44
"major major major major" = disable auto threshold (in 2pass)? Hmm... ;)Major major = remove the limiter when not using automatic thresholding.

I could also make it so that the default sensitivity depends on which pass mode you choose, but for now, set it yourself.

This is the reason why I call it "major major":

http://img255.imageshack.us/img255/2779/sanstitre1rl1.gif

DeathTheSheep
20th January 2008, 00:49
Ah, I was wondering when you'd post another one of your aniGifs. Now with contrast enhancement!! But I realize you still forgot to mention what the other two "major"s were for. I have a suggestion: adaptive smexiness profile. ;)

Well, to tell you the truth, I don't see any clear benefit in keeping the limiter--even in auto mode.

Sagekilla
20th January 2008, 00:53
Oh curses.. I just started a new encode a few minutes ago, now I have to go check this new update out.

Dark Shikari
20th January 2008, 00:55
Ah, I was wondering when you'd post another one of your aniGifs. Now with contrast enhancement!! But I realize you still forgot to mention what the other two "major"s were for. I have a suggestion: adaptive smexiness profile. ;)

Well, to tell you the truth, I don't see any clear benefit in keeping the limiter--even in auto mode.Under the current system, its there for very good reason. This is because if the frame is flat, it'll try to raise the quantizers of the few complex blocks in the frame by a huge amount--causing problems, since it won't lower the blocks in the rest of the frame enough. It *correctly* calculated the *difference* in quantizers between the flat and complex blocks, except that since it wasn't willing to vastly lower the quantizer of the frame, the complex blocks would end up with far too high a quantizer.

Sagekilla
20th January 2008, 00:57
Under the current system, its there for very good reason. This is because if the frame is flat, it'll try to raise the quantizers of the few complex blocks in the frame by a huge amount--causing problems, since it won't lower the blocks in the rest of the frame enough. It *correctly* calculated the *difference* in quantizers between the flat and complex blocks, except that since it wasn't willing to vastly lower the quantizer of the frame, the complex blocks would end up with far too high a quantizer.

Hmm just read a few posts up. How come this AQ improvement won't work on CRF?

lexor
20th January 2008, 01:02
Automatic thresholding, the default sensitivity option, helps keep the bitrate sane in CRF mode, but in 2pass you can do much better, and use a static sensitivity. Using --aq-sensitivity 30 or similar will result not only in flat areas of individual frames getting more bits, but also entire flat FRAMES getting more bits.
I am a bit confused by that wording. It seems to allow possibility of the entire movie's bitrate going up (if it decides all frames need tuning up, I know unlikely, but as a theoretic possibility). Does it? Dark, you said that that "new" image above is larger, but is it true for the entire movie, or have the bits simply been re-allocated between frames?

Dark Shikari
20th January 2008, 01:04
I am a bit confused by that wording. It seems to allow possibility of the entire movies bitrate going up (if it decides all frames need tuning up, I know unlikely, but as a theoretic possibility). Does it? Dark, you said that that "new" image above is larger, but is it true for the entire movie, or have the bit simply been re-allocated between frames?Static sensitivity allows bits to be re-allocated between frames, but gives no guarantee that the bitrate of the movie is anywhere close to that of what you would get without CRF.

It *works* on CRF, but it will not keep the bitrate anywhere near to the original (or at least, one cannot guarantee it).

lexor
20th January 2008, 01:07
Static sensitivity allows bits to be re-allocated between frames, but gives no guarantee that the bitrate of the movie is anywhere close to that of what you would get without CRF.

It *works* on CRF, but it will not keep the bitrate anywhere near to the original (or at least, one cannot guarantee it).

Eh, I'm not the guy concerned about CRF, I do 2pass encodes. :)

On that note, will MeGUI's bitrate calculator still be correct (or as correct as it is for official builds)? I mean can I still rely on getting a more or less certain file size out of a 2pass?

Atak_Snajpera
20th January 2008, 01:18
It *works* on CRF, but it will not keep the bitrate anywhere near to the original (or at least, one cannot guarantee it).

Hehe It sounds like old Haali's AQ :)

DeathTheSheep
20th January 2008, 01:21
Sounds like pretty much anything that redefines the allocation of bits between frames contrary to x264 defaults. :)

Dark Shikari
20th January 2008, 01:30
Eh, I'm not the guy concerned about CRF, I do 2pass encodes. :)

On that note, will MeGUI's bitrate calculator still be correct (or as correct as it is for official builds)? I mean can I still rely on getting a more or less certain file size out of a 2pass?Yes, bitrate mode in general (1pass or 2pass) should probably still be pretty accurate at this point.

Slightly less than normal, but not by much.
Sounds like pretty much anything that redefines the allocation of bits between frames contrary to x264 defaults. :)Yup.

It'll be interesting to see what Dakaz thinks of this mode ;)

Atak_Snajpera
20th January 2008, 01:34
Dakaz ? Do we know him? :/

Dark Shikari
20th January 2008, 01:36
Dakaz ? Do we know him? :/Many don't, but he and the rest of Avail Media's encoding division are behind quite a bit of x264, especially interlaced mode :p

DeathTheSheep
20th January 2008, 01:40
He also offered a bounty for people who could produce good, novel x264 code. (http://mailman.videolan.org/pipermail/x264-devel/2007-May/003055.html) And of course by bounty I mean...you know.
Now I just have to cross my fingers and hope the satd patch will come magically rolling in. The guts of this thing are truly, in lack of a better term, mathemagical. (Not to mention easy to break beyond repair, if you don't know what you're doing).

ToS_Maverick
20th January 2008, 01:40
Static sensitivity allows bits to be re-allocated between frames, but gives no guarantee that the bitrate of the movie is anywhere close to that of what you would get without CRF.

It *works* on CRF, but it will not keep the bitrate anywhere near to the original (or at least, one cannot guarantee it).

who cares? with Haali's AQ it was clear, that if you enable it, the bitrate with CRF will rise.

i consider it that way: std x264 is not at an optimum.

why do you want to stick to a same frame size? if a frame needs more bits to look good, so be it... as long as the ratecontrol has no problem with it ;)

Dark Shikari
20th January 2008, 01:56
who cares? with Haali's AQ it was clear, that if you enable it, the bitrate with CRF will rise.

i consider it that way: std x264 is not at an optimum.

why do you want to stick to a same frame size? if a frame needs more bits to look good, so be it... as long as the ratecontrol has no problem with it ;)Remember what happened with the previous AQ?

Sometimes, video size would rise drastically, other times it would drop drastically.

The point being, that if most of the bits in your video are in blocks that are above the variance threshold you set, your bitrate will drop--and if most of the bits are in blocks that are below, your bitrate will rise.

slavickas
20th January 2008, 02:25
He also offered a bounty for people who could produce good, novel x264 code. (http://mailman.videolan.org/pipermail/x264-devel/2007-May/003055.html) And of course by bounty I mean...you know
...
http://upload.wikimedia.org/wikipedia/commons/3/31/BountyBars.jpg
?
edit: my 264 post :D

Dark Shikari
20th January 2008, 02:26
I think he means:

http://ecx.images-amazon.com/images/I/51HpkVvqKwL.jpg

ToS_Maverick
20th January 2008, 02:27
wait a minute... with a too low --aq-sensitivity xx the bitrate could DROP? well... that's a fact that i didn't know.

my dream of a CRF mode would be, set it at 18 and your encode comes out transparent. if we could reach this goal, we would have a "lame -V2" H264 codec :D

Dark Shikari
20th January 2008, 02:30
wait a minute... with a too low --aq-sensitivity xx the bitrate could DROP? well... that's a fact that i didn't know.

my dream of a CRF mode would be, set it at 18 and your encode comes out transparent. if we could reach this goal, we would have a "lame -V2" H264 codec :DThis is because AQ sensitivity sets a midpoint variance, above which the QP is raised, and below which the QP is lowered.

Technically, one could apply the entire "automatic sensitivity" aspect of my algorithm to the ENTIRE VIDEO on the first pass, and then use that sensitivity on the second pass.

ToS_Maverick
20th January 2008, 02:40
is my thinking correct:
your auto sensitivity redistributes the bits (lowers/raises quants) from areas that are more complex to lower complex areas.
what's bothering me is, if there is a scene, like the last .gif you posted, where there are no bits to redistribute, what are you going to do?

is there a reliable metric (like ssim), to find out where the problematic parts are, and give them more bits?

of course we could do a lot of testing and tweaking, to find out a good sensitivity that we could use for our encodes, but that would be empirical, and will surely get rejected by aku for SVN...

Dark Shikari
20th January 2008, 02:42
is my thinking correct:
your auto sensitivity redistributes the bits (lowers/raises quants) from areas that are more complex to lower complex areas.
what's bothering me is, if there is a scene, like the last .gif you posted, where there are no bits to redistribute, what are you going to do?

is there a reliable metric (like ssim), to find out where the problematic parts are, and give them more bits?

of course we could do a lot of testing and tweaking, to find out a good sensitivity that we could use for our encodes, but that would be empirical, and will surely get rejected by aku for SVN...Sensitivity is arbitrary, in a sense; as long as you don't put a limiter on how QPs are changed.

No matter what sensitivity is chosen, the QPs chosen in a particular frame will be exactly the same, except scaled up or down by a constant value.

DeathTheSheep
20th January 2008, 03:25
Hm, it's odd how .45 behaves differently with the same commandline as .431 with the line removed. .431 made the file 4204KB, 0.9747881 SSIM, while your new patch with same threshold made it 4205KB, 0.9747586 SSIM. How did the result become slightly bigger and worse in SSIM, if just the threshold was removed? Maybe change in the code for interlace fixes?

Dark Shikari
20th January 2008, 03:31
Hm, it's odd how .45 behaves differently with the same commandline as .431 with the line removed. .431 made the file 4204KB, 0.9747881 SSIM, while your new patch with same threshold made it 4205KB, 0.9747586 SSIM. How did the result become slightly bigger and worse in SSIM, if just the threshold was removed? Maybe change in the code for interlace fixes?There was a slight change to fix a bug when variance was equal to zero.

DeathTheSheep
20th January 2008, 03:42
Note to self: you cannot say if(floating point value == 0) before calling x264_cpu_restore()... ;)
http://www.biffleys.com/Strategy/pictures/bounty.jpg

desta
20th January 2008, 03:46
Some test results, using AQ with varying thresholds.

--pass 2 --bitrate 4100 --stats "D:\TESTS\test.aq1.sens-auto.stats" --keyint 240 --min-keyint 24 --ref 7 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -3,-2 --subme 7 --trellis 1 --analyse all --8x8dct --vbv-maxrate 25000 --me umh --merange 24 --threads auto --thread-input --progress --no-dct-decimate --output "D:\TESTS\test.aq1.sens-auto.mkv" "D:\TESTS\test.aq1.sens-auto.avs" --aq-strength 1

x264 [info]: slice I:19 Avg QP:17.37 size: 53151 PSNR Mean Y:50.05 U:50.90 V:52.20 Avg:49.86 Global:47.70
x264 [info]: slice P:782 Avg QP:19.27 size: 27715 PSNR Mean Y:45.76 U:48.71 V:49.06 Avg:46.53 Global:46.00
x264 [info]: slice B:519 Avg QP:20.61 size: 10718 PSNR Mean Y:45.11 U:48.61 V:49.06 Avg:45.90 Global:45.07
x264 [info]: mb I I16..4: 21.1% 67.2% 11.7%
x264 [info]: mb P I16..4: 10.4% 30.0% 4.6% P16..4: 24.2% 12.1% 3.2% 0.1% 0.0% skip:15.4%
x264 [info]: mb B I16..4: 2.3% 5.2% 0.9% B16..8: 32.6% 1.9% 4.3% direct: 5.0% skip:47.9%
x264 [info]: 8x8 transform intra:66.2% inter:57.1%
x264 [info]: direct mvs spatial:94.8% temporal:5.2%
x264 [info]: ref P 67.3% 14.7% 6.9% 3.8% 2.9% 2.5% 1.8%
x264 [info]: ref B 76.9% 11.4% 4.6% 2.7% 1.8% 1.5% 1.1%
x264 [info]: SSIM Mean Y:0.9831273
x264 [info]: PSNR Mean Y:45.567 U:48.706 V:49.105 Avg:46.331 Global:45.626 kb/s:4104.32
----------

the following tests use the same options as above, with the only difference being...

--aq-sensitivity 30

x264 [info]: slice I:19 Avg QP:26.11 size: 55517 PSNR Mean Y:49.61 U:50.68 V:51.93 Avg:49.46 Global:46.92
x264 [info]: slice P:782 Avg QP:29.36 size: 27478 PSNR Mean Y:44.90 U:48.13 V:48.45 Avg:45.72 Global:44.85
x264 [info]: slice B:519 Avg QP:30.41 size: 10933 PSNR Mean Y:44.33 U:48.02 V:48.45 Avg:45.17 Global:44.20
x264 [info]: mb I I16..4: 22.6% 63.8% 13.7%
x264 [info]: mb P I16..4: 10.2% 30.8% 5.2% P16..4: 24.9% 11.6% 2.9% 0.1% 0.0% skip:14.4%
x264 [info]: mb B I16..4: 2.7% 5.5% 1.0% B16..8: 33.4% 1.9% 4.4% direct: 4.9% skip:46.3%
x264 [info]: 8x8 transform intra:65.9% inter:54.0%
x264 [info]: direct mvs spatial:93.3% temporal:6.7%
x264 [info]: ref P 67.0% 14.6% 7.0% 3.8% 3.1% 2.6% 1.9%
x264 [info]: ref B 76.2% 11.6% 4.8% 2.8% 1.9% 1.6% 1.2%
x264 [info]: SSIM Mean Y:0.9822634
x264 [info]: PSNR Mean Y:44.746 U:48.122 V:48.500 Avg:45.558 Global:44.605 kb/s:4100.14
----------

--aq-sensitivity 25

x264 [info]: slice I:19 Avg QP:23.89 size: 56126 PSNR Mean Y:49.75 U:50.76 V:52.03 Avg:49.60 Global:47.08
x264 [info]: slice P:782 Avg QP:27.18 size: 27493 PSNR Mean Y:44.97 U:48.15 V:48.49 Avg:45.78 Global:44.93
x264 [info]: slice B:519 Avg QP:28.25 size: 10881 PSNR Mean Y:44.43 U:48.05 V:48.48 Avg:45.21 Global:44.26
x264 [info]: mb I I16..4: 24.6% 62.4% 13.0%
x264 [info]: mb P I16..4: 10.1% 30.9% 5.1% P16..4: 24.8% 11.7% 2.9% 0.1% 0.0% skip:14.4%
x264 [info]: mb B I16..4: 2.6% 5.6% 0.9% B16..8: 33.4% 1.8% 4.4% direct: 4.8% skip:46.4%
x264 [info]: 8x8 transform intra:66.2% inter:54.5%
x264 [info]: direct mvs spatial:93.4% temporal:6.6%
x264 [info]: ref P 67.1% 14.6% 7.0% 3.8% 3.1% 2.6% 1.9%
x264 [info]: ref B 75.9% 11.7% 4.8% 2.9% 1.9% 1.6% 1.2%
x264 [info]: SSIM Mean Y:0.9823568
x264 [info]: PSNR Mean Y:44.825 U:48.149 V:48.536 Avg:45.612 Global:44.677 kb/s:4099.58
----------

--aq-sensitivity 20

x264 [info]: slice I:19 Avg QP:21.63 size: 54153 PSNR Mean Y:49.61 U:50.66 V:51.92 Avg:49.46 Global:47.00
x264 [info]: slice P:782 Avg QP:24.51 size: 27609 PSNR Mean Y:45.04 U:48.20 V:48.53 Avg:45.85 Global:45.03
x264 [info]: slice B:519 Avg QP:25.63 size: 10797 PSNR Mean Y:44.47 U:48.09 V:48.52 Avg:45.27 Global:44.34
x264 [info]: mb I I16..4: 22.9% 64.3% 12.8%
x264 [info]: mb P I16..4: 10.2% 31.0% 5.0% P16..4: 24.8% 11.6% 2.9% 0.1% 0.0% skip:14.4%
x264 [info]: mb B I16..4: 2.6% 5.5% 0.9% B16..8: 33.4% 1.8% 4.4% direct: 4.8% skip:46.7%
x264 [info]: 8x8 transform intra:66.4% inter:55.0%
x264 [info]: direct mvs spatial:93.8% temporal:6.2%
x264 [info]: ref P 67.1% 14.6% 7.0% 3.8% 3.0% 2.6% 1.9%
x264 [info]: ref B 75.9% 11.6% 4.9% 2.8% 1.9% 1.6% 1.2%
x264 [info]: SSIM Mean Y:0.9824983
x264 [info]: PSNR Mean Y:44.883 U:48.192 V:48.573 Avg:45.673 Global:44.765 kb/s:4101.04
----------


Source:
- http://img413.imageshack.us/img413/2694/source66fo1.png
- http://img526.imageshack.us/img526/551/source165am9.png
- http://img526.imageshack.us/img526/1001/source358ij5.png
- http://img237.imageshack.us/img237/3574/source490ut0.png
- http://img242.imageshack.us/img242/9720/source615hn0.png

Auto Sensitivity:
- http://img248.imageshack.us/img248/2976/x264sensauto66iy7.png
- http://img248.imageshack.us/img248/4585/x264sensauto165en5.png
- http://img301.imageshack.us/img301/4273/x264sensauto358is1.png
- http://img99.imageshack.us/img99/6886/x264sensauto490uw8.png
- http://img99.imageshack.us/img99/8848/x264sensauto615el2.png

Sensitivity 30:
- http://img99.imageshack.us/img99/3128/x264sens3066bp5.png
- http://img412.imageshack.us/img412/5105/x264sens30165lw9.png
- http://img262.imageshack.us/img262/4485/x264sens30358rc9.png
- http://img262.imageshack.us/img262/9405/x264sens30490qd9.png
- http://img237.imageshack.us/img237/8185/x264sens30615yt0.png

Sensitivity 25:
- http://img149.imageshack.us/img149/8880/x264sens2566my7.png
- http://img101.imageshack.us/img101/5821/x264sens25165eb6.png
- http://img237.imageshack.us/img237/9567/x264sens25358un8.png
- http://img266.imageshack.us/img266/4593/x264sens25490ad9.png
- http://img101.imageshack.us/img101/6435/x264sens25615rr8.png

Sensitivity 20:
- http://img293.imageshack.us/img293/9208/x264sens2066za7.png
- http://img293.imageshack.us/img293/6128/x264sens20165hv3.png
- http://img293.imageshack.us/img293/7093/x264sens20358hq7.png
- http://img207.imageshack.us/img207/346/x264sens20490ko7.png
- http://img207.imageshack.us/img207/1899/x264sens20615dl4.png

Dark Shikari
20th January 2008, 03:52
Interesting results, but it seems there aren't any particularly flat scenes there to test with, as there with some of the earlier sources (where the static sensitivity method did much better).

DeathTheSheep
20th January 2008, 03:57
A few comments:
- Perhaps the true power of the AQRC is realized with QP encoding. When it's the only QP-altering force at the helm (rather than x264's rate control, which is shifting quantizers one way as the AQ shifts them the other).
- How about trying 25 instead of either 20 (too low) or 30 (too high, IMHO, at least for my sources/bitrates)?

Dark Shikari
20th January 2008, 03:58
A few comments:
- Perhaps the true power of the AQRC is realized with QP encoding. When it's the only QP-altering force at the helm (rather than x264's rate control, which is shifting quantizers one way as the AQ shifts them the other)The issue here is that QP mode doesn't take account into things like the fact that higher QPs can be used right before an I-frame.

DeathTheSheep
20th January 2008, 04:14
I'm not so sure that's such a good idea in the first place, especially when you have a lot of I-frames packed into the same place. I frame "flickering" and such become prevalent and distracting...I always hated that in videos. Always. Wonder why I use QP? :p And in anime, such a mechanism is inherently worthless, since any such QP hike is easily noticeable on the static backgrounds/choppy cartoon animations. The purpose of keyframe boost is to raise the quality on I-frames in proportion to surrounding frames (p and especially b), and with your patch, the "smearing" and detail loss (in the form of blurring, sometimes) doesn't occur anyway.
Egh. Maybe that's a rant or something, psht.

akupenguin
20th January 2008, 04:25
I'm not so sure that's such a good idea in the first place, especially when you have a lot of I-frames packed into the same place.
Then don't pack lots of I-frames in the same place. There can't be any I-frame flicker if I-frames appear only at scenecuts.
And in anime, such a mechanism is inherently worthless, since any such QP hike is easily noticeable on the static backgrounds/choppy cartoon animations.
Increasing the QP just before an I-frame does absolutely nothing to static backgrounds, because the background has no changes to code at any QP.

desta
20th January 2008, 04:33
Interesting results, but it seems there aren't any particularly flat scenes there to test with, as there with some of the earlier sources (where the static sensitivity method did much better).
Well that was primarily why I was testing really. As Sharktooth said earlier, your new AQ really does look like a good reason to completely ditch CQM's. I just wanted to see which sensitivity would yield better overall results, as people aren't typically always going to be encoding from flat/soft/low-detail sources.

- How about trying 25 instead of either 20 (too low) or 30 (too high, IMHO, at least for my sources/bitrates)?
Added test results.

Sharktooth
20th January 2008, 04:40
auto still looks better to me.

Dark Shikari
20th January 2008, 04:41
The question is, how can we get the benefit of auto while still having good quality in scenes like the two sample images I posted earlier, and the animated GIF?

I could have the ability for auto to limit how low its threshold goes, so as to add more bits to problematic frames... then we could ideally get the best of both worlds.

DeathTheSheep
20th January 2008, 04:42
Oops, I meant non-static backgrounds. :) To elaborate, imagine a clip in which the screen is moving (shaking, panning, etc) wherein an animated character does something or an exaggerated motion occurs that triggers a keyframe without actually undergoing a scene change. This is bad enough, but when some aspect of the frame is changing (like the characters move around on the static background), the moving part visibly loses quality against the static, unchanging background. This is an eyesore of sorts if it happens enough, which it seems invariably to do in some anime. And here tweaking the scene change threshold just makes things worse, extending other GOPs to huge numbers of frames and missing actual scene cuts.
If this is an inaccessible example, consider in crf mode when there is an extended sequence of fast but uniform motion (more than even that in CruNcher's clip), and keyframes are inserted during the motion...

Then don't pack lots of I-frames in the same place.
And though that's the goal, those --keyint 9999 users out there are pretty rare, and for good reason.

Sharktooth
20th January 2008, 04:42
The question is, how can we get the benefit of auto while still having good quality in scenes like the two sample images I posted earlier, and the animated GIF?

I could have the ability for auto to limit how low its threshold goes, so as to add more bits to problematic frames... then we could ideally get the best of both worlds.
dark masking?

lexor
20th January 2008, 05:07
auto still looks better to me.

Actually it seems form desta's images that auto does the best at foreground (faces, etc) and 30 does the best with the backgrounds. Now, if only we could get both... compute the difference between auto calculated value and 30, and bump auto value by half that distance towards 30 (unless it's already higher)? Talking nonsense here.

Of course the problem with those test images is the relatively high bitrate (4mbps), this new AQ would be put to a better test if it was a bit more bitrate starved.

CruNcher
20th January 2008, 05:10
People should note that the OLd AQ results http://forum.doom9.org/showthread.php?p=1089048#post1089048 look more visual pleasing in a ABR encode @ 3 Mbit to the eyes (fine edges) then those of the New AQ, as it looses quality there (hard edges) and makes it less watchable (limited or non limited makes no difference).
New AQ in ABR looks almost exactly the same (hard edges) as like in the CRF Encode Shoots (CRF 25).

DeathTheSheep
20th January 2008, 05:10
Actually, that might not really be nonsense... I was thinking...

How low can you go, with dark [Shikari] masking.

A best of both worlds approach figuring in automatic sensitivity while still freely giving more bits to frames that need them most, according to the threshold.

Maybe this can be combined with the two-pass mode you mentioned earlier--the first pass would decide on what general threshold to apply to the video in its entirety, and the 2nd pass can apply that threshold in general, yet have the freedom to shuffle bits around elsewhere as well, as would best benefit the video according to the established global threshold and whatever incidental threshold may prove favorable (predictive weighted average)?

DeathTheSheep
20th January 2008, 05:23
Wait a minute. When exactly did you address this variance == 0 bug? After 0.431?

Dark Shikari
20th January 2008, 05:43
Wait a minute. When exactly did you address this variance == 0 bug? After 0.431?0.45. I have no idea what happened beforehand, but now if variance is zero, it just uses the QP from the previous MB.

DeathTheSheep
20th January 2008, 05:47
Simply baffling. I suppose this is testament to the axiom that SSIM, bitrate and such are truly fickle entities indeed.

Dark Shikari
20th January 2008, 07:25
News of the day: Not using AQ on your first pass is extremely bad, especially if you're not using adaptive sensitivity.

In other news, water does seem to be somewhat wet and slippery.

Razorholt
20th January 2008, 08:50
Would it make sense to use AQ auto in first pass and AQ + sensitivity 30 in second pass - or the other way around?

Dark Shikari
20th January 2008, 08:59
Would it make sense to use AQ auto in first pass and AQ + sensitivity 30 in second pass - or the other way around?No, same sensitivity in both passes.

Also, I'd say 30 is probably too high, after some testing.

Maybe 20 or 25.

Dark Shikari
20th January 2008, 09:20
Conclusion after encoding a few dozen scenes of The Matrix: static sensitivity 20-25 or so is nearly always better than automatic, but one needs to use the same sensitivity on both passes.

ChronoCross
20th January 2008, 09:43
News of the day: Not using AQ on your first pass is extremely bad, especially if you're not using adaptive sensitivity.

In other news, water does seem to be somewhat wet and slippery.

I think the way you put it in the chat room was better.

<Dark_Shikari> also, it turns out
<Dark_Shikari> not using AQ on your first pass
<Dark_Shikari> = EPICFAILURE
<Dark_Shikari> don't do it
<Dark_Shikari> or you might be the next one to join the epic failtrain

Dreassica
20th January 2008, 13:30
Is there a way it doesn't work in megui?
I copied the patched x264.exe in the tolls dir, but it wont render anything but if I use the usual settings that do work with standard x264.exe.


This is log from megui:

[Information] Log for job1
-[NoImage] Job type: video
-[Information] [20-1-2008 13:30:29] Started handling job
-[Information] [20-1-2008 13:30:32] Preprocessing
-[NoImage] Job commandline: "G:\Program Files\megui\tools\x264\x264.exe" --qp 16 --ref 16 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,1 --subme 7 --trellis 2 --analyse all --8x8dct --me esa --threads auto --thread-input --sar 1:1 --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\sample.mkv" "D:\\sample.avs" --aq-strength=0.7
-[Information] [20-1-2008 13:30:34] Encoding started
-[Information] [20-1-2008 13:30:34] Job completed


Maybe I'm just stupid and missing something really obvious.

desta
20th January 2008, 13:31
Conclusion after encoding a few dozen scenes of The Matrix: static sensitivity 20-25 or so is nearly always better than automatic, but one needs to use the same sensitivity on both passes.
Just as a fyi, those tests I did earlier used the same AQ strength & sensitivity in both passes, and the tests with static sensitivity actually had the quants go up in the 2nd pass.

fields_g
20th January 2008, 14:01
Dreassica,

"--aq-strength=0.7" is wrong.
Get rid of the "=" sign.
"--aq-strength 0.7" is your answer.

Dreassica
20th January 2008, 14:10
Dreassica,

"--aq-strength=0.7" is wrong.
Get rid of the "=" sign.
"--aq-strength 0.7" is your answer.

Tried it, still doesn't start rendering. It's certainly not aq causing this, If I take it out from custom commandline in Megui, it still does this.

Dark Shikari
20th January 2008, 14:31
Just as a fyi, those tests I did earlier used the same AQ strength & sensitivity in both passes, and the tests with static sensitivity actually had the quants go up in the 2nd pass.This is because x264 measures only framewide quantizers, not block quantizers, and as such with adaptive quantization the quantizer readings can be extremely misleading.

With AQ, the average quantizers will generally be much lower than x264 displays.

fields_g
20th January 2008, 14:34
Dreassica,

Try using RAWAVC as your container. Also let us know if there is something weird in your "Standard Error Stream" in your log.

salehin
20th January 2008, 14:39
Here is a set comparisons.. I hope this helps. The source was BBC HD, Power of the Planet h264 1080p- encoded around 480 frames in 720p. Please note that the clip doesn't contain too many dark sequences.

If possible, please advise other possible ways of improving it.

aq_test with DarkShikari's v.45 aq 0.9, sens 20:

Avs Scipt

SetMemoryMax(128)
clip1=dgdecode_mpeg2source("J:\temp\crf test.d2v",info=3).ColorMatrix(hints=true,interlaced=true).tfm(order=0).tdecimate(hybrid=1).ConverttoYV12()

a=clip1.crop(2, 8, -2, -2)
clip2=DeGrainMedian(a, limitY=2,limitUV=3,mode=1)

GrainOptimizer(clip2).LimitedSharpenFaster(Strength=150).Spline36Resize(1280, 720)


Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 1 --bitrate 4800 --stats "K:\planet test\aq_test.stats" --bframes 3 --b-pyramid --direct auto --filter -3,-3 --subme 1 --analyse none --me dia --threads auto --thread-input --sar 1:1 --cqmfile "C:\Program Files\megui\Custom Matrices\Prestige.cfg" --progress --no-dct-decimate --output NUL "K:\planet test\aq_test.avs" --aq-strength 0.9 --aq-sensitivity 20
--[Information] [1/20/2008 11:52:46 AM] Encoding started
Standard output stream
Standard error stream
avis [info]: 1280x720 @ 25.00 fps (489 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
x264 [info]: slice I:6 Avg QP:21.33 size:123021 PSNR Mean Y:46.63 U:48.01 V:49.72 Avg:47.22 Global:46.50
x264 [info]: slice P:265 Avg QP:25.22 size: 35899 PSNR Mean Y:40.57 U:46.54 V:47.86 Avg:41.86 Global:41.33
x264 [info]: slice B:218 Avg QP:26.15 size: 11726 PSNR Mean Y:40.46 U:45.89 V:47.69 Avg:41.70 Global:41.32
x264 [info]: mb I I16..4: 36.1% 0.0% 63.9%
x264 [info]: mb P I16..4: 28.7% 0.0% 0.0% P16..4: 62.6% 0.0% 0.0% 0.0% 0.0% skip: 8.7%
x264 [info]: mb B I16..4: 3.4% 0.0% 0.0% B16..8: 24.3% 0.0% 0.0% direct:35.0% skip:37.3%
x264 [info]: final ratefactor: 25.61
x264 [info]: direct mvs spatial:79.8% temporal:20.2%
x264 [info]: SSIM Mean Y:0.9598064
x264 [info]: PSNR Mean Y:40.597 U:46.270 V:47.804 Avg:41.855 Global:41.367 kb/s:5238.34
encoded 489 frames, 1.18 fps, 5238.76 kb/s

-[Information] Log for job36 (video, aq_test.avs -> aq_test with DS v.45 aq 0.9, sens 20.mkv)
--[Information] [1/20/2008 11:59:52 AM] Started handling job
--[Information] [1/20/2008 11:59:52 AM] Preprocessing
Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 2 --bitrate 4800 --stats "K:\planet test\aq_test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -3,-3 --subme 7 --trellis 1 --analyse all --8x8dct --me umh --threads auto --thread-input --sar 1:1 --cqmfile "C:\Program Files\megui\Custom Matrices\Prestige.cfg" --progress --no-dct-decimate --output "K:\planet test\aq_test with DS v.45 aq 0.9, sens 20.mkv" "K:\planet test\aq_test.avs" --aq-strength 0.9 --aq-sensitivity 20
--[Information] [1/20/2008 11:59:59 AM] Encoding started
Standard output stream
Standard error stream
avis [info]: 1280x720 @ 25.00 fps (489 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
x264 [info]: slice I:6 Avg QP:20.33 size:106716 PSNR Mean Y:46.13 U:48.86 V:50.38 Avg:46.97 Global:46.73
x264 [info]: slice P:265 Avg QP:22.55 size: 34281 PSNR Mean Y:41.89 U:47.37 V:48.66 Avg:43.13 Global:42.73
x264 [info]: slice B:218 Avg QP:23.57 size: 9752 PSNR Mean Y:41.32 U:46.51 V:48.34 Avg:42.54 Global:42.35
x264 [info]: mb I I16..4: 1.8% 93.3% 4.9%
x264 [info]: mb P I16..4: 0.1% 5.2% 0.2% P16..4: 54.9% 24.2% 9.7% 0.9% 0.5% skip: 4.2%
x264 [info]: mb B I16..4: 0.0% 0.4% 0.0% B16..8: 36.8% 1.2% 3.8% direct: 2.9% skip:54.9%
x264 [info]: 8x8 transform intra:93.5% inter:82.6%
x264 [info]: direct mvs spatial:80.3% temporal:19.7%
x264 [info]: ref P 69.0% 15.2% 7.6% 4.9% 3.4%
x264 [info]: ref B 83.4% 10.2% 2.9% 2.2% 1.3%
x264 [info]: SSIM Mean Y:0.9635958
x264 [info]: PSNR Mean Y:41.687 U:47.005 V:48.535 Avg:42.914 Global:42.586 kb/s:4846.87
encoded 489 frames, 0.41 fps, 4847.54 kb/s
--[Information] Final statistics
Desired video bitrate: 4800 kbit/s
Obtained video bitrate (approximate: 4850 kbit/s

oooooooooooooooo

aq_test with x264 r719 SVN aq 0.3.mkv

Applied patches (by techhouse):
x264_2pass_vbv.diff
x264_aq-brdo.diff
x264_faster-dia.diff
x264_fp-eta.01.r680.diff
x264_hrd_pulldown.diff
x264_me-prepass.diff
x264_satd_fpel.11.diff
x264_thread_pool.r680.diff

** note: it's not a cef built- i named the file incrrectly**
Avs Scipt: same

Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 1 --bitrate 4800 --stats "K:\planet test\aq_test.stats" --bframes 3 --b-pyramid --direct auto --filter -3,-3 --subme 1 --analyse none --me dia --threads auto --thread-input --sar 1:1 --cqmfile "C:\Program Files\megui\Custom Matrices\Prestige.cfg" --progress --no-dct-decimate --output NUL "K:\planet test\aq_test.avs" --aq-strength 0.3

--[Information] [1/20/2008 12:25:38 PM] Encoding started
Standard output stream
Standard error stream
avis [info]: 1280x720 @ 25.00 fps (489 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
x264 [info]: slice I:6 Avg QP:21.00 size:121756 PSNR Mean Y:46.78 U:48.09 V:49.83 Avg:47.35 Global:46.77
x264 [info]: slice P:265 Avg QP:24.60 size: 36377 PSNR Mean Y:40.74 U:46.70 V:48.04 Avg:42.03 Global:41.48
x264 [info]: slice B:218 Avg QP:25.56 size: 12186 PSNR Mean Y:40.62 U:46.02 V:47.85 Avg:41.86 Global:41.50
x264 [info]: mb I I16..4: 39.0% 0.0% 61.0%
x264 [info]: mb P I16..4: 27.2% 0.0% 0.0% P16..4: 62.2% 0.0% 0.0% 0.0% 0.0% skip:10.6%
x264 [info]: mb B I16..4: 3.1% 0.0% 0.0% B16..8: 23.9% 0.0% 0.0% direct:36.1% skip:36.9%
x264 [info]: final ratefactor: 25.19
x264 [info]: direct mvs spatial:80.7% temporal:19.3%
x264 [info]: SSIM Mean Y:0.9606545
x264 [info]: PSNR Mean Y:40.765 U:46.414 V:47.976 Avg:42.022 Global:41.523 kb/s:5327.96
encoded 489 frames, 1.22 fps, 5328.38 kb/s
[1/20/2008 12:32:40 PM] Job completed
--[Information] [1/20/2008 12:32:40 PM] Postprocessing
---[Information] Deleting intermediate files
-[Information] Log for job37 (video, aq_test.avs -> aq_test with cef709 aq 0.3.mkv)
--[Information] [1/20/2008 12:32:40 PM] Started handling job
--[Information] [1/20/2008 12:32:40 PM] Preprocessing

Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 2 --bitrate 4800 --stats "K:\planet test\aq_test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -3,-3 --subme 7 --trellis 1 --analyse all --8x8dct --me umh --threads auto --thread-input --sar 1:1 --cqmfile "C:\Program Files\megui\Custom Matrices\Prestige.cfg" --progress --no-dct-decimate --output "K:\planet test\aq_test with cef709 aq 0.3.mkv" "K:\planet test\aq_test.avs" --aq-strength 0.3
--[Information] [1/20/2008 12:32:56 PM] Encoding started
Standard output stream
Standard error stream
avis [info]: 1280x720 @ 25.00 fps (489 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
x264 [info]: slice I:6 Avg QP:20.17 size:102501 PSNR Mean Y:46.06 U:48.76 V:50.33 Avg:46.90 Global:46.66
x264 [info]: slice P:265 Avg QP:22.35 size: 34217 PSNR Mean Y:41.94 U:47.40 V:48.69 Avg:43.18 Global:42.79
x264 [info]: slice B:218 Avg QP:23.34 size: 9897 PSNR Mean Y:41.38 U:46.53 V:48.35 Avg:42.60 Global:42.40
x264 [info]: mb I I16..4: 2.2% 93.1% 4.6%
x264 [info]: mb P I16..4: 0.1% 5.5% 0.2% P16..4: 53.4% 23.4% 9.9% 1.0% 0.5% skip: 5.9%
x264 [info]: mb B I16..4: 0.0% 0.4% 0.0% B16..8: 35.7% 1.2% 3.9% direct: 3.1% skip:55.8%
x264 [info]: 8x8 transform intra:94.1% inter:83.1%
x264 [info]: direct mvs spatial:83.0% temporal:17.0%
x264 [info]: ref P 68.9% 15.2% 7.6% 4.9% 3.4%
x264 [info]: ref B 83.5% 10.3% 2.8% 2.1% 1.2%
x264 [info]: SSIM Mean Y:0.9639119
[info]: PSNR Mean Y:41.741 U:47.029 V:48.558 Avg:42.964 Global:42.642 kb/s:4842.55
encoded 489 frames, 0.46 fps, 4843.22 kb/s
Final statistics
Desired video bitrate: 4800 kbit/s
Obtained video bitrate (approximate: 4846 kbit/s

Dreassica
20th January 2008, 14:42
Dreassica,

Try using RAWAVC as your container. Also let us know if there is something weird in your "Standard Error Stream" in your log.

No on both counts.

--crf 16 --ref 16 --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,1 --subme 7 --trellis 1 --analyse all --8x8dct --me esa --threads 8 --thread-input --progress --no-psnr --no-ssim --output "output" "input"

Dark Shikari
20th January 2008, 14:43
Unfortunately one cannot tell much from logs in this sort of situation; its really all about how it looks.

Yoshiyuki Blade
20th January 2008, 14:50
Tried it, still doesn't start rendering. It's certainly not aq causing this, If I take it out from custom commandline in Megui, it still does this.

As fields_g stated, select RAWAVC as your container (it will output as a .264 file). Also make sure pthreadGC2.dll is in the same folder as Dark's build of x264. If its not there, copy pthreadGC2.dll from the "ffmpeg" folder in the MeGUI directory, which is also in the "tools" folder like "x264"

I've been running a few anime encodes with the v0.45 and it seems great. At the same settings and bitrate with --aq-strength 1.0 --aq-sesitivity 25, there was much less blockyness with v0.45, although it seems to come at the cost of overall quality. On v0.44 the scenes look nicer, but it looks rather nasty in dark areas. Simply swapping out build v0.44 with v0.45 with the exact same configuration reduced blockyness by a lot, but the overall crispness is lost. Perhaps it needs more bitrate.

Dreassica
20th January 2008, 14:55
Pasting the commandline in a dos prompt gives me the error that pthreadGC2.dll can't be found. Odd as standard x264 seems to work fine.

EDIT

Downloaded said dll and put it in same dir as x264.exe, NOW it works finally.

Dark Shikari
20th January 2008, 14:56
Pasting the commandline in a dos prompt gives me the error that pthreadGC2.dll can't be found. Odd as standard x264 seems to work fine.That's because incompetent me can't seem to get a static build working. Get PthreadGC2.dll here (http://mirror05.x264.nl/Dark/force.php?file=./pthreadGC2.dll).

Dreassica
20th January 2008, 15:16
That's because incompetent me can't seem to get a static build working. Get PthreadGC2.dll here (http://mirror05.x264.nl/Dark/force.php?file=./pthreadGC2.dll).

Yea found that out on my own. Rendering finally now. :)

Yoshiyuki Blade
20th January 2008, 15:17
EDIT

Downloaded said dll and put it in same dir as x264.exe, NOW it works finally.

Great, now double-check and make sure RAWAVC is the output container instead of mp4, or itll error on the 2nd pass. I wasted several hours of testing by not doublechecking :D.

salehin
20th January 2008, 15:19
Unfortunately one cannot tell much from logs in this sort of situation; its really all about how it looks.
Here they are
With DarkShikari's v.45: aq_test with DS v.45 aq 0.9, sens 20.mkv (http://www.sendspace.com/file/vdx2rn)
*Without DarkShikari: aq_test with x264 r719 SVN aq 0.3.mkv (http://www.sendspace.com/file/rke9ij)

*Applied patches in the svn:


x264_2pass_vbv.diff
x264_aq-brdo.diff
x264_faster-dia.diff
x264_fp-eta.01.r680.diff
x264_hrd_pulldown.diff
x264_me-prepass.diff
x264_satd_fpel.11.diff
x264_thread_pool.r680.diff

Dark Shikari
20th January 2008, 15:24
Here they are
With DarkShikari's v.45: aq_test with DS v.45 aq 0.9, sens 20.mkv (http://www.sendspace.com/file/vdx2rn)
*Without DarkShikari: aq_test with x264 r719 SVN aq 0.3.mkv (http://www.sendspace.com/file/rke9ij)

*Applied patches in the svn:Looking at the quantizer distribution its clear neither of these actually used my AQ. This isn't surprising, given that "x264_aq-brdo.diff" isn't my patch, its Haali's.

salehin
20th January 2008, 15:26
eek!! i'll remove the svn from the x264 folder and post new link later

Update: Here is the proper one. Only your v.45 is in the MeGUIs x264 folder- renamed as x264.exe

(Proper) aq_test with DS v.45 aq 0.9, sens 20.mkv (http://www.sendspace.com/file/jyavnu)

Log

Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 1 --bitrate 4800 --stats "K:\planet test\aq_test.stats" --bframes 3 --b-pyramid --direct auto --filter -3,-3 --subme 1 --analyse none --me dia --threads auto --thread-input --sar 1:1 --cqmfile "C:\Program Files\megui\Custom Matrices\Prestige.cfg" --progress --no-dct-decimate --output NUL "K:\planet test\aq_test.avs" --aq-strength 0.9 --aq-sensitivity 20

[1/20/2008 2:37:33 PM] Encoding started
avis [info]: 1280x720 @ 25.00 fps (489 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
x264 [info]: slice I:6 Avg QP:24.83 size:118889 PSNR Mean Y:45.43 U:47.34 V:49.11 Avg:46.12 Global:44.64
x264 [info]: slice P:270 Avg QP:29.72 size: 35574 PSNR Mean Y:39.36 U:45.62 V:46.89 Avg:40.67 Global:39.95
x264 [info]: slice B:213 Avg QP:30.15 size: 11561 PSNR Mean Y:39.46 U:44.99 V:46.81 Avg:40.70 Global:39.96
x264 [info]: mb I I16..4: 29.2% 0.0% 70.8%
x264 [info]: mb P I16..4: 29.3% 0.0% 0.0% P16..4: 62.8% 0.0% 0.0% 0.0% 0.0% skip: 7.9%
x264 [info]: mb B I16..4: 3.9% 0.0% 0.0% B16..8: 25.1% 0.0% 0.0% direct:33.7% skip:37.3%
x264 [info]: final ratefactor: 29.88
x264 [info]: direct mvs spatial:77.9% temporal:22.1%
x264 [info]: SSIM Mean Y:0.9580282
x264 [info]: PSNR Mean Y:39.480 U:45.366 V:46.880 Avg:40.751 Global:39.990 kb/s:5227.32
encoded 489 frames, 1.36 fps, 5226.36 kb/s
[1/20/2008 2:43:45 PM] Job completed


Log for job41 (video, aq_test.avs -> aq_test with DS v.45 aq 0.9, sens 20.mkv)
[1/20/2008 2:43:47 PM] Started handling job
[1/20/2008 2:43:47 PM] Preprocessing
Job commandline: "C:\Program Files\megui\tools\x264\x264.exe" --pass 2 --bitrate 4800 --stats "K:\planet test\aq_test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 3 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -3,-3 --subme 7 --trellis 1 --analyse all --8x8dct --me umh --threads auto --thread-input --sar 1:1 --cqmfile "C:\Program Files\megui\Custom Matrices\Prestige.cfg" --progress --no-dct-decimate --output "K:\planet test\aq_test with DS v.45 aq 0.9, sens 20.mkv" "K:\planet test\aq_test.avs" --aq-strength 0.9 --aq-sensitivity 20
[1/20/2008 2:44:01 PM] Encoding started
avis [info]: 1280x720 @ 25.00 fps (489 frames)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 Cache64
x264 [info]: slice I:6 Avg QP:23.67 size:105442 PSNR Mean Y:45.08 U:48.10 V:49.69 Avg:45.97 Global:45.58
x264 [info]: slice P:270 Avg QP:26.36 size: 33967 PSNR Mean Y:40.84 U:46.60 V:47.84 Avg:42.10 Global:41.61
x264 [info]: slice B:213 Avg QP:27.25 size: 9408 PSNR Mean Y:40.55 U:45.73 V:47.58 Avg:41.77 Global:41.46
x264 [info]: mb I I16..4: 1.9% 94.1% 4.0%
x264 [info]: mb P I16..4: 0.2% 6.6% 0.1% P16..4: 50.8% 28.8% 9.2% 0.3% 0.1% skip: 3.8%
x264 [info]: mb B I16..4: 0.0% 0.4% 0.0% B16..8: 39.1% 1.2% 4.0% direct: 3.1% skip:52.2%
x264 [info]: 8x8 transform intra:95.5% inter:81.8%
x264 [info]: direct mvs spatial:71.4% temporal:28.6%
x264 [info]: ref P 70.1% 14.4% 7.4% 4.8% 3.4%
x264 [info]: ref B 82.8% 10.4% 3.1% 2.3% 1.5%
x264 [info]: SSIM Mean Y:0.9639779
x264 [info]: PSNR Mean Y:40.766 U:46.242 V:47.749 Avg:42.004 Global:41.577 kb/s:4829.26
encoded 489 frames, 0.44 fps, 4828.30 kb/s
Final statistics
Desired video bitrate: 4800 kbit/s
Obtained video bitrate (approximate: 4831 kbit/s

DeathTheSheep
20th January 2008, 16:33
So you're saying you managed to successfully apply the satd and me-prepass patches to svn-r719? A lot of the variables in me.c don't even refer to the quite same things anymore... casting pointers into ints and whatnot, that's what the compiler complains of.

In other news, water does seem to be somewhat wet and slippery.
No, I don't believe you. (Why would you say that of all things?) This doesn't have anything to do with an "epic failtrain," does it? :p

Sagekilla
20th January 2008, 16:42
Well now, this is quite an oddity.. I tested --aq-strength 1 with sensitivity of 0 and 20 on a randomly picked scene. (Which also happened to be a very dark, flat scene.) To my surprise.. Sensitivity of 20 more than doubled the bitrate used under crf mode! And this is with the 0.45 AQ.

G:\Movies\Battlestar Galactica>x264_aq --keyint 1000 --crf 18 --ref 6 --mixed-re
fs --no-fast-pskip --bframes 16 --bime --weightb --b-pyramid --b-rdo --8x8dct --
subme 7 --me umh --trellis 1 --aq-strength 1 --aq-sensitivity 20 --threads auto
--thread-input --progress --output "video.264" --pass 1 --stats "stats.log" "sou
rce.avs"
avis [info]: 864x480 @ 23.98 fps (720 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 3DNow!
x264 [info]: slice I:3 Avg QP:14.67 size: 61484 PSNR Mean Y:50.83 U:52.00
V:53.61 Avg:51.36 Global:51.34
x264 [info]: slice P:351 Avg QP:17.30 size: 28768 PSNR Mean Y:49.25 U:50.53
V:52.70 Avg:49.85 Global:49.71
x264 [info]: slice B:366 Avg QP:19.85 size: 9467 PSNR Mean Y:47.62 U:49.54
V:51.63 Avg:48.35 Global:48.30
x264 [info]: mb I I16..4: 21.5% 69.4% 9.1%
x264 [info]: mb P I16..4: 4.7% 12.5% 2.2% P16..4: 35.2% 27.5% 17.0% 0.0% 0
.0% skip: 0.9%
x264 [info]: mb B I16..4: 0.5% 1.4% 0.5% B16..8: 33.8% 3.8% 10.1% direct:
5.8% skip:44.1%
x264 [info]: 8x8 transform intra:64.0% inter:47.0%
x264 [info]: ref P 73.5% 12.2% 5.6% 3.5% 2.6% 2.6%
x264 [info]: ref B 88.4% 7.9% 1.6% 0.9% 0.6% 0.6%
x264 [info]: SSIM Mean Y:0.9915203
x264 [info]: PSNR Mean Y:48.426 U:50.032 V:52.158 Avg:49.096 Global:48.940 kb/s:
3662.19

encoded 720 frames, 2.98 fps, 3661.24 kb/s


G:\Movies\Battlestar Galactica>x264_aq --keyint 1000 --crf 18 --ref 6 --mixed-re
fs --no-fast-pskip --bframes 16 --bime --weightb --b-pyramid --b-rdo --8x8dct --
subme 7 --me umh --trellis 1 --aq-strength 1 --aq-sensitivity 0 --threads auto -
-thread-input --progress --output "video2.264" --pass 1 --stats "stats.log" "sou
rce.avs"
avis [info]: 864x480 @ 23.98 fps (720 frames)
x264 [info]: using cpu capabilities: MMX MMXEXT SSE SSE2 SSE3 3DNow!
x264 [info]: slice I:3 Avg QP:14.67 size: 45170 PSNR Mean Y:49.09 U:50.68
V:52.41 Avg:49.72 Global:49.71
x264 [info]: slice P:351 Avg QP:17.30 size: 11716 PSNR Mean Y:47.14 U:49.62
V:51.95 Avg:47.99 Global:47.89
x264 [info]: slice B:366 Avg QP:19.85 size: 3612 PSNR Mean Y:45.90 U:48.76
V:50.91 Avg:46.80 Global:46.77
x264 [info]: mb I I16..4: 23.0% 68.2% 8.8%
x264 [info]: mb P I16..4: 5.4% 7.5% 2.4% P16..4: 51.1% 20.4% 7.8% 0.0% 0
.0% skip: 5.4%
x264 [info]: mb B I16..4: 0.5% 0.9% 0.2% B16..8: 39.4% 1.4% 3.3% direct:
1.8% skip:52.5%
x264 [info]: 8x8 transform intra:50.6% inter:65.8%
x264 [info]: ref P 69.6% 14.1% 6.9% 3.8% 2.9% 2.7%
x264 [info]: ref B 86.2% 8.4% 2.4% 1.2% 1.0% 0.8%
x264 [info]: SSIM Mean Y:0.9874901
x264 [info]: PSNR Mean Y:46.517 U:49.186 V:51.424 Avg:47.392 Global:47.289 kb/s:
1483.77

encoded 720 frames, 3.59 fps, 1482.82 kb/s


Any thoughts on this?

salehin
20th January 2008, 16:43
So you're saying you managed to successfully apply the satd and me-prepass patches to svn-r719? A lot of the variables in me.c don't even refer to the quite same things anymore... casting pointers into ints and whatnot, that's what the compiler complains of.


No, I don't believe you. (Why would you say that of all things?) :p

Sorry DtS- i don't know what or how to reply, as I'm no expert into this sort of thing. I just got that patched svn from another place. The accompanied text file mentions that those patches are included in that x264

techhouse's x264 (http://x264.tk/)

DeathTheSheep
20th January 2008, 17:20
Ah, I see. Then would you direct me to this place of mystery? :) Thanks.

Sagekilla: Yes, raising the sensitivity tends to increase the bitrate quite a bit. Try sensitivity 30 and be shocked. (EPICSHOCK.) Lowering the sensitivity (not 0, that's automatic I presume) can actually lower the bitrate significantly, because the threshold acts like a centerpoint--if the input is above this centerpoint, it gets more data data, and vise versa.

bob0r
20th January 2008, 17:38
If you want to test AQ, i guess you should use x264 svn + AQ patch only:
x264.721.dark.aq.rdrc.0.45.exe (http://files.x264.nl/AQ/x264.721.dark.aq.rdrc.0.45.exe) (pthreads/mp4 = yes, made with make fprofiled)

DeathTheSheep
20th January 2008, 20:17
You said it, bob0r. Most of the patches in that build were broken anyway--even calling fpel cmp satd would crash the program, and for good reason. The old patch would increment the memory addresses of some of the variables in the new revision, obviously leading to a crash even before the first keyframe could be composed. And it's no easy fix--the mechanism has been practically redesigned, not to mention rearranged. I'm here with 681, custom built, march core2, cyborg gcc 4.3 [rev6], custom makefprofiled commandlines, tweaked function grouping priorities, etc, which is still pretty darn fast and works like a charm, save for a non-determinism or two with MT.

But like I've indicated earlier, combining some of the old patches with this new AQ leads to a synergistic effect, yielding a significantly greater quality increase than any used separately.

Sagekilla
20th January 2008, 22:53
Dark Shikari, I can confirm that your new AQ does work well on animated video. I'm encoding Dark Fury right now and the quality is very good across clip so far.

Atak_Snajpera
20th January 2008, 23:07
I've converted 1080p anime and it looks very well. Size even lower than without AQ

jvt40
20th January 2008, 23:28
Both clips looked fine (watching from bit outdated 17" screen :p) , difference running them side by side was Proper DS was slightly warmer (more red!!) , could be seen on jungle colour , sky and mountains , which is more truthful is matter of opinion .

eek!! i'll remove the svn from the x264 folder and post new link later

Update: Here is the proper one. Only your v.45 is in the MeGUIs x264 folder- renamed as x264.exe

(Proper) aq_test with DS v.45 aq 0.9, sens 20.mkv (http://www.sendspace.com/file/jyavnu)

Dark Shikari
21st January 2008, 04:07
Well now, this is quite an oddity.. I tested --aq-strength 1 with sensitivity of 0 and 20 on a randomly picked scene. (Which also happened to be a very dark, flat scene.) To my surprise.. Sensitivity of 20 more than doubled the bitrate used under crf mode! And this is with the 0.45 AQ.This is why I said nonzero sensitivity isn't for CRF mode, because it doesn't maintain bitrate.

Morte66
22nd January 2008, 01:21
This is why I said nonzero sensitivity isn't for CRF mode, because it doesn't maintain bitrate.

But that's much the same as the AQ in the current x264 mainstream build. --crf 18 --aq-strength 0.3 --aq-sensitivity 5 gives something like double the bitrate of plain crf 18, depending on the material, and it's about enough to control blocking/banding on a revealing display.

Nobody knows what bitrate is going to come out of crf anyhow, that's the point of it. You encode to a target quality, not a target size. AQ changes the definition of "quality" from PSNR to something more concerned with subjective perception of smooth areas.

CruNcher
22nd January 2008, 01:40
Exactly that's also my fear as my tests show exactly this behaveiour, no doub't it has much better detail preservation @ low bitrates, but it comes at a price for more unbalanced quality between the whole final thing. I think exactly that's what the goal should be with a AQ "balanced percepted visual quality". Under normal viewing conditions for this lossy stuff it's hard anyway to see slight blocking in motion, but banding is imidietly visible under any viewing condition. Sure you can try to use a Higher Compression factor but this again comes most of the times @ a heavy speed loss (Encoding/Decoding).

Dark Shikari
22nd January 2008, 01:45
Exactly that's also my fear as my tests show exactly this behaveiour, no doub't it has much better detail preservation @ low bitrates, but it comes at a price for more unbalanced quality between the whole final thing. I think exactly that's what the goal should be with a AQ "balanced percepted visual quality". Under normal viewing conditions for this lossy stuff it's hard anyway to see slight blocking in motion, but banding is imidietly visible under any viewing condition. Sure you can try to use a Higher Compression factor but this again comes most of the times @ a heavy speed loss (Encoding/Decoding).Wait, you're criticizing my AQ for banding, even though banding is what it handles far better than any previous algorithm?

:rolleyes:

This is getting pointlessly retarded. Its as if you ran out of valid criticisms for my AQ (of which there are many), because your comments seem just completely nonsensical. Lowering quantizers in areas with banding problems is not going to make the banding worse!

CruNcher
22nd January 2008, 01:49
http://forum.doom9.org/showthread.php?p=1089048#post1089048

look @ those scenes in the samples you've got and tell me what your New AQ does wrong then :)
Possible that i visually interpreting those results wrong as banding but they look like as this for me because of the fine gradients of noise that suddenly become blocks

Dark Shikari
22nd January 2008, 01:53
http://forum.doom9.org/showthread.php?p=1089048#post1089048

look @ those scenes in the samples you've got and tell me what your New AQ does wrong then :)
Possible that i visually interpreting those results wrong as banding but they look like as this for me because of the fine gradients of noise that suddenly become blocksThis is because you used automatic sensitivity, which intentionally doesn't allow bits to be moved between frames. That means if a frame does genuinely need more bits, and you can't just move the bits from elsewhere in the frame to solve the quality problem, it won't get fixed.

Use a static sensitivity with twopass, and try again.

CruNcher
22nd January 2008, 01:56
But i don't want to use twopass, because of the Speed reason i mentioned above :) and i tried every combination possible with ABR and your New AQ and it results allways in those Visual results it doesn't change for this particular spots (and those are really nasty they ruin the complete visual interpretation of this subjectively).
Psy optimizations aren't about Metrics or that you have super duper detail preservation in one part of the final result, they are about the balanced Look & Feel of the whole thing. That's why im also not really impressed @ the moment with the Parkrun results seeing this now, i mean it should be consistent improvement and not only for 1 spot and such a scene as Parkrun is a very special thing because also it's only 1 part scene and not 1 part of something much bigger around it :)

Dark Shikari
22nd January 2008, 01:57
But i don't want to use twopass, because of the Speed reason i mentioned above :) and i tried every combination possible with ABR and your New AQ and it results allways in those Visual results it doesn't change for this particular spots.Then use CRF with static sensitivity. Just remember the filesize isn't guaranteed to stay where you want it--much like Haali's AQ, of course.

Try sensitivity 20 if you want to try to stay near the original filesize.

Inventive Software
22nd January 2008, 02:25
Remind me... what's RCRD? And how much of a quality benefit does SATD with this patch give over a vanilla build?

Dark Shikari
22nd January 2008, 02:34
Remind me... what's RCRD? And how much of a quality benefit does SATD with this patch give over a vanilla build?RCRD is a ratecontrol method that finds the optimal quantizer distribution, RD-wise.

SATD patch hasn't been updated to be compatible with r717+ yet. Benefit would be the same as before.

Inventive Software
22nd January 2008, 02:38
Thanks for the info clear-up. :)

Dark Shikari
22nd January 2008, 20:15
Link to 1080p sample encoded with the new AQ added to the original post.

DeathTheSheep
22nd January 2008, 21:56
Could you also upload an RCRD sample? I'm sure people would be more eager to try it out if they see some results justifying the speed loss. After all, a lot of work and #x264dev chat has apparently gone into it.

In a few days' time, we'll have working satd too, and with very well-tweaked thresholds, might I add. Or shouldn't I spoil the surprise? One can hope prepass will be tackled by someone, too (heck, if nobody's up to fixing prepass by that time, I'll look into it myself, though whether or not I can possibly whip that on to x264's new code I'm obviously quite unsure about...let's hope the problem isn't too deeply rooted). This will give the community a much better reason to use RDRC--an insane option among insane options--since they'll have others to stack on top of it to mitigate their fears.

Dark Shikari
22nd January 2008, 22:07
Could you also upload an RCRD sample? I'm sure people would be more eager to try it out if they see some results justifying the speed loss. After all, a lot of work and #x264dev chat has apparently gone into it.

In a few days' time, we'll have working satd too, and with very well-tweaked thresholds, might I add. Or shouldn't I spoil the surprise? One can hope prepass will be tackled by someone, too (heck, if nobody's up to fixing prepass by that time, I'll look into it myself, though whether or not I can possibly whip that on to x264's new code I'm obviously quite unsure about...let's hope the problem isn't too deeply rooted). This will give the community a much better reason to use RDRC--an insane option among insane options--since they'll have others to stack on top of it to mitigate their fears.RDRC sample (http://momupload.com/files/69785/touhou.mp4.html) with AQ.

DeathTheSheep
22nd January 2008, 22:19
MOMupload? MOM?! Boy oh boy and by golly george, there's a mom upload now of all things? And what's more, I see you used them over mediafire...ouch. :\

What, a 100MB upload limit to the former?

Dark Shikari
22nd January 2008, 22:19
MOMupload? MOM?! Boy oh boy and by golly george, there's a mom upload now of all things? And what's more, I see you used them over mediafire...ouch. :\

What, a 100MB upload limit to the former?Correct, size limit :p

DeathTheSheep
22nd January 2008, 22:25
Well, 180KB/sec, that's nice. (Results after the break, ladies and gentlemen!) ;)

[edit]
Oh my god, is that you playing the game? I watched it two and half times already... This is MAD SKILLZ to the max. Tell me it's not you playing the game...tell me...

As for RCRD, this seems like an inordinately complex source, and all that rapid motion (and R4D D061NG sk111z) were handled...erm, well. But since I haven't made any such video game encodes before (or even seen many of them), I'm not quite sure what to expect here. Maybe a test run using CruNcher's infamous test clip is in order, eh?

But seriously. This game is 73H 1337 H4x0rz. And the soundtrack kicks Major General Asce.

CruNcher
23rd January 2008, 02:57
Hmm could you guys please post wich of these looks better to you on the first sight and then also your GFX Card, Monitor ,Operating System, Player Application,Decoder and used Renderer as information. Thx in Advance :)

Please don't look framewise just subjectively rate them on the first sight :)

http://mirror05.x264.nl/CruNcher/force.php?file=./1.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./2.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./3.mkv

Btw: If you want to see allways correct representation results in Windows with a Directshow Player use CoreAVC as Decoder :)

Dark Shikari
23rd January 2008, 02:58
Oh my god, is that you playing the game? I watched it two and half times already... This is MAD SKILLZ to the max. Tell me it's not you playing the game...tell me...Hell no, I got that off Stage6 (download link, for your own test encodes--source = 12.5 megabit DivX (http://www.stage6.com/user/SeaWeeze/video/1652397/Touhou-10-Mountain-of-Faith---Extra-Reimu-B)). I can't even beat Perfect Cherry Blossom on Normal difficulty let alone Ludicrous or Extra!

Note that its not as bad as it looks, as the hitbox in the game (like all similar games in the genre) is extremely small.
As for RCRD, this seems like an inordinately complex source, and all that rapid motion (and R4D D061NG sk111z) were handled...erm, well. But since I haven't made any such video game encodes before (or even seen many of them), I'm not quite sure what to expect here. Maybe a test run using CruNcher's infamous test clip is in order, eh?

But seriously. This game is 73H 1337 H4x0rz. And the soundtrack kicks Major General Asce.The game series is called Touhou Project, and you can get it (with the English patch) and play it on non-insane difficulty :p There's 10 games in the series now, Perfect Cherry Blossom is the easiest to find of them.

I love the Touhou clip because its absolute murder for any video encoder.

The main benefit of AQ in that clip is in the backgrounds (which were blurred to death without it). The main benefit of RCRD was more intelligent I/B-frame offset decision.

CruNcher
23rd January 2008, 05:09
Holly Bible :D
http://s3.directupload.net/images/080123/qon9i85i.png <- oldaq
http://s6.directupload.net/images/080123/bzj6hm86.png <- newaq

There is still Darkness also with it (in the real meaning) :) (in motion it can hurt sometimes especialy if it's directly in the ROI as here)
http://s5.directupload.net/images/080123/4fpl8zs5.png <- oldaq
http://s5.directupload.net/images/080123/omgt9f7o.png <- newaq

When this blocking problems gonna get fixed i jump around all night and day (when it's possible to balance it out efficiently without destroying it's purpose (takeing bits from one area allocateing to another)) :D
http://s5.directupload.net/images/080123/hgbfhnta.png <- oldaq
http://s1.directupload.net/images/080123/jnbkrecp.png <- newaq

Let's love it again ;)
http://s5.directupload.net/images/080123/m5aava2r.png <- oldaq
http://s5.directupload.net/images/080123/xh6fd92z.png <- newaq

Old AQ ( --aq-strength 1.0 --aq-sensitivity 15)

x264 [info]: final ratefactor: 32.15
x264 [info]: 8x8 transform intra:19.1% inter:49.5%
x264 [info]: direct mvs spatial:98.5% temporal:1.5%
x264 [info]: ref P 82.8% 11.2% 6.0%
x264 [info]: ref B 91.0% 9.0%
x264 [info]: SSIM Mean Y:0.9780284
x264 [info]: PSNR Mean Y:44.630 U:45.375 V:48.696 Avg:45.190 Global:44.796 kb/s:
2944.99

encoded 9336 frames, 8.66 fps, 2952.16 kb/s

New AQ (--aq-strength 1.0 --aq-sensitivity 20)

x264 [info]: final ratefactor: 34.11
x264 [info]: 8x8 transform intra:20.9% inter:50.6%
x264 [info]: direct mvs spatial:98.2% temporal:1.8%
x264 [info]: ref P 84.3% 10.2% 5.6%
x264 [info]: ref B 91.8% 8.2%
x264 [info]: SSIM Mean Y:0.9787061
x264 [info]: PSNR Mean Y:43.929 U:45.177 V:48.162 Avg:44.574 Global:43.988 kb/s:
3014.86

encoded 9336 frames, 8.53 fps, 3022.31 kb/s

New AQ (--aq-strength 0.5 --aq-sensitivity 20)

x264 [info]: final ratefactor: 30.68
x264 [info]: 8x8 transform intra:18.1% inter:51.1%
x264 [info]: direct mvs spatial:98.7% temporal:1.3%
x264 [info]: ref P 84.3% 10.1% 5.6%
x264 [info]: ref B 93.1% 6.9%
x264 [info]: SSIM Mean Y:0.9802665
x264 [info]: PSNR Mean Y:45.205 U:45.745 V:49.185 Avg:45.725 Global:45.209 kb/s:
3020.03

encoded 9336 frames, 8.72 fps, 3027.55 kb/s

New AQ (--aq-strength 0.1 --aq-sensitivity 20)

x264 [info]: final ratefactor: 28.51
x264 [info]: 8x8 transform intra:15.4% inter:51.4%
x264 [info]: direct mvs spatial:98.9% temporal:1.1%
x264 [info]: ref P 83.8% 10.4% 5.8%
x264 [info]: ref B 93.8% 6.2%
x264 [info]: SSIM Mean Y:0.9804187
x264 [info]: PSNR Mean Y:45.724 U:45.997 V:49.670 Avg:46.197 Global:45.673 kb/s:
3023.08

encoded 9336 frames, 8.72 fps, 3030.47 kb/s

New AQ (--aq-strength 0.1)

x264 [info]: final ratefactor: 28.40
x264 [info]: 8x8 transform intra:15.4% inter:51.4%
x264 [info]: direct mvs spatial:98.9% temporal:1.1%
x264 [info]: ref P 83.8% 10.4% 5.8%
x264 [info]: ref B 93.8% 6.2%
x264 [info]: SSIM Mean Y:0.9804481
x264 [info]: PSNR Mean Y:45.729 U:45.999 V:49.670 Avg:46.200 Global:45.682 kb/s:
3026.98

encoded 9336 frames, 8.67 fps, 3034.38 kb/s

New AQ (--strength 0.5)

x264 [info]: final ratefactor: 28.24
x264 [info]: 8x8 transform intra:16.8% inter:51.3%
x264 [info]: direct mvs spatial:98.7% temporal:1.3%
x264 [info]: ref P 84.1% 10.2% 5.7%
x264 [info]: ref B 93.4% 6.6%
x264 [info]: SSIM Mean Y:0.9804409
x264 [info]: PSNR Mean Y:45.506 U:45.891 V:49.456 Avg:45.998 Global:45.485 kb/s:
3033.67

encoded 9336 frames, 8.64 fps, 3041.13 kb/s

New AQ (--strength 1.0)

x264 [info]: final ratefactor: 28.58
x264 [info]: 8x8 transform intra:19.0% inter:50.8%
x264 [info]: direct mvs spatial:98.5% temporal:1.5%
x264 [info]: ref P 84.2% 10.1% 5.6%
x264 [info]: ref B 92.6% 7.4%
x264 [info]: SSIM Mean Y:0.9798093
x264 [info]: PSNR Mean Y:44.864 U:45.574 V:48.843 Avg:45.410 Global:44.888 kb/s:
3044.61

encoded 9336 frames, 8.60 fps, 3052.14 kb/s

All in all this is unbeliveable Detail Preservation (my wildest dreams since Grasprite come true) (ignoring it's side effects, for sure also partly coused by the insane bitrate) :)

nm
23rd January 2008, 08:47
Hmm could you guys please post wich of these looks better to you on the first sight and then also your GFX Card, Monitor ,Operating System, Player Application,Decoder and used Renderer as information. Thx in Advance :)

Please don't look framewise just subjectively rate them on the first sight :)

http://mirror05.x264.nl/CruNcher/force.php?file=./1.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./2.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./3.mkv

Order from best to worst: 1, 2, 3
The first sample is noticeably better than the others. I saw difference between 2 and 3 only by scaling up the video to see the details.

Gfx card: NVIDIA GeForce4 Ti4200
Monitor: 21" CRT (resolution 1600x1200)
OS: Linux (NVIDIA binary blob X.org drivers)
Player/Decoder: MPlayer/libavcodec
Renderer: OpenGL output (same results with XVideo)

Boardlord
23rd January 2008, 12:13
Hi all!

Has anyone tried the AQ on a concert footage? I tried to backup David Gilmour's "Remember that night" concert. I thought it would be a good test for Dark's AQ. I am sorry in advance that I have no pictures, but I am writing from my workplace, as I won't have net at home till March.
On the problems: the picture looked good, *very* little blocking, I was amazed! But, there were randomly appearing little black dot-like artifacts in the picture for a few secs, which then vanished, reappeared and so on. Tried it with Trellis, on-off (although Dark said Trellis doesn't matter). Aq strength was 0.8, sensitivity was tried at auto and 20 - same result. I used bob0r's compile. Setting weren't extreme (ref:5, bframes:3 (spatial), subme: 6, threads: 3, b-pyramid, no fast pskip, the rest were megui defaults)If needed, I'll post some pictures tomorrow, along with an x264 log. Of course the black dots were not present when AQ was not used... but blocks were :(

salehin
23rd January 2008, 13:46
Hi all!

Has anyone tried the AQ on a concert footage? I tried to backup David Gilmour's "Remember that night" concert. I thought it would be a good test for Dark's AQ. I am sorry in advance that I have no pictures, but I am writing from my workplace, as I won't have net at home till March.
On the problems: the picture looked good, *very* little blocking, I was amazed! But, there were randomly appearing little black dot-like artifacts in the picture for a few secs, which then vanished, reappeared and so on. Tried it with Trellis, on-off (although Dark said Trellis doesn't matter). Aq strength was 0.8, sensitivity was tried at auto and 20 - same result. I used bob0r's compile. Setting weren't extreme (ref:5, bframes:3 (spatial), subme: 6, threads: 3, b-pyramid, no fast pskip, the rest were megui defaults)If needed, I'll post some pictures tomorrow, along with an x264 log. Of course the black dots were not present when AQ was not used... but blocks were :(

Try with bobr's again by using weak aq (not more than 0.50) and sensitivity of 20-25. Compare to Dark's build with aq 1.0 and sens .25 (posted in the 1st post), it gave me better results. However, note that my source contains very little dark sequence (see sample tests with other settings- links posted above)

update: perhaps you can use one of the CQMs (see the custom matrix bit (http://forum.doom9.org/showpost.php?p=1056555&postcount=3)) to see if it improves the transparency

Morte66
23rd January 2008, 20:32
@DS

I thought you might be interested in this clip from 3 Colours White (http://www.mediafire.com/?dm1gmc1d2fl) (ffv1 in avi). It's been cropped/levelled/deblocked/denoised/debanded from DVD. It's my "worst reasonable case for AQ" material. I expect an encoder to handle it gracefully, if it's to get used for sight-unseen DVD backup.

It's assumed that it will be played back with Deband or GradFunkMirror() in ffdshow, which means it just has to avoid aggregating small blocks into bigger blocks.

Dark Shikari
23rd January 2008, 21:59
@DS

I thought you might be interested in this clip from 3 Colours White (http://www.mediafire.com/?dm1gmc1d2fl) (ffv1 in avi). It's been cropped/levelled/deblocked/denoised/debanded from DVD. It's my "worst reasonable case for AQ" material. I expect an encoder to handle it gracefully, if it's to get used for sight-unseen DVD backup.

It's assumed that it will be played back with Deband or GradFunkMirror() in ffdshow, which means it just has to avoid aggregating small blocks into bigger blocks.
That was easy (http://www.mediafire.com/?5d1fywox22w). Just 600 kilobits, CRF mode, was plenty.

On this topic, it seems CRF mode seems to work fine with AQ if you use a reasonable sensitivity (20).

Blue_MiSfit
23rd January 2008, 22:48
Hmm could you guys please post wich of these looks better to you on the first sight and then also your GFX Card, Monitor ,Operating System, Player Application,Decoder and used Renderer as information. Thx in Advance :)

Please don't look framewise just subjectively rate them on the first sight :)

http://mirror05.x264.nl/CruNcher/force.php?file=./1.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./2.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./3.mkv

Btw: If you want to see allways correct representation results in Windows with a Directshow Player use CoreAVC as Decoder :)


I like them in order. 1 is best followed by 2 and 3. Double checked.

8800GT, BenQ 24" LCD, Windows XP, Media Player Classic, CoreAVC, Haali Renderer.

Did the comparison at full screen (Scaled to 1920x1200). While 2 and 3 seemed to have more details in high frequency areas, the flat areas lost detail, and had irregular noise.

1 seemed to have the most even picture, and if there was any detail loss in the high frequency areas, it didn't distract me at all. The loss of detail in flat areas in the other versions WAS very distracting.

Excellent work! Let's see those settings!

~MiSfit

ToS_Maverick
23rd January 2008, 22:58
That was easy (http://www.mediafire.com/?5d1fywox22w). Just 600 kilobits, CRF mode, was plenty.

On this topic, it seems CRF mode seems to work fine with AQ if you use a reasonable sensitivity (20).

i can second that, i use --aq-strength 1.0 --aq-sensitivity 15 in CRF mode, works great!

DeathTheSheep
24th January 2008, 00:17
15? Didn't he say that was too low? If 20-25 works better, with bias towards the lower end, wouldn't that imply 21-22?

Just beating some weighted averages around. Hoohoo!

Dark Shikari
24th January 2008, 00:21
15? Didn't he say that was too low? If 20-25 works better, with bias towards the lower end, wouldn't that imply 21-22?

Just beating some weighted averages around. Hoohoo!I have no idea. You'll have to test.

Ideally, a good "middle value" would result in true Constant Quality, since it would appropriately adjust CRF based on the variance of the video, which defines what bitrate it "really" needs. Technically any decent value of sensitivity would do this--its just that the bitrate would not necessarily be comparable to non-AQ CRF.

Atak_Snajpera
24th January 2008, 00:39
http://mirror05.x264.nl/CruNcher/force.php?file=./1.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./2.mkv
http://mirror05.x264.nl/CruNcher/force.php?file=./3.mkv

1 looks a lot better than 2 or 3.

MPC + FFDShow HQ-RGB32 forced

i can second that, i use --aq-strength 1.0 --aq-sensitivity 15 in CRF mode, works great!
I fully agree with you. 1.0 and 15 seems to be well balanced. However I still prefer auto sensitivity. Better quality (compared to no AQ) plus smaller files in CRF.

Sagekilla
24th January 2008, 02:13
Dark Shikari, would it be possible to hack in AQ to be used for RC under CRF mode? (Acronym city!) I'm sure if you did that, you could get even closer to a truer "constant quality"

Dark Shikari
24th January 2008, 02:17
Dark Shikari, would it be possible to hack in AQ to be used for RC under CRF mode? (Acronym city!) I'm sure if you did that, you could get even closer to a truer "constant quality"Actually, thinking about it... it should be even better than CRF mode, because it properly takes into account the quantizer needed to reach a constant quality.

Try sensitivity 20 on CRF mode.

lexor
24th January 2008, 02:37
hey guys, is anyone else having problems getting the files from x264.nl? I can't get there for a few days now. I thought there was a site hiccup, but seeing how a new video was added there (the one in OP) not everyone is having these issues. Or is it intermittent and I always try it when it's down?

DeathTheSheep
24th January 2008, 02:47
Wouldn't a "truer" constant quality be reached using QP and threshold? (To answer my own question, yeah, pretty much. No qcomp hiccups to worry about here, or artificial degradation near keyframes -.-).
:)

CruNcher
24th January 2008, 03:54
@Dark Shikari
A strange idea came to my mind :D wouldn't it be possible to implement both AQs @ the same time? yours to enhance the current picture (distribution on the frame) and the old one todo the rest (distribution between the frames)? hmm something like a Mixed-Mode AQ (the most efficient of both worlds combined) :D

Dark Shikari
24th January 2008, 03:59
@Dark Shikari
A strange idea came to my mind :D wouldn't it be possible to implement both AQs @ the same time? yours to enhance the current picture (distribution on the frame) and the old one todo the rest (distribution between the frames)? hmm something like a Mixed-Mode AQ :DWouldn't work. The old AQ didn't change framewide QP, it just changed it (in an extremely strong manner) on a few blocks in the frame.

Also, 0.46 is out, see original post for changes.

fields_g
24th January 2008, 04:37
Actually, thinking about it... it should be even better than CRF mode, because it properly takes into account the quantizer needed to reach a constant quality.

Try sensitivity 20 on CRF mode.

I WAS trying out the most current version (.45) until SOMEONE wanted to update it! :) Anyway... I'm using my favorite AQ testing material (POTC2 first 34 seconds.. aka the new Disney Logo) @ 1080p. Here's my base command line:

--crf 18 --ref 16 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 7 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me esa --merange 12 --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input" --aq-strength 1.0

Here are some results:
Sensitivity ....Filesize
-------------------------------
Auto.............20.0 MB
10................18.5 MB
11................23.8 MB
15................50.6 MB
25................79.5 MB
NoAQ............20.5 MB


As you can see, on this clip, the sensitivity really blows up the bitrate. Auto does a really good job of following the NoAQ bitrate. When you guys suggest 20, does this produce a filesize that is approximately equal to auto/NoAQ on your clips?

Visually inspecting the results, it is obvious that Auto needs more help. I much prefer the 15 than the Auto or 11 results. This leads me to believe that your suggestion of 20 might not be so bad! Just hoping this doesn't always mean a 3x bitrate at a given CRF!

CruNcher
24th January 2008, 04:47
fields_g see it's almost a real VBR mode you get the best possible quality but with it CRF becomes non predictable anymore DABR i think isn't possible this way (or actually has to be new ploted). :) At least the whole CRF shifts the higher you set the Sensitivity CRF 19 with for example --aq-sensitivity 40 would get very very big ;) i would suggest for a Real VBR that reaches Visual Losslesness (Transparency) 50dB here as a good target to test alot of source (the most complex you can find) and see how it comes out CRF 19 and @ wich Sensitivity :)

fields_g
24th January 2008, 04:50
I guess there is a reason that it is an AQ test sample for me. It isn't a cakewalk to fix. The bitrate is showing that!

CruNcher
24th January 2008, 05:01
25................79.5 MB
how much dB did you reached with it with your settings?

fields_g
24th January 2008, 05:07
Playing around earlier I made sure to hit the ssim / pnsr boxes, but this run I didn't. I'll rerun some of these and get those.


EDIT: sensitivity 25 - SSIM .9931239 - PSNR Y55.808 U57.585 V57.585 Avg56.259 Global50.669
Though AQ has to be judged subjectively.

Dark Shikari
24th January 2008, 05:21
OK folks, here's a challenge for you!

In order to put the AQ in SVN, we're going to need to decide on a good general sensitivity to use that would keep bitrate relatively constant on a large sample of different types of video. That is, a flat video might get higher bitrate, and a complex one lower, but in general, on a large sample of a bunch of random stuff, it shouldn't change the bitrate at all in CRF mode.

So, grab yourself a ton of selecteveryrange() and find me a sensitivity that doesn't change bitrate at all on a combination of many sources. Once we get that sensitivity, I will likely remove the sensitivity option completely and simply have two modes--automatic and static.

fields_g
24th January 2008, 05:41
OK folks, here's a challenge for you!

In order to put the AQ in SVN, we're going to need to decide on a good general sensitivity to use that would keep bitrate relatively constant on a large sample of different types of video. That is, a flat video might get higher bitrate, and a complex one lower, but in general, on a large sample of a bunch of random stuff, it shouldn't change the bitrate at all in CRF mode.

Is the idea to also get rid of --aq-strength also? Should we be testing at 1.0 or a variety of strengths and comparing to NoAQ?

Dark Shikari
24th January 2008, 05:45
Is the idea to also get rid of --aq-strength also? Should we be testing at 1.0 or a variety of strengths and comparing to NoAQ?Nah, just compare strength 1.0 to strength 0, since 1.0 is meant to be a good default.

AQ strength should always be adjustable.

CruNcher
24th January 2008, 05:45
does someone has the EBU testsequence they test broadcast codecs with i think that would be perfect a mix of the schumacher and alot of other testclips that is :)

http://www.ebu.ch/en/technical/hdtv/test_sequences.php

desta
24th January 2008, 05:46
I'm assuming DS means finding a sensitivity that gives the same final bitrate as using no AQ. The sensitivity being the same or pretty close when applied to various samples.

Edit: too late.

burfadel
24th January 2008, 10:09
Who else thinks it would be better, in general, to have aq-strength 1.0 set as default (that is, without the need to actually specify it at all), but still have it adjustable by the addition of the --aq-strength x.x command?

Dark Shikari
24th January 2008, 10:10
Who else thinks it would be better, in general, to have aq-strength 1.0 set as default (that is, without the need to actually specify it at all), but still have it adjustable by the addition of the --aq-strength x.x command?I'm going to leave in AQ strength. Having it on by default might definitely be useful, if akupenguin would allow it.

CruNcher
24th January 2008, 10:23
Btw Dark i fixed all this problems in those scenes by combining your aq with the prestige matrix :) now i have both of best worlds your enhanced detail preservation + a fix for those ROI problems im uber happy (no banding situations anymore) :)

http://mirror05.x264.nl/CruNcher/force.php?file=./roi-fixed-newaq.mkv <- combined result still 0.45 :)

This is the most advanced lossy encode i ever did :)

Morte66
24th January 2008, 11:34
OK folks, here's a challenge for you!

In order to put the AQ in SVN, we're going to need to decide on a good general sensitivity to use that would keep bitrate relatively constant on a large sample of different types of video. That is, a flat video might get higher bitrate, and a complex one lower, but in general, on a large sample of a bunch of random stuff, it shouldn't change the bitrate at all in CRF mode.

Let me see if I understand this...

Current CRF implies a certain level of quality (crispness, fine detail retention etc), but the judgement of quality places more weight on complex material relative to flat areas than humans do. Current CRF (without AQ) is not a constant quality mode as judged by humans.

With current CRF/AQ, CRF achieves the same level of quality on complex stuff and AQ throws extra bitrate at flat areas. It maintains the quality of plain CRF on complex material, and improves flat material with extra bitrate. Overall bitrate rises. It gets closer to constant quality as perceived by humans. [I like this -- for me the point of CRF is to give constant quality by spending whatever bitrate is necessary.]

VAQ, however, nabs bitrate from complex areas to improve flat areas (both within and optionally between frames). So if CRFVAQ (hey, a new acronym) maintains the bitrate of plain CRF, will it look worse on complex stuff and better on smooth? Will a particularly shadowy episode of the X-Files look less crisp and detailed in its complex areas than one which is visually busy? Will "CRF 18" no longer imply a certain level of crispness and fine detail on the complex stuff? Or have I completely misunderstood it?

DarkZell666
24th January 2008, 12:53
From what I gathererd, an important change is rather the method used to distinguish/define what's complex and what isn't.

You'd think the new AQ munches fine details since it supposedly takes bitrate out of complex scenes, but CruNcher's screenshots actually show some extra detail on people's faces and in flying particles :)


Just my 0.02€ ^^ :helpful:

akupenguin
24th January 2008, 13:42
@Morte66
Current CRF is constant quality as measured by one particular metric (not psnr or ssim or any of the normal metrics, but an ad-hoc one implicit in the CRF algorithm). It's similar bitrate on average as the same value of CQP, because I multiplied the CRF scale by a magic number to make that true, not by anything inherent in the algorithm.
CRF+HaaliAQ is constant quality as measured by another metric. It's always at least a little higher bitrate per CRF value than plain CRF because HaaliAQ has a negative average QP bias, but that's just a UI tuning issue and could be fixed if desired.
CRF+VAQ(static) is constant quality as measured by another metric. If you pick some random value of aq-sensitivity, it won't be equal on average, but will have some bias. We're trying to find a sensitivity value such that that bias averages to 0, thus it will hopefully be about the same bitrate on average as a given value of plain CRF if we do the tuning right (unless we choose to keep psy quality the same instead of bitrate).
CRF+VAQ(auto) does the frame-wise bit allocation using the same metric as plain CRF, but reallocates bits within the frame using the VAQ metric.

In short, there's no substantive difference between "improves smooth stuff at the cost of complex stuff" and "improves smooth stuff while keeping complex stuff the same". The only criterion AQ can be judged by is the relative distribution of quality. Any bias in the total quality is just a user interface issue separate from the algorithm.

Morte66
24th January 2008, 13:56
In short, there's no substantive difference between "improves smooth stuff at the cost of complex stuff" and "improves smooth stuff while keeping complex stuff the same". The only criterion AQ can be judged by is the relative distribution of quality. Any bias in the total quality is just a user interface issue separate from the algorithm.

*digests*

OK, I see what you mean. That makes good sense to me. Thank you.

So if I found that...

- plain crf 18 creates a 350MB file with satisfactory complex stuff but unsatisfactory smooth stuff

- crf 18 with Haali --aq-strength 0.3 --aq-sensitivity 5 creates a 600MB file which is satisfactory on complex and flat

- crf 18 with variance --aq-strength 1.0 --aq-sensitivity 15 creates a 350MB file which is better overall than either of the above, but occasionally loses a little on the complex stuff

... then I ought to try using variance AQ and changing crf to 17, hoping to get quality that is equal or better in every respect to the 600MB Haali AQ encode in perhaps 400MB?

LoRd_MuldeR
24th January 2008, 14:17
@Morte66
Current CRF is constant quality as measured by one particular metric (not psnr or ssim or any of the normal metrics, but an ad-hoc one implicit in the CRF algorithm). It's similar bitrate on average as the same value of CQP, because I multiplied the CRF scale by a magic number to make that true, not by anything inherent in the algorithm.
CRF+HaaliAQ is constant quality as measured by another metric. It's always at least a little higher bitrate per CRF value than plain CRF because HaaliAQ has a negative average QP bias, but that's just a UI tuning issue and could be fixed if desired.
CRF+VAQ(static) is constant quality as measured by another metric. If you pick some random value of aq-sensitivity, it won't be equal on average, but will have some bias. We're trying to find a sensitivity value such that that bias averages to 0, thus it will hopefully be about the same bitrate on average as a given value of plain CRF if we do the tuning right (unless we choose to keep psy quality the same instead of bitrate).
CRF+VAQ(auto) does the frame-wise bit allocation using the same metric as plain CRF, but reallocates bits within the frame using the VAQ metric.

In short, there's no substantive difference between "improves smooth stuff at the cost of complex stuff" and "improves smooth stuff while keeping complex stuff the same". The only criterion AQ can be judged by is the relative distribution of quality. Any bias in the total quality is just a user interface issue separate from the algorithm.

Thanks for the explanation. Very interesting post :)

ToS_Maverick
24th January 2008, 14:22
OK folks, here's a challenge for you!

In order to put the AQ in SVN, we're going to need to decide on a good general sensitivity to use that would keep bitrate relatively constant on a large sample of different types of video. That is, a flat video might get higher bitrate, and a complex one lower, but in general, on a large sample of a bunch of random stuff, it shouldn't change the bitrate at all in CRF mode.

So, grab yourself a ton of selecteveryrange() and find me a sensitivity that doesn't change bitrate at all on a combination of many sources. Once we get that sensitivity, I will likely remove the sensitivity option completely and simply have two modes--automatic and static.

hey i got some results for you!

aq10 means --aq-strength 1.0
sens15 means --aq-sensitivity 15
and so on

BlackPearlSample (3622 frames):

Black.Pearl.Sample test crf 18 aq00.mkv 35,3 MB
Black.Pearl.Sample test crf 18 aq10 sens13.mkv 35 MB
Black.Pearl.Sample test crf 20 aq10 sens15.mkv 32,6 MB
Black.Pearl.Sample test crf 22 aq10 sens18.mkv 33,3 MB
Black.Pearl.Sample test crf 24 aq10 sens21.mkv 32,1 MB
Black.Pearl.Sample test crf 26 aq10 sens25.mkv 32,3 MB


SAW 1 DVD comp check (1488 frames):

SAW test crf 18 aq00.mkv 15,5 MB
SAW test crf 18 aq10 sens12.mkv 15,7 MB
SAW test crf 18 aq10 sens13.mkv 19,4 MB


Klick from HDTV comp check (1548 frames):

klick comp test crf 18 aq00.mkv 49,7 MB
klick comp test crf 18 aq10 sens12.mkv 42,5 MB
klick comp test crf 18 aq10 sens13.mkv 50,9 MB


what i found out:
- 3 steps in sensitivity roughly equals 2 steps in CRF
- sensitivity 13 seems to be a good pick (for now) if you want the same CRF filesize with and without AQ

Chabb
24th January 2008, 15:05
Would it be right to use AQ with lowered deadzones and custom quantization matrixes? Please express any pro et contra.

Morte66
24th January 2008, 15:46
Entourage Season 2 Episode 11 with AviSynth levels/deblock/denoise/deband -> huffYUV, then x264 crf 18 encode. FWIW I've put what I thought of the quality in brackets.

Variance AQ build 46 but no AQ set: 372MB (unsatisfactory)
Variance AQ strength 1.0 sensitivity 15: 326MB (about the same quality but smaller)
Variance AQ strength 1.0 sensitivity 17: 378MB (barely satisfactory, only saw problems if I looked for them)
Variance AQ strength 1.0 sensitivity 20: 516MB (comfortable, maybe diminished returns)

Well, based on that completely inadequate pool of data I'd make --aq-sensitivity a parameter that defaults to 17.

@TOS_Maverick: you got a much lower number. Did you pre-process in any way? Maybe because I deblocked/denoised/debanded my video has more "smooth/flat" in it.

DeathTheSheep
24th January 2008, 17:05
Would it be right to use AQ with lowered deadzones and custom quantization matrixes? Please express any pro et contra.

Pro:
Btw Dark i fixed all this problems in those scenes by combining your aq with the prestige matrix :) now i have both of best worlds your enhanced detail preservation + a fix for those ROI problems im uber happy (no banding situations anymore) :)

http://mirror05.x264.nl/CruNcher/force.php?file=./roi-fixed-newaq.mkv <- combined result still 0.44 :)

This is the most advanced lossy encode i ever did :)

Contra: Not much has been said about deadzones, so more testing has to be done.

CruNcher
24th January 2008, 20:13
It seems with useing a custom matrix you can have an influence on how Darks NewAQ behaves on the Frame for example what you will see here is that both combined are like a AI AQ for this testcut (it seems almost to decide magicaly were the scene could band and then allocates more bits into this area)
This is perfect Visually as it enhances allways the right area :)
So if there is no banding possibility it will enhance (Facial Details) if their could banding happen (it does enhance the background with the gradients).
I know it's not AI but the balance that i shifted this way by useing a custom matrix, tough still 1 scene gives me major headaches Visually ;)

Noaq
http://s5.directupload.net/images/080124/9u87knfx.png
http://s6.directupload.net/images/080124/g7owp2vh.png (this scene is the Visual killer, especialy when the viewing system is wrong calibrated)
http://s2.directupload.net/images/080124/rqsiveqs.png
http://s5.directupload.net/images/080124/qj6gkjzb.png
http://s1.directupload.net/images/080124/fi6pqiwd.png
http://s3.directupload.net/images/080124/9xfz7l2t.png
http://s3.directupload.net/images/080124/bf8kl9uw.png

Newaq
http://s2.directupload.net/images/080124/idkvrpus.png
http://s5.directupload.net/images/080124/ge4gmtph.png
http://s6.directupload.net/images/080124/8apvaqwh.png
http://s2.directupload.net/images/080124/j32kt8zb.png
http://s3.directupload.net/images/080124/pe7f62ej.png
http://s5.directupload.net/images/080124/sf3up4fl.png
http://s4.directupload.net/images/080124/4qr7pot6.png

But i think i still need to balance it out a little as you can see ringing (to sharp) appearing, btw Prestige Matrix does the shifting in this case very precise dunno what some have against this matrix it seems visually optimized entirely for grain preservation and it does a good job i can also see no artifacts @ all, coused by it (except the sligth ringing effect coused by overoptimizing the source visually with this) :)

Morte66
24th January 2008, 20:37
Maybe because I deblocked/denoised/debanded my video has more "smooth/flat" in it.

So I ran it again with all the crud left in, just doing levels. This time:

crf 18 no AQ: 623MB
crf 18 VAQ strength 1.0 sensitivity 17: 932MB (sensitivity 17 matched sizes when cleaned up)
crf 18 VAQ strength 1.0 sensitivity 13: 559MB (13 seemed closest for TOS_Maverick's tests)
crf 18 VAQ strength 1.0 sensitivity 14: 648MB (and 14 is closest for my uncleaned DVD)

So if I feed in a middle of the road DVD with no cleanup, I get results fairly similar to TOS_Maverick (sensitivity ~13 matches no AQ). For a cleaned up DVD, which doesn't have crud all over the smooth/dark content making it un-smooth, it needed to be 3 or 4 higher.

bob0r
24th January 2008, 20:47
AQ patch 0.47: http://akuvian.org/src/x264/x264_aq_var.47.diff

x264.721.dark.aq.0.47.exe (http://files.x264.nl/AQ/x264.721.dark.aq.0.47.exe) (pthreads/mp4 = yes, made with make fprofiled)

Terranigma
24th January 2008, 20:48
Thanks for that bob0r. :)

Dark Shikari
24th January 2008, 20:49
AQ patch 0.47: http://akuvian.org/src/x264/x264_aq_var.47.diff

x264.721.dark.aq.0.47.exe (http://files.x264.nl/AQ/x264.721.dark.aq.0.47.exe) (pthreads/mp4 = yes, made with make fprofiled)Can you guys do your testing again with this patch? It vastly cleaned up the code, and more importantly, it fixed a rounding error with low QPs which should somewhat improve visual quality in flat areas. However, this may change the results of your sensitivity tests.

DeathTheSheep
24th January 2008, 21:03
The fact that this patch has been worked on by akupenguin (and is now hosted on his site) of no small significance or import, and perhaps signals the coming of something momentous.

A rounding error was fixed? What's the likelihood of any quality difference occurring at high QP?

Dark Shikari
24th January 2008, 21:05
The fact that this patch has been worked on by akupenguin (and is now hosted on his site) of no small significance or import, and perhaps signals the coming of something momentous.

A rounding error was fixed? What's the likelihood of any quality difference occurring at high QP?I'm referring to the lower QPs *in the video*, i.e. the flat areas that get lowered a lot. Areas with low variance.

Not low QPs as a general statement.

Previously, variance was calculated as (SSD - (SAD / 16) ^2). Now its calculated as (SSD - SAD^2 / 256).

DeathTheSheep
24th January 2008, 21:29
Ouch. In that case, there's almost guaranteed to be a [somewhat] positive difference on the very places that count most.

Morte66
24th January 2008, 22:12
Can you guys do your testing again with this patch?

Arrgh! Just as I've nearly finished the 1080p test. ;)

I did the Taurus media samples, which are uncompressed footage from a digital camera. They're very busy, not much smooth/flat stuff, so they're not really a good surrogate for film/TV. But I've never been able to resist testing them since I spent three whole days downloading them.

crf 18 no AQ: 228MB
crf 18 VAQ strength 1.0 sensitivity 17: 381MB
crf 18 VAQ strength 1.0 sensitivity 14: 203MB

That's with 0.46.

DeathTheSheep
24th January 2008, 22:41
.47 makes the files balloon in size at the same threshold I was using before (22 in this particular test case).

Seems quite a bit slower, but maybe this can be explained by the fact that my "old" settings apparently correspond to different things now.

[edit]
[I said edit, where's the edit notifier at the bottom? ...that's better.]
SSIM just...went...through...the...roof...... (I'm going to have to re-shingle now, damn it).

Dark Shikari
24th January 2008, 22:42
.47 makes the files balloon in size at the same threshold I was using before (22 in this particular test case).

Seems quite a bit slower, but maybe this can be explained by the fact that my "old" settings apparently correspond to different things now.Yeah, due to rounding, 0.46 wasn't giving flat blocks low enough quantizers.

ditche
24th January 2008, 23:11
I don't understand...

I've encoded a movie (duration : 1h52) with theses settings :
--qp 22 --ref 10 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --bime --weightb --direct auto --filter -2,-1 --trellis 2 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads 3 --thread-input --sar 1:1 --progress --no-dct-decimate --no-psnr --no-ssim --output "D:\VIDEO\rip\video.mp4" "D:\VIDEO\rip\video.avs" --aq-strength 1.0
CRF : 22, resolution : 688x304, x264 721 0.45

Results : 339 Mo @ CRF22 for 1h52.
Fun, tiny size, but...

Nop, because the result is very bad...

http://img134.imageshack.us/img134/7029/snapshot20080124230054ib5.th.jpg (http://img134.imageshack.us/my.php?image=snapshot20080124230054ib5.jpg)

http://img134.imageshack.us/img134/94/snapshot20080124230249cm7.th.jpg (http://img134.imageshack.us/my.php?image=snapshot20080124230249cm7.jpg)

My other encodings with same settings give me larger files but a good quality...
:confused:


Have you an idea ? Thanks. :)

:helpful:

Dark Shikari
24th January 2008, 23:13
Those are somewhat odd settings you're using, plus, you're using QP mode, not CRF mode. Trellis 2 doesn't help detail retention, either.

I suspect you're just not giving it enough bits.

Atak_Snajpera
24th January 2008, 23:13
Since when --qp 22 means --crf 22 ?

ditche
24th January 2008, 23:23
Since when --qp 22 means --crf 22 ?
Arf !! :)
I make a little confusion between qp & crf... :p

LoRd_MuldeR
24th January 2008, 23:25
Since when --qp 22 means --crf 22 ?

ditche, according to your command-line you were using --qp 22, but you said "CRF : 22" :p

ditche
24th January 2008, 23:29
Yeah, I'm confused...

It's almost midnight here, tomorrow i'll try with theses settings, is it OK for you (± Shartooth settings :p) ?

--crf 22.0 --ref 3 --mixed-refs --bframes 16 --b-pyramid --bime --weightb --filter -2,-1 --subme 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-psnr --no-ssim --output "output" "input" --aq-strength 1.0 --aq-sensitivity 15

:)


Edit :
Ah, it's looks so much better on my last sample test. :)

Dark Shikari
24th January 2008, 23:33
Those are really bizarre settings--UMH/8x8dct/many refs, but subme 1?!