Log in

View Full Version : Variance AQ Megathread (AQ v0.48 update--defaults changed)


Pages : 1 2 3 4 5 6 7 8 9 10 11 [12] 13 14 15 16 17 18 19

burfadel
25th January 2008, 23:58
With my clips I also ended up at --aq-sensitivity 9 with --aq-strength 1.0 to get similar bitrates to pure crf. But the settings don't work well on all clips. If I use them on Clerks 2 for instance I end up with a small really bad looking file.

Thats exactly what I found with some small tests I did as well! Looking at the resultant file is much better than relying on the numbers alone :)

Lowering the CRF from 24 to even 18 didn't completely help the situation, and you certainly don't want to keep finding the 'sweet spot' every time you encode something.

The lower the sensitivity the lower the quality of the resultant file, I thought this AQ was an 'additive' function (add more bits where needed, subtract from where not needed), and not a 'base' function (every block has a higher quality with a higher AQ).

For instance, if you select a sensitivity of 1, the result from the test I did earlier was a file of 45kb and just a complete mess.

Just for clarification, what frames does AQ operate on? does it operate on keyframes alone to have a higher quality base for the p and b frames to be based on, or does it operate on all frames?

CruNcher
26th January 2008, 00:00
MAMA (--aq-strength 1.0 --aq-sensitivity 20) CRF 32

x264 [info]: SSIM Mean Y:0.9858374
x264 [info]: PSNR Mean Y:45.823 U:46.553 V:49.433 Avg:46.350 Global:45.818 kb/s:
11161.54

encoded 9336 frames, 7.81 fps, 11164.45 kb/s

CRF 38 maybe?

Dark Shikari
26th January 2008, 00:05
The lower the sensitivity the lower the quality of the resultant file, I thought this AQ was an 'additive' function (add more bits where needed, subtract from where not needed), and not a 'base' function (every block has a higher quality with a higher AQ).

For instance, if you select a sensitivity of 1, the result from the test I did earlier was a file of 45kb and just a complete mess.That's because in order to add or subtract, it needs to know where its baseline is.
Just for clarification, what frames does AQ operate on? does it operate on keyframes alone to have a higher quality base for the p and b frames to be based on, or does it operate on all frames?All frames.

burfadel
26th January 2008, 00:24
Ah ok! I found that a sensitivity of around 17 is the lowest I'd want to go with anything, anything lower than that and for standard definition encodes the visual quality dropped too much in certain scenes. A lower strength also seems to be a good idea! Once the strength is lowered to around 0.6 at sensitivity 17 the file size is very close to without AQ without quality loss. That seems to be the best combination for this particular clip.

Yoshiyuki Blade
26th January 2008, 00:30
nurbs posted a graph that shows the bitrate across different AQ strengths, which seems to be helpful here (post #525). Strength of 0.5 produces the smallest filesize at a given CRF and sensitivity. Other than strength 0.1, its a nice parabolic pattern (above and below 0.5, the bitrate starts going up).

EDIT: Is high strength/low sensitivity any different (visually) than low strength/high sensitivity?

DeathTheSheep
26th January 2008, 00:45
Actually, I've started to use sensitivity as a means by which to fine-tune the desired QP, and since x264 doesn't accept fraction QP, it is of use indeed for comparison purposes; that is, in order to reach a desired filesize to some degree of exactitude, one can simply input a fractional sensitivity instead of worrying over the effects of differences in rate control mechanism behavior.

However, I can't quite mirror DS's statement when it comes to QP. Lowering the nominal QP while lowering threshold produces strikingly different results than a higher QP and sensitivity (at nearly identical resultant filesizes). Again, these findings pertain to nominal QP mode.

Dark Shikari
26th January 2008, 00:47
Actually, I've started to use sensitivity as a means by which to fine-tune the desired QP, and since x264 doesn't accept fraction QP, it is of use indeed for comparison purposes; that is, in order to reach a desired filesize to some degree of exactitude, one can simply input a fractional sensitivity instead of worrying over the effects of differences in rate control mechanism behavior.

However, I can't quite mirror DS's statement when it comes to QP. Lowering the nominal QP while lowering threshold produces strikingly different results than a higher QP and sensitivity (at nearly identical resultant filesizes). Again, these findings pertain to nominal QP mode.This shouldn't be the case unless there is QP clipping going on; i.e. --qpmin is limiting some QPs.

MasterNobody
26th January 2008, 01:44
As I understand the main purpose of ac_energy_mb function is calculation of (X - Xa)^2 where X is pixel value and Xa is average value of pixel. Now it is calculated as X^2 - Xa^2 (ssd - sad^2). But mathematically correct formula would be (X - Xa)^2 = X^2 + Xa^2 - 2*X*Xa (ssd + sad^2 - 2*sqrt(ssd)*sad). So I made modification of AQ which use this formula for ac_energy_mb calculation (also it has other cosmetic changes, and by default it use automatic sensitivity because in my opinion every source need its own sensitivity and you never find value which would be good for any source). I don't know would it have better or worse quality so it needs thorough testing and comparison with not modified AQ.

patch: http://stashbox.org/75410/x264_aq_var.47.mod.diff
exe (pthread = yes, mp4 = no, fprofiled = no):http://stashbox.org/75411/x264.721.dark.aq.0.47.mod.exe

burfadel
26th January 2008, 02:24
As I understand the main purpose of ac_energy_mb function is calculation of (X - Xa)^2 where X is pixel value and Xa is average value of pixel. Now it is calculated as X^2 - Xa^2 (ssd - sad^2). But mathematically correct formula would be (X - Xa)^2 = X^2 + Xa^2 - 2*X*Xa (ssd + sad^2 - 2*sqrt(ssd)*sad). So I made modification of AQ which use this formula for ac_energy_mb calculation (also it has other cosmetic changes, and by default it use automatic sensitivity because in my opinion every source need its own sensitivity and you never find value which would be good for any source). I don't know would it have better or worse quality so it needs thorough testing and comparison with not modified AQ.

patch: http://stashbox.org/75410/x264_aq_var.47.mod.diff
exe (pthread = yes, mp4 = no, fprofiled = no):http://stashbox.org/75411/x264.721.dark.aq.0.47.mod.exe

I agree with the automatic sensitivity, especially with your modified patch :) It seems to work much better for the clip I was using before, and the filesize is very close to without AQ! I tried strengths 0.5 and 1, at least for lower resolution clips, including DVD resolution, your modified AQ seems to work better!
(I hope that doesn't sound too rude to Dark_Shikari and Akupenguin, I'm just giving credit to someone giving very worthwhile input into the codec!)

Dark Shikari
26th January 2008, 02:31
I agree with the automatic sensitivity, especially with your modified patch :) It seems to work much better for the clip I was using before, and the filesize is very close to without AQ! I tried strengths 0.5 and 1, at least for lower resolution clips, including DVD resolution, your modified AQ seems to work better!
(I hope that doesn't sound too rude to Dark_Shikari and Akupenguin, I'm just giving credit to someone giving very worthwhile input into the codec!)The inherent problem with automatic sensitivity is that it avoids giving more bits to frames that really need it, such as flat, dark frames; this leads to, in many cases, this AQ performing worse than the old (Haali's) AQ.

One could possibly use automatic sensitivity with a min-sensitivity feature to ensure it never drops below a certain value.

burfadel
26th January 2008, 02:47
The min sensitivity feature would be a good idea, it would help resolve the problem I had with the other clip! What do you think of Masternobody's modification? It seemed to work okay with me, but then again its trying to find an adequate balance amongst all clips :)

radius
26th January 2008, 03:39
Hi, from what I tested the default sensitivity at 20 is a bit aggressive, compared with automatic sensitivity there's a small yet noticeable compression loss, stuff like text suffers at 20 or more, min-sensitivity thing looks like a good idea :)

Terranigma
26th January 2008, 04:01
...and automatic sensitivity would be 0 like with strength?

akupenguin
26th January 2008, 04:22
As I understand the main purpose of ac_energy_mb function is calculation of (X - Xa)^2 where X is pixel value and Xa is average value of pixel. Now it is calculated as X^2 - Xa^2 (ssd - sad^2). But mathematically correct formula would be (X - Xa)^2 = X^2 + Xa^2 - 2*X*Xa (ssd + sad^2 - 2*sqrt(ssd)*sad).
No, the current formula is correct.
http://en.wikipedia.org/wiki/Variance

Yoshiyuki Blade
26th January 2008, 04:30
Heres my results from an anime test encode:

Rurouni Kenshin, episode 48 (aggressively filtered/resized)

CRF 25 (no AQ): 157 MB
CRF 25 --aq-strength 1.0 --aq-sensitivity 8: 95.6 MB
CRF 25 --aq-strength 1.0 --aq-sensitivity 9: 124 MB
CRF 25 --aq-strength 1.0 --aq-sensitivity 10: 161 MB
CRF 25 --aq-strength 1.0 --aq-sensitivity 12: 259 MB

Looking purely by numbers, sensitivity of 10 is the closest to the original bitrate. Quality is significantly lower across the entire episode, but dark areas look much better.

DeathTheSheep
26th January 2008, 05:23
The inherent problem with automatic sensitivity is that it avoids giving more bits to frames that really need it, such as flat, dark frames; this leads to, in many cases, this AQ performing worse than the old (Haali's) AQ.
The same is true of .47 with sensitivity, compared to .45. (I hope the reintroduction of the constant will make it into the next release). Really, it's odd how the correct rounding was so stingy on the frames that needed it most in my samples.

burfadel
26th January 2008, 06:42
No, the current formula is correct.
http://en.wikipedia.org/wiki/Variance

So the results I got with some quick testing weren't 'accurate'? I didn't and don't have time today for more thorough tests, on the outset it just seems to work better (then again, it may have just like the particular clips I threw at it)!

Dark Shikari
26th January 2008, 06:49
So the results I got with some quick testing weren't 'accurate'? I didn't and don't have time today for more thorough tests, on the outset it just seems to work better (then again, it may have just like the particular clips I threw at it)!That's odd, as in my experience fixing the rounding made low QPs even lower.

akupenguin
26th January 2008, 07:49
So the results I got with some quick testing weren't 'accurate'? I didn't and don't have time today for more thorough tests, on the outset it just seems to work better (then again, it may have just like the particular clips I threw at it)!

I'm not saying that AC energy aka variance is the optimal heuristic for AQ. I'm saying that what I calculated is variance, and what you calculated isn't. In particular, the X in the 3rd term of "X^2 + Xa^2 - 2*X*Xa" is avg(X), not sqrt(avg(X^2)).

Morte66
26th January 2008, 10:49
Especially since moving sensitivity is equivalent to changing CRF :)

Hmm. That raises a concern. I'm not 100% sure about this yet, still gnawing at it.

I've been doing some actual encoding (not just tests). I used --aq-sensitivity 10 which seems like a good weighted average -- I got lots of 9 and the occasional 13 when matching sizes.

A lot of the results are excellent. All the results are excellent for the bitrate. But there's an issue...

The stuff that needed sensitivity 13 to match sizes in the testing is coming out quite small at sensitivity 10, and and although it's admirable in terms of blocking/banding and dark/smooth detail it's rather soft and lacking in texture. Woven cotton shirts become smooth polyester. It almost looks like I did an encode at crf 20/21 and chucked in loads of Haali AQ control the blocks/bands. That would need at least 50% more bitrate, which is obviously a score for Varaince AQ.

But the thing is, with Haali's AQ you could say --crf 18 and know it would be sharp enough, and you could say --aq-strength 0.3 --aq-sensitivity 5 and know that whatever blocking/banding came out would be controllable by ffdshow DeBand. You didn't have to inspect the video or run test encodes to find a set of parameters that would give satisfactory quality. I am concerned that this might not be the case with variance AQ.

Dark Shikari
26th January 2008, 10:52
We may have found a problem underlying some of the problems with VAQ.

In particular, the qcomp setting already varies frame quantizers based on the SATD score of the frame. So its basically doing what AQ is doing also, in a sense overweighting and underweighting some frames.

I would suggest trying qcomp=1 in the meantime, and seeing whether it improves it or not. We're working on a fix.

In particular, qcomp = 0.6 + AQ strength 0 should have a similar QP-moving effect (on frame level, not block level) as qcomp = 1.0 + AQ strength 0.28 (according to akupenguin's math).

Edit: More information. Qcomp = 1 is equivalent to CQP mode. AQ in a sense does the same job as qcomp, so it would make sense to disable qcomp (set it to 1) when using decently strong AQ. For now, try settings like qcomp=1,aq strength=0.6, and experiment with AQ sensitivity.

ToS_Maverick
26th January 2008, 14:03
here you go again, some results:

Black.Pearl.Sample test crf 18 aq00 qcomp10.mkv 48,8 MB
Black.Pearl.Sample test crf 18 aq06 sens12 qcomp10.mkv 53,3 MB
Black.Pearl.Sample test crf 18 aq10 sens11 qcomp10.mkv 53,6 MB

Black.Pearl.Sample test crf 18 aq00 qcomp06.mkv 35,3 MB
Black.Pearl.Sample test crf 18 aq06 sens09 qcomp10.mkv 37,1 MB
Black.Pearl.Sample test crf 18 aq10 sens09 qcomp10.mkv 36,9 MB

with the 37 MB encodes, 0.6 is better in the foreground, 1.0 is better in the background, visually. 0.6 delivers a better SSIM

Hellworm
26th January 2008, 14:12
So Qcomp = 1 does constant quant and AQ with sensivity = high then does constant quality, completely ignoring the bitrate? Is this correct? I sort of followed the thread, but didn't have much time to test and I'd find a "real" constant quality very useful, as crf often doesn't produce a constant quality, even with Qcomp = 1

Edit: read again and high sensivity seems to mean ignore bitrate, so whats the maximum sane sensivity?

DeathTheSheep
26th January 2008, 22:30
And I was right all along, practically. QP is best for VAQ, is what I said. :)

Anyway, I got satd/tesa and me-prepass working with 721. But I have a question about prepass (optimization) I'd like to discuss with you, Dark Shikari (read: Good Morning! You've got mail!). :D

And how goes adding the so-called erroneous "rounding constant" back in? Like I've said, for anime, 0.47 doesn't help where it should, instead throwing bits at frames which clearly don't need them. See gabext.com/samples/.

Dark Shikari
26th January 2008, 22:46
So Qcomp = 1 does constant quant and AQ with sensivity = high then does constant quality, completely ignoring the bitrate? Is this correct? I sort of followed the thread, but didn't have much time to test and I'd find a "real" constant quality very useful, as crf often doesn't produce a constant quality, even with Qcomp = 1

Edit: read again and high sensivity seems to mean ignore bitrate, so whats the maximum sane sensivity?Ignoring clipping of QPs, sensitivity rising is equivalent to CRF dropping. As I said, try AQ strength 0.5-0.6, AQ sensitivity 15, qcomp 1.

Morte66
27th January 2008, 00:04
All range tests, qcomp 1.0 with levels/deblock/denoise/deband...

Entourage 211 DVD

crf 18 no AQ: 17.7MB
crf 18 VAQ strength 1.0 sensitivity 9: 12.5 MB
crf 18 VAQ strength 1.0 sensitivity 13: 23.2 MB
crf 18 VAQ strength 1.0 sensitivity 11: 18.1 MB
crf 18 VAQ strength 0.6 sensitivity 11: 15.8 MB
{crf 18 qcomp 0.6 no AQ: 14.4 MB}

Deadwood 307 DVD (unusually dark and shadowy episode)

crf 18 no AQ: 8.81MB
crf 18 VAQ strength 1.0 sensitivity 11: 14.1 MB
crf 18 VAQ strength 1.0 sensitivity 9: 10.3 MB
crf 18 VAQ strength 1.0 sensitivity 8: 8.52 MB
crf 18 VAQ strength 0.6 sensitivity 8: 7.75 MB
{crf 18 qcomp 0.6 no AQ: 12.7 MB}


Trois Couleurs Blanc DVD

crf 18 no AQ: 37.2 MB
crf 18 VAQ strength 1.0 sensitivity 8: 28.0 MB
crf 18 VAQ strength 1.0 sensitivity 10: 45.9 MB
crf 18 VAQ strength 1.0 sensitivity 9: 37.0 MB
crf 18 VAQ strength 0.6 sensitivity 9: 29.5 MB
{crf 18 qcomp 0.6 no AQ: 35.7 MB}


Irreversible DVD

crf 18 no AQ: 44.1 MB
crf 18 VAQ strength 1.0 sensitivity 9: 47.0 MB
crf 18 VAQ strength 1.0 sensitivity 8: 36.6 MB
crf 18 VAQ strength 0.6 sensitivity 9: 39.1 MB
{crf 18 qcomp 0.6 no AQ: 42.6 MB}


Battlestar Galactica Razor DVD

crf 18 no AQ: 70.0 MB
crf 18 VAQ strength 1.0 sensitivity 9: 66.5 MB
crf 18 VAQ strength 1.0 sensitivity 10: 82.4 MB
crf 18 VAQ strength 0.6 sensitivity 9: 58.8 MB
{crf 18 qcomp 0.6 no AQ: 68.0 MB}


The Magic Mile MPEG2 SDTV

crf 18 no AQ: 22.2 MB
crf 18 VAQ strength 1.0 sensitivity 9: 18.4 MB
crf 18 VAQ strength 1.0 sensitivity 10: 23.1 MB
crf 18 VAQ strength 0.6 sensitivity 10: 19.6 MB
{crf 18 qcomp 0.6 no AQ: 18.5 MB}


Nine with outliers again.

Dark Shikari
27th January 2008, 00:17
How about comparing to no-AQ CRF *with* qcomp on default? (i.e. qcomp = 0.6)

Morte66
27th January 2008, 00:45
How about comparing to no-AQ CRF *with* qcomp?

{added}

{busy for next ~20 hours}

Jawed
27th January 2008, 00:53
Sigh, ignore, comprehension...

Jawed

Yoshiyuki Blade
27th January 2008, 02:49
I have a n00b question. What does lowerering the AQ strength do besides create a smaller file size at a given sensitivity? lol. Say from 1.0 to the (now suggested) 0.5-0.6. There's not much I know beyond that.

Dark Shikari
27th January 2008, 02:52
I have a n00b question. What does lowerering the AQ strength do besides create a smaller file size at a given sensitivity? lol. Say from 1.0 to the (now suggested) 0.5-0.6. There's not much I know beyond that.Because the new VAQ redistributes quantizers; unlike the old AQ, it doesn't just lower them. On many sources, this means a decrease in file size.

DeathTheSheep
27th January 2008, 02:53
Strength 0.5 to 0.6 is now recommended for 0.47? Looks like I missed something here. ;)

Yoshiyuki Blade
27th January 2008, 02:57
Strength 0.5 to 0.6 is now recommended for 0.47? Looks like I missed something here. ;)

I wouldn't say recommended, but DS has suggested it a few posts ago, as well as setting qcomp to 1 for the time being. Unfortunately, it's gonna be hours till I get some results O_O!

Sagekilla
27th January 2008, 03:09
Because the new VAQ redistributes quantizers; unlike the old AQ, it doesn't just lower them. On many sources, this means a decrease in file size.

See, in my eyes AQ was an algorithm that rearranged quants in a frame to get the best quality, not one that would just "lower quants" like Haali's AQ does.


On a side note, if qcomp is set to 1 and aq-strength is varied, isn't that just making AQ the rate control algorithm?

akupenguin
27th January 2008, 04:08
Yes. Static-mode AQ is very similar algorithm-wise to CRF, it just uses a different complexity metric and applies to MBs.

fields_g
27th January 2008, 05:09
I have a n00b question. What does lowerering the AQ strength do besides create a smaller file size at a given sensitivity? lol. Say from 1.0 to the (now suggested) 0.5-0.6. There's not much I know beyond that.

I want anyone to feel free to correct me.

FIRST think of Sensitivity.... What level of flatness should be considered necessary adjustment. THEN.. knowing what needs to be changed, Strength determines how radical the adjustment is.

Higher Sensitivity, more should be fixed.
Higher Strength, more radically the attempt to fix it.
This is why while lowering one of these and raising the other, you can get a file with the same bitrate.

Yoshiyuki Blade
27th January 2008, 06:14
From what I've tested so far, AQ on anime look much better with strong settings (such as --aq-strength 1.0 --aq-sensitivity 20). Even though it makes file sizes large at a given crf, it still looks nice at smaller file sizes too, though doing that will compromise overall image quality.

Setting qcomp 1, aq str 0.5, and sensitivity 20 looks nice, and the file size is small, but dark areas don't get improved much in comparison. I'm going to re-run the same settings but change strength to 1.0 and see how it turns out. Sorry that I can't post numbers, I'm unorganized and everything's a mess :D. I'll try and get a consistent order going.

Morte66
27th January 2008, 10:25
Slight diversion from crf: any thoughts about qcomp and sensitivity for 2-pass encode to a target size with 0.47?

Dark Shikari
27th January 2008, 10:28
Slight diversion from crf: any thoughts about qcomp and sensitivity for 2-pass encode to a target size with 0.47?qcomp = 1, sensitivity = 10-15? Sensitivity is much less meaningful with 2pass.

burfadel
27th January 2008, 10:33
I did some tests on 'A knight's tale' and 'the 6th day', on both occasions I found the best aq strength and sensitivity, in regards to these two clips as strength 0.6 and senstivity 17.6 to maintain the same file size (very slightly larger, I mean by about 0.5 percent) and be of good quality. I didn't set qcomp.

Again, setting the sensitivity lower than that, even with qcomp being 1, the filesize was way undersized. The quality was also noticeable lower, but was about right in terms of its file size.

Setting aq strength to 1 (auto sens.) with the same clips ended up with doubling the file size.

I keep ending up with a sensitivity of around 17 for strengths of say, 0.6, and slightly lower at around 15 or 16 for strength 1

Dark Shikari
27th January 2008, 10:51
Before I head out for the day, let me throw up a concept to be shot down (in the full knowledge that I've no idea what I'm talking about). This describes functionality, not necessarily algorithm:

The encoder chugs along, frame after frame. Within each individual frame, it calculates a spatial bitrate distribution according to the new Variance AQ's definition of quality. Then at the whole frame level, it measures the quality according to the vanilla CRF metric, the one that's not PSNR or SSIM but "implicit in the algorithm", and varies the whole-frame bit budget to hit a quality level measured that way. So VAQ does spatial bitrate distribution within frames, and non-AQ does temporally varying bitrate allocation across frames.

I wonder if that would give the excellent smooth/dark area handling by using VAQ spatially within frames, then make sure the frames are crisp enough by using the current CRF metric (which seems more dependable re crispness than VAQCRF 0.47).Using CRF for frame QP distribution and VAQ for within-frame bit distribution can be done using --aq-sensitivity 0 and adjusting qcomp to however strong you want CRF to be (lower = stronger) ;)

Morte66
27th January 2008, 11:08
Using CRF for frame QP distribution and VAQ for within-frame bit distribution can be done using --aq-sensitivity 0 and adjusting qcomp to however strong you want CRF to be (lower = stronger) ;)

I thought that approach had a problem on frames with a small amount of sharp detail that gets clobbered by variance AQ to feed the large amount of smooth stuff?

I was thinking particularly of CRF measuring quality after VAQ has done spatial bitrate distribution, so it would feed extra bitrate in from the top to resharpen the sharp bits of frames like that. Or is that what it does anyhow?

{I seem to have deleted my original post by accident (!), but DS quoted it whole.}

Jawed
27th January 2008, 13:15
Yes. Static-mode AQ is very similar algorithm-wise to CRF, it just uses a different complexity metric and applies to MBs.
Should VAQ constitute an x264 mode alongside crf, qp, bitrate, pass ?

Jawed

Dark Shikari
27th January 2008, 13:17
Should VAQ constitute an x264 mode alongside crf, qp, bitrate, pass ?

JawedWell, VAQ works with all of those modes (though, technically, qcomp=1 and CRF is equivalent to QP mode)... so not really, I think.

Jawed
27th January 2008, 15:48
A few months back I experimented with qcomp in conjunction with HAQ (pre-processed to PC Levels + MVDegrain2). It certainly produced interesting results:


AQ 0.3 5 qcomp 0 qcomp 1 source bitrate
24 1827.9 2272.1 1621.0 4762.7
Black Pearl 2270.7 1671.9 2924.5 3639.8
Carnivale 2485.2 2115.3 2942.0 6623.8
Hana & Alice 1 2102.7 2557.1 1958.2 4015.5
Monk 3207.5 2132.0 4358.8 8354.6
War of the Worlds 6829.3 3047.2 11542.4 7059.3
X-Files 1951.0 2275.5 1693.9 4821.1
Average 2953.5 2295.9 3863.0 5611.0


Note all three test encodes are with the same HAQ setting, --aq-strength 0.3 --aq-sensitivity 5. The other x264 settings were:

--crf 16 --ref 5 --mixed-refs --no-fast-pskip --bframes 8 --b-pyramid --b-rdo --bime --weightb --subme 6 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --merange 32 --no-dct-decimate

qcomp is a bit surprising. Maybe you guys are used to it.

I've attached the spreadsheet containing a bit more detail.

Jawed

bob0r
27th January 2008, 18:54
AQ patch 0.47: http://akuvian.org/src/x264/x264_aq_var.47.diff

x264.735.dark.aq.0.47.exe (http://files.x264.nl/AQ/x264.735.dark.aq.0.47.exe) (pthreads/mp4 = yes, made with make fprofiled)

Should work i guess, patch did apply.

nurbs
27th January 2008, 19:07
I did some encodes with qcomp = 1. With aq-strenght 0.5 and sensitivity 10 it gives me similar filesizes to my previous encodes with normal qcomp and without aq. The videos look very good.

Yoshiyuki Blade
27th January 2008, 20:43
After some more testing, older anime with lots of noise and details look better (in my opinion) with high CRF and high AQ strength vs low CRF and low AQ strength (at a fixed sensitivity). Both results have their ups and downs.

-Low CRF and Low AQ strength gives a cleaner overall picture as expected, AQ but hardly seems to address ugly blocking in dark areas.

-High CRF and High AQ strength results in a worse overall image quality (as expected), but dark areas look much better.

This trend happens at lower bitrates. At high bitrates, the results look very similar to each other (very good quality across the board).

I'm going to edit this post in a couple hours with some examples. In the meantime, this is what I have so far:

--crf 28.0 --qcomp 1 aq-strength 0.5 aq-sensitivity 20: 188 MB (Sample 1 (http://www.mediafire.com/?2axiwywmdzm)) (Sample 2 (http://www.mediafire.com/?49ne1jndn61))
--crf 33.0 --qcomp 1 aq-strength 1.0 aq-sensitivity 20: 199 MB (Sample 1 (http://www.mediafire.com/?djnj12he71x)) (Sample 2 (http://www.mediafire.com/?e6yj4kjy1jz))
--crf 34.0 --qcomp 1 aq-strength 1.0 aq-sensitivity 20: 161 MB

Update: Uploaded the samples. To be honest, none of them are satisfactory lol, but I expect them to look much better in a 2-pass encode. I did a 2-pass encode a while ago (before patch .47) at lower bitrates than those, and it still looks better than both.

I'm currently re-running tests with 2-pass at a higher bitrate not only to keep the final filesizes very close to each other (there's an 11 MB bias in the tests above), but to narrow the gap on the differences between high CRF/high AQ strength vs low CRF/low AQ strength. The next set of results will look much easier on the eyes.

Raere
27th January 2008, 21:38
Are low AQ values recommended as a general psychovisual improvement at this point, or is it still just a tradeoff for lower overall quality for better dark areas?

Dark Shikari
27th January 2008, 21:40
Are low AQ values recommended as a general psychovisual improvement at this point, or is it still just a tradeoff for lower overall quality for better dark areas?VAQ is supposed to be a general psychovisual improvement, period. Too high AQ strengths may backfire though.