Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th May 2009, 22:52   #1  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 511
AutoVAQ patch testing

I think it is the time for testing my experimental AutoVAQ patch (updated 03 version which doesn't overwrite standard VAQ and add "--aq-mode 2" option for activating AutoVAQ) in wider audience than users of #x264dev IRC-channel and small Russian-speaking group of people. The main purpose of this patch was to make algorithm automaticaly choose close to optimal strength of AQ (on per frame base) and by this increase quality of encoding (without need of manually finding optimal strength for every source).
Patch does not add or remove any x264 options but replace the current offical VAQ implementation. The algorithm is tweaked to "--aq-strength 1.0" (which is default) and changing of this parameter is not recommended.
For testing you may use:
- my reference x264 CLI build based on x264 revision 1148 (non fprofiled, without MP4 support, no other patches)
- Komisar's kAVAQ builds
- LoRd_MuldeR's libx264 binaries for Avidemux
- or build x264 yourself with AutoVAQ patch
For comparison you may need clean x264 CLI build.

Here is comparison of current offical VAQ (StdVAQ) and my AutoVAQ builds on two rather different samples (for which optimal strength of StdVAQ differ considerably):

Parkrun encodes of Parkrun source with base options:
Quote:
--fps 25 --threads 3 --8x8dct --b-adapt 2 --bframes 4 --b-pyramid --weightb --ref 3 --mixed-refs --me umh --subme 9 --trellis 2 --no-fast-pskip
without Psy options (--psy-rd 0.0:0.0):
Quote:
StdVAQ --crf 28.00 --aq-strength 1.0 SSIM Y: 0.8383909 APSNR: 28.364 OPSNR: 28.001 kb/s: 1568.94
StdVAQ --crf 26.15 --aq-strength 1.5 SSIM Y: 0.8486719 APSNR: 28.151 OPSNR: 27.764 kb/s: 1568.91
StdVAQ --crf 24.65 --aq-strength 2.0 SSIM Y: 0.8514804 APSNR: 27.749 OPSNR: 27.292 kb/s: 1571.41
AutoVAQ --crf 30.23 --aq-strength 1.0 SSIM Y: 0.8586000 APSNR: 28.077 OPSNR: 27.724 kb/s: 1567.00
with Psy options (--psy-rd 1.0:0.3):
Quote:
StdVAQ --crf 29.33 --aq-strength 1.0 SSIM Y: 0.8316043 APSNR: 27.694 OPSNR: 27.318 kb/s: 1570.51
StdVAQ --crf 27.56 --aq-strength 1.5 SSIM Y: 0.8395926 APSNR: 27.439 OPSNR: 27.039 kb/s: 1571.78
StdVAQ --crf 26.18 --aq-strength 2.0 SSIM Y: 0.8397329 APSNR: 26.980 OPSNR: 26.509 kb/s: 1573.75
AutoVAQ --crf 31.58 --aq-strength 1.0 SSIM Y: 0.8500254 APSNR: 27.386 OPSNR: 27.021 kb/s: 1568.49
Azumanga encodes of Azumanga source with base options:
Quote:
--frames 1156 --fps 25 --threads 3 --8x8dct --b-adapt 2 --bframes 4 --b-pyramid --weightb --ref 3 --mixed-refs --me umh --subme 9 --trellis 2 --no-fast-pskip
without Psy options (--psy-rd 0.0:0.0):
Quote:
StdVAQ --crf 29.55 --aq-strength 0.5 SSIM Y: 0.9921343 APSNR: 41.893 OPSNR: 41.173 kb/s: 297.54
StdVAQ --crf 28.00 --aq-strength 1.0 SSIM Y: 0.9922015 APSNR: 41.575 OPSNR: 40.809 kb/s: 297.78
StdVAQ --crf 26.63 --aq-strength 1.5 SSIM Y: 0.9919121 APSNR: 41.084 OPSNR: 40.249 kb/s: 297.91
StdVAQ --crf 25.50 --aq-strength 2.0 SSIM Y: 0.9911979 APSNR: 40.387 OPSNR: 39.479 kb/s: 298.04
AutoVAQ --crf 26.78 --aq-strength 1.0 SSIM Y: 0.9921364 APSNR: 41.837 OPSNR: 40.989 kb/s: 297.46
with Psy options (--psy-rd 1.0:0.3):
Quote:
StdVAQ --crf 31.13 --aq-strength 0.5 SSIM Y: 0.9903588 APSNR: 41.438 OPSNR: 40.598 kb/s: 297.76
StdVAQ --crf 29.67 --aq-strength 1.0 SSIM Y: 0.9901783 APSNR: 40.989 OPSNR: 40.099 kb/s: 298.11
StdVAQ --crf 28.55 --aq-strength 1.5 SSIM Y: 0.9893354 APSNR: 40.279 OPSNR: 39.318 kb/s: 298.30
StdVAQ --crf 27.60 --aq-strength 2.0 SSIM Y: 0.9880089 APSNR: 39.444 OPSNR: 38.410 kb/s: 298.44
AutoVAQ --crf 28.35 --aq-strength 1.0 SSIM Y: 0.9903360 APSNR: 41.409 OPSNR: 40.418 kb/s: 297.76
I wouldn't make any conclusions from these tests. Make your own comparison (on other sources) and write yours results and conclusions.

Last edited by MasterNobody; 30th June 2009 at 09:48.
MasterNobody is offline   Reply With Quote
Old 13th May 2009, 23:01   #2  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,956
Hi. I noticed that with the AutoVAQ patch I get significant smaller files in CRF mode. Like ~15% smaller at CRF=22.

Of course I can lower the CRF value by one or two in order to compensate. But is this intended?
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 13th May 2009 at 23:06.
LoRd_MuldeR is offline   Reply With Quote
Old 13th May 2009, 23:09   #3  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 511
Quote:
Originally Posted by LoRd_MuldeR View Post
Hi. I noticed that with the AutoVAQ patch I get significant smaller files in CRF mode. Like ~15% with CRF=22.

Of course I can lower the CRF value by one or two in order to compensate. But is this intended?
Yes, this is intended that it changes CRF/size ratio and you need probably find new CRF which give good quality (and it can not only decrease size but also increase it on other sources). Look at my comparison. For Parkrun sample it increases size on the same CRF and for Azumanga sample it decreases size on the same CRF.

Last edited by MasterNobody; 13th May 2009 at 23:14.
MasterNobody is offline   Reply With Quote
Old 13th May 2009, 23:11   #4  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
When I eventually commit this, it will (for the fourth time) rebalance CRF.
Dark Shikari is offline   Reply With Quote
Old 13th May 2009, 23:15   #5  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,956
Okay. Thanks for the info.

(BTW: May it be possible to apply this patch in a way that doesn't need GCC 4.x and the "force_align_arg_pointer" attribute to prevent crash? Tried using "x264_stack_align()", but it only accepts one param)
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 13th May 2009 at 23:20.
LoRd_MuldeR is offline   Reply With Quote
Old 13th May 2009, 23:56   #6  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,956




Encoding settings: cabac=1 / ref=8 / deblock=1:0:0 / analyse=0x3:0x133 / me=umh / subme=9 / psy_rd=1.0:0.2 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-3 / threads=6 / nr=0 / decimate=1 / mbaff=0 / bframes=5 / b_pyramid=1 / b_adapt=2 / b_bias=0 / direct=3 / wpredb=1 / keyint=250 / keyint_min=25 / scenecut=40 / rc=2pass / bitrate=1500 / ratetol=1.0 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / ip_ratio=1.40 / pb_ratio=1.30 / aq=1:1.00

I find the Non-AutoVAQ version more pleasing with this source
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 14th May 2009 at 00:07.
LoRd_MuldeR is offline   Reply With Quote
Old 14th May 2009, 00:08   #7  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Single-frame comparisons are completely useless for this patch for the following reasons:

1. The patch inherently reverts AQ's effect on redistribution of bits. This may reduce the quality of some frames and increase the quality of others.

2. This patch changes AQ on a per-frame basis. In theory, some frames could get worse.
Dark Shikari is offline   Reply With Quote
Old 14th May 2009, 00:13   #8  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,956
Somehow I need to compare and it's really hard to compare two encodes in full playback speed. So I step through the interleaved video frame by frame. How do you do it?

Also I can find many frames like that in my encode where the Non-AutoVAQ version looks more pleasing. But I hardly see any frames that actually look better with AutoVAQ.
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Old 14th May 2009, 00:22   #9  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Quote:
Originally Posted by LoRd_MuldeR View Post
Somehow I need to compare and it's really hard to compare two encodes in full playback speed. So I step through the interleaved video frame by frame. How do you do it?

Also I can find many frames like that in my encode where the Non-AutoVAQ version looks more pleasing. But I hardly see any frames that actually look better with AutoVAQ.
Pick a whole bunch of different frames.

Also, adjust the AQ strength. If it is overall worse, try adjusting the AQ strength until you can make it look better. If you can't, then it's worse, if you can, then its default is bad.
Dark Shikari is offline   Reply With Quote
Old 14th May 2009, 00:27   #10  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Specifically, there are three things about this patch that have to be tested separately:

1. Is its default AQ value good?

2. Is the ratio between AQ values in different frames good? That is, is the "auto" part of AutoVAQ good?

3. Is the distribution of quantizers in a single frame, ignoring the AQ strength of that frame, good? This patch is not really VAQ, as it uses a different metric. I'd call it more like "Hadamard Variance Adaptive Quantization", or HVAQ. Or VHAQ.
Dark Shikari is offline   Reply With Quote
Old 14th May 2009, 00:28   #11  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 511
Quote:
Originally Posted by LoRd_MuldeR View Post
Also I can find many frames like that in my encode where the Non-AutoVAQ version looks more pleasing. But I hardly see any frames that actually look better with AutoVAQ.
I would recommend to compare CRF-encodes with close bitrates (because of problem with few first frames in 2-pass encoding and high quant for first I-frame) or at least at long samples (with more than three I-frames). Also it would be good if you provide the source sample if you think that at it AutoVAQ gives considerably worse results than StdVAQ. How long is this sample?
MasterNobody is offline   Reply With Quote
Old 14th May 2009, 00:29   #12  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,956
Quote:
Originally Posted by Dark Shikari View Post
Pick a whole bunch of different frames.
Well, I scanned through the entire video and I easily spotted worse looking frames, but no better looking ones.

Quote:
Originally Posted by Dark Shikari View Post
Also, adjust the AQ strength. If it is overall worse, try adjusting the AQ strength until you can make it look better. If you can't, then it's worse, if you can, then its default is bad.
Will try to do some more tests with different AQ strengths...
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.


LoRd_MuldeR is offline   Reply With Quote
Old 14th May 2009, 00:33   #13  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 12,956
Quote:
Originally Posted by MasterNobody View Post
I would recommend to compare CRF-encodes with close bitrates (because of problem with few first frames in 2-pass encoding and high quant for first I-frame) or at least at long samples (with more than three I-frames). Also it would be good if you provide the source sample if you think that at it AutoVAQ gives considerably worse results than StdVAQ. How long is this sample?
From all that I learned using CRF for comparison is a bad idea. Especially because the AutoVAQ version would be ~15% smaller in size. Using two 2-Pass encodes of the same (average) bitrate is highly recommended. Also my test sequence was a 5000 frame clip (taken from an even longer source). Long enough I guess. And the screenshot above was not taken from the very first part of the sequence.
__________________
There was of course no way of knowing whether you were being watched at any given moment.
How often, or on what system, the Thought Police plugged in on any individual wire was guesswork.



Last edited by LoRd_MuldeR; 14th May 2009 at 00:36.
LoRd_MuldeR is offline   Reply With Quote
Old 14th May 2009, 00:38   #14  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 511
Quote:
Originally Posted by LoRd_MuldeR View Post
From all that I learned using CRF for comparison is a bad idea.
I am not talking about comparison on the same CRF but on the CRFs that gives close bitrate.
Quote:
Originally Posted by LoRd_MuldeR View Post
Also my test sequence was a 5000 frame clip (taken from an even longer source). Long enough I guess.
Yes, this is long enough.
MasterNobody is offline   Reply With Quote
Old 14th May 2009, 02:42   #15  |  Link
kemuri-_9
Compiling Encoder
 
kemuri-_9's Avatar
 
Join Date: Jan 2007
Posts: 1,348
hmm...

i would gander that if the new algo permanently replaces the current algo, there's probably gonna be (yet another) rush of complaints,
with some of them being 'I want the old version back because of xxxx!'

I also didn't notice anything in the patch code to prevent the aq-strength from being altered,
this would be currently due to the testing state of the patch to allow modifications for further testing,
but it has already been stated to be optimized for aq-strength=1.0,
so will the final version prevent such changes or at least warn when the value is changed?

I would say the ride would be less bumpy if it was instituted as a new aq-mode rather than completely replacing the current algorithm...
even if the majority consensus says the new version is better, chances are there will be some that claim the old version is better and would like to keep using it.
(On this point, what is the consensus as of yet?)


more on the testing sides of things when i find the time....
How long should a test sample be for it to be long enough to properly test the new algorithm? (and what should we expect to occur with short samples?)

and as a person who generally does 2pass bitrate encodes, what changes should i expect to occur (ssim/psnr/visual changes, etc.)? (as the bitrate won't change like it is in CRF)
__________________
custom x264 builds & patches | F@H | My Specs

Last edited by kemuri-_9; 14th May 2009 at 02:48.
kemuri-_9 is offline   Reply With Quote
Old 14th May 2009, 09:24   #16  |  Link
Sharktooth
Mr. Sandman
 
Sharktooth's Avatar
 
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
if the old method is objectively visually better than the new one i dont see any reason why the new method should replace the old one.
just a couple of point of SSIM score do not justify ANY visual detail loss.
on the other hand, if the new method is visually consistently better than the old one, go on...

Last edited by Sharktooth; 14th May 2009 at 09:33.
Sharktooth is offline   Reply With Quote
Old 14th May 2009, 10:43   #17  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,235
If the lower quality is attributed to a lower bitrate of ~15 percent (may vary) for a given CRF, it sounds reasonable to say that for a set bitrate the quality would be noticeably improved. It would also be reasonable to suggest to patch x264 to increase the bitrate used in CRF mode since you now have some spare, to the point where the image quality is actually improved (and still probably saving some bitrate)! It doesn't matter if the SSIM is reasonably higher in this case, the main point would be the increased quality at a given CRF whilst still saving a few percentage points of quality for most encodes. Complaints only occur if people have to change the CRF they use from say, 24 to 22 to make up the shortfall in quality.

To summarise, regardless of bitrate saving it would be more important to adjust CRF mode so people can use the same settings in CRF mode and have better quality, without using more bitrate!
burfadel is offline   Reply With Quote
Old 14th May 2009, 10:51   #18  |  Link
audyovydeo
Registered User
 
audyovydeo's Avatar
 
Join Date: Apr 2007
Posts: 464
Quote:
Originally Posted by burfadel View Post
If the lower quality is attributed to a lower bitrate of ~15 percent (may vary) for a given CRF, it sounds reasonable to say that for a set bitrate the quality would be noticeably improved. It would also be reasonable to suggest to patch x264 to increase the bitrate used in CRF mode since you now have some spare, to the point where the image quality is actually improved (and still probably saving some bitrate)! It doesn't matter if the SSIM is reasonably higher in this case, the main point would be the increased quality at a given CRF whilst still saving a few percentage points of quality for most encodes. Complaints only occur if people have to change the CRF they use from say, 24 to 22 to make up the shortfall in quality.

To summarise, regardless of bitrate saving it would be more important to adjust CRF mode so people can use the same settings in CRF mode and have better quality, without using more bitrate!

Improvements are always welcome.
On the other hand, given the time-consuming process of empirically finding a new CRF value range by each user, the less CRF changes, the better.
As things stand, it would seems wise (as in : user-friendly) to add this new patch as an option, as has been suggested in the earlier post

cheers
audyovydeo
audyovydeo is offline   Reply With Quote
Old 14th May 2009, 10:58   #19  |  Link
nurbs
Registered User
 
Join Date: Dec 2005
Posts: 1,457
Well, there is already an --aq-mode option, so I don't see how making this optional can be a problem. Looks like an interesting change. I'll have time to test this at the weekend.
nurbs is offline   Reply With Quote
Old 14th May 2009, 11:00   #20  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,689
Quote:
Originally Posted by nurbs View Post
Well, there is already an --aq-mode option, so I don't see how making this optional can be a problem. Looks like an interesting change. I'll have time to test this at the weekend.
Yes, if there are any reasons to keep the old algorithm, I will make both available as --aq-modes.
Dark Shikari is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.