Variance AQ Megathread (AQ v0.48 update--defaults changed)

[DeathTheSheep]

Does anyone know where I can find 0.46? It seems to have vanished, and I'd like to upgrade from 0.45.

[edit] w007

- var += ssd - (sad * sad >> (i?6:8));
+ var += ssd - ((sad/16) * (sad/16));


I win. :p

Y4yz 4 err0r-ous (0[)3 |=+\X/

[burfadel]

Anyone else notice that in general, the higher the input resolution he lower the sensitivity required? So a sensitivity of say, 9 is good for 1920x1080, whereas for much lower resolutions a 16 or 17 etc is required? Also the strength affects whether a higher or lower sensitivity is more beneficial.

That makes a fixed sensitivity almost impossible, maybe an automatic sensitivity that can adjust to the input resolution and AQ strength?

[Dark Shikari]

Does anyone know where I can find 0.46? It seems to have vanished, and I'd like to upgrade from 0.45.

[edit] w007

- var += ssd - (sad * sad >> (i?6:8));
+ var += ssd - ((sad/16) * (sad/16));


I win. :p

Y4yz 4 err0r-ous (0[)3 |=+\X/That will fail because i?6:8 compensates for the different sizes of the SSDs in different planes, while your code does not.
Anyone else notice that in general, the higher the input resolution he lower the sensitivity required? So a sensitivity of say, 9 is good for 1920x1080, whereas for much lower resolutions a 16 or 17 etc is required?

That makes a fixed sensitivity almost impossible, maybe an automatic sensitivity that can adjust to the input resolution?This reflects the fact that lower resolutions generally have higher average block variances, since there are more edges and fewer large areas of flat detail.

[DeathTheSheep]

Do you have a link to 0.46? :)

If not, how do I add this compensation?

[Dark Shikari]

Do you have a link to 0.46? :)

If not, how do I add this compensation?Divide by 8 for chroma planes, 16 for luma planes.

[DeathTheSheep]

Divide which part? The whole expression, the (sad/16)^2 chunk only, or...? And how can I test with a neat little "colon question mark" if (like something below)?

var += ssd - ((sad/16) * (sad/16)) / (i?8:16);


Is this even the right syntax for such a trick? And is there a neat bitshift >> or whatnot for "divide by 16"? Admittedly, I know very little when it comes to syntax/speed notation.

[burfadel]

This reflects the fact that lower resolutions generally have higher average block variances, since there are more edges and fewer large areas of flat detail.

I found that:
--aq-strength 0.3 --aq-sensitivity 16.5
Seems to maintain an almost identical file size to not having AQ on for both very low res and Full HD encodes, although strength 0.3 is very little. With even a strength of 0.6 the difference is too great between low and high resolutions to the normal, non AQ builds. - that is, what works for high res doesn't work for low res and vice versa.

[akupenguin]

var += ssd - ((sad/16) * (sad/16)) / (i?8:16);

No, and even if you corrected it it would just bring back the imprecision.
The compensation would be:
- unsigned int var=0
+ unsigned int var=42
for some value of 42. Or better yet, add it after the computation, so that the satd shortcut still works.

[CruNcher]

hmm is it possible that the Visual Quality decreased? i see ringing now @ Face Edges (something i saw last time with ASP but not used to see with H.264) aq-strength of 1.0 only lower strength avoid this now.

[burfadel]

hmm is it possible that the Visual Quality decreased? i see ringing now @ Face Edges (something i saw last time with ASP but not used to see with H.264) aq-strength of 1.0 only lower strength avoid this now.

Have you checked the file size with and without AQ? I found at a strength of 1 NO senstivity suited all scenarios. There was a very large difference in terms of comparing it with its non-AQ counterpart between HD and resolution such as 512x384. What sensitivity at that strength works for one DOES NOT work for the other.

Even at a strength of 0.6, the sensitivities made a large difference with different resolutions. The only figure that worked perfectly for low and high resolutions for me, was strength 0.3 and sensitivity 16.5. (16 was giving a slightly lower file size, 16.5 fractionally larger, which is ok!).

It did help in the dark areas still, although its very weak at those settings, although I'd have to agree with you about strength 1, it did make the faces etc a bit less in terms of quality. That doesn't occur with strength 0.3. It was as if I raised the CRF in a way... Problem is to have an ideal quality in the dark areas, taking bits away from bright areas will lower its quality, and it seems having a strong AQ takes too much away from the bright areas. I'm not even sure that it maintains a CRF of say 24, with aq enabled at strength 1, it lowers the bright area quality and the filesize can become way undersized - or oversized!

Try strength 0.3 and sensitivity 16.5 and see whether that helps you like it did with me :) - and its ideal for both very low res and full HD

[DeathTheSheep]

Since the one posted here earlier wasn't at all valid, here's my updated version:
Updated me-prepass (unoptimized):
--- common/common.c Sun Jan 27 13:39:08 2008
+++ common/common.c Sun Jan 27 13:39:08 2008
@@ -441,6 +441,8 @@
p->analyse.i_mv_range_thread = atoi(value);
OPT2("subme", "subq")
p->analyse.i_subpel_refine = atoi(value);
+ OPT2("me-prepass", "meprepass")
+ p->analyse.i_me_prepass = atobool(value);
OPT("bime")
p->analyse.b_bidir_me = atobool(value);
OPT("chroma-me")
@@ -879,6 +881,7 @@
s += sprintf( s, " analyse=%#x:%#x", p->analyse.intra, p->analyse.inter );
s += sprintf( s, " me=%s", x264_motion_est_names[ p->analyse.i_me_method ] );
s += sprintf( s, " subme=%d", p->analyse.i_subpel_refine );
+ s += sprintf( s, " me-prepass=%d", p->analyse.i_me_prepass );
s += sprintf( s, " brdo=%d", p->analyse.b_bframe_rdo );
s += sprintf( s, " mixed_ref=%d", p->analyse.b_mixed_references );
s += sprintf( s, " me_range=%d", p->analyse.i_me_range );
--- x264.c Sun Jan 27 13:39:08 2008
+++ x264.c Sun Jan 27 13:39:08 2008
@@ -232,7 +232,8 @@
H1( " --mvrange-thread <int> Minimum buffer between threads [-1 (auto)]\n" );
H0( " -m, --subme <integer> Subpixel motion estimation and partition\n"
" decision quality: 1=fast, 7=best. [%d]\n", defaults->analyse.i_subpel_refine );
- H0( " --b-rdo RD based mode decision for B-frames. Requires subme 6.\n" );
+ H0( " --me-prepass Run an ME prepass on predictors. Requires subme 3 or higher.\n");
+ H0( " --b-rdo RD based mode decision for B-frames. Requires subme 6 or higher.\n" );
H0( " --mixed-refs Decide references on a per partition basis\n" );
H1( " --no-chroma-me Ignore chroma in motion estimation\n" );
H1( " --bime Jointly optimize both MVs in B-frames\n" );
@@ -398,6 +399,7 @@
{ "mvrange", required_argument, NULL, 0 },
{ "mvrange-thread", required_argument, NULL, 0 },
{ "subme", required_argument, NULL, 'm' },
+ { "me-prepass", no_argument, NULL, 0 },
{ "b-rdo", no_argument, NULL, 0 },
{ "mixed-refs", no_argument, NULL, 0 },
{ "no-chroma-me", no_argument, NULL, 0 },
--- x264.h Sun Jan 27 13:39:08 2008
+++ x264.h Sun Jan 27 13:39:08 2008
@@ -220,6 +220,7 @@
int i_mv_range; /* maximum length of a mv (in pixels). -1 = auto, based on level */
int i_mv_range_thread; /* minimum space between threads. -1 = auto, based on number of threads. */
int i_subpel_refine; /* subpixel motion estimation quality */
+ int i_me_prepass; /* run an ME prepass on predictors */
int b_bidir_me; /* jointly optimize both MVs in B-frames */
int b_chroma_me; /* chroma ME for subpel and mode decision in P-frames */
int b_bframe_rdo; /* RD based mode decision for B-frames */
--- encoder/me.c Sun Jan 27 13:39:08 2008
+++ encoder/me.c Sun Jan 27 20:25:50 2008
@@ -69,6 +69,23 @@
COPY3_IF_LT( bpred_cost, cost, bpred_mx, mx, bpred_my, my ); \
}

+#define COST_MV_HPEL2( mx, my, cost ) \
+{ \
+ int stride = 16; \
+ uint8_t *src = h->mc.get_ref( pix, &stride, m->p_fref, m->i_stride[0], mx, my, bw, bh ); \
+ cost = h->pixf.fpelcmp[i_pixel]( m->p_fenc[0], FENC_STRIDE, src, stride ) \
+ + p_cost_mvx[ mx ] + p_cost_mvy[ my ]; \
+}
+
+#define COST_MV_HPEL3( mx, my) \
+{ \
+ int stride = 16; \
+ uint8_t *src = h->mc.get_ref( pix, &stride, m->p_fref, m->i_stride[0], mx, my, bw, bh ); \
+ int cost = h->pixf.fpelcmp[i_pixel]( m->p_fenc[0], FENC_STRIDE, src, stride ) \
+ + p_cost_mvx[ mx ] + p_cost_mvy[ my ]; \
+ COPY3_IF_LT( bestcost, cost, bestx, mx, besty, my ); \
+}
+
#define COST_MV_X3_DIR( m0x, m0y, m1x, m1y, m2x, m2y, costs )\
{\
uint8_t *pix_base = p_fref + bmx + bmy*m->i_stride[0];\
@@ -171,8 +188,13 @@
int mv_y_min = h->mb.mv_min_fpel[1];
int mv_x_max = h->mb.mv_max_fpel[0];
int mv_y_max = h->mb.mv_max_fpel[1];
+ int mv_x_min4 = h->mb.mv_min_fpel[0]<<2;
+ int mv_y_min4 = h->mb.mv_min_fpel[1]<<2;
+ int mv_x_max4 = h->mb.mv_max_fpel[0]<<2;
+ int mv_y_max4 = h->mb.mv_max_fpel[1]<<2;

#define CHECK_MVRANGE(mx,my) ( mx >= mv_x_min && mx <= mv_x_max && my >= mv_y_min && my <= mv_y_max )
+#define CHECK_MVRANGE4(mx,my) ( mx >= mv_x_min4 && mx <= mv_x_max4 && my >= mv_y_min4 && my <= mv_y_max4 )

const int16_t *p_cost_mvx = m->p_cost_mv - m->mvp[0];
const int16_t *p_cost_mvy = m->p_cost_mv - m->mvp[1];
@@ -183,19 +205,88 @@
pmy = ( bmy + 2 ) >> 2;
bcost = COST_MAX;

- /* try extra predictors if provided */
- if( h->mb.i_subpel_refine >= 3 )
- {
- COST_MV_HPEL( bmx, bmy );
- for( i = 0; i < i_mvc; i++ )
+ /* try extra predictors if provided */
+ if( h->mb.i_subpel_refine >= 3 )
+ {
+ COST_MV_HPEL( bmx, bmy );
+ if(!h->param.analyse.i_me_prepass)
+ {
+ for( i = 0; i < i_mvc; i++ )
+ {
+ const int mx = x264_clip3( mvc[i][0], mv_x_min*4, mv_x_max*4 );
+ const int my = x264_clip3( mvc[i][1], mv_y_min*4, mv_y_max*4 );
+ if( mx != bpred_mx || my != bpred_my )
+ COST_MV_HPEL( mx, my );
+ }
+ }
+ else
{
- int mx = mvc[i][0];
- int my = mvc[i][1];
- if( (mx | my) && ((mx-bmx) | (my-bmy)) )
+ for( i = 0; i < i_mvc; i++ )
{
- mx = x264_clip3( mx, mv_x_min*4, mv_x_max*4 );
- my = x264_clip3( my, mv_y_min*4, mv_y_max*4 );
- COST_MV_HPEL( mx, my );
+ const int mx = x264_clip3( mvc[i][0], mv_x_min*4, mv_x_max*4 );
+ const int my = x264_clip3( mvc[i][1], mv_y_min*4, mv_y_max*4 );
+ int doSearch = 1;
+ int j;
+ for(j = 0; j < i; j++)
+ {
+ if(mvc[i][0] == mvc[j][0] && mvc[i][1] == mvc[j][1]) doSearch = 0;
+ }
+ if( ( mx != bpred_mx || my != bpred_my ) && doSearch)
+ {
+ int bestcost;
+ int bestx = mx;
+ int besty = my;
+ COST_MV_HPEL2( mx, my, bestcost );
+ COPY3_IF_LT( bpred_cost, bestcost, bpred_mx, bestx, bpred_my, besty );
+ if(bestcost < 2*bpred_cost)
+ {
+ int n;
+ int dir = -2;
+ COST_MV_HPEL2(bestx-4,besty,costs[0]);
+ COST_MV_HPEL2(bestx-2,besty+4,costs[1]);
+ COST_MV_HPEL2(bestx+2,besty+4,costs[2]);
+ COST_MV_HPEL2(bestx+4,besty,costs[3]);
+ COST_MV_HPEL2(bestx+2,besty-4,costs[4]);
+ COST_MV_HPEL2(bestx-2,besty-4,costs[5]);
+ COPY2_IF_LT( bestcost, costs[0], dir, 0 );
+ COPY2_IF_LT( bestcost, costs[1], dir, 1 );
+ COPY2_IF_LT( bestcost, costs[2], dir, 2 );
+ COPY2_IF_LT( bestcost, costs[3], dir, 3 );
+ COPY2_IF_LT( bestcost, costs[4], dir, 4 );
+ COPY2_IF_LT( bestcost, costs[5], dir, 5 );
+ if( dir != -2 )
+ {
+ static const int hex2[8][2] = {{-2,-4}, {-4,0}, {-2,4}, {2,4}, {4,0}, {2,-4}, {-2,-4}, {-4,0}};
+ bestx += hex2[dir+1][0];
+ besty += hex2[dir+1][1];
+ for( n = 1; n < i_me_range && CHECK_MVRANGE4(bestx, besty); n++ )
+ {
+ static const int mod6[8] = {5,0,1,2,3,4,5,0};
+ const int odir = mod6[dir+1];
+ COST_MV_HPEL2(hex2[odir+0][0]+bestx,hex2[odir+0][1]+besty,costs[0]);
+ COST_MV_HPEL2(hex2[odir+1][0]+bestx,hex2[odir+1][1]+besty,costs[1]);
+ COST_MV_HPEL2(hex2[odir+2][0]+bestx,hex2[odir+2][1]+besty,costs[2]);
+ dir = -2;
+ COPY2_IF_LT( bestcost, costs[0], dir, odir-1 );
+ COPY2_IF_LT( bestcost, costs[1], dir, odir );
+ COPY2_IF_LT( bestcost, costs[2], dir, odir+1 );
+ if( dir == -2 )
+ break;
+ bestx += hex2[dir+1][0];
+ besty += hex2[dir+1][1];
+ }
+ }
+ COST_MV_HPEL3(bestx+2,besty-2);
+ COST_MV_HPEL3(bestx+2,besty);
+ COST_MV_HPEL3(bestx+2,besty+2);
+ COST_MV_HPEL3(bestx,besty-2);
+ COST_MV_HPEL3(bestx,besty+2);
+ COST_MV_HPEL3(bestx-2,besty-2);
+ COST_MV_HPEL3(bestx-2,besty);
+ COST_MV_HPEL3(bestx-2,besty+2);
+ COPY3_IF_LT(bpred_cost,bestcost,bpred_mx,bestx,bpred_my,besty);
+ }
+ }
}
}
bmx = ( bpred_mx + 2 ) >> 2;

Or better yet, add it after the computation, so that the satd shortcut still works.
After the computation in line 69, like this?
var += ssd - (sad * sad >> (i?6:8)) + k;

or just:

var += ssd - (sad * sad >> (i?6:8));
// SATD to represent the block's overall complexity (bit cost) for intra encoding.
// exclude the DC coef, because nothing short of an actual intra prediction will estimate DC cost.
if( var && satd )
*satd += h->pixf.satd[pix](flat, 0, h->fenc->plane[i]+offset, stride) - sad/2;
+ var += 42;
}
return var;

where k=42, an arbitrary constant?

[edit]
Your code above (var=42) made almost no difference in quality, SSIM, or QP distribution, and certainly didn't reinstate the behavior of .45, for better or for worse mathematically, you might say, though visually worse in my case. At constant QP, the bitrate/threshold ratio also remains the same; the algorithm still requires a vastly lower threshold to reach the same bitrate as .45 at less SSIM and quality--same as normal .47--a sure sign it's not doing what it's intended to.

At the risk of reintroducing imprecision (and what tremendously vast imprecision it must be to produce what in some cases amounts to such polar opposite bit distributions), I still seek to recreate a rounding/distribution similar to that present in .45, but perhaps, if possible, more mathematically accurate behavior.

[sysKin]

Lol, as much as 42 is the answer to everything, it's not the answer here. It's in the order of several thousands I'm guessing.

[zbutsam]

hmm is it possible that the Visual Quality decreased? i see ringing now @ Face Edges (something i saw last time with ASP but not used to see with H.264) aq-strength of 1.0 only lower strength avoid this now.

I have to report the same thing. I did a 2-pass encode with aq-strength at 1.0 and default sensitivity and in the video with AQ enabled there was a lot more ringing around people's faces than the one without AQ. Perhaps the bitrate was too low for my source and where the No-AQ version became blurred, the AQ version produced ringing.

[Dark Shikari]

I have to report the same thing. I did a 2-pass encode with aq-strength at 1.0 and default sensitivity and in the video with AQ enabled there was a lot more ringing around people's faces than the one without AQ. Perhaps the bitrate was too low for my source and where the No-AQ version became blurred, the AQ version produced ringing.AQ 1.0 is too strong. Use the recommended settings: strength 0.5, sensitivity 13, qcomp 1.

[DeathTheSheep]

Lol, as much as 42 is the answer to everything, it's not the answer here. It's in the order of several thousands I'm guessing.

Hmm, true that. :) But after all, I've tried on the order of up to 50000 already to limited success. Very high values do skew QP distribution indeed (throwing in any sizable constant into any rate control mechanism would almost have to), but trying to guess at some random, arbitrary constant isn't quite optimal unless I know exactly what constant .45 did introduce, and how exactly that supposed "constant" actually varies per frame/decision.

[zbutsam]

AQ 1.0 is too strong. Use the recommended settings: strength 0.5, sensitivity 13, qcomp 1.

:thanks:

I tried again with the settings you recommended and everything looks in order now.
Could you please add these new recommended settings on the first post of this thread?

[Dark Shikari]

:thanks:

I tried again with the settings you recommended and everything looks in order now.
Could you please add these new recommended settings on the first post of this thread?Done.

[Poopoo]

Can you please tell me where I can get this SSIM thingy ?
Thanks !

[Sagekilla]

SSIM calculation is built into x264. In fact, x264 calculates it by default. You have to use --no-ssim to disable it. When your encode finishes, it'll display the SSIM along with a bunch of other data.

[Yoshiyuki Blade]

AQ 1.0 is too strong. Use the recommended settings: strength 0.5, sensitivity 13, qcomp 1.

I've tried several different levels of AQ strength and sensitivity, but I still can't seem to reproduce the same quality as I did with v0.45 whether its in CRF mode or 2-pass (anime). High strength and high sensitivity is the closest I've gotten, but the overall visual quality turns out so bad, it's not a worthwhile tradeoff anymore. With v0.45 I used strength 1.0, sensitivity 20, and qcomp 0.6, and the improvement in dark areas was definitely worth the cost of some overall quality. It didn't look nearly as bad as it does now.

Using the current recommended settings (strength 0.5, sensitivity 13-15, qcomp 1), the quality is good, but dark areas don't get addressed very well if at all.

At the moment, nothing looks remotely satisfying unless I really up the bitrates (about 500 megabytes for 25-minute footage), in which case it will look good across the board. Perhaps this version of AQ is more optimized for real-life footage?

However, before I feel confident on my experiences, I'm gonna have to do another encode with v0.45. Be back in like 10 hours with my experiences lol :D.

[Dark Shikari]

I've tried several different levels of AQ strength and sensitivity, but I still can't seem to reproduce the same quality as I did with v0.45 whether its in CRF mode or 2-pass (anime). High strength and high sensitivity is the closest I've gotten, but the overall visual quality turns out so bad, it's not a worthwhile tradeoff anymore. With v0.45 I used strength 1.0, sensitivity 20, and qcomp 0.6, and the improvement in dark areas was definitely worth the cost of some overall quality. It didn't look nearly as bad as it does now.

Using the current recommended settings (strength 0.5, sensitivity 13-15, qcomp 1), the quality is good, but dark areas don't get addressed very well if at all.

At the moment, nothing looks remotely satisfying unless I really up the bitrates (about 500 megabytes for 25-minute footage), in which case it will look good across the board. Perhaps this version of AQ is more optimized for real-life footage?

However, before I feel confident on my experiences, I'm gonna have to do another encode with v0.45. Be back in like 10 hours with my experiences lol :D.That's very odd.

Can you encode one clip with 0.45 at settings you like, and at 0.47 at the same bitrate with settings you like, and post both .h264 files here for me to look at? Upload them to Mediafire or something.

[DeathTheSheep]

No, not really odd at all. Why do you think I'm trying to "get back" the old behavior? Nostalgic reasons? ;)

I still can't seem to reproduce the same quality as I did with v0.45

Yep, that's exactly my point. Good to see that people are reaching the same conclusion on wider scales. :) That's why I'm back to 0.46, which works like a dream. If you're interested, adding a constant 17,000 after the SATD "trick" in the ac_energy_mb function seems to bring things back to the bitrate/threshold ballpark, but still doesn't allocate bits where they're needed in anime, so the quality, long story short, will still suck.

VAQ < 0.47 was a freak occurrence of an erroneous miracle-rounding, which, for some yet-unknown reason, worked amazingly well on anime. Odd, isn't it?

[Yoshiyuki Blade]

That's very odd.

Can you encode one clip with 0.45 at settings you like, and at 0.47 at the same bitrate with settings you like, and post both .h264 files here for me to look at? Upload them to Mediafire or something.

Will do. Is this (http://files.x264.nl/AQ/force.php?file=./x264.721.dark.aq.rdrc.0.45.exe) the old v0.45 patch? I hope it wasn't ninja-edited over time or anything. ;)

[Dark Shikari]

Will do. Is this (http://files.x264.nl/AQ/force.php?file=./x264.721.dark.aq.rdrc.0.45.exe) the old v0.45 patch? I hope it wasn't ninja-edited over time or anything. ;)Probably not, since that's what it says.

[DeathTheSheep]

Ninja-edited? :eek:

You could use .46, which is more cleaned up, streamlined, and compatible with latest x264 revisions: http://pastebin.com/f22a03c76

I found this baby myself, by the way. It was a long, painful search. :D

[Yoshiyuki Blade]

In the meantime, I've uploaded clips of an older encode I had lying around as reference, which I'm confident used v0.45 of the patch.

To roughly illustrate my impressions, I'll compare the older clip to the results I posted earlier in this thread here (http://forum.doom9.org/showthread.php?p=1092791#post1092791).

(Patch v0.45) --qcomp 0.6 aq-strength 1.0 aq-sensitivity 20: 179 MB (Sample 1 (http://www.mediafire.com/?dfumwzjee9z)) (Sample 2 (http://www.mediafire.com/?fmnbtj1yuhw))

Keep in mind the descrepancy in this comparison:

- For one, this older sample is the result of a 2-pass encode, while the others were done in CRF mode. I think the average CRF for this sample is about 32.53.
- Besides the settings I listed in the description, there may be other differences in the settings used. If there is, it's probably minor. EDIT: One other major difference I spotted is that Sharktooth's AVC CQM was used on the older encode while the others didn't use any (flat), though I doubt it could make this drastic of a difference.
- The file sizes between the 3 tests are very different from each other. However, this older sample happens to be the smallest in file size and still looks subjectively better (in my opinion at least).

The overal impression of the older clip seems more well-balanced. Low strength in v0.47 looks better where the details are prominent, but the background areas dont look very good. Higher strength (1.0), looks similar to the v0.45 version, but the overall impression is worse, and this is considering its 20 MB larger than the v0.45 sample. Pay special attention to Yumi's face 11 seconds into the 2nd sample. It looks best with v0.47 strength 0.5, but the dark scenes look like crap. Second best is the v0.45 sample, which also addressed blockyness in the dark scene very well.

Anyhow, I'm currently re-encoding with v0.45 and will do the same with v0.47 to keep everything more consistent. By the way, do you really need a .264 file? For some stupid reason, I like to encode at 1280x720 even on 4:3 material so the video will be stretched unless I resize them (while putting them into an mkv container) :D

Ninja-edited? :eek:

You could use .46, which is more cleaned up, streamlined, and compatible with latest x264 revisions: http://pastebin.com/f22a03c76

I found this baby myself, by the way. It was a long, painful search. :D

I should check it out. I actually never used v0.46 because it was updated to v0.47 soon afterwards.

[Dark Shikari]

Well, if you give me an MKV, I'll just demux it anyways since Streameye doesn't accept MKVs.

[bob0r]

source:
720p50_parkrun_ter.yuv (ftp://ftp.ldv.e-technik.tu-muenchen.de/pub/test_sequences/720p/720p50_parkrun_ter.yuv)

commandlines:
start /belownormal /b /w x264aq0.48.exe --pass 1 --aq-strength 0.0 --threads auto --bitrate 5000 --deblock 0:0 --bframes 16 --direct auto --me dia --ref 1 --subme 1 --no-dct-decimate --partitions none --progress --fps=50 --output NUL 720p50_parkrun_ter.yuv 1280x720
start /belownormal /b /w x264aq0.48.exe --pass 2 --aq-strength 0.0 --threads auto --bitrate 5000 --deblock 0:0 --bframes 16 --direct auto --b-pyramid --bime --weightb --b-rdo --me umh --ref 5 --mixed-refs --subme 7 --trellis 1 --analyse all --8x8dct --no-fast-pskip --progress --fps=50 --output x264noaq.parkrun.mkv 720p50_parkrun_ter.yuv 1280x720

start /belownormal /b /w x264aq0.48.exe --pass 1 --threads auto --bitrate 5000 --deblock 0:0 --bframes 16 --direct auto --me dia --ref 1 --subme 1 --no-dct-decimate --partitions none --progress --fps=50 --output NUL 720p50_parkrun_ter.yuv 1280x720
start /belownormal /b /w x264aq0.48.exe --pass 2 --threads auto --bitrate 5000 --deblock 0:0 --bframes 16 --direct auto --b-pyramid --bime --weightb --b-rdo --me umh --ref 5 --mixed-refs --subme 7 --trellis 1 --analyse all --8x8dct --no-fast-pskip --progress --fps=50 --output x264aq.parkrun.mkv 720p50_parkrun_ter.yuv 1280x720

result:
AQ strength 0.0(disabled) versus 0.5(default), 50fps 5000kbit:
x264.736.aq.0.48.strenght.0.0.parkrun.5000kbit.50fps.mkv (http://files.x264.nl/AQ/x264.736.aq.0.48.strenght.0.0.parkrun.5000kbit.50fps.mkv)
x264.736.aq.0.48.strenght.0.5.parkrun.5000kbit.50fps.mkv (http://files.x264.nl/AQ/x264.736.aq.0.48.strenght.0.5.parkrun.5000kbit.50fps.mkv)

AQ 0.48 .exe in first post.
changelog: 0.48: AQ strength 0.5, sensitivity 13 made the defaults. Updated to r736. Qcomp is now scaled based on AQ strength automatically.

[buzzqw]

@bob0r

is possibile to have a new build (0.48) with nal/hrd/pulldown ?

thanks!

BHH

[bob0r]

@bob0r

is possibile to have a new build (0.48) with nal/hrd/pulldown ?

thanks!

BHH

- x264 revision 736
- AQ_0.48.diff
- x264_2pass_vbv.0.diff
- x264_hrd_pulldown.04.diff

x264.736.dark.aq.0.48-2pass_vbv.0-hrd_pulldown.04.exe (http://files.x264.nl/AQ/x264.736.dark.aq.0.48-2pass_vbv.0-hrd_pulldown.04.exe) pthreads/gpac/fprofiled = yes

[buzzqw]

thanks bob0r!

BHH

[Atak_Snajpera]

AQ strength 0.5, sensitivity 13 made the defaults

Does it mean that AQ is always on?

[bob0r]

Does it mean that AQ is always on?

Yes, with the AQ 0.48 patch.
To disable it: --aq-strength 0

[buzzqw]

Yes, with the AQ 0.48 patch.

:eek: wow

is this to be definitive (aka in x264 svn) ?

BHH

[vpupkind]

- x264 revision 736
- AQ_0.48.diff
- x264_2pass_vbv.0.diff
- x264_hrd_pulldown.04.diff

Can you please share the sources?

[ToS_Maverick]

why is 0.5 now the default? did you change something with the strength-scale?

[Dark Shikari]

why is 0.5 now the default? did you change something with the strength-scale?No, its just because I'd rather be conservative and choose something that will always work and never be over the top than to choose something that is too strong in many cases.

Especially with 0.47, 1.0 seems too strong for most ordinary sources.

[Inventive Software]

0.5 compared to 1.0's a bit like using subme 5 instead of subme 7. (Something that appeals to all and can be changed if it isn't suitable.)

That's an analogy BTW, not a direct comparison. ;)

[DeathTheSheep]

- The file sizes between the 3 tests are very different from each other. However, this older sample happens to be the smallest in file size and still looks subjectively better (in my opinion at least).

The overal impression of the older clip seems more well-balanced. Low strength in v0.47 looks better where the details are prominent, but the background areas dont look very good. Higher strength (1.0), looks similar to the v0.45 version, but the overall impression is worse, and this is considering its 20 MB larger than the v0.45 sample. Pay special attention to Yumi's face 11 seconds into the 2nd sample. It looks best with v0.47 strength 0.5, but the dark scenes look like crap.

Yep. As I described it, the problem is threefold:
1. Bad bit/QP distribution in regard to scenes that need it.
2. Huge filesize at same crf/cqp/threshold--at significantly less SSIM.
3. Far more erratic visual quality, favoring parts of frames that don't visibly benefit from AQ.

In a number of cases, entire clips look visibly better (or even more commonly, not visibly worse) with this new AQ turned off! Why waste more bits (read: larger files) on something that doesn't help much? Actually, believe it or not, your clip is actually better off under 0.47 than some of the 0.47 clips I've made.

Here's something interesting: If you play around with 0.46 (the "official" test build is on mirror05.x264.nl/dark), you'll get even better results than those you're probably getting with 0.45. :)

[Dark Shikari]

In a huge number of cases, entire clips look visibly better (or even more commonly, not visibly worse) with this new AQ turned off! Why waste more bits (read: larger files) on something that doesn't help much?I have never ever seen this ever except in the case of low-bitrate animation.

Please give proof and examples.

Also, I need a 0.47 version of these animated clips to compare to, so I can see what the problem is. Old versions alone just won't do.

[Yoshiyuki Blade]

I just reviewed an encode (same anime episode as the other samples) with AQ v0.48, and it actually looks very good at default settings. I compared the following:

AQ v0.45 --qcomp 1.0 --aq-strength 1.0 --aq-sensitivity 20 average CRF 30.24: 279 MB
AQ v0.48 --qcomp 1.0 --aq-strength 0.5 --aq-sensitivity 13 average CRF 26.64: 279 MB

Everything else constant.

The results are really comparable to each other, and the current patch may provide better quality. I'm not quite sure about that yet, but if the lower average CRF is any indication, it should.

Did anything major happen between 0.47 and 0.48 besides changing the defaults? It makes me wonder if Sharktooth's AVC CQM contributed to the quality difference (it's the only major difference in my earlier comparisons). The two tests above both have the CQM.

I'm currently re-testing at lower bitrates to see how well the current patch scales at lower bitrates.

[Sagekilla]

Between 0.47 and 0.48, according to the main post anyway, the biggest change seems to be qcomp being based off aq-strength.

[Dark Shikari]

Between 0.47 and 0.48, according to the main post anyway, the biggest change seems to be qcomp being based off aq-strength.But qcomp=1 will be qcomp=1 regardless of AQ strength, so that's not a change in the case of his settings.

[Morte66]

@DS
So what qcomp are you recommending now, for crf and two pass?

[Dark Shikari]

@DS
So what qcomp are you recommending now, for crf and two pass?Leave it at default, AQ will automatically raise it.

For AQ > 0.28, qcomp will be 1 automatically.
For clarity, I was only referring to low bitrate-animation.

And in pertinence to using 0.47, does 0.48 use the same algorithm? If so, an upgrade could well be in order anyway. :DYes, 0.48 is the same as 0.47 except with changed defaults and the automatic qcomp scaling.

[DeathTheSheep]

Holy crap, that was a fast response! You know, I'm actually going to repeat the bulk of my above post here, since I didn't have time to edit it:

For clarity, I was only referring to low bitrate-animation in my previous assertions.

And in pertinence to using 0.47, does 0.48 use the same algorithm? If so, an upgrade could well be in order anyway. (Answered, yes).

I'm not quite sure about that yet, but if the lower average CRF is any indication, it should.
No, a program that reports something like "lower average CRF" is useless in this case, because "lower average" says almost nothing by itself when considering both the statistical and visual entirety of the results. It's the distribution of lower (and higher) QPs across frames/scenes that matters. For all we know, the QPs could be lowering in areas where they shouldn't be, and the higher ones are relegated to areas where they harm quality. I was just saying that actually seems to be taking place for me in the context of said anime sources.

For live footage, results seem to be roughly the same across patch versions when tweaked properly.

[vpupkind]

After trying 0.48 I noticed that the bitrate constraints are no longer obeyed (which might be explained by qcomp=1)

[Dark Shikari]

After trying 0.48 I noticed that the bitrate constraints are no longer obeyed (which might be explained by qp=1)Bitrate constraints?

[burfadel]

On the help it says --aq-strength is defaulted to 0.5, but you still have to actually enter a value for the strength for AQ to be turned on...

To run default settings, wouldn't it be easier to either:
- Have it on by default :D (I think many would agree), and turn it off by '--aq off' or
- Be able to turn it on by just '--aq'? (on by default when set, but still allow '--aq on', and when set use the default settings.

The other options can remain, it just seems silly having it as a default 0.5 when you have to state it anyway!

[Dark Shikari]

On the help it says --aq-strength is defaulted to 0.5, but you still have to actually enter a value for the strength for AQ to be turned on...

To run default settings, wouldn't it be easier to either:
- Have it on by default :D (I think many would agree), and turn it off by '--aq off' or
- Be able to turn it on by just '--aq'? (on by default when set, but still allow '--aq on', and when set use the default settings.

The other options can remain, it just seems silly having it as a default 0.5 when you have to state it anyway!Uh, it works just fine here without specifying it...