Which frame looks better?

[Dark Shikari]

The following frames are from 5 different encodes plus a reference frame (not indicated which is which). Not all are necessarily various modifications of x264--in fact, they might all be different codecs... I'm not telling for now. If you figure it out, don't say, because that sort of eliminates the point of this post.

For the following 6 images, comment on what you like and don't like about each, and which one(s) you think are the best. Look very carefully at them--find blocks in the background, ringing, sharpness, grain retention, etc. Comment on them--I want to know what is wrong with each of them, and what is right--not just that one sucks and another is good!

Also note that I will tell which is which later. One of them is the original.

1: http://img233.imageshack.us/img233/6760/89164279aw2.png
2: http://img72.imageshack.us/img72/899/22269025xa2.png
3: http://img63.imageshack.us/img63/3974/93797124ps4.png
4: http://img295.imageshack.us/img295/5980/39562005yz4.png
5: http://img295.imageshack.us/img295/159/94900362yd7.png
6: http://img295.imageshack.us/img295/3848/96015227wh9.png

They are all from streams at the same ABR bitrate. The original source was quite grainy, so all the grain shown is from the original. No FGM was used.

[Sergey A. Sablin]

do you mind to post first image of the same resolution as others?

[Dark Shikari]

do you mind to post first image of the same resolution as others?I accidentally overcropped the frame--let me fix it.

Edit: Frame fixed and a 5th and a 6th. I have also posted that a reference frame hidden among them, rather than leaving you to decide on your own whether there was one or not.

[Japhsoncross]

frame 2 doesn't have the same vertical position as other frames, but its quality is almost the same as frame 1. frame 3,4,5 seem to be compressed a bit. at least the background is smoothed in frame 3,4,5. it seems frame 4,5 show more grain in the background than 4, but they don't have better quality on the center part of the qicture as frame 4. i think frame 3 is a bit better than frame 5 since i can see contours in frame 5.

---------------------------
after posting, i found 6 frame totally. i'll check them again.

[le_canz]

Pic 2 clearly seems to be the smoother to me.

Pic 6 seems the most blocky (at least the most obvious, and pic 5 is nearly as blocky.

Pic 1 and 2 are close except that 2 is smoother, but they're not blocky.

Pic 3 has a more grain than 4, they're not blocky.

I think I prefer picture 3.

I hope it will be helpful :)

[CruNcher]

yep definately 3 but takeing also the size into account (size/quality balance seems perfect) for this 1 frame at least, but how about in motion ?

[ToS_Maverick]

2 is smooth and flawless
1 and 3 are very close, 3 is probably a bit better
4
6
5 is the worst

[*.mp4 guy]

I completely agree with ToS_Maverick.

[Wishbringer]

1 has most details for me and a nice remaining grain in the background.

can't decide if 5 or 6 looks worst in background for me... think 5

hand in 2 is brighter than 1, more shiny (but not better, i prefer 1)

3 - 6 is overbrightend at hand, reducing details. 4 - 6 are worst.

5 has worst grain, then 4, 6, 3, 2, 1

So I think 1 is original, 2 has AQ.
5 is without optimization, then ranking 3, 4, 6 (better to worse)

[Egladil]

1 is best for me, sharp and nice background
2 is too smooth
3 looks very good too, has a bit less grain preserved than 1 in detailed areas
4 bad background
5 grain in background lost.
6 bad background

my preference:
1, 3, 5, 2, (4, 6)

[Brother John]

1 is best.
2 is a little smoother and less detailled than 1.
3 preserves detail a tiny bit better than 2 in the left part of the picture but pays by being slightly smoother and more blocky on the right (the dark blue area).
4 is smoother than 2/3 and has noticeable blocks.
5 and 6 are even more blocky.

My favourite is 1 and I'm about indifferent between 2-3. 4-6 all have too many blocks.

[check]

There's definately a green cast to 2. 2 is a little more blocky as well. See the brown to light blue transition in the upper right.
3 seems to have banding in the gradients. See the brown to light blue transition in the upper right. The high detail areas seem sharper.
4 is blockier in the bottom right again... looks like it would be extra noticeable in motion, but fixable with deband in ffdshow. I think there's a little more detail in the high noise areas... but it's hard to tell.
5 is nice and noise free, there are some underlying gradients that are more pronounced now, but looks reasonably nice otherwise. Probably rather high quality compared to the input. I'd imagine it would be a little flickery in motion.
6 has the green cast again. Blocky in the gradients. Just looks worse in general.

My pick would be 1, because it doesn't have any obvious flaws, and the background is detailed. 5 would be nice if there was a little more bits in the background too.

[Selur]

I go with:
1, 3, 5, 2, (4, 6)
same reason as Egladil

[addit]

My preference order:

1) Sharpest image - best foreground detail and best background grain retention.
3) Almost as good foreground as 1) but a subtly smoother/less grainy background.
2) Better background grain retention than 2) but a softer, smoother foreground hence worse than 3)
4) Slight blocking in background, foreground seems alright
6) Slightly more blocking in the background.
5) Worst blocking in the background...

Adam

[honai]

The original source was quite grainy, so all the grain shown is from the original.

Not the best source for this test, to be honest. The Chronicles of Riddick was one of the first A-list HD-DVD titles to use VC-1, and given the fact that they worked straight off the digital master this release is sub-par for various reasons:


grain retention is actually quite bad
slightly wrong levels, and hue is off as well
obviously the source material has been softened all over the place


Also, the VC-1 tool chain wasn't really mature when this title was produced.

A much better choice would be either one of (in no particular order):

Black Snake Moan
Casino Royale
The Departed
Apocalypto
Mission Impossible 3
Pirates of the Caribbean 2
X-Men 3


Otherwise you'd be only amplifying the problems of the source material.

[Dark Shikari]

Not the best source for this test, to be honest. The Chronicles of Riddick was one of the first A-list HD-DVD titles to use VC-1, and given the fact that they worked straight off the digital master this release is sub-par for various reasons:


grain retention is actually quite bad
slightly wrong levels, and hue is off as well
obviously the source material has been softened all over the place


Also, the VC-1 tool chain wasn't really mature when this title was produced.

A much better choice would be either one of (in no particular order):

Black Snake Moan
Casino Royale
The Departed
Apocalypto
Mission Impossible 3
Pirates of the Caribbean 2
X-Men 3


Otherwise you'd be only amplifying the problems of the source material.Its from the DVD, not the HD-DVD :)

[honai]

Oops! ;-)

[reepa]

2 and 6 have the same greenish color cast. 2 retains most of the grain. 5 has obliterated the grain. 1 and 3 are similar - they get the second place in grain retention. 4 and 6 are slightly worse, 4 being better. My ranking from nicest to least nice: 2, 1/3, 4, 6, 5.

[Mtz]

1 and 3, or 3 and 1.

enjoy,
Mtz

[fields_g]

Image 3 is 720x352 instead of 720x356. Image 2 have a slight vertical shift down. Could this be fixed? I know this sounds really picky.


My RANKINGS:

1=3>2>4>6>5

[Dark Shikari]

Image 3 is 720x352 instead of 720x356. Image 2 have a slight vertical shift down. Could this be fixed? I know this sounds really picky.

I also noticed that 1 and 2 are darker than the others... I wonder if this points to different encoding....
3 has been fixed for you.

[IgorC]

How usefull is compare still images?
There can be changes between frames during playback those nobody don't get in count. A lot of issues would be invisible. Flickering, not smooth motion etc...

[Dark Shikari]

How usefull is compare still images?
There can be changes between frames during playback those nobody don't get in count. A lot of issues would be invisible. Flickering, not smooth motion etc...
First of all, not all of the frames are from streams which I even have; though I won't post which is which yet, one of them is from the latest in-house version of the Ateme encoder, and for obvious reasons bobobolo refused to give me anything more than a screenshot :p

The motion is smooth on all of them as far as I can tell, none are particularly bad in terms of motion. We're mainly looking at lack of blocking, sharpness, grain retention, etc here.

[foxyshadis]

Proof, if anyone needed it, that I'm an undiscerning video consumer: I can't tell a difference between 1, 2, and 3, aside from that color cast. 4 is okay, I dislike the banding of 5 & 6 more than I dislike microblock artifacts.

[Dark Shikari]

Eh, I think there have been enough responses, I'll tell which is which.

This is a scene in which my new AQ (not the one in the other thread--yet another new algorithm) doesn't really shine that much, because the old AQ is already so good at dark/blue blocking artifacts.

1: My new AQ algorithm at full strength.
2: Original
3: Old AQ at full strength.
4: New AQ at half strength.
5: No AQ at all.
6: Ateme's latest in-house encoder with psyopts enabled

Since more people answered 1) than 2) as their best, I think I can declare 1 to be basically transparent and 3 probably pretty damn close.

[Egladil]

Hm why is the source smoother than the encodes? did you apply sharpening?

[Dark Shikari]

Hm why is the source smoother than the encodes? did you apply sharpening?Nope, but I noticed this in a number of places throughout my encode--my AQ seems to actually sharpen the source slightly for some reason. Many edges seem slightly sharper.

It could be because x264 is rounding up some of the higher frequencies, due to the lower quantizer, and so it looks sharper.

[addit]

Alright enough showmanship! Give us the patch already so we can try it out for ourselves.:devil:

Adam

[Daodan]

Yeah, :stupid:

Nr 1 was gonna be my pick as well (heh, the higher quantizers reduced some of the blocks in bright areas from the source).
Also question, in the other AQ thread, was the source also untouched? Because the encode with old aq was visibly sharper then the source.

[Dark Shikari]

Yeah, :stupid:

Nr 1 was gonna be my pick as well (heh, the higher quantizers reduced some of the blocks in bright areas from the source).
Also question, in the other AQ thread, was the source also untouched? Because the encode with old aq was visibly sharper then the source.Yup, untouched. All the sharpening is due solely to the rounding up of high-frequency coefficients, I think.

Here is the new AQ code; no modifications are required to any file other than analyze.c.

static int x264_subtract_dctq( int16_t dct[8][8], int16_t origdct[8][8], int dctweight[8][8] )
{
int i, j, t = 0;
for( i=0; i<8; i++ )
for( j=0; j<8; j++ )
{
unsigned int s = (abs(dct[i][j] - origdct[i][j]) * x264_dct8_weight_tab[1+(i<<3)+j]);
s = (s*s) / ((abs(origdct[i][j])+10)* x264_dct8_weight_tab[1+(i<<3)+j]);
t += (s >> dctweight[i][j]);
}
return t;
}

static int x264_sum_dctq( int16_t dct[8][8], int dctweight[8][8] )
{
int i, j, t = 0;
for( i=0; i<8; i++ )
for( j=0; j<8; j++ )
{
t += (abs(dct[i][j]) * x264_dct8_weight_tab[1+(i<<3)+j]) >> dctweight[i][j];
}
return t;
}

/*****************************************************************************
* x264_adaptive_quant:
* check if mb is "flat", i.e. has most energy in low frequency components, and
* adjust qp down if it is
*****************************************************************************/
void x264_adaptive_quant( x264_t *h, x264_mb_analysis_t *a )
{
DECLARE_ALIGNED( static uint8_t, zero[FDEC_STRIDE*8], 16 );
DECLARE_ALIGNED( int16_t, dct[8][8], 16 );
DECLARE_ALIGNED( int16_t, origdct[8][8], 16 );
int dctweight[8][8];
int i,j,k;
float fc;
int total = 0;
int qp = h->mb.i_qp, qp_adj;
int weightqp[16] = {0,0,qp/50,qp/40,qp/30,qp/20,qp/15,qp/12,qp/10,qp/8,qp/6,qp/5,qp/4,qp/3,qp/2,qp};
for( i=0; i<8; i++ )
for( j=0; j<8; j++ )
dctweight[i][j] = weightqp[i+j];
for( i=0; i<4; i++ )
{
h->dctf.sub8x8_dct8( dct, h->mb.pic.p_fenc[0] + (i&1)*8 + (i>>1)*FENC_STRIDE, zero );
for( j=0; j<8; j++ )
for( k=0; k<8; k++ )
origdct[j][k]=dct[j][k];
h->quantf.quant_8x8( dct, h->quant8_mf[CQM_8IY][qp], h->quant8_bias[CQM_8IY][qp] );
h->quantf.dequant_8x8( dct, h->dequant8_mf[CQM_8IY], qp );
total += x264_subtract_dctq(dct,origdct,dctweight);
}
x264_cpu_restore( h->param.cpu );
fc = expf(-1e-6 * total);
qp_adj = (int)(qp * ((2.0 * h->param.analyse.f_aq_strength / pow(2 - fc, h->param.analyse.f_aq_sensitivity))));
int newQP = qp - qp_adj;
h->mb.i_qp = a->i_qp = X264_MAX(X264_MIN(X264_MAX(X264_MIN(newQP,3*qp/2),0),h->param.rc.i_qp_max),h->param.rc.i_qp_min);
h->mb.i_chroma_qp = i_chroma_qp_table[x264_clip3( h->mb.i_qp + h->pps->i_chroma_qp_index_offset, 0, 51 )];
a->i_lambda = i_qp0_cost_table[h->mb.i_qp];
a->i_lambda2 = i_qp0_cost2_table[h->mb.i_qp];
}

Note that my weights, while obviously an educated guess (higher QP = higher frequencies matter less, lower QP = higher frequencies matter more), are just that, a guess. If someone can come up with a good way to better choose weights, my AQ could get considerably better.

Here's how it works:

1. Get the DCT of the current 8x8 block.
2. Quantize, and then dequantize, the DCT from 1). Call it DCT2.
3. Calculate (abs( DCT2(i) - DCT(i) ) ^ 2) / DCT(i) and then shift by the appropriate weight value. Do this for each of the 64 coefficients and add them up.
4. Do the above for all 4 8x8 blocks and add up the results.
5. This total is the weight used for the AQ algorithm. At this point very little is changed from the original AQ except various constants.

[ToS_Maverick]

ha, i knew i was right ;)

@Dark Shikari:
what's the difference of your latest AQ version, and the version i tested and made screenshots with?

the reason i ask is, that i was quite satisfied with the version i tested, cause it prevented the slightest details in the background from getting quantized away. the only downside was, that it got slightly blocky at high constrast areas.

[lexor]

As you might remember from our little discussion in the new AQ thread, I like clear picture with sharp details.

1 = blurry mess, especially around edges, area of the left of the ball is too dark
2 = slight improvement on 1, especially on the ball, but not much
3 = has the greatest sharpness, provides greatest detail = best picture perceptually (I'm not claiming it's original, just that it looks the best). Overall the best grain-feel to it
4 = very close to 3, but bottom right is a bit oversmoothed, same darkness problem on left as 1
5 = almost as good as 3, but ball is slightly blurrier, detail on the mesh to the left of the arm that holds the ball is slightly shifted (shifted noise, not better or worse than 3), ridges to the right of ball blurier, too much blur in bottom right area
6 = same as 5 on the figure, but more blur on ball and ridges right of ball, and too much colour separation in bottom right again, almost like sharp gradient.

So 3 >> 6 > 5 > 4 > 2 > 1

>> meaning it's much better than the rest, no contest if I had to pick one.

/EDIT: oops didn't notice second page before posting, I guess contest is over :( No offense, you do great work, but I hope this patch doesn't get committed then, since 3 is old AQ

/EDIT2: I can't believe people are picking #1, it's so incredibly blurry, do you all like that?

[Dark Shikari]

ha, i knew i was right ;)

@Dark Shikari:
what's the difference of your latest AQ version, and the version i tested and made screenshots with?

the reason i ask is, that i was quite satisfied with the version i tested, cause it prevented the slightest details in the background from getting quantized away. the only downside was, that it got slightly blocky at high constrast areas.
The old version had some issues in high contrast areas that could not be solved with the algorithm it used.

My current method is probably better at both the background and on high-detail areas, because it doesn't raise quants. I found that raising quants is generally a bad idea when using a DCT algorithm, because DCT itself cannot be guaranteed to find the quants that "should" be raised, and one or two mistakes give you one or two horrible blocks.

[Dark Shikari]

Also, frame sizes:

1: 26k
2: Original
3: 26k
4: 20k
5: 18k
6: 12k

They're not very meaningful without knowing the sizes of nearby frames, of course (if Ateme used fewer B-frames, for example, that would explain the lower P-frame size), but they're both a good comment on the ratecontrol ability of the encoder and the efficiency.

[Terranigma]

/EDIT2: I can't believe people are picking #1, it's so incredibly blurry, do you all like that?
Better to be smooth than to be sharp and noisy.

[*.mp4 guy]

Nope, but I noticed this in a number of places throughout my encode--my AQ seems to actually sharpen the source slightly for some reason. Many edges seem slightly sharper.

It could be because x264 is rounding up some of the higher frequencies, due to the lower quantizer, and so it looks sharper.

I don't beleive this is the case, I have noticied many times that bluring can actually lead to a greater perception of sharpness; generally what apears to happen is that when you remove small details/gradients, but leave the big ones, the big details look more prominent and, thus, sharper. Psychovisually, a detail on a flat background is more noticible then a detail on a detailed background.

Deen can sometimes have this effect, as can most masked/thresholded blurs, X264 always exhibits this to some extent, with high deadzones + the flat matrix being the most obvious.

[Dark Shikari]

I don't beleive this is the case, I have noticied many times that bluring can actually lead to a greater perception of sharpness; generally what apears to happen is that when you remove small details/gradients, but leave the big ones, the big details look more prominent and, thus, sharper. Psychovisually, a detail on a flat background is more noticible then a detail on a detailed background.

Deen can sometimes have this effect, as can most masked/thresholded blurs, X264 always exhibits this to some extent, with high deadzones + the flat matrix being the most obvious.That is an interesting idea...

Another thought I had: does it cost more bits to retain grain, or to remove that grain, add dither, and use ridiculously low deadzone settings (adaptive deadzone) to retain that dither?

[desta]

/EDIT2: I can't believe people are picking #1, it's so incredibly blurry, do you all like that?
I can't believe you think 3 and 5 look the same. No offense, but are you looking at the same pics as everyone else?

3. http://img63.imageshack.us/img63/3974/93797124ps4.png

5. http://img295.imageshack.us/img295/159/94900362yd7.png

[morph166955]

Here is the new AQ code; no modifications are required to any file other than analyze.c.


Any chance of getting a diff off of r680?

[Dark Shikari]

Any chance of getting a diff off of r680?I'll get around to it later--the code has no changes in that area between r676 and 680 as far as I know, so it should be a simple copy-paste job.

[Sagekilla]

6 seems to have changed the color in the background. 4 and 5 have banding. 2 seems to have a very small loss in quality.

Aside from that I can discern no other differences.

[lexor]

I can't believe you think 3 and 5 look the same. No offense, but are you looking at the same pics as everyone else?

Where did I say they are the same? I said 5 is almost as good, it has nothing to do with being the same or not. And the comment was mostly meant for the objects not for backgrounds (in which case 3 is much better than 5 and 6 as I noted)

As for:
Better to be smooth than to be sharp and noisy.
I do want a little noise to replace grain that would be there in any decent source (and killed off by filter to help encode better) motion will alleviate the little noise anyway. And I take sharp and grainy over smooth and blurry any day, since smoothness itself is an artifact of upscaling DVDs (SD footage in general) to high monitor resolutions.

[woah!]

can i ask how you came up with 720x356 ??? image seems streched vertically no???

my hd version is 2:35:1 ratio when cropped.... 720x306 ?

[fields_g]

can i ask how you came up with 720x356 ??? image seems streched vertically no???

my hd version is 2:35:1 ratio when cropped.... 720x306 ?

Your HD version has a PAR of 1:1. This source was a anamorphic DVD. I'm not exactly sure but the dvd decoder would either:
a) squeeze this image vertically down to 720x306 on playback
or
b) the stretch it horizontally to 836x356 on playback

Either way getting 2.35:1. In order to eliminate the stretches as a source of the visual problems, the aspect is off with this sample.

[DeathTheSheep]

6: Ateme's latest in-house encoder with psyopts enabled
...at 12kb/frame. Hmm. Something tells me their psychovisual system only produces its desired effects at framerates > 10fps (meaning not so good when you actually freeze-frame it). Just a little reminder from 2004's beta...

And by latest do you mean their H264 Encoder Suite or their newer Ateme File Encoder (broadband)?
Also, maybe Mainconcept Reference Pro should've been given a shot. ;)

[Dark Shikari]

...at 12kb/frame. Hmm. Something tells me their psychovisual system only produces its desired effects at framerates > 10fps (meaning not so good when you actually freeze-frame it). Just a little reminder from 2004's beta...

And by latest do you mean their H264 Encoder Suite or their newer Ateme File Encoder (broadband)?
Also, maybe Mainconcept Reference Pro should've been given a shot. ;)By latest, I mean the latest experimental in-house version that is so sekrit that I couldn't even get more than a single frame and its size and type in terms of information.

[Gabriel_Bouvigne]

...at 12kb/frame. Hmm. Something tells me their psychovisual system only produces its desired effects at framerates > 10fps (meaning not so good when you actually freeze-frame it). Just a little reminder from 2004's beta...
I'm not objecting to the fact that a single frame could be a bad quality indicator of a video (what about stability?), but you are not seriously expecting the psychovisuals to be identical/close to the 2004 ones?

[Dark Shikari]

Also, though there's still tons of tuning to do before I post anything... adaptive deadzone has been implemented :p

[Gabriel_Bouvigne]

1: My new AQ algorithm at full strength.
2: Original
3: Old AQ at full strength.


Since more people answered 1) than 2) as their best, I think I can declare 1 to be basically transparent and 3 probably pretty damn close.

Transparency is not a matter of preference, but the fact that you can not spot differences between the source and modified/encoded version.
If people said that they like 1) better than 2), then it means that it's not transparent (in the context of a single, non moving frame), but you have no way to assess transparency by this test.

[Dark Shikari]

Transparency is not a matter of preference, but the fact that you can not spot differences between the source and modified/encoded version.
If people said that they like 1) better than 2), then it means that it's not transparent (in the context of a single, non moving frame), but you have no way to assess transparency by this test.Probably, though to me I actually cannot tell the difference without extremely close inspection.

There are some other scenes with cases in which it is clearly not 100% transparent; I'm working on them.