Which frame looks better?

[Wishbringer]

With a good TFT monitor with high contrast you can easily see the differences between 1 and 2.
So it's not 100% transparent, but all other pictures aren't too.
1 comes far nearer to 2 than any other picture.
On a CRT monitor you really have to search for differences.

[lexor]

Probably, though to me I actually cannot tell the difference without extremely close inspection.

There are some other scenes with cases in which it is clearly not 100% transparent; I'm working on them.

You kidding right? 2 is waaay brighter than 1, I'd call changing the contrast of the image a pretty big screw up. I'm beginning to think people got bad monitors around here (or at least poorly calibrated).

[Wishbringer]

You kidding right? 2 is waaay brighter than 1, I'd call changing the contrast of the image a pretty big screw up. I'm beginning to think people got bad monitors around here (or at least poorly calibrated).

So people see different results:
1 and 2 are both at same darkness level for me.
3 to 6 are way brighter for me

And I have a S-PVA Panel and not a TN+Film one.

[fields_g]

I thought there was a really big change in brightness/contrast also BUT.... I found the problem was my own machine.
Under IE, 1 and 2 were darker at first. Later, under IE, 1-3 were darker. This made me suspicious. Loaded up firefox and all were the same. I don't know if this phenomon is what is happening to you guys also.

[Dark Shikari]

Any overall brightness difference is entirely coincidental or my mistake.

[foxyshadis]

Well, the original DVD snap appears to have been screenshotted using another color matrix, so it comes out a bit off. (Actually I just checked, it looks the same as the others when using 601->709 conversion.)

[DeathTheSheep]

Just as an aside, what movie was that screenshot taken from? Actually, what the heck is in that frame? A tower? Grrr...

[Dark Shikari]

Just as an aside, what movie was that screenshot taken from? Actually, what the heck is in that frame? A tower? Grrr...Its from Chronicles of Riddick, the intro sequence. Its one of the alien guys grabbing some sword thing.

[DeathTheSheep]

Aha! I thought it was some alien and some staff/sword, but then I second-guessed myself... :p.

Well anyway, screen 1 looks the best to me, too, so congratulations on the AQ even though it doesn't apply to my INSANE-ly low bitrates. :D

...probably.

[Dark Shikari]

Aha! I thought it was some alien and some staff/sword, but then I second-guessed myself... :p.

Well anyway, screen 1 looks the best to me, too, so congratulations on the AQ even though it doesn't apply to my INSANE-ly low bitrates. :D

...probably. I could definitely make a "low-bitrate" AQ, but it would have to be totally different. Here's one idea I had:

1. In the first pass, store the locations of the macroblocks that use up the most bits, and how many bits they use relative to the rest of the frame.
2. In the second pass, raise the quants on these blocks proportional to the amount of bits they used the first time around.

At low bitrates, you can deal with losing quality in some places if it means greater quality everywhere else.

[RaynQuist]

unsigned int s = (abs(dct[i][j] - origdct[i][j]) * x264_dct8_weight_tab[1+(i<<3)+j]);
s = (s*s) / ((abs(origdct[i][j])+10)* x264_dct8_weight_tab[1+(i<<3)+j]);
t += (s >> dctweight[i][j]);


Optimization ideas:

int s = (dct[i][j] - origdct[i][j]);
s = (s*s) / (abs(origdct[i][j])+10) * x264_dct8_weight_tab[1+(i<<3)+j];
t += (s >> dctweight[i][j]);
1. abs before squaring is redundant, unless you need that s to be unsigned. What's the range of DCT coefficients? Can they be negative?
2. x264_dct8_weight_tab mathematically cancel out.


As far as the algorithm goes, this doesn't makes sense to me:
difference^2 / original

If you're scaling by the original I would do:
(difference / original)^2

I would also argue that the difference doesn't need to be scaled by the original. I think 3->0 is as much of an error as 203->200. So I say just do:
difference^2


Also, about weights:


int weightqp[16] = {0,0,qp/50,qp/40,qp/30,qp/20,qp/15,qp/12,qp/10,qp/8,qp/6,qp/5,qp/4,qp/3,qp/2,qp};
for( i=0; i<8; i++ )
for( j=0; j<8; j++ )
dctweight[i][j] = weightqp[i+j];


1. Max of i+j is 14, so your weightqp has 1 extra number.
2. You can just have dctweight be a constant and multiply by qp when you use it so you don't have to recompute (as much) everytime the function is called.
3. Do we need to multiply weight by qp? aq_strength already multiplies qp when calculating qp_adj; I don't think aq_sensitivity should scale with qp.
4. I don't think shifting by the weight gives enough precision and I think it's too aggresive. I say multiply by the weights and just tweak the weights to achieve your intended scaling curve.

[Dark Shikari]

I would also argue that the difference doesn't need to be scaled by the original. I think 3->0 is as much of an error as 203->200. So I say just do:
difference^2

No, this is an important tenant of the algorithm--a flat block has much higher error relative to its actual frequency values. The same error is much more tolerable in a "non-flat" block.


Also, about weights:

1. Max of i+j is 14, so your weightqp has 1 extra number.
2. You can just have dctweight be a constant and multiply by qp when you use it so you don't have to recompute (as much) everytime the function is called.
3. Do we need to multiply weight by qp? aq_strength already multiplies qp when calculating qp_adj; I don't think aq_sensitivity should scale with qp.
4. I don't think shifting by the weight gives enough precision and I think it's too aggresive. I say multiply by the weights and just tweak the weights to achieve your intended scaling curve.
Yeah, the weights are completely arbitrary and should really be redone based on some actual thought/testing.

Not sure what you mean by 3).

2) can't be done because the QP can change from frame to frame, so I'd have to at least redo it for each frame.

Otherwise those are definitely some good optimizations.

[RaynQuist]

No, this is an important tenant of the algorithm--a flat block has much higher error relative to its actual frequency values. The same error is much more tolerable in a "non-flat" block.

I see what you're doing now and I like it.


2) can't be done because the QP can change from frame to frame, so I'd have to at least redo it for each frame.

I meant something like:

int weightqp[16] = {0,0,1/50,1/40,1/30,1/20,1/15,1/12,1/10,1/8,1/6,1/5,1/4,1/3,1/2,1};
for( i=0; i<8; i++ )
for( j=0; j<8; j++ )
dctweight[i][j] = weightqp[i+j];

static int x264_subtract_dctq()
{
...
t += (s * dctweight[i][j]);
...
}

void x264_adaptive_quant()
{
...
for()
{
total += x264_subtract_dctq()
}
total *= qp;
...
}

If we multiply by weight instead of shifting then qp can be factored out. Of course weightqp needs to be changed so it's not all zero.
Another idea is that since we only have 52 qp's we can just precompute them all.


Not sure what you mean by 3).

Using qp to influence "how much aq this block needs" is already done here:

qp_adj = (int)(qp * ((2.0 * h->param.analyse.f_aq_strength / pow(2 - fc, h->param.analyse.f_aq_sensitivity))));

I'm asking if we really need to use qp in the weights also, because then we'd be scaling by qp twice.

And I forgot something:

for( j=0; j<8; j++ )
for( k=0; k<8; k++ )
origdct[j][k]=dct[j][k];

This can use a memcopy or something.

[Dark Shikari]

The reason I used QP in the weights was I *guessed* that when encoding at lower QPs, you'd need to take into account the high frequency components of the source more than at high QPs.

As I said, the weights need actual testing, and perhaps some input from someone with a lot of experience with DCT encoding.

[kad77]

Hello all, I've read through this thread, and perhaps I missed someone saying this...

#1 has *very* noticeably decreased luminosity in at least the upper left of the frame compared to #2 and #3. The decreased brightness is not uniform to all of #1, but this color/brightness distortion would need to be fixed before switching away from the old algorithm.

FWIW, I liked 2,(3/1),4,5,6 before I read the results, although #1 dealt with grain better than #3 except for the darkening of the image.

#3 seemed to enhance the grain too much, and #1 kept edges defined while subtlety lessening grain (probably luminosity tampering again). #4 artificially increased brightness in the lower part that was originally dark, which is a distortion in the other direction of #1 and negatively affected the scene.

#5 blocky, blah, #6 seemed very low bitrate + blocky, and it turned out in fact that it was.

Nice job, keep up the good work! I think your new algo has promise.

[Sagittaire]

Dark Shikari, you can write a patch for your new AQ ...

[addit]

Any news on this? I've just encountered a grain heavy source that would be perfect to test this with .:)

[Dark Shikari]

Any news on this? I've just encountered a grain heavy source that would be perfect to test this with .:)I've been quite busy with real-life work lately, though I've been working on my GrainOptimizer some.

Perhaps in a bit I'll get back to working on my AQ and optimizing it for grain, and not just truly flat areas.

[Ranguvar]

They are close enough that I don't see the point - only after staring at it for many minutes can I discern any difference at all, so there's no need to be picky during encoding, go with the fastest :)