Idea, To reduce ringing using psy trellis

[*.mp4 guy]

X = kept frequency
O = dicarded frequency

Hypothetical frequency retention with psy trellis
X X X X X X X O
X X X X O O O O
X X X X X O O O
X X O O O X O O
X X O O O O O O
X X O O O X O X
X O O X X O X O
X X X X X O X X

Proposed additional rule, to reduce ringing, by sacrificing sharpness:
do not keep a frequency that is preceded directly by a discarded frequency, The effect would "cascade" so, that if a frequency is dropped by the rule, and the rule would cause additional frequencies to be dropped based on this, they are dropped aswell.

Hypothetical frequency retention with psy trellis and additional rule
X X X X X X X O
X X X X O O O O
X X X X O O O O
X X O O O O O O
X X O O O O O O
X X O O O O O O
X O O O O O O O
X O O O O O O O

Consequency: less high frequency retention (bluring), less orphaned frequencies, greater consistency of frequency distribution (less ringing)


This will probably need to be tweaked a bit to keep it from over
zealously pruning frequencies in some cases (I'm thinking noise/texture in mostly empty blocks).

[Dark Shikari]

Bad idea; would almost surely destroy fine detail retention; I don't see how such a rule takes into account the fact that one can indeed have higher frequencies that have near-zero neighbors. In fact, if we follow your rule to the letter (drop, rather than merely choose the lower of the two frequency values available), it could result in massive visual artifacting in cases where the coefficient was considerably larger than 1.

Additionally, trellis works in zigzag order to begin with, so your spatial rule is probably not possible anyways.

[*.mp4 guy]

You won't ever get considerable artifacting from eliminating the last two overall frequency bins, especially if you consider the most important part of them would be protected.


X X X X X X O O
X X X X X X O O
X X X X X X O O
X X X X X X O O
X X X X X X O O
X X X X X X O O
O O O O O O O O
O O O O O O O O
(http://img361.imageshack.us/my.php?image=lasttwobinswr9.png)

X X X O X X X X
X X X O X X X X
X X X O X X X X
O O O O X X X X
X X X X X X X X
X X X X X X X X
X X X X X X X X
X X X X X X X X
(http://img529.imageshack.us/my.php?image=only3rdbinfw4.png)
(click on code boxes to see example images, created with dctfilter)

Also, you can have high frequencies without preceding frequencies being present in large amount, but that only happens in mostly flat blocks where the high frequencies are texture. Blocks that contain features, specifically transients, will never have orphaned high frequencies.

[Dark Shikari]

But there's one problem with your approach; you're quantizing actual pixel data, not motion-compensated residuals.

When you deal with residuals instead of pixel data, things get a lot more complicated, and the normal spatial rules no longer necessarily apply.

(and, as you said, your approach breaks in textured blocks--a truly good approach would deal well with both transcients and textured blocks.)

Also, are you entirely sure that psy-trellis's problems are even from the highest frequencies? I would think, given that its trellis, that those would be highly weighted against due to bit cost.

[*.mp4 guy]

Motion compensated data isn't as different as people like to blather on about. When you get down to it, all that really matters is the psychovisual impact of different frequencies, it doesn't matter if you maul a frequency delta version of a low frequency, or a spatial version, the end result is always a block artifact, they are just slightly different looking blocks artifacts, the same generally extends to ringing and other artifacts, but gets a bit more complicated.

I'm not sure how it would react on texture, for all I know it might help there aswell, I don't have anyway to test what a nonlinear system like this would do.

about ringing: ringing is actually caused by a sort of garbled version of the gibbs effect you get when you lowpass an image, the garbling happens because you get frequencies dropped that are very close to frequencies that arent droped, in a semi random manner. By enforcing a consistency requirement, you will get closer to realising a simple lowpass-esque artifact, which generally, I find, are much less annoying then ringing.


[edit]I would argue that the removal of the inconsistencies by dropping more high frequencies makes the image look much better:


X X X O O O O O
X X X O O O O O
X X X O O O O O
O O O O O O O O
O O O O O O O O
O O O O O O O O
O O O O O O O O
O O O O O O O O
(http://img73.imageshack.us/my.php?image=3rdandupremovedep2.png)

[Dark Shikari]

Motion compensated data isn't as different as people like to blather on about.How about just solve the problem by doing it psy-trellis style; consider the value of these optimizations on the reconstructed frequency domain rather than the residual domain? We already have a framework for it with psy-trellis; no reason not to use it further.about ringing: ringing is actually caused by a sort of garbled version of the gibbs effect you get when you lowpass an image, the garbling happens because you get frequencies dropped that are very close to frequencies that arent droped, in a semi random manner. By enforcing a consistency requirement, you will get closer to realising a simple lowpass-esque artifact, which generally, I find, are much less annoying then ringing.Lowpassing is generally much better than ringing; best is blurring of the transcient instead of ringing, which is similar to a lowpass.

The idea sounds reasonable, but there is at least one technical issue with doing something remotely similar to the above using reconstructed frequency values: trellis runs in reverse order, from the highest to lowest frequency, so it doesn't know how the lowest frequencies have been quantized yet.

Now, you could simply be lazy and do pure deadzone quantizer to represent the reconstructed frequency coefficients, and see if that works. My thought was to say "if left or top neighbor of this frequency was quantized towards a lower absolute level value than the source, bias towards the same direction in the current frequency, with the bias determined by lambda and the amount the neighbor was quantized towards a lower absolute level."

So, for example, let's say the left frequency residual was 0.6 and the top frequency residual was -0.8, quantized. The left predicted residual was 0.3 and the top was 0.5. So the source values would be 0.9 and -0.3.

Now, let us say that the left frequency residual was quantized down (0.6 -> 0) and the top frequency residual was quantized up (-0.8 -> -1). This means our bias is (0.9 - 0.3) + (0.3 - 0.5), or 0.4. Multiply by this by some lambda, and subtract this from the cost of the mode which results in a lower abs_level in the reconstructed frame.

[*.mp4 guy]

Now, you could simply be lazy and do pure "deadzone zero" quantizer to represent the reconstructed frequency coefficients, and see if that works. My thought was to say "if left or top neighbor of this frequency was quantized towards a lower absolute level value than the source, bias towards the same direction in the current frequency, with the bias determined by lambda and the amount the neighbor was quantized towards a lower absolute level."

That sounds like a more robust way of handling the situation. Since this will likely be more usefull in some cases (artificial sources) I think Idealy the strength should have a user adjustable range from "do nothing" through "do moderate clamping" to "zero everything to the left or bottom of a zeroed frequency, and reduce all frequencies to the left or bottom of a reduced frequency by the same or larger percentile margin".

[edit]
So, for example, let's say the left frequency residual was 0.6 and the top frequency residual was -0.8, quantized. The left predicted residual was 0.3 and the top was 0.5. So the source values would be 0.9 and -0.3.

Now, let us say that the left frequency residual was quantized down (0.6 -> 0) and the top frequency residual was quantized up (-0.8 -> -1). This means our bias is (0.9 - 0.3) + (0.3 - 0.5), or 0.4. Multiply by this by some lambda, and subtract this from the cost of the mode which results in a lower abs_level in the reconstructed frame.
That sounds good for a base strength, to increase or decreas from. Also this doesn't just apply to direct neighbors, it should aply to ALL frequencies to the left or bottom of the reduced frequencies. In a "geometric" fashion.

[Dark Shikari]

That sounds good for a base strength, to increase or decreas from. Also this doesn't just apply to direct neighbors, it should aply to ALL frequencies to the left or bottom of the reduced frequencies. In a "geometric" fashion.Why don't you hop on #x264dev on Freenode to discuss this in more detail? It'll be a lot easier to come up with a robust solution in a more real-time environment.

[*.mp4 guy]

I've recently moved, and I am dealing with a wireless network, for which no offensive explatives are available to sufficiently describe its horribleness, but its actually working right now, so IRC could be possible.