Preserving details & grain - worse settings superior? [pics]

[hristoff2]

I wonder how that could be explained :eek:

This is not about 'what looks better' but why x264 acts like this since in theory the better / more effective algorithm should end visually superior, right?
take a look at the left upper arm for example

(x264 rev. 680)
same script except:


--trellis 2 --me umh
http://nwgat.net/woot/files/51/comparisonwtf/trellis2umh_thumb.png (http://nwgat.net/woot/files/51/comparisonwtf/trellis2umh.png)
adjusted brightness (making it more visible)
http://nwgat.net/woot/files/51/comparisonwtf/trellis2umh_adjust_thumb.png (http://nwgat.net/woot/files/51/comparisonwtf/trellis2umh_adjust.png)

--trellis 1 --me hex
http://nwgat.net/woot/files/51/comparisonwtf/trelli1hex_thumb.png (http://nwgat.net/woot/files/51/comparisonwtf/trelli1hex.png)
adjusted brightness (making it more visible)
http://nwgat.net/woot/files/51/comparisonwtf/trelli1hex_adjust_thumb.png (http://nwgat.net/woot/files/51/comparisonwtf/trelli1hex_adjust.png)

___
quantizers: avg. difference below .5%

- source wasn't top notch but it's just about the grain anyway

[Dark Shikari]

Its because of --trellis 2. Trellis optimizes specifically for PSNR, often decimating detail in the process. Trellis 2 is particularly notorious for this.

Grain, by definition, has very little effect on PSNR; as a result it generally gets "optimized" out by Trellis.

[Sharktooth]

if you want to preserve noise or grain then disable any R/D optimizations such as trellis and subme > 5.
optimizing for metrics usually means "flattening" the picture by a certain amount unless the encoder supports psychovisual compression models (x264 doesnt).

[Terranigma]

optimizing for metrics usually means "flattening" the picture by a certain amount unless the encoder supports psychovisual compression models (x264 doesnt).
Do you happen to know where one might find the algorithm for this so that they may create a patch for x264? ;)

[Sagekilla]

Do you happen to know where one might find the algorithm for this so that they may create a patch for x264? ;)

I'd imagine this would take quite a bit of work and would take a while to develop properly, and I don't think you'd be able to "find" an algorithm that does it either :(

[Dark Shikari]

I'd imagine this would take quite a bit of work and would take a while to develop properly, and I don't think you'd be able to "find" an algorithm that does it either :(Oh, its quite possible.

One way might be to strategically replace uses of Sum of Squared Differences with Sum of Absolute Differences, which tends to be more grain-friendly.

[Sharktooth]

Do you happen to know where one might find the algorithm for this so that they may create a patch for x264? ;)
That stuff is usually proprietary and patented...

[akupenguin]

Since when does x264 care about patents? Proprietary only matters insofar as it affects how hard it would be to find the algorithm.

[Kurtnoise]

I'd imagine this would take quite a bit of work and would take a while to develop properly, and I don't think you'd be able to "find" an algorithm that does it either :(
Dreams come true (http://manao4.free.fr/x264_grain.0.diff) ?

[Tack]

I'm delighted to see work being done with FGM on x264! Pardon my ignorance, but does this require support in the decoder? And if so, does libavcodec's h264 decoder yet have that support?

[Dark Shikari]

I'm delighted to see work being done with FGM on x264! Pardon my ignorance, but does this require support in the decoder? And if so, does libavcodec's h264 decoder yet have that support?Nope, but you can't have the chicken without the egg. The support will come when x264 implements it.

[burfadel]

On slightly different note, but still in 'preserving detail', I'm wondering about the AQ patch? I know Akupenguin is relunctant to put it in the SVN, and I understand the reasons behind it with CBR/ABR (2 pass or more) modes - as I understand it may stuff up the bitrate, but what about constant quality mode?

If it were enabled by default in constant quality mode, or only available in constant quality mode, the problems with it affecting CBR/ABR would be avoided plus it could be submitted to SVN...?

Or is the problem surrounding the AQ patch of a different nature?

Also, I was wondering about Cef's builds, his are still back at 682. Is your fast ref search patch going to be part of Cef's next build...? how is the progress going with the fast-ref search and it being submitted? sorry about all the questions just got a lot of things to encode and was wondering whether it was worth waiting for these great features! - I know waiting negates faster encoding, but there's a couple of series of tv shows that if I do borrow them from a friend it would be a for a short time, so I'd need the maximum speed possible. - and before there's any comments on that these shows aren't available on DVD!

Anyways, thanks for the great work, keep it up! we're all very appreciative :)

[Yoshiyuki Blade]

Wow, rather old thread brought back here :). It does bring up a question though.

On the subject of detail preservation, does DCT Decimate also do anything significant to details too? I substituted --trellis 2 with it (aka didn't add --no-dct-decimate) , but I'm not exactly sure if it'll botch details the same way --trellis 2 did.

[Dark Shikari]

Wow, rather old thread brought back here :). It does bring up a question though.

On the subject of detail preservation, does DCT Decimate also do anything significant to details too? I substituted --trellis 2 with it (aka didn't add --no-dct-decimate) , but I'm not exactly sure if it'll botch details the same way --trellis 2 did.DCT decimate is more useful than trellis 2 in most respects, though it does lose some detail. Basically what it does is in cases when there is very very little change between a block in one frame and the next, and its all not-really-noticeable high frequency change, it decimates that change to nothing to save bits.

[Yoshiyuki Blade]

DCT decimate is more useful than trellis 2 in most respects, though it does lose some detail. Basically what it does is in cases when there is very very little change between a block in one frame and the next, and its all not-really-noticeable high frequency change, it decimates that change to nothing to save bits.

Ah I see. Thanks for the info. I figured there is some quality loss, but the quality/bitrate ratio makes up for it. I think I tested out no trellis and no dct decimate once at a given filesize. Not a pretty result if I remembered correctly :D.

[Zanejin]

On slightly different note, but still in 'preserving detail', I'm wondering about the AQ patch?

Or is the problem surrounding the AQ patch of a different nature?

I believe that akupenguin dislikes the current AQ algorithm primarily because it's an empirical work-around for something that should in theory work. It does, however, save the day most of the time.

[ToS_Maverick]

Dark Shikari, you mentioned some time ago, that you could change some metrics in x264 to preserve more grain? i think it was about SATD and SAD and you wanted to use the absolute differnce...

did you work on this, any progress ;)

[Dark Shikari]

Dark Shikari, you mentioned some time ago, that you could change some metrics in x264 to preserve more grain? i think it was about SATD and SAD and you wanted to use the absolute differnce...

did you work on this, any progress ;)From my experience the biggest way that grain gets "eaten up" is through the quantization itself; the best way you can help the grain is by using a quantization algorithm designed to retain it. Low deadzone settings can help this.

Modifying x264 to retain more grain would be difficult; the best way would probably be to add some sort of quantization algorithm designed for grain. Well, of course, FGM would truly be the best way.

[ToS_Maverick]

what really confuses me is, how statellite broadcasters are able to conserve not really all grain, but the dirty look. most of the streams have quants in the 20-26 range. what's interesting is, that they get blocky very quickly, if there's a lot of motion.

would be really interesting to know, how they do that.

and if x264 gets FGM, it would once again demonstrate, that it is THE state-of-the-art H.264 codec :D

[foxyshadis]

Either deadzone or a custom matrix that allocates more than the default to detail (and thus less to smoothness, leading to blocking). Sometimes you can get that effect by lowering motion search, but that mainly works by introducing extra quantization noise to hide the removed noise.

[Sagittaire]

Dreams come true (http://manao4.free.fr/x264_grain.0.diff) ?

how use this patch (command line in zone?)

[*.mp4 guy]

Either deadzone or a custom matrix that allocates more than the default to detail (and thus less to smoothness, leading to blocking).

Minor quible, the three major tradeoffs a matrix makes are between detail, blocking, and ringing, though ringing is partially hidden by the inloop deblocker (just like blocking). There are a lot of other attributes a matrix can have, but they are mostly irrelevant here.

[RaynQuist]

how use this patch (command line in zone?)

Looking at the patch and a bit of guessing, it's:

--grainfile [filename]


and the file format is

zone=[start frame]:[end frame]
method=["ar" or "dct"]
blending=["multiplicative" or "additive"]
plane Y:
interval:[low intensity],[high intensity],[param1],[param2],[param3],[param4],[param5],[param6]
interval: ...
..
plane U:
interval: ...
..
plane V:
interval: ...
..
zone=[start frame]:[end frame]
..


The patch basically parses that file and writes it to the SEI header. There's no grain detection and x264 encodes the video as it normally would.

I think

[Dark Shikari]

Looking at the patch and a bit of guessing, it's:

--grainfile [filename]


and the file format is

zone=[start frame]:[end frame]
method=["ar" or "dct"]
blending=["multiplicative" or "additive"]
plane Y:
interval:[low intensity],[high intensity],[param1],[param2],[param3],[param4],[param5],[param6]
interval: ...
..
plane U:
interval: ...
..
plane V:
interval: ...
..
zone=[start frame]:[end frame]
..


The patch basically parses that file and writes it to the SEI header. There's no grain detection and x264 encodes the video as it normally would.

I thinkOf course, given there is now an Avisynth filter for grain modelling that generates the appropriate parameters, we're basically all the way to FGM :D

[AGDenton]

Of course, given there is now an Avisynth filter for grain modelling that generates the appropriate parameters, we're basically all the way to FGM :D

I've been looking around, and I can't find any filter like this. What's its name ?

[ToS_Maverick]

well there is this thread about retention outside the encoder:
http://forum.doom9.org/showthread.php?t=121422

and there is Dark Shikari's GrainOptimizer:
http://forum.doom9.org/showthread.php?t=130611

[Dark Shikari]

well there is this thread about retention outside the encoder:
http://forum.doom9.org/showthread.php?t=121422

and there is Dark Shikari's GrainOptimizer:
http://forum.doom9.org/showthread.php?t=130611Its called GrainTools, I believe. I think Manao coded it?

[AGDenton]

Its called GrainTools, I believe. I think Manao coded it?

Just found it, on the exact same page as the FGM patch... The only problem is that it doesn't seem to output to a file, only debug. I wonder how one can collect debug output on Wine..

[MfA]

Grain, by definition, has very little effect on PSNR; as a result it generally gets "optimized" out by Trellis.
Grain has just as much effect on PSRN as anything else of the same magnitude, it gets optimized out because it's harder to code.

[Dark Shikari]

Grain has just as much effect on PSRN as anything else of the same magnitude, it gets optimized out because it's harder to code.No... grain is by definition material that has a lot of variation but with a low standard deviation. So every single pixel in the grain has a considerable deviation from the source, but no single pixel has a lot. Since the sum of squares favors a few pixels with very high deviation against a lot of pixels with low deviation, grain gets optimized out.

[MfA]

Yes but the large differences get coded anyway, in fact with trellis more low magnitude differences can be coded for the simple reason that the quantizer can be lower ... and grain will still be thrown away while differences with even smaller contributions can stay. I wasn't saying what you said about it's effect on PSNR was little was untrue, I was merely trying to point out for innocent bystanders that is only half the story of why it gets thrown away. It's about rate-distortion, not just distortion.