PSY settings for grain tuning

[excellentswordfight]

Hi,

As most users of x265 are aware no-sao together with a slow presets (slow/slower) are pretty much a must to achieve high quality output of "film" content, I personally also add a slight deblock reduction (-1,-1). for high quality SDR encodes aq-mode 1 also seems to perform the best.

So far so good, and I think a lot of users can agree on those settings. And I do not think at all that it produces something that is subpar to standard x264 tune film encodes.

But something that I see a lot of diversity on is the psy settings, this also something that the old tune grain preset (that tbh should be removed if they wont update it with sane settings) cranks up. So if a tune film preset would be something like "--no-sao --deblock -1,-1" I was wondering what a tune grain preset would look like.

So I encoded a part of criterion collections release of mulholland drive, with a vast number of combinations of psy-rd psy-rdoq combinations and values (about 20 of them), to see if i could find some sort of optimal values for grainy content. But to my surprise, even at low values both these started to amplify grain, as well as making changes to the structure of the grain. I was aware of this issue, but was under the assumption that it happened at far higher values. When I actually made comparison to the source the default psy-rd 2 psy-rdoq 1 was the closest, i.e. least amount of change when switching between sources. Cause I have used vales in the 2-4 range in the past for grainy content, but after this I think I might stop that.

So the question is, why is it rather common to see higher values? For those who increase this, what methodology have you used to evaluate those settings? Am I missing something? Is this just like a lot of other settings, that were used in the early days of x265 were detail retention was hard were pretty much every relevant option possible were tweaked to try to combat it just keep floating around like the no-strong-intra-smoothing?

[Z2697]

Is this just like alot of other settings, that were used in the early days of x265 were detail retention was hard were pretty much every option possible relevant were tweaked to try to combat it that just keep floating around like the no-strong-intra-smoothing?

Pretty much yes.

And no, the psychovisual optimization is not the root cause of "distorted grain structure". The motion compensation itself is. The grain by its nature can't be efficiently predicted.

[GeoffreyA]

I also think high values are common because many follow, perhaps with limited or no testing, the conventional advice of popular guides.

As for the source, Criterion's restoration of Mulholland Drive carries deadly grain, the 1080p heavier than the 2160p version. I've put this film on hold till I find a way to encode it well!

[hellgauss]

I also use very low psy, even with grain, I prefer to lower the -crf (but I only encode anime).

I read that rskip=0, or rskip=1 with low threshold is also helpful.

But in general, the best method to keep grain is... to remove it, especially fine grain! This allows to lower the crf without make bitrate explode (otherwise, just keep the source). As a preprocessing filter I use removegrain=1 (undot) or =2, or =2 applied twice. For codec settings the two options nr-intra and inter are very effective. Try e.g. nr-intra=50 and nr-inter=150 for test, and then tune.

As a personal opinion i think that grain is more evident on luma channel, so another option that you can test are chroma-qp-offsets=1 (I would not go higher)

[microchip8]

I use custom settings with high values of psy-rd. (4.0) and psy-rdoq (15.0). Why? Because every time I encode a clean source with standard or low values, I get banding. That's why I use such high values that completely eliminate banding. Don't argue with me as I won't change them! :)

[Z2697]

I use custom settings with high values of psy-rd. (4.0) and psy-rdoq (15.0). Why? Because every time I encode a clean source with standard or low values, I get banding. That's why I use such high values that completely eliminate banding. Don't argue with me as I won't change them! :)

Then your source is "not clean". You destroyed the dithering pattern, and trying to save it with high psy value.
And this thread isn't for discussion of "clean source".

[benwaggoner]

I also use very low psy, even with grain, I prefer to lower the -crf (but I only encode anime).
Anime itself has quite different tuning than natural image film/video. What anime are you working with that has a whole lot of grain? Is it intentional, or just the result of old processes used in old titles?

I read that rskip=0, or rskip=1 with low threshold is also helpful.Yeah, recursion skipping works better with content that doesn't have most of the energy in the smallest TU sizes. Which is the opposite of fine grain.

But in general, the best method to keep grain is... to remove it, especially fine grain! This allows to lower the crf without make bitrate explode (otherwise, just keep the source). As a preprocessing filter I use removegrain=1 (undot) or =2, or =2 applied twice. For codec settings the two options nr-intra and inter are very effective. Try e.g. nr-intra=50 and nr-inter=150 for test, and then tune.
Yeah, if the content has grain that isn't creative intent, removing it is the right thing to do creatively and compression-wise. I see a lot of excessive grain in 4K & 8K remasters, where they have all the grain that was on the negative, which is a lot more grain than the original creatives would have seen projected on their perf screen at 14 foot-lamberts. There is no value in preserving grain that the people who approved the final cut didn't see.

As a personal opinion i think that grain is more evident on luma channel, so another option that you can test are chroma-qp-offsets=1 (I would not go higher)
FWIW, for film content in HDR, I find negative chroma offsets are useful, and sometimes required.

[hellgauss]

My first meet with grain has been Nier Automata (blu ray), but I encoded bad since I was not aware of what to do.

Now I'm planning to rip Perfect Blue 4K remastered edition (SDR), which has really a lot of fine grain.

I always used chroma offsets=1, but I did only one encode in hdr, and it was very easy and very compressible. Thank you for your suggestion.

[benwaggoner]

My first meet with grain has been Nier Automata (blu ray), but I encoded bad since I was not aware of what to do.

Now I'm planning to rip Perfect Blue 4K remastered edition (SDR), which has really a lot of fine grain.

I always used chroma offsets=1, but I did only one encode in hdr, and it was very easy and very compressible. Thank you for your suggestion.
Fine but sharp grain is the work of Satan. It pretty much disappears in any downscale due to low pass filtering, so any sort of lookahead based on downscaled frames doesn't see it. And so the final frame is encoded using really catastrophically wrong lookahead data.

If it is fine light grain things can be okay (but use a -1 chroma offset for HDR), but if it's heavy, like randomizing the 6+ least significant bits it can become pretty uncompressable, and it'll get noise reduced by quantization if not something before.

x264's --nr-inter is nice as it acts as an adaptive deadzone, and those denoises exactly the temporal noise that is hardest to compress. It's not as "clean" as a really good temporal denoiser in preprocessing, but can work in a pinch or if a prior denoise couldn't do enough. Basically it ignores small details that change randomly every frame. So it can persevere the feel of grain to some degree, but the grain will "swirl" and bounce around less.

You generally need to use some --nr-intra with it to avoid keyframe strobing; as suggested above something in the 1:4 to 1:2 range seems to work well. Or if --nr-inter is low enough, you may not need any --nr-intra at all. You want to only use as much --nr-intra as you need to, because it actually does reduce spatial detail.

The other common issue with grain is that it throws off block size/shape estimations with a lot of threshold variance. So the same flat-but-grainy area can wind up looking very different frame to frame. That's where turning off --rksip, --tskip-fast, and limiting CU size with --ctu 32 can help. In some particularly challenging cases, setting --min-cu-size 8 has helped.

The right solution for the industry is Film Grain Simulation. I hope we get a codec with a reliably good implementation of it someday. AV1's is the best so far, but can't emulate larger grain in high resolution well and has a repeated pattern problem that's finicky to get around.

[Z2697]

That's where turning off --rksip, --tskip-fast, and limiting CU size with --ctu 32 can help. In some particularly challenging cases, setting --min-cu-size 8 has helped.

tskip is designed to code clean, sharp and high-contrast edges more efficiently, so it's better used in "screen content" where such edges are abundant and not moving or transforming, thus tskip is better disabled completely, for basically any regular content.
I mean, I assume it's not gonna hurt quality, but it's a complete waste of computation power, and time.

min-cu-size is already 8 by default... only if you choose preset ultrafast the value is increased to 16.

The right solution for the industry is Film Grain Simulation. I hope we get a codec with a reliably good implementation of it someday. AV1's is the best so far, but can't emulate larger grain in high resolution well and has a repeated pattern problem that's finicky to get around.

It's good when you see grains as "all grains look like the same", like how PNS in audio encoding is built upon.
But I think the topic of this thread is to "care about" the grains' look...

[GeoffreyA]

I remember that -rd 6 could alter the grain unnaturally in some circumstances.

[Z2697]

I remember that -rd 6 could alter the grain unnaturally in some circumstances.

Compared to?

[GeoffreyA]

-rd 4. I could be wrong.

[benwaggoner]

tskip is designed to code clean, sharp and high-contrast edges more efficiently, so it's better used in "screen content" where such edges are abundant and not moving or transforming, thus tskip is better disabled completely, for basically any regular content.
I mean, I assume it's not gonna hurt quality, but it's a complete waste of computation power, and time.
--rskip is also useful for things like credits and cel animation. I've seen it provide some value for clean natural image content, but not as much. And yeah, it's deadly with heavy grain.

min-cu-size is already 8 by default... only if you choose preset ultrafast the value is increased to 16.
Since --tu-* defaults to 3 in slower and veryslow, if you set --ctu to 32 you can wind up with 4x4 CUs. But yeah, not at the default medium. I tend to default to slower and iterate from there.

It's good when you see grains as "all grains look like the same", like how PNS in audio encoding is built upon.
But I think the topic of this thread is to "care about" the grains' look...
Right. Grain is actual physical particles with shape and edges if working at a high enough resolution. Placement is spatially and temporally random, but the grain particles themselves are pretty distinct based on film stock and other things.

[Z2697]

Since --tu-* defaults to 3 in slower and veryslow, if you set --ctu to 32 you can wind up with 4x4 CUs. But yeah, not at the default medium. I tend to default to slower and iterate from there.

IIRC, smallest CU is 8x8, TU can go down to 4x4.
And tu-*-depth is a bit weird, it's not a hard limit of minimal TU size (as calculated from CTU).