To grain or not to grain, that is the question

[GeoffreyA]

Aiming for high-quality encoding of films at 1080p—with or without grain—I've settled on these parameters for the most part, along with 10-bit encoding. The tool of choice is FFmpeg, but for the sake of familiarity, here are the x265 CLI translations.

--preset slow --crf 18/21 --level-idc 41 --no-sao --deblock=-1:-1 --aq-mode 1 --subme 5 --max-merge 5 --keyint 240 --min-keyint 24 --rc-lookahead 240 --bframes 8

For the encoder-stumping element of grain, previous threads recommend:

--rskip 2 --rskip-edge-threshold 2


Will --rskip 2 have any negative effects on clean, non-grainy material?
Are there any quality drawbacks of using --ctu 32, and would --merange 26 be the correct accompanying value for a CTU of that size?
Performance is a constraint; so --preset slower isn't practical for the most part. Having said that, --rskip 2 regains quite a bit of speed, some of which could be spent on slower parameters, if not in the realm of diminishing returns.


Many thanks for your time!

[john33]

On topic, I think, but from a slightly different perspective. I am of an age that my formative years of still and cine photography were all based on film. The unavoidable consequence of that was grain. This was something that became worse the faster the emulsion and it was not revered in any way, it was an artefact you could do nothing about. My simple question, therefore, is why are people so pre-occupied with preserving something which we would have got rid of, back in the day, had we had the means to do so?

[rwill]

On topic, I think, but from a slightly different perspective. I am of an age that my formative years of still and cine photography were all based on film. The unavoidable consequence of that was grain. This was something that became worse the faster the emulsion and it was not revered in any way, it was an artefact you could do nothing about. My simple question, therefore, is why are people so pre-occupied with preserving something which we would have got rid of, back in the day, had we had the means to do so?

Grain is an artistic style element.

[GeoffreyA]

That's an excellent question that could be answered in different ways. Your argument is that grain is an artefact, similar to the crackle of a record or the distortion of a cassette—elements that shouldn't be there.

Videophile thinking has, perhaps, led to the venerating of grain. Blu-ray reviews repeat the same hackneyed language: "It's got a fine layer of grain, adding to the filmic effect." Others would say that grain adds "warmth" to the picture; it's aesthetically pleasing, though one can't explain why. I fall into this category, feeling that grain brings a desirable, fleeting quality, an artefact reflecting the medium, like the texture of a canvas. That's not to say we don't have examples of the highest cinematography without it, Blade Runner 2049, and instances of rubbish camera work with it: RoboCop. At the other end of the spectrum are those who maintain that grain is repulsive to the eyes.

Removing grain is challenging to do right, so some would rather have their encoders suffer the debilitating, swirling particles than put up with a television, plastic-like picture, which is thought to be ugly. Many, on good grounds, recoil from, and bitterly lament, the smothered picture of The Lord of the Rings

[microchip8]

1) rskip=2 does not have a negative impact on clean source. I use it all the time with no visible issues. I use a threshold of 1 for more accuracy and it's still "speedy"

2) yes, 26 for merange is correct when you use a ctu of 32. The formula to calculate merange is: ctu size - 4(luma) - 2(chroma) (- 1 if me=hex is used) -- so in case of a ctu=32 you have 32 - 4 - 2 = 26 (provided you don't use me-hex where you have to lower it by one). There are no drawbacks using a ctu of 32 for resolutions below 4K. For 4K and up, ctu of 64 is recommended.

3) see 1)

[john33]

@GeoffreyA: Thanks for your response. I guess your comments are as good an answer as any. I fall into the camp that prefers to eliminate grain as I don't perceive it as being remotely artisitic, but, as you say, each to their own.

[GeoffreyA]

1) rskip=2 does not have a negative impact on clean source. I use it all the time with no visible issues. I use a threshold of 1 for more accuracy and it's still "speedy"

2) yes, 26 for merange is correct when you use a ctu of 32. The formula to calculate merange is: ctu size - 4(luma) - 2(chroma) (- 1 if me=hex is used) -- so in case of a ctu=32 you have 32 - 4 - 2 = 26 (provided you don't use me-hex where you have to lower it by one). There are no drawbacks using a ctu of 32 for resolutions below 4K. For 4K and up, ctu of 64 is recommended.

3) see 1)

Thanks. I was impressed with the speed-up from rskip=2, so wondered whether it could be set for all content. And ctu=32 is helping utilisation; I think it was a couple per cent faster.

@GeoffreyA: Thanks for your response. I guess your comments are as good an answer as any. I fall into the camp that prefers to eliminate grain as I don't perceive it as being remotely artisitic, but, as you say, each to their own.

Nowadays, I'm inclining towards toning down grain, considering how much trouble it causes encoders. That, in itself, is quite a challenge, but KNLMeansCL has given decent results. Too often, denoising results in an overly-softened picture or patchy grain.

[microchip8]

Yes, rskip=2 gives a large speedboost - I use it on all kinds of content and have not comes across any issues. The same can be said about limit-modes when using rect (and amp, which requires rect)

[hellgauss]

I think your setup is quite good.

A few question/comments:

- It is often suggested "--subme 5" . Is there any evidence that 5 is better than 4, also considering that encoding speed is a constraint? 4 is the default for veryslow, while 5 is for placebo. According to the subme table in
https://x265-repo.readthedocs.io/en/latest/cli.html
in order to get a securely better option than 4 you should go to 7.

- In the lower spectrum of crf (18-19) I would try also no-strong-intra-smoothing

-rskip 2 with threshold 2 is ok. Perhaps if you need speed you can rise threshold to 3.

[StainlessS]

Some people think that grain is 'nice', me is not in that camp.

[Z2697]

If it's the "real grain", from the film or sensor, then I'm mostly fine.
But post processing grain, unless it's for a good reason, I don't like them.
Fake the film "aesthetic" through out whole movie, is not a good reason.
Just my personal bias.

rskip doesn't care if it's grainy of clean, but some DCT artifact on clean video may be more obvious than grainy video.
There's no real reason to keep merange below max CU size... well, except the need for speed.

[GeoffreyA]

Yes, rskip=2 gives a large speedboost - I use it on all kinds of content and have not comes across any issues. The same can be said about limit-modes when using rect (and amp, which requires rect)

Luckily, --limit-modes was enabled by default on --preset slow, improving performance.

I think your setup is quite good.

A few question/comments:

- It is often suggested "--subme 5" . Is there any evidence that 5 is better than 4, also considering that encoding speed is a constraint? 4 is the default for veryslow, while 5 is for placebo. According to the subme table in
https://x265-repo.readthedocs.io/en/latest/cli.html
in order to get a securely better option than 4 you should go to 7.

- In the lower spectrum of crf (18-19) I would try also no-strong-intra-smoothing

-rskip 2 with threshold 2 is ok. Perhaps if you need speed you can rise threshold to 3.

Good point. At --subme 5, we are losing half- and quarter-pixel iterations, compared to 4, so it might be wiser to choose either 4 or 7. I had used 5 because it was in --preset placebo and not a massive loss of speed, albeit some. Regarding visual differences between 4 and 5, I can't seem to see any in the short clip I looked at.

Some people think that grain is 'nice', me is not in that camp.

Yes, grain can be quite controversial. It certainly has been a curse to encoders! Little by little, I'm being pulled into the no-grain camp. Jokes aside, there are older movies where it was not stylistic but incidental. Many from the '80s. Then we get contemporary movies shot on film but not looking grainy (Tenet, that boring Oppenheimer, Nope). There's a rarer category where grain was part and parcel of the look, such as The Ring. Lynch's films are an interesting case. I reckon that Mulholland Drive could survive all right without grain, whereas Lost Highway, it would be a loss, taking away from the noir feeling.

If it's the "real grain", from the film or sensor, then I'm mostly fine.
But post processing grain, unless it's for a good reason, I don't like them.
Fake the film "aesthetic" through out whole movie, is not a good reason.
Just my personal bias.

rskip doesn't care if it's grainy of clean, but some DCT artifact on clean video may be more obvious than grainy video.
There's no real reason to keep merange below max CU size... well, except the need for speed.

I agree. There should be a reason, or it should reflect the medium. What do you think of Dune's process, where they printed the digital footage to film and scanned it for an "epic film" effect? Personally, I think the picture looks rather smothered.

[microchip8]

Are we talking about grain or noise? Because grain is different from noise. It's thicker and moves differently than noise. I don't mind a bit of noise (helps reduce certain things like banding if not enough precision and bitrate) but I detest grain. The same way how I hate motion judder with a strong passion.

[GeoffreyA]

Grain, which, indeed, many do detest. I think it looks bad when too thick. As for motion judder, it makes one shudder :)

[jpsdr]

I can understand how people may like grain (i'm not in that case), so i remove it on my projects. The issue i have is the film decay, with old age. I have seen Blu-Ray of movies that were not so "uggly" when i saw them in theatre in my youth when they were out. My purpose in doing this is trying to retreive the youth of the film.

After it's gain vs loss. There is no magic, even if you're using excellent denoiser program to remove grain, there will always be details loss. The better the program is, the best it can keep details (or minimise details loss), but again, no magic.
So, is the grain more annoying than a small details loss ? It's up to you.
The more grain is important, the more there is risk of details loss.
What i regret is that Blu-Ray releases, either they remove grain, either they keep, but they don't offer the possibility to choose. Produce 2 discs, one with grain, another without. Most releases keep grain, and it's better to not remove it than remove it badly.

Grain and noise are different things, but in a video, they share at least more than 95% the same properties (but the bigger the true resolution of video is, the less they share), this is why you can use denoiser program to remove grain.

[GeoffreyA]

Most releases keep grain, and it's better to not remove it than remove it badly.

That's what gets on the nerves of many, when it's so poorly done that it looks like plastic. I believe the late remasters of Terminator and Aliens had this treatment, though I haven't personally seen it except for screenshots. The Two Towers has at least one badly-smeared shot, along with other scenes, particularly of faces (i.e., no VFX), that are quite sharp.

Correction: Apparently, Aliens was sharpened with AI or something; I'm not too sure.

[Z2697]

Are we talking about grain or noise? Because grain is different from noise. It's thicker and moves differently than noise. I don't mind a bit of noise (helps reduce certain things like banding if not enough precision and bitrate) but I detest grain. The same way how I hate motion judder with a strong passion.

The root cause maybe different, but the "randomness" is the same, I mean, each frame is independent from the others.
If it moves differently, then maybe temporal denoise is applied, or some sort of different noise is added to it.
Or just the inherent nature of the post-process with films, before the digital evolution. There may be multiple layers of film, or transfer from film to film...

[Z2697]

What do you think of Dune's process, where they printed the digital footage to film and scanned it for an "epic film" effect? Personally, I think the picture looks rather smothered.

It's certainly something that a STEM brain cannot understand :rolleyes:
Oh wait, is it... similar to glitch art? I'm starting to understand here...
But glitch arting the whole movie is too much for me :scared:
(glitch arting? is that a word?)

[jpsdr]

I believe the late remasters of Terminator and Aliens.
I've bought 4k Blu-Ray of Aliens and Abyss.
There was some critics about being denoised.
It's true there is not a lot of grain left. So few that in the first place i thought i would make my own version, but when i saw the video i said to myself "Yeah, nice, i don't have to do it !":p
Nevertheless, for me, the picture is fine, i liked the result, and didn't find it poorly done.

[benwaggoner]

Grain is an artistic style element.
Sometimes. Moreso in recent content since it has to be added on purpose. But for older content it was just there. And given the softness of an old 14 foot lambert projection onto a perf screen, fine detail that's crystal clear on a 4K TV may have been unseen by the original creators.

The past saw everything through a low pass filter.

[benwaggoner]

I've bought 4k Blu-Ray of Aliens and Abyss.
There was some critics about being denoised.
It's true there is not a lot of grain left. So few that in the first place i thought i would make my own version, but when i saw the video i said to myself "Yeah, nice, i don't have to do it !":p
Nevertheless, for me, the picture is fine, i liked the result, and didn't find it poorly done.
Having had to improve encoding on those two titles before Cameron's team would allow them to be streamed, I am very confident that any release has exactly as much grain as he wants it to have. He has more oversight on his content than anyone else I've heard of.

[benwaggoner]

Are we talking about grain or noise? Because grain is different from noise. It's thicker and moves differently than noise. I don't mind a bit of noise (helps reduce certain things like banding if not enough precision and bitrate) but I detest grain. The same way how I hate motion judder with a strong passion.
More specifically, grain is a very common type of noise. Sensor gain is another common kind of noise.

[GeoffreyA]

I've bought 4k Blu-Ray of Aliens and Abyss.
There was some critics about being denoised.
It's true there is not a lot of grain left. So few that in the first place i thought i would make my own version, but when i saw the video i said to myself "Yeah, nice, i don't have to do it !":p
Nevertheless, for me, the picture is fine, i liked the result, and didn't find it poorly done.

I'm glad you enjoy it. From what I read, it seems divisive, more because of the digital processing that seems to have been applied.

[GeoffreyA]

Having had to improve encoding on those two titles before Cameron's team would allow them to be streamed, I am very confident that any release has exactly as much grain as he wants it to have. He has more oversight on his content than anyone else I've heard of.

Apparently, the problem, if problem there be, is not so much the grain but digital processing, leading to a strange mix of cleanness and, at times, uncanny sharpness, especially in faces. Some viewers maintain that it stands out, changing the feeling of the film, and that AI scaling or sharpening is suspected to have had a hand in it.

https://www.avforums.com/reviews/aliens-4k-blu-ray-review.21742/
https://www.avforums.com/threads/discussion-of-our-review-of-aliens-4k-blu-ray.2495788/

Cameron hit back at fans who criticised the new transfer of Aliens. Sure, he is the director and has the right and knowledge to do as he wishes with his films, and certainly, fans can make a moan for nothing; but if they are pointing out concrete problems, surely it is less than ideal.

[GeoffreyA]

It's certainly something that a STEM brain cannot understand :rolleyes:
Oh wait, is it... similar to glitch art? I'm starting to understand here...
But glitch arting the whole movie is too much for me :scared:
(glitch arting? is that a word?)

Well, I knew the thing but not the term for it, glitch art. :)

With Dune, they didn't "glitch art" the whole movie—or was that Mr. Villeneuve's intent!—but gave it a certain look, hard to describe. Perhaps fittingly dusty, as the sands of Arrakis, and the distance of time.

[excellentswordfight]

Aiming for high-quality encoding of films at 1080p—with or without grain—I've settled on these parameters for the most part, along with 10-bit encoding. The tool of choice is FFmpeg, but for the sake of familiarity, here are the x265 CLI translations.

--preset slow --crf 18/21 --level-idc 41 --no-sao --deblock=-1:-1 --aq-mode 1 --subme 5 --max-merge 5 --keyint 240 --min-keyint 24 --rc-lookahead 240 --bframes 8

For the encoder-stumping element of grain, previous threads recommend:

--rskip 2 --rskip-edge-threshold 2


Will --rskip 2 have any negative effects on clean, non-grainy material?
Are there any quality drawbacks of using --ctu 32, and would --merange 26 be the correct accompanying value for a CTU of that size?
Performance is a constraint; so --preset slower isn't practical for the most part. Having said that, --rskip 2 regains quite a bit of speed, some of which could be spent on slower parameters, if not in the realm of diminishing returns.


Many thanks for your time!
1) rskip=2 does not have a negative impact on clean source. I use it all the time with no visible issues. I use a threshold of 1 for more accuracy and it's still "speedy"
Just fyi for me Rskip 2 has always resulted in pixel shift. It does not have negative impact during normal viewing mind you so the speed-increase could be a worthy tradeoff, but when i compare stills were rskip is the only difference I can see that pixels are "moved" from the source. This change doesnt sit well with me personally so I have stopped using it.

When you say it does not have a netagative impact have you done isolated pixel-peeping tests?

[GeoffreyA]

Just fyi for me Rskip 2 has always resulted in pixel shift. It does not have negative impact during normal viewing mind you so the speed-increase could be a worthy tradeoff, but when i compare stills were rskip is the only difference I can see that pixels are "moved" from the source. This change doesnt sit well with me personally so I have stopped using it.

When you say it does not have a netagative impact have you done isolated pixel-peeping tests?

That is concerning indeed. Does it occur on every frame? (So it would be easy to test.)

[microchip8]

Just fyi for me Rskip 2 has always resulted in pixel shift. It does not have negative impact during normal viewing mind you so the speed-increase could be a worthy tradeoff, but when i compare stills were rskip is the only difference I can see that pixels are "moved" from the source. This change doesnt sit well with me personally so I have stopped using it.

When you say it does not have a netagative impact have you done isolated pixel-peeping tests?

No, I have not. I don't usually compare stills or do pixel inspection.

Have you done any tests with the accuracy by lowering the rskip-edge-threshold ?

[excellentswordfight]

That is concerning indeed. Does it occur on every frame? (So it would be easy to test.)
I put it on my todo list to do more exhaustive testing (which was never done :P ), but I do remember that i noticed it on stuff like strains of hair and similar details, were the whole strain got shifted, so probably edge-related.

Having had to improve encoding on those two titles before Cameron's team would allow them to be streamed, I am very confident that any release has exactly as much grain as he wants it to have. He has more oversight on his content than anyone else I've heard of.
That could be the case, but then its even more perplexing that pretty much all his older titles have a problematic history of releases, especially the 4K ones.

I actually bought both Aliens and Terminator 2 on uhd-bluray, and to my disappointment, I actually used my bluray copies when I rewatched them... And even Titanic, which is otherwise one of the better ones - had this insane shot were whatever "AI" tool they used turned Rose in to an old painting.

https://ibb.co/232m4RHk
How the next cut looks like for reference
https://ibb.co/HDPfT59t

At the end of the day I think he just has some fixation with trying to make these title look modern, and in the process ends up in the middle, which to me is the worst of two worlds.

Im gonna give a shoutout to Jackie Brown here, absolutely incredible uhd restoration/transfer of a 90s title shot on 35mm with a very pleasing amount and good looking/encoded grain.

[microchip8]

Aliens in particular has a lot more grain in the FHD Blu-ray than in the 4K one. The latter has been cleaned up a lot. I'm not sure I like it. It looks a bit of plastic to me, especially that scene in the beginning where Ripley wakes up in the all-white ship hospital room

[GeoffreyA]

Aliens in particular has a lot more grain in the FHD Blu-ray than in the 4K one. The latter has been cleaned up a lot. I'm not sure I like it. It looks a bit of plastic to me, especially that scene in the beginning where Ripley wakes up in the all-white ship hospital room

I put it on my todo list to do more exhaustive testing (which was never done :P ), but I do remember that i noticed it on stuff like strains of hair and similar details, were the whole strain got shifted, so probably edge-related.


That could be the case, but then its even more perplexing that pretty much all his older titles have a problematic history of releases, especially the 4K ones.

I actually bought both Aliens and Terminator 2 on uhd-bluray, and to my disappointment, I actually used my bluray copies when I rewatched them... And even Titanic, which is otherwise one of the better ones - had this insane shot were whatever "AI" tool they used turned Rose in to an old painting.

https://ibb.co/232m4RHk
How the next cut looks like for reference
https://ibb.co/HDPfT59t

At the end of the day I think he just has some fixation with trying to make these title look modern, and in the process ends up in the middle, which to me is the worst of two worlds.

Im gonna give a shoutout to Jackie Brown here, absolutely incredible uhd restoration/transfer of a 90s title shot on 35mm with a very pleasing amount and good looking/encoded grain.

I haven't seen the new transfer of Aliens in action, but I think I need to and compare with the older release. It's evident that some form of AI tooling has been involved in these remasters. He's not the only director, I think, who's fond of such "revisionism."

[Z2697]

rskip can't create pixel shift technically, it doesn't touch the pixel data. All it does is just as the name says, skipping recursion (of CU partitioning) based on some metrics.

[Z2697]

My take on AI is that no-AI is best AI.
I mean, AI is evolving, give us the "natural" material so in the future we can use better AI on it (if we even want to at all), not stuck on whatever the inferior AI result that got pressed onto the disc.
:rolleyes:

Or have to dow- I mean buy re-re-re-re-release version of it.

[excellentswordfight]

That is concerning indeed. Does it occur on every frame? (So it would be easy to test.)
rskip can't create pixel shift technically, it doesn't touch the pixel data. All it does is just as the name says, skipping recursion (of CU partitioning) based on some metrics.
Im pixel-peeping way to hard trying to replicate this, so dont mind me... Not sure if it might have been worse in conjuncture with some other setting or maybe in an older version or that I was just (un)lucky when I noticed it in the first place.

Pixel-shifting is probably a bad description, its like in some cases objects "move" and is one or two pixels off, but now when searching for it, I can find it with rskip 1 as well, just at a less extent, so more like an accuracy/compression issue. Cause looking at it now (at stills, zoomed mind you), it does hit compression, and has more artifacts and issues in general. So for example the version with rskip 2 introduced compression issues in the background as the character moved across it in TOS, something not present in the control encode https://ibb.co/9m7BdFT0. Could be that at low crf/high bitrate levels these difference are even more negligible and the speed-gain would be a reasonable tradeoff.

[GeoffreyA]

My take on AI is that no-AI is best AI.
I mean, AI is evolving, give us the "natural" material so in the future we can use better AI on it (if we even want to at all), not stuck on whatever the inferior AI result that got pressed onto the disc.
:rolleyes:

Or have to dow- I mean buy re-re-re-re-release version of it.

A take I agree with.

With or without AI, a director has all the right in the world to change a film according to his or her taste, but that does not make it infallibly better. There are examples of revisionism changing works that were all right. The present version of The Matrix, sporting a teal-like grading, comes to mind.

Sometimes, there is subjective improvement. The Criterion version of Lost Highway has a browner hue than the familiar, vibrant version. Revisionism? Perhaps. But we learn that this was closer to the photography on set, with the chocolate filters on the lenses, and I think lends itself better to the noir atmosphere.

[Hellboy.]

First thanks. I learned a lot from everyone here.

For 1080p I use vbv-maxrate=50000 and vbv-bufsize=50000, is that ok or is it better to use one higher than the other? (I use CRF)

[GeoffreyA]

First thanks. I learned a lot from everyone here.

For 1080p I use vbv-maxrate=50000 and vbv-bufsize=50000, is that ok or is it better to use one higher than the other? (I use CRF)

I think that would be all right because those are the defaults for Level 4.1 High. The next step up would be for > 1080p.

[GeoffreyA]

Im pixel-peeping way to hard trying to replicate this, so dont mind me... Not sure if it might have been worse in conjuncture with some other setting or maybe in an older version or that I was just (un)lucky when I noticed it in the first place.

Pixel-shifting is probably a bad description, its like in some cases objects "move" and is one or two pixels off, but now when searching for it, I can find it with rskip 1 as well, just at a less extent, so more like an accuracy/compression issue. Cause looking at it now (at stills, zoomed mind you), it does hit compression, and has more artifacts and issues in general. So for example the version with rskip 2 introduced compression issues in the background as the character moved across it in TOS, something not present in the control encode https://ibb.co/9m7BdFT0. Could be that at low crf/high bitrate levels these difference are even more negligible and the speed-gain would be a reasonable tradeoff.

Is it happening on the I, P, or B frames, or any of them?

[Z2697]

Is it happening on the I, P, or B frames, or any of them?

rskip (either one) doesn't affect I frames.
But I guess that's hardly a good news, we don't do all-intra encoding that often right?
And what if we want to speed up all-intra encoding this way?
Is it a bug or a feature? Maybe worth reading some code to try to find out...

[microchip8]

I think excellentswordfight is seeing things that are not there. But regardless, without some form of rskip, encoding really slows down a lot even on my 12c/24t CPU doing Full HD encodes. I'm happy with rskip=2 and the lowest possible threshold of 1

[Z2697]

DCT ringing/noise on "incufficient recursion depth" is real and is a problem with rskip, or even without rskip :rolleyes:
rskip makes this problem slightly (hopefully) more likely to happen.

But some slight "morphing" is normal for motion estimation based codecs, unless you have so many bits to signal all / most of the residuals.