How best to resize and heavily degrain?

[tppytel]

I'm reencoding some movies at low bitrates (trying for 720p, ~1500 mbps HEVC video) to do remote streaming with Plex. I'm not expecting great quality at such low bitrates, just something casually watchable. I've only done a few movies so far, but have gotten acceptable results running the original bluray source through MDegrain1 or 2 (if necessary), resizing down to 720p with Spline36Resize, and encoding with x265 at crf20. I should probably just do this with 2-pass encoding, but it's interesting to observe what bitrates crf20 picks for me.

However, I'm having trouble with 12 Angry Men, which has very strong film grain (clip here (https://www.dropbox.com/s/rt62c44xcrbqox6/clip.mkv?dl=0)). I've tried MDegrain, DegrainMedian, dfttest, and some others with various settings, but none of them come close to removing the grain prior to encoding. I tried just throwing it to a 2-pass 1500mbps encode (slow speed, no other settings) with

DeGrainMedian(mode=1,limitY=16)
Blur(0.5)
Spline36Resize

I guess the result isn't unwatchable, but I can see the encoder struggling valiantly to keep patches of grain in the picture and it's pretty distracting. Any suggestions for how to better degrain before encoding? Or for x265 settings that would better prevent it from trying to maintain the film grain?

Thanks.

[Taurus]

Oi, Oi, Oi,
what a mess!:mad::p
This is not grain, it is dancing grain!
Try MCDegrain or Temporaldegrain.
Just a warning: They are deadslow but worth all the CPU Cycles:D.
And hard to setup right.
Just did a small test torture encode with your sample:
1280x720 bicubicresize, x264 crf20 medium preset:
2Mb filesize.
with x264 crf17 very slow preset:
3.2Mb
dont use the grain tuning...
and voila all the grain is gone.

[SaurusX]

I guess you haven't tried KNLMeansCL? It will kill the grain and still retain much of the detail. Just be sure to crank up the D (temporal) parameter.

[LemMotlow]

That sample is grossly oversharpened. Strong degrainers will work (slow, as Taurus says) but remove all the grain and you'll have banding in large areas like background walls and shadows. Likely you'll have to add something like GradFun3 and some very fine-grain dithered noise (AddGrainC) or it will look worse when re-encoding at low bitrates.

[tppytel]

Thanks for the suggestions. TemporalDegrain does indeed work wonders, but yeah... wow, it's slow. Here's (https://www.dropbox.com/s/glsvi0wcuduqn4j/TemporalDegrain.mkv?dl=0) a clip after a straight

TemporalDegrain(degrain=3, ov=4, blksize=16)
Spline36Resize(1200,720)

run through a 2-pass x265 encode for 2mbps. Any other suggestions for improvement? I'm not sure if there's banding on the wall behind the actors, or the actual wall is just a little smudgy. Would further debanding help any?

edit: Also, FWIW, the clip is a straight rip of the Criterion Collection BD for this film. The grain does indeed seem pretty over the top.

[Lyris]

Oi, Oi, Oi,
what a mess!:mad::p
This is not grain, it is dancing grain!

Never heard of "dancing grain" before :) The grain in that example is just standard film grain (quite a lot of it) plus sharpening (or "aperture correction" or whatever the euphemism of the era was!)

[Taurus]

Never heard of "dancing grain" before

So just call it "pulsating grain" vs static grain :p
I got quite pleasing results with:

Temporaldegrain(degrain=3,sigma=16,pel=2,blksize=8,sad1=400,sad2=320,hq=2)
BicubicResize(1280,720)

Did not test on x265, x264 only at crf 17/very slow.
And i dont see any banding, just the smudge on the walls :D

And remember: This script is a resource hog.
You have to decide if it's worth the hazzle....

[burfadel]

It's one of the things I'm trying to achieve with mClean https://forum.doom9.org/showthread.php?t=174804

Something that can suit most cirumstances without too much faffing around trying to find the best settings which you'll find aren't really 'best' as other areas of the video require different settings. You can adjust the removal amount by adjusting the thSAD. Since it's oversharp, I would suggest the settings mClean (sharp=0, deband=3), if you do decide to try it. It's still a work in progress :). Also maybe try a higher thSAD, such as thSAD=600. There is a setting, rn, that adds back some altered noise to avoid the picture looking flat. If there is heavy grain it might need lowering a bit to say, 10. Deband=3 applies both f3kdb debander (needs the latest available build) and a balanced autoadjust setting that applies mild auto gain, colour gain, correct etc. Requires the latest MVtools, Masktools, Avisynth, Autoadjust, fftw, FFT3DFilter, DCTFilter, RGTools, f3kdb. Lots I know... but was necessary and future additions to the script won't require additional plugins.

There are two key points about grain. Firstly, as far as the encoder and filters are concerned, it's detail. The tempoeral degrainers work by seeing the difference between frames and removing frequently changing detail (grain). Spatial degrainers only distinguish grain by the size, and not whether it changes. The end result is a softer picture. The issue with removing grain is that the grain covers the picture detail and 'construction', with something that is heavily grainy it's more about making it acceptably watchable and also be friendly compressibility wise. If you do manage to remove all the grain typically you would just end up with a flat picture and mushy detail. In other words, you probably won't get it perfect per se, in that you will still have grain, but you will hopefully get it to a point of being somewhat acceptable.

[StainlessS]

I have not looked at source clip, but used to like to use a Very Light touch of spatial denoise, followed by temporal denoise,
eg VagueDenoiser (still my fav spatial denoiser, and quite fast), and then maybe MCDegrain or some other.
I'm not a fan of grainy clips (it dont look like detail to me, just noise).

[lansing]

knlmeanscl should be your best choice in term of both speed and quality, if you have a modern graphic card. Well there was not much options to choose from anyway, temporaldragin, mctd and knlmeanscl, but the prior 2 are very slow.

knlmeanscl(d=1, h=10, a=3, s=4)

[Katie Boundary]

Spline36 is a sharpening filter. Never use sharpening filters for downscaling. Try bilinear maybe? It won't get you all the way there but it'll make things easier for your denoisers.

[StainlessS]

Spline36 is a sharpening filter.

I think that you will find that Spline36Resize is considered neutral, whereas Spline64Resize is considered sharpening.

[Katie Boundary]

Spline36Resize is considered neutral

Does it have negative lobes?

[wonkey_monkey]

Oh here we go...

[FranceBB]

Spline36 is a sharpening filter. Never use sharpening filters for downscaling. Try Bilinear


I think that you will find that Spline36Resize is considered neutral, whereas Spline64Resize is considered sharpening.


I generally use Lanczos to downscale and Spline to upscale only. Anyway, I think Spline36 could be used to downscale a footage, but not if there's text (credits, captions etc), 'cause last time I used it with a 1080p that had text to downscale it to 720p, it screwed up (ringing, mostly).

[Katie Boundary]

I generally use Lanczos to downscale

Not a good idea unless you're planning to re-upscale later.

[crystalfunky]

knlmeanscl should be your best choice in term of both speed and quality, if you have a modern graphic card. Well there was not much options to choose from anyway, temporaldragin, mctd and knlmeanscl, but the prior 2 are very slow.

knlmeanscl(d=1, h=10, a=3, s=4)

What do you think about FFT3d?
I tried your settings with knlmeans and the picture was very blurry without any details.

[shodan5000]

MCTemporalDenoise has worked the best for me. Excellent denoising while keeping detail. Very slow though. If MCTD could ever be made to make use of a GPU I'd be a happy camper.

[lansing]

What do you think about FFT3d?
I tried your settings with knlmeans and the picture was very blurry without any details.

That setting was for op's grainy source, you'll need to tweak it according to yours.

FFT3d is not better than knlmeans in terms of final output. Nowadays FFT3dfilter is used mainly for intermediate clip in a filter chain. For example, if a filter chain requires a clean, grainless clip for further processing, that's where fft3dfilter fits in, it's fast and get the job done without caring much of quality.

The only case where fft3dfilter can be use as a standalone denoiser is when you're looking to get rid of compression artifacts and mosquito noises.

[Forteen88]

Why do most people use Spline36Resize & Lanczos resizers when MadVR says that Jinc resizing is much better (MadVR says that Jinc-resizing got much less aliasing and ringing, and a little less sharping)?
Or is MadVR's developer wrong about this?

[StainlessS]

Why do most people use ...
Maybe because Spline36Resize is builtin.

[Forteen88]

Maybe because Spline36Resize is builtin.Oh, it seems to be also because "Downscaling isn't currently implemented.", on the Jinc Avisynth-plugin page here at Doom9.
But maybe this updated version do handle downscale,
http://avisynth.nl/index.php/JincResize

[gispos]

Thanks for the suggestions. TemporalDegrain does indeed work wonders, but yeah... wow, it's slow. Here's (https://www.dropbox.com/s/glsvi0wcuduqn4j/TemporalDegrain.mkv?dl=0) a clip after a straight

TemporalDegrain(degrain=3, ov=4, blksize=16)
Spline36Resize(1200,720)

For me a number better, less noise, more details and sharper

Spline36Resize (1280, 720)
MCTemporalDenoise (settings = "very high", sigma = 20, strength = 100, tovershoot = 1, GPU = true) # GPU = True if you card supports
sharpen (0.20) # or not

[crystalfunky]

So I'm trying to degrain 12 angry men as well, but without resizing.
I tried a lot of filters: MCTD, DFTest, FFT3d with different settings, HQDN3D, KNLMeansCL with different settings, TNLMeans, xNLMeans, and some more.
So far TemporalDegrain2 gave me the best quality. Now I just need to re-sharpen the image a bit. For that I'm using aWarpSharp2 with type=1 setting.
If I'm done and happy with the result, I will post some pictures.

Settings:
TemporalDegrain2(degrainTR=2, postFFT=3, postSigma=3)
aWarpSharp2(thresh=128, blur=2, type=1, depth=16, chroma=3)

Do I need to change anything in the filters for a black&white movie?

[crystalfunky]

I will switch to another sharpening filter. aWarpSharp2 will blur the picture a lot.

Regarding degraining: I noticed that I loose some of the HDR brightness in some scenes.
Can I avoid this with some settings somehow?

[tormento]

Do I need to change anything in the filters for a black&white movie?
SMDegrain (tr=9, thSAD=600, refinemotion=false, n16=true, mode=0, contrasharp=false, PreFilter=4, truemotion=true, plane=4, chroma=true)
You can increase thSAD up to 1000 and, to gain a few fps, set plane=0 and chroma=false but you will lose some compression.

If you see some smearing near moving light sources, you can use refinemotion=true.

On extreme (and it's not this case), you can add before SMDegrain
Spresso(10,30).STpresso(10,30,4,22,8,49,1,mc=true)
and lower tr=6 but, I repeat, this is not the case.

[DTL]

Why do most people use Spline36Resize & Lanczos resizers when MadVR says that Jinc resizing is much better (MadVR says that Jinc-resizing got much less aliasing and ringing, and a little less sharping)?
Or is MadVR's developer wrong about this?

At first madvr is mostly viewing software. So it mostly use resize-upscalers (restoring sample-coded content to as close to analog form as possible). For upscaling Jinc works about good. So author of madvr wrote about resizers for upsampling process.

But downscaling is content-production work and much more important process. Its settings and used resampler depends on source and on downsize ratio.

In good case the content-producer with downscaling must check its result with upscaling (some sort of) and in best case provide metadata of which upscaler was used for content testing.

Also current JincResize builds for Avisynth is very slow in compare with V+H resampler built-in in avisynth. It can be used for downsampling work in case user pay attention on spectrum preparation before using this downsampler. For spectrum preparation many low-pass configurable filters can be used - from GeneralConvolution with appropriate kernel and many FFT-based, etc. Though to create and fine-adjust kernel like 5x5 and larger for GeneralConvolution is hard enough work for many users.

The only good applicable production-downscaler in standard Avisynth without pre-processing with correct low-pass is GaussResize with low enough p-parameter about 15. It creates about good conditioned against ringing/alising content. Though it is soft enough. Also it creates non-peaked/haloing result. In 2020 there was made SinPowResize built in ResampleMT plugin pack used V+H fast processing and it have controlled peaking/haloing with keeping ringing low. Its recommended p-parameter value is about from 2.5 (more sharp and peaking is max) to 3.0..3.2 (less peaking and lower sharpness). With p> 3.2..3.4 it mostly close to simple Gauss.

Also currently is work in progress for true-2D (planar) resizer used same Gauss and form of SinPow-like kernels. May it will be option to JincResize because of using same resampler core or may be separated forks. It will make a bit better results in compare with V+H resamplers built-in in Avisynth. Though still significally slower due to number of operations for planar convolution.

In all cases of downsize work the author must check its result for ringing/alising with upsampled form at least 2x or larger with typical or 'reference' upsamplers. Typical is Lanczos with taps around 3 I think or may be even Bicubic and 'reference' for V+H processing is Sinc (it has computational errors if very few numbers of taps used - about less 50..70) or SincLin2 in ResampleMT plugins pack - it have workaround for small samples number and provides at least taps/2 full-strike Sinc lobes. So SincLin2 may be used with taps about 10..20 without that bugs exposed. For planar/2D upsamplers for content checking we currently have mostly JincResize only and it is close to Lanczos in V+H.

These all are 'linear' content-independent processing. For 'non-linear' resizers life is more complicated. So it is good if downsampled content author provides data on which target upsampler is recommended for this content viewing.