TemporalDegrainV2

[ErazorTT]

Hi all,

I want to share an updated version of the script TemporalDegrain.

The reason for the update was to combine TemporalDegrain with the relevant parts of QTGMC:
Although MDegrain of MVTools with Didees contra-sharpening already performes great, the additional stage of motion compensated fourier denoising can do wonders.
Together with this, the original motion estimation is also much improved using the exceptional approach of QTGMC instead of the very mushy intermediate stream which was losing some vectors in sceenes with low contrast and diverging movement.

With this optional fourier denoising setting the result can be up to completely grainless, much better compressable and together with the optional dithering it is guaranteed to produce no banding, yielding a very natural look in total.

Here a comparison of a scene of s2e1 of Breaking Bad, starting with the original [505KB for the JPEG still]:


v1.23: TemporalDegrain(degrain=2) [310KB for the JPEG still]


v2.0: TemporalDegrain2(degrainTR=2,postFFT=3,postSigma=3) [216KB for the JPEG still]

Compared to V1.23 the out-of-focus regions of V2.0 have almost no remaining grain anymore (can be even better seen on the moving picture) helping compressablity (see the JPEG file sizes as a proxy).
The in-focus regions however have still all details of the original frame.

And now the comparison of why I think it was time for an improved motion estimation over v1 of TemproralDegrain.
original from Breaking Bad s1e1, the camera moves following the car:


v1.23: TemporalDegrain(degrain=2)

The motion of the low contrast tarmac is not captured well and thus the details are being washed away.

v2.10: TemporalDegrain2(degrainTR=2)

More of the details of the tarmac are preserved.

In this thread further down are some movie samples: here
I have also made some performance measurements: here

Greets,
enjoy!

Changelog:

Dec 20, 2021: v2.3.4 (if you ever had a reason to use something else than postFFT 0, 1 or 2 try again now!)
- dithering is delayed to after contraSharpending which increases quality of dithering, decreasing banding and noise (postDither=2), old behaviour is regained with postDither=1
- postDither argument has been inverted! [new postDither = (-1) * old postDither] <--- if you ever explicitly set the postDither argument, read this line

Dec 21, 2021: v2.4.0 (This and later releases need AviSynth 3.7.1+)
- add BM3D_CPU & BM3D_CUDA (based on BM3D prefilter of Dogway's SMDegrain) and mods by kedautinh12 (Aug 12 - Dec 19)

Dec 21, 2021: v2.4.1
- small speed increase for BM3D (thanks to kedautinh12)

Jan 05, 2022: v2.4.2
- allow enabling or disabling of chroma and luma filtering for BM3D (thanks to kedautinh12)
- allow denoising on different planes at the two stages (degrainPlane vs postPlane)

Feb 15, 2022: v2.4.3
- add limit r to 3 when CUDA=true for BM3D (based on BM3D prefilter of Dogway's SMDegrain) and mods by kedautinh12
- add preset for BM3D (based on BM3D prefilter of Dogway's SMDegrain) and mods by kedautinh12

Apr 05, 2022: v2.5.0
- allow distinct postFFT and postSigma for luma and chroma

May 01, 2022: v2.6.0
- clean chroma more by also using the first degrain step
- tweak scene change detection factors

May 02, 2022: v2.6.1
- fix grainLevelSetup which was broken in previous release

May 10, 2022: v2.6.2
- some fixes and cleanup

Jun 06, 2022: v2.6.3
- increase possible range of grainLevel
- allow for multiple usages of this script

Feb 11, 2023: v2.6.4
- always undot before postFFT stage, increasing FFT efficiency by quite a bit when degrain was disabled
- mention suggested settings inside the script file

Feb 28, 2023: v2.6.6
- cleaning chroma only in one step, to ameliorate the chroma shift bug in mvtools MDegrain

Jan 14, 2024: v2.6.7
- forward device_type and device_id to KNLMeansCL

Link to newest version: >here<
All current and old versions can be downloaded: >here<

[tormento]

Originally Posted by ErazorTT

I want to share an updated version of the script TemporalDegrain.

I will try ASAP and tell you results.

You did a little mistype in the comments:

# KNLMeansCLL: https://github.com/Khanattila/KNLMeansCL/releases            #

I am a bit confused too about the role of KNLMeansCL in the script. You state as optional but I can't see any knob or boolean to use it, even if I see trace of it in the code.

I am a bit puzzled why you use FFT3DFilter and FFT3DGPU and not only one of them.

I'd like also to know if the script is high bitdepth aware or internally working in high bit precision or 8 bit only.

I humbly suggest to use thumbnails for the images, not to send to hell thread formatting

[johnmeyer]

I've tried a lot of noise reduction and degraining plugins and scripts. You really cannot tell much from static shots because many of these approaches can cause really weird artifacts that can only be seen when you watch the resulting video. This is because you sometimes get correlation between grain in adjacent frames which results in really weird alignment of some of the residual grains, as well as other artifacts, none of which are apparent in a single frame.

So, it would be useful to have a side-by-side before/after video.

[ErazorTT]

Thanks a lot for the quick feedback!!

Originally Posted by tormento

You did a little mistype in the comments:

Oh I will correct that.

Originally Posted by tormento

I am a bit confused too about the role of KNLMeansCL in the script. You state as optional but I can't see any knob or boolean to use it, even if I see trace of it in the code.

Yeah, right I forgot to mention that in the file. It is being set by postFTT=4.

Originally Posted by tormento

I am a bit puzzled why you use FFT3DFilter and FFT3DGPU and not only one of them.

I was asuming FFT3DGPU would be a faster version of FFT3DFilter but if I am not mistaken currently only FFT3DFilter is still maintained, thus I though to leave the decission to the user.

Originally Posted by tormento

I'd like also to know if the script is high bitdepth aware or internally working in high bit precision or 8 bit only.

I have made some of the necessary changes to the inner working as a first step to supporting high bitdepth but I am not there yet. Thats what I plan to do as a next step.

Originally Posted by tormento

I humbly suggest to use thumbnails for the images, not to send to hell thread formatting

Let me see if I can figure out how to do that...

Originally Posted by johnmeyer

...it would be useful to have a side-by-side before/after video.

Yes very true, I will prepare some samples

[StainlessS]

Let me see if I can figure out how to do that...

PostImage.org allows anonymous upload, but also easy to make account.
Below, HotLink for forum, and then Thumbnail for forum below that.

[ErazorTT]

Originally Posted by StainlessS

PostImage.org allows anonymous upload, but also easy to make account.

Awesome

[tormento]

Originally Posted by ErazorTT

Awesome

I use lensdump too.

[Forteen88]

Thanks! Is this script of yours also great against "dancing grain"?
I've noticed that I can barely stand "dancing grain" in videos anymore :P
Oh, I remember that the movie "The Good, the Bad and the Ugly" Blu-ray had really much annoying grain, I could barely concentrate on the movie.
EDIT: But this does not mean that I hate all grain. I can accept a bit of grain if it's not too much!

[hello_hello]

Interesting.....

For a long time QTGMC has been my "go to" noise filter. Not that it's perfect, as being a de-interlacer it can sometimes create unwanted side effects "normal" denoising scripts don't, but where there's movement it's blurring to noise removal ratio tends to be pretty good. Only the other day I found myself wondering how hard it'd be to modify QTGMC to remove the de-interlacing completely for progressive video, and if it was possible, would it adversely effect it's ability to denoise.

Anyhow..... for a quick and very unscientific comparison. The "original video" is the script output with only resizing applied. The filtered screenshots were taken after encoding with x264, CRF18, Preset Slower(ish). I'm not sure why I did it that way, but I guess ultimately it's all about how the encoded version looks.

Thumbnails so as not to upset the scrolling police (and because it is annoying). It looks like you need to click on a thumbnail to open an image on the PostImage site, then right click and select "view image" for the full size version.

Screenshot 1. The camera was panning down a little. The playing card top/left shows the most detail difference.
Screenshot 2 was static.
Screenshot 3 had lots of movement. The stripes on the guy's shirt would be typical noise removal victims.

Edit: Whoops. The first three screenshots were off by a frame relative to the others. They're fixed.


Original 1080p video resized to 720p.
Spline36Resize(1280,536)








Spline36Resize(1280,536)
QTGMC(InputType=1, EzDenoise=1.5)
GradFun3()








Spline36Resize(1280,536)
TemporalDegrainV2()
GradFun3()








Spline36Resize(1280,536)
TemporalDegrain()
GradFun3()

[ErazorTT]

I updated the first post to include a comparison concerning the motion estimation! Since this was one of the main driving points I figured it would be good to show.

Originally Posted by hello_hello

Screenshot 1. The camera was panning down a little. The playing card top/left shows the most detail difference.
Screenshot 2 was static.
Screenshot 3 had lots of movement. The stripes on the guy's shirt would be typical noise removal victims.

Hm QTGMC yeilds better results for 1 and 3. I wonder why... What version of QTGMC were you using?
Could you please try to run again with version 2.0.1 and play with options meBlksz=8/16 and/or meTM=true/false for TemporalDegrain2?
(Use the new updated version 2.0.1 from the first post. With the function renamed to TemporalDegrain2 it should also be easier to compare to v1)

[ErazorTT]

Originally Posted by johnmeyer

So, it would be useful to have a side-by-side before/after video.

Here you have the motion pricture you requested. I used x264 with crf 16 at "Very Slow" for all three:
Original
TD1.23: TemporalDegrain(degrain=2)
TD2.10: TemporalDegrain2(degrainTR=2,postFFT=3,postSigma=3,postDither=-1)

Be sure to actually download the files and do not just watch the preview in the browser! The quality of the browser preview is miserable..

Originally Posted by Forteen88

... Is this script of yours also great against "dancing grain"?

Yes you should be able to go from "mostly gone" to "OMG WFT, how is that even possible that a movie looks that clean"
Look at the files in this post, the grain is gone.

[Taurus]

@ErazorTT:
Please upload your Temporaldegrain2.zip to a filehoster.
It can take days, weeks or months for "Attachments Pending Approval"
Or just show the avsi in quotes here in this thread.
Thank you.

[ErazorTT]

@Taurus
Great you made me aware of that! I just updated the first post accordingly. I wonder however why the attachment could be downloaded right after I opened the thread...

[tebasuna51]

If you modify the first post I can't know when I need approve the attachment, sorry.
Approved now.

I only read new post.

[ErazorTT]

@tebasuna51: I have seen in other threads that this is the way people seem to operate, namely having the first post up to date. What would then be the best way to update the attachments? Do other let you know that they want something approved?

[tebasuna51]

When avsi text is bigger than 16000 chars (like TemporalDegrain2.avsi is) can't be included like text in the post, then you can use:

1) Pastebin (the user can read the source before download it)

2) Other upload free sites: DriveGoogle, Mediafire, Sendspace,... (like you make here)

3) If attach please make a new post with something like:

"New version X.Y attached in first post, please approve it"

[hello_hello]

Originally Posted by ErazorTT

What version of QTGMC were you using?

QTGMC 3.357s.

Originally Posted by ErazorTT

Could you please try to run again with version 2.0.1 and play with options meBlksz=8/16 and/or meTM=true/false for TemporalDegrain2?
(Use the new updated version 2.0.1 from the first post. With the function renamed to TemporalDegrain2 it should also be easier to compare to v1)

I didn't bother uploading screenshots with TemporalDegrain2(meTM=true) as the blurring always increased. It appears to have been true for the last version of the script. Were there any other changes aside from making meTM=false the default? It seems to be better now.

Two of the screenshots are the same frames as before, although this time the screenshots are taken directly from the script output and I increased QTGMC's noise removal a bit as I think it had an advantage there.

The new screenshot is of Val Kilmer's face as he was moving a little. Often the blurring of detail on someone's skin increases dramatically when they move, but it seems to be on a par with QTGMC now. There's a halo above the brim of his hat on the left side. It's in the source but largely hidden by noise. If there's an existing halo, QTGMC sometimes enhances it more than I'd like. TemporalDegrain2 enhanced it too, but so far, not quite as much.

QTGMC 3.357s takes functionality from SMDegrain to expand the levels to full range. I think it's only used for the motion search. The idea is to make it more accurate, but I don't know how much difference it makes. Any thoughts there?

Rather than upload 15 screenshots and post thumbnail links, here's a zip file.

Screenshots.zip (13.1MB)

[StainlessS]

Another moderator suggested using "Report Post", to report where new attachment requires approval,
Report Post is found below avatar of the offending post as Red triangle with ! symbol inside.

[ErazorTT]

Originally Posted by hello_hello

Were there any other changes aside from making meTM=false the default?

Setting truemotion to false was the only functional change.

Originally Posted by hello_hello

It seems to be better now.

Thats really awesome! I pretty much forgot that truemotion switch in MVTools. So, huge thanks for you screenshots. Looking at those and comparing back and forth with the ones from QTGMC somehow reminded me of that again.

Originally Posted by hello_hello

QTGMC 3.357s takes functionality from SMDegrain to expand the levels to full range. I think it's only used for the motion search. The idea is to make it more accurate, but I don't know how much difference it makes. Any thoughts there?

Hm I will have to analyze that. Right off the top of my head I do not know what exactly you mean. I will look into the code and come back if I see something.

[hello_hello]

Originally Posted by ErazorTT

Hm I will have to analyze that. Right off the top of my head I do not know what exactly you mean. I will look into the code and come back if I see something.

It expands levels from TV range to PC range for the MSuper clip. I don't know how much difference it makes. Have a look under line #187 here.

https://pastebin.com/u1xsPLwK
# Default Auto-Prefilter - Luma expansion TV->PC (up to 16% more values for motion estimation)

Maybe now I understand why I've always thought SMDegrain and other mvtools2 based denoising scripts blur too much when there's motion. A quick look at the SMDegrain script indicates TrueMotion is disabled by default for resolutions above standard definition (although it still blurs more than QTGMC), and it's enabled by default for mvtools2 itself.
It's disabled by default for MCTemporalDenoise and MCTemporalDenoiseMod (MCTD), so maybe that's why they tend to blur less (from memory, I haven't used them for a while).
I hadn't played with the TrueMotion option myself until now.

Cheers.