About the Script MDegrain2i2 [Archive]

Genji

4th March 2024, 05:17

I was searching about noise reduction and found this page.
https://www.vegascreativesoftware.info/us/forum/another-better-avisynth-noise-reduction-script--73868/

I was intrigued by the content, so I looked up "MDegrain2i2" in doom9, and the script in this post seems to be the most recent one.
https://forum.doom9.org/showthread.php?p=1990681#post1990681

I have two questions about this script.

1. about the speed of operation
I usually use Amatsukaze.
https://github.com/nekopanda/Amatsukaze
https://github.com/rigaya/Amatsukaze

Amatsukaze was created by nekopanda, but he is no longer active, so rigaya is currently updating it.
rigaya has tried twice now to get it to work with the latest Avisynth Plus, but so far has not been successful.
https://rigaya34589.blog.fc2.com/blog-entry-1791.html

Therefore, it is assumed to work with Avisynth Neo.
https://github.com/nekopanda/AviSynthPlus

The Vegas Community post I mentioned above is from 2009, and at that time it says I am getting over 90fps.
However, in my environment, I only get around 10fps.
Is there a way to improve this, or is it just not possible with Avisynth Neo?
The script is below.

SetDeviceOpt(DEV_CUDA_PINNED_HOST)
dsrc = AMT_SOURCE.OnCPU(2)
pass = Select(AMT_PASS, 1, 3)
AMT_PRE_PROC = (AMT_PASS < 1)
dsrc.KFMDeint(mode=2, pass=pass, ucf=true, nr=true, svp=false, thswitch=3, cuda=true, is120=False, dev=AMT_DEV, filepath=AMT_TMP)
AssumeBFF()
if(AMT_PASS != 1) { return last }
KDeblock(qpclip=dsrc.QPClip().OnCPU(2),quality=3,str=-1.0,bratio=0,thr=38,sharp=True)

MDegrain2i2(8,0,400,0).OnCPU(2)

ConvertBits(14)
KTemporalNR(3, 1)
KDeband(25, 1, 2, true)
ConvertBits(10, dither=0)
if(IsProcess("AvsPmod.exe")) { ConvertBits(8, dither=0) }
OnCUDA(2, AMT_DEV)

I just added the MDegrain2i2 line to the settings output by Amatsukaze.
I also tried YadifMod2 without KFMDeint, but it was even worse, about 8 fps.
MDegrain2i2(8,0,400,0).Prefetch(6).OnCPU(2)
Thus, adding Prefetch improved the performance by 1-2 fps, but it stopped deadlocked in the middle of the process.

I'm very happy with the output on AvsPmod and would like to use it regularly, so if there is a way to speed up the process, please let me know.

2. about updating scripts
When I search for "MDegrain2i2", there are so many posts scattered around that it is hard to tell which one is the latest.
Is this script available on blogs, Github, etc.?

johnmeyer

4th March 2024, 06:50

Gosh, that takes me back to the days when I posted in the Vegas forum almost every day. I think I had over 6,000 posts by the time I quit.

As for your question on speed, pay close attention to my post: that speed I reported was on SD video, not HD. The video I was working on was only 720x480 and was 30 fps interlaced.

1920x1080 60 fps progressive is going to be a LOT slower, and 4K video will be slower than that.

The original MDegrain2i2 script is found in the example given on the MVTools2 Wiki:

http://www.avisynth.nl/users/fizick/mvtools/mvtools2.html

I tweaked it, but most certainly did not invent it.

Finally, if your goal is to get the best noise reduction while creating the fewest artifacts (something I talked about at length in my first post in that thread) you might want to consider other noise reducers. I expect that other people will help out by suggesting some alternatives. There has been a lot of progress in AVISynth noise reduction since my 2019 post in the Vegas forum.

Mounir

4th March 2024, 08:32

Yep there has been a lot since 2k9 like KNLMEANS, NEATVIDEO V5 (GPU ready), SPOTLESS and more

lollo2

4th March 2024, 14:18

The whole concept of 2i2, i.e. separate fields and apply filtering to even fields and odd fiels is wrong in principle:

https://forum.doom9.org/showthread.php?p=1691595#post1691595

https://forum.doom9.org/showthread.php?p=1618309#post1618309

http://forum.doom9.net/showthread.php?p=1921993#post1921993

Best way is to deinterlace the video, apply the temporal or spatial/temposal denoiser, and eventually interlace back if needed

Genji

5th March 2024, 07:47

Simply because it's HD, it's only 10 fps.
It's been 15 years since 2009 and HD is still a heavy processing resolution.
I like the processing result with clear edges, but 10fps is not enough for daily use.
I have been pointed out that this method is wrong, so I will give up on MDegrain2i2.

Neat Video is too smooth, so I was wondering if there is an h.264 version of grav1synth that would be good, or some other script that would reproduce the grain of the original source in the same way.
I thought it would be a good way to do it, since the file size would shrink and the details would remain.

DTL

6th March 2024, 11:09

2i2 is the very simple example function for for interlaced sources processing without deinterlacing.

"The whole concept of 2i2, i.e. separate fields and apply filtering to even fields and odd fiels is wrong in principle:"

It makes acceptable quality for some use cases. Also about 2 times faster because processing fields 1/2 of height. Also to (partially) fix half field MVs jitter mvtools have internal flags for field-based sources.

Also the most sad thing about deinterlace first - it also requires best possible motion search for motion compensation. And attempt to motion search with original input noised content also provides search errors. So possible better approach is (much slower) multi-generation processing with pre-filter stage of deinterlace+denoise or denoise only and use this clip as possibly better reference while doing next generation motion search for second stage of processing (again deinterlace and denoise).

Dual-input MAnalyse for multi-generation motion search with different src and ref clips already available - https://forum.doom9.org/showthread.php?p=1992298#post1992298

So it is now possible to create 'super-slow' new modes for still widely used QTGMC deinterlacing script with at least 2 generations of MVs search.

Genji

6th March 2024, 12:31

After giving up on MDegrain2i2, I was searching further on the web for noise reduction that can be used regularly and found this information.
https://www.dpreview.com/forums/thread/4461481

It is packaged for VirtualDub2, but I extracted the necessary files and ran it through AvsPmod and got very good output results.
It is a combination of existing plugins and scripts, but very well tuned.
The encoding speed using Amatsukaze is also very good.
Files that were 10 fps with MDegrain2i2 are now 30 fps with Denoiser_Medium.
Denoiser_Anime seems to be a heavy processor and doesn't speed up the process, but it is acceptable.
I like it very much and will give it a trial run for regular use.

lollo2

6th March 2024, 20:21

2i2 is the very simple example function for for interlaced sources processing without deinterlacing.

"The whole concept of 2i2, i.e. separate fields and apply filtering to even fields and odd fiels is wrong in principle:"

It makes acceptable quality for some use cases. Also about 2 times faster because processing fields 1/2 of height. Also to (partially) fix half field MVs jitter mvtools have internal flags for field-based sources.

No acceptable quality for me.

Also the most sad thing about deinterlace first - it also requires best possible motion search for motion compensation. And attempt to motion search with original input noised content also provides search errors. So possible better approach is (much slower) multi-generation processing with pre-filter stage of deinterlace+denoise or denoise only and use this clip as possibly better reference while doing next generation motion search for second stage of processing (again deinterlace and denoise).

Dual-input MAnalyse for multi-generation motion search with different src and ref clips already available - https://forum.doom9.org/showthread.php?p=1992298#post1992298

So it is now possible to create 'super-slow' new modes for still widely used QTGMC deinterlacing script with at least 2 generations of MVs search.

My approach is different: I (temporaly) deinterlace with QTGMC using lossless option, so the original fields (and then the frame) are preserved. I then apply the denoiser. I finally interlace back discharging the additional frames that have been created by QTGMC and its motion vectors.
In other words, I denoise the "original" full frame using both fields ;)

Sharc

6th March 2024, 20:32

No acceptable quality for me.

My approach is different: I (temporaly) deinterlace with QTGMC using lossless option, so the original fields (and then the frame) are preserved. I then apply the denoiser. I finally interlace back discharging the additional frames that have been created by QTGMC and its motion vectors.
In other words, I denoise the "original" full frame using both fields ;)
Which method is 'better' depends very much on the source (and on the denoiser). Try both, then decide.

lollo2

6th March 2024, 21:16

I always experiment possible options with my sources, not being a fan of any ;)

johnmeyer

7th March 2024, 02:26

My approach is different: I (temporaly) deinterlace with QTGMC using lossless option, so the original fields (and then the frame) are preserved. I then apply the denoiser. I finally interlace back discharging the additional frames that have been created by QTGMC and its motion vectors.
In other words, I denoise the "original" full frame using both fields ;)This sounds very intriguing, but I don't understand how you can denoise the original interlaced and separated fields using results from QTGMC. If you can really do that, you would get the best of both worlds: the improved accuracy of denoising fields that are from the same moment in time (thanks to QTGMC), but avoid the inevitable visual degradation which QTGMC causes because of its motion estimation.

I have always avoided QTGMC because I don't like what motion estimation does, and prefer the artifacts from denoising odd and even fields separately and then weaving the result back together (as is done in the 2i2 script).

Do you have a few lines of sample code you could post?

lollo2

7th March 2024, 12:17

Do you have a few lines of sample code you could post?

Very basic processing

video_org=Avisource(<filename>.avi)

### de-interlacing
deinterlaced=video_org.AssumeTFF().QTGMC(lossless=1)

### denoising
denoised=deinterlaced.TemporalDegrain2(degrainTR=3)

### interlacing
interlaced=denoised.AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

return(interlaced)

You can check that QTGMC(lossless=1) is lossless on your source with a simple comparison:

video_org=Avisource(<filename>.avi)

deinterlaced=video_org.AssumeTFF().QTGMC(lossless=1)
interlaced=deinterlaced.AssumeTFF().SeparateFields().SelectEvery(4,0,3).Weave()

### check difference between original video and original video after QTGMC_lossless and re-interlacing ###
difference_video_org_interlaced=Subtract(video_org, interlaced).Levels(65, 1, 255-64, 0, 255, coring=false)

stackhorizontal(\
subtitle(video_org,"video_org",size=20,align=2),\
subtitle(interlaced,"interlaced",size=20,align=2),\
subtitle(difference_video_org_interlaced,"difference",size=20,align=2)\
)

an example: https://ibb.co/FgMJ7T2

johnmeyer

7th March 2024, 19:37

Thanks for that.

I had not paid much attention to the Lossless=1 QTGMC settings. I just looked at the QTGMC code, and the QTGMC_MakeLossless function is where "lossless" happens. It looks like the following is the key line of code:processed = Interleave( srcFields, newFields ).SelectEvery(4, 0,1,3,2 ).Weave()If I am reading this correctly, "lossless" refers to one of the two interleaved pairs (upper or lower, depending on settings) which is passed through to the output, untouched by either motion estimation or noise reduction. However, the other field IS motion estimated, but not denoised. You are then free to apply the denoiser of your choice to this progressive result without having denoising on top of QTGMC denoising.

Then, if you wish, you can take it back to interlaced by decimating via SelectEvery(), as you do in your sample script, and then reweaving back to interlaced.

So, if I am understanding, you use QTGMC in the most conservative setting possible, and half the result of its deinterlacing is 100% the same as the original.

I assume that you could try to get an even better result, at the cost of speed, by using one of the super-high quality settings for the motion estimation so that the field which is NOT the same as the original has as few artifacts as possible.

lollo2

7th March 2024, 22:52

Not sure what you mean. What you leave untouched is the whole frame, not only the fields. Or if you prefere, the original fields.
QTGMC is clear about the processing:
"Lossless mode 1 restores the *exact* pixels of the source into the output (provided NoiseRestore=0). This recovers a liitle more source detail but can introduce shimmering, minor combing, noise etc." [from http://avisynth.nl/index.php/QTGMC#Source_Match_.2F_Lossless]
The additional frame introduced (because double frame rate) is somehow "new" (but based on the original fields and the motion vectors).

In practice, if I want to deinterlace and restore video I use QTGMC at its full potential (no lossless, but with its denoise filtering reduced to the minimum possible) and then apply a denoiser. This guarantee the best final results (if not over-processed).

If I want to keep the video interlaced for whatever reason (but restored) I apply a temporary lossless deinterlacing (QTGMC lossless) to feed the denoiser with the proper temporal sequence, and then discharge the "new" (denoised) frame, and interlace back after. If I did not use QTGMC to retain the "original pixels" this whole approach had little sense and would be much better to deinterlace, denoise and keep the deinterlaced video, as in the deinterlace and denoise processing above.

johnmeyer

8th March 2024, 00:50

DTL

8th March 2024, 07:51

"QTGMC has done absolutely nothing?"

In doube-framerate mode it mean QTGMC only produces new lines for full frame and keeps original field lines ? But if user not uses new lines at the output maybe it is more simple (and faster) not use QTGMC at all and simply insert black or grey 50% lines as second field for double framerate and feed to temporal denoiser ? It will also doubles job of denoiser. Maybe it simply attempt to fix possible vertical jitter of MVs in simply separated fields sequence (though it also somehow fixed inside mvtools by adjusting MVs with fields-based source).

I still not sure if new generated by QTGMC lines (based on not perfect motion estimation on noised input source) can help the next stage motion estimation engine in temporal denoiser to do its job better. The temporal denoiser can also work on the frames with black or 50% grey lines inserted at the place of skipped field in double framerate frame-based clip.

lollo2

8th March 2024, 12:11

QTGMC has done absolutely nothing?

Just run the comparison script and you will understand immediately what QTGMC is doing in lossless mode, better than words.

The information in the original fields are preserved, and new fields are built to build the double frame rate.
When you SeparateFields().SelectEvery(4,0,3).Weave() you discharge what QTGMC created and return back to original source.
The difference is 0.

Now run the same with QTGMC() instead of QTGMC(lossless=1) and there are differences!

The denoiser is applied while the (lossless) deinterlacing is still there, then is uses the full temporal data in the right sequence.

In doube-framerate mode it mean QTGMC only produces new lines for full frame and keeps original field lines ? But if user not uses new lines at the output maybe it is more simple (and faster) not use QTGMC at all and simply insert black or grey 50% lines as second field for double framerate and feed to temporal denoiser?

The denoiser uses the full frame, built across the data present in the fields, rather then half of it when you use instead "separateFields()" or "separateFields.selectOdd() / separateFields.selectEven()". A black second field would not then been appropriate.

Sharc

8th March 2024, 12:45

I still not sure if new generated by QTGMC lines (based on not perfect motion estimation on noised input source) can help the next stage motion estimation engine in temporal denoiser to do its job better. The temporal denoiser can also work on the frames with black or 50% grey lines inserted at the place of skipped field in double framerate frame-based clip.

One may want to use this script to compare the methods, then take the best and leave the rest :)

AVISource("blabla") #your source filter
converttoYV16(interlaced=true)
source=last

a=source.QTGMC(lossless=1)
a=a.<your denoiser()>
a=a.separatefields().selectevery(4,0,3).weave().subtitle("QTGMC+filtering",align=2) #re-interlace

b=source.separatefields()
e=b.selecteven().<your denoiser>
o=b.selectodd().<your denoiser>
b=interleave(e,o).weave().subtitle("even/odd+filtering",align=2)

view1=stackhorizontal(source.subtitle("source",align=2),a,b)
view2=interleave(source.subtitle("source",align=2),a,b)

return view1 #view1 or view2

lollo2

8th March 2024, 14:44

Yes Sharc, always experiment with your own material rather then following generic guidelines

DTL

9th March 2024, 11:08

lollo2

9th March 2024, 12:59

"The denoiser uses the full frame, built across the data present in the fields, rather then half"

I agree it may produce some better quality at some use cases (depending on source and quality of motion estimation and compensation and it also depends on source sharpness and noise levels).

It is the opposite. Most of the source benefit from the larger amount of data provided to the denoiser. Maybe for few cases where there is no motion across the fields, the field-approach denoise can match the previous.

In my experience sharpness does not really play a role here, and in any case, after a denoise a light sharpening is (almost) always needed.

Method 'b' of

b=source.separatefields()
e=b.selecteven().<your denoiser>
o=b.selectodd().<your denoiser>
b=interleave(e,o).weave()

possibly will save most of sharpness/details on static areas (also may depends on denoiser used and settings).

I do not see how working with half data can save more details, everything else being equal.

Also the typical MDegrain2i2 uses method 'c' of

c=source.separatefields()
c.<your denoiser>
c.weave()

and it also may be compared with a and b for quality/performance. It expected to have max performance with single denoise engine and half V-size and no deinterlacer used.

SeparateFields().<denoise>.Weave() is not even good for temporal filtering, not only for spatial or spatial/temporal denoising.
https://forum.doom9.org/showthread.php?p=582059#post582059 The whole thread is interesting.

Just to summarize my view about the deinterlacing of interlaced video with spatial ot spatial/temporal denoiser:

- Spatial filter:
SeparateFields()
<filtering>
Weave()

- Temporal filter:
<filtering>

or

<Bobbing deinterlacer>
<filtering>
SeparateFields()
SelectEvery(4,0,3)
Weave()

second is better

- Spatial-Temporal filter:
separateFields()
SelectEven().<filtering>
SelectOdd().<filtering>
Interleave()
Weave()

or

<Bobbing deinterlacer>
<filtering>
SeparateFields()
SelectEvery(4,0,3)
Weave()

second is better

Edit: I see you have edited your post and added a method using the deinterlaced clip as reference for MVtool. I will experiment that!

Sharc

9th March 2024, 13:37

For the some better quality of processing noised sources the other possible method is about:

gen1_progressive=<deinterlace+denoise(src))>
gen1_interlaced=Re_interlace(gen1_progressive)

gen2_progressive=QTGMC(src, reference=gen1_interlaced)
gen2_denoised=mvtools_denoise(src=gen2_progressive, reference=gen1_progressive)

Re_interlace(gen2_denoised)

So the motion estimation engine at gen2 will work with original source unchanged and reference clip partially denoised but also partially damaged by distortions/errors at gen1 processing.
Interesting proposal. I guess I have to try it .....

DTL

9th March 2024, 16:15

"Most of the source benefit from the larger amount of data provided to the denoiser."

Temporal denoise is simply extending data accumulation time over the current frame exposure time interval. With a true frame-based clip any line of the previous and next frame contains real valuable data about the object. And any sample can be fully useful if motion compensation is perfect.

With interlaced scan the previous and next line of upsampled to full frame size clip do not have real camera data and only filled with fake data interpolated by QTGMC from some area (nearby lines) or other frames or even simply neural-network generated by NNEDI resize. So applying temporal denoise over double-rate deinterlaced clip may or may not give expected quality. Because denoiser will operate with half of somehow interpolated data - not real captured by camera.

"I do not see how working with half data can save more details, everything else being equal."

If camera and objects are static and motion estimation correctly creates zero MVs - blending of the same lines from different frames will not degrade sharpness. Because typical source of sharpness degradation at temporal denoising is errors in motion compensation. Also denoiser will operate with real camera captured data for best performance for given tr-value.

"SeparateFields().<denoise>.Weave() is not even good for temporal filtering"

It is a frequently used solution because of good performance and not very bad quality. It can degrade V-sharpness somehow. But cameras with interlaced scan typically also have degraded V-sharpness to decrease V-aliasing. So total degradation of V-sharpness is typically not very visible.

"I will experiment that!"

It is still pseudo-script because it is required to update QTGMC script to use MAnalyse with SuperCurrent input as second frame source for MVs search. Also denoise script used too. And required to use at least -eXX builds of mvtools2 with modified MAnalyse for dual-input. https://forum.doom9.org/showthread.php?p=1992298#post1992298

Sharc

9th March 2024, 16:38

With interlaced scan the previous and next line of upsampled to full frame size clip do not have real camera data and only filled with fake data interpolated by QTGMC from some area (nearby lines) or other frames or even simply neural-network generated by NNEDI resize. So applying temporal denoise over double-rate deinterlaced clip may or may not give expected quality. Because denoiser will operate with half of somehow interpolated data - not real captured by camera.
These are exactly my thoughts (filtering based on real camera data only) and rationale why making tests with the actual video and filters (denoiser, sharpener ....) etc. for 'important' videos.
One will always spot differences between the various methods. Sometimes it is difficult to decide which method is (visually) 'better' (residual noise, artifacts ...). In this case I usually 'default' to the fastest method (no deinterlacing), unless I have a good reason to deinterlace anyway.

Sharc

9th March 2024, 16:57

It is still pseudo-script because it is required to update QTGMC script to use MAnalyse with SuperCurrent input as second frame source for MVs search. Also denoise script used too. And required to use at least -eXX builds of mvtools2 with modified MAnalyse for dual-input. https://forum.doom9.org/showthread.php?p=1992298#post1992298
Uggh yes. Beyond my skills. Looking forward to its development :)

lollo2

9th March 2024, 19:44

Because denoiser will operate with half of somehow interpolated data - not real captured by camera.

Which is good enough to make the denoising more effective.

If camera and objects are static and motion estimation correctly creates zero MVs - blending of the same lines from different frames will not degrade sharpness.

I agree on this, but except having a shot of a stone in a valley or an interview with static actors, that's more the exception then a rule.

Because typical source of sharpness degradation at temporal denoising is errors in motion compensation.

Not only. The temporal denoising erases object that are not present across the temporal radius, and sometimes it is not only the noise behaving like that but also small details. Out of contest anyhow.

Also denoiser will operate with real camera captured data for best performance for given tr-value.

Which really gives no advantage.

It is a frequently used solution because of good performance and not very bad quality. It can degrade V-sharpness somehow. But cameras with interlaced scan typically also have degraded V-sharpness to decrease V-aliasing. So total degradation of V-sharpness is typically not very visible.

In my experiment and with my sources the degradation is there, so it is not an option for me.

DTL

9th March 2024, 20:51

"https://forum.doom9.org/showthread.p...059#post582059"

It was post of 2004 year - the beginning of mvtools. As I see currently after 20 years mvtools have some internal workaround for processing sequence of fields after SeparateFields to save from this vertical jitter. But the pel param must be 2 or 4.

fieldShift value computed based of clip property IsFieldBased and pel value.
https://github.com/pinterf/mvtools/blob/d8bdff7e02c15a28dcc6e9ef2ebeaa9d16cc1f56/Sources/MVAnalyse.cpp#L615

Where compute_fieldshift is https://github.com/pinterf/mvtools/blob/d8bdff7e02c15a28dcc6e9ef2ebeaa9d16cc1f56/Sources/ClipFnc.cpp#L43 - it only works if pel > 1 .

So input clip must have property FieldBased and pel must be > 1.

Though not all temporal filters may support such tweaks - so it applicable to mvtools and may be not valid for other temporal denoisers.