How would I remove grains from a video ?

[Batman007]

Hi guys

I got a movie of 1987 having grains in it .... How do I remove those grains ?
Please give me plugins and filters for DeGraining ....


Thank you .......

[StainlessS]

http://avisynth.nl/index.php/External_filters#Denoisers

I like (others will differ)

Spatial: VagueDenoise().

SpatioTemporal: FFT3DFilter(), HqDN3D() [fast], MCDegrain script.

MCDegrain by Didee (requires MVTools v2.0, heavy duty grain removal)

Function MCDegrain(clip c, int "frames")
{ # By Didee, http://forum.doom9.org/showthread.php?p=1508289#post1508289

    frames = default(frames, 2)
    bs = (c.width>960) ? 16 : 8
    super = c.MSuper(pel=2, sharp=1)
    backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=bs, overlap=bs/2)
    backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=bs, overlap=bs/2)
    backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=bs, overlap=bs/2)
    forward_vec3 = MAnalyse(super, isb = false, delta = 3, blksize=bs, overlap=bs/2)    
    (frames<=0) ? c :\
    (frames==1) ? c.MDegrain1(super, backward_vec1,forward_vec1,thSAD=400) :\
    (frames==2) ? c.MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) :\
              c.MDegrain3(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400)
    return(last)
}

Function MCDegrainSharp(clip c, int "frames", float "bblur", float "csharp", bool "bsrch")
{ # Based on MCDegrain By Didee, http://forum.doom9.org/showthread.php?t=161594
  # Also based on DiDee observations in this thread: http://forum.doom9.org/showthread.php?t=161580 
  # "Denoise with MDegrainX, do slight sharpening where motionmatch is good, do slight blurring where motionmatch is bad"
  # In areas where MAnalyse cannot find good matches, the blur() will be dominant.
  # In areas where good matches are found, the sharpen()'ed pixels will overweight the blur()'ed pixels
  # when the pixel averaging is performed. 
    frames = default(frames, 2)
    bblur  = default(bblur, 0.6)
    csharp = default(csharp, 0.6)
    bsrch  = default(bsrch, true)
    bs = (c.width>960) ? 16 : 8
    
    c2 = c.blur(bblur)
    super = bsrch ? c2.MSuper(pel=2, sharp=1) : c.MSuper(pel=2, sharp=1)
    super_rend  = c.sharpen(csharp).MSuper(pel=2, sharp=1,levels=1)
    backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=bs, overlap=bs/2)
    backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=bs, overlap=bs/2)
    backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=bs, overlap=bs/2)
    forward_vec3 = MAnalyse(super, isb = false, delta = 3, blksize=bs, overlap=bs/2)    
    (frames<=0) ? c :\
    (frames==1) ? c2.MDegrain1(super_rend, backward_vec1,forward_vec1,thSAD=400) :\
    (frames==2) ? c2.MDegrain2(super_rend, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) :\
              c2.MDegrain3(super_rend, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400)
    return(last)
}

[Witzig]

I second StainlessS.

On a side note: grain is detail. In my opinion, especially with old movies, the movies look better with the grain preserved. Just some food for thought.

[johnmeyer]

Completely removing grain can make the film look artificial and odd. I suggest that your goal should be grain reduction, and not removal.

There are several threads about restoring old film, and several excellent scripts for not only reducing grain, but also removing dirt, removing gate weave, improving color, correcting gamma, etc. Here is a link to one of those threads:

The power of Avisynth: restoring old 8mm films

[Stormborec]

Removing grain = removing details.

Fast, but destroying: removegrain(1) / (2) / (3) / (4)

Compromise: fft3dfilter(sigma=x) /Fastdegrain(x) /Edgecleaner(x)

Slow - just experimental: MCtemporaldegrain/ Dfttest

[Sparktank]

I also agree grain should be touched ever so lightly or not at all.

A prime example, funnily enough, is a 1987 film of Predator.
I read so many reviews when the "Ultimate Hunter Edition" came out only to find out it's heavily DNR'ed and looked like wax to so many people.

I went store to store to look for the first release of the movie, with the grain.
It was significantly cheaper since the UHE was available.

I've personally never dealt with grain removal, but I would go with any of the above for their experience.


EDIT: Assuming the source is progressive:
You can add this right after the source filter so you can trim the video down to 5 minutes or less for test encoding with the various degraining filters:

### NTSC VALUES # (24000 / 1001 = 23.97602397602398)
## 30s=719 | 1min=1438 | 2min=2877 | 3min=4315 | 4min=5754 | 5min=7192 | 10min=14385 ########
SelectRangeEvery(every=1500, length=71, offset=0, audio=false).Trim(0, 7192)

The number sign (#) is a comment sign that gets ignored in scripts.

71 seconds will give you over a minute of video frames, which should do well to judge a scene long enough.
I luck out during scene changes.

The Trim at the end you can change from 7192 (5min) to any thing else, as long as it's equal to the source frame rate (23.976, 24, 25; all after deinterlacing/detelecining).

You'll get 5min of a test encode using each variation of degraining and you can judge yourself what suits your own needs, as grain is a highly subjective nature rather than an objective nature.

It would look like this really...

degrain.avs

SetMemoryMax(512)
LoadPlugin(whatever)

FFVideoSource("source.mkv", cachefile="source.mkv.ffindex")
## TFM order | -1 auto | 0 BFF | 1 TFF #
#tfm(order=0, mode=5, slow=2, pp=7, field=0).TDecimate(mode=1, hybrid=1).Vinverse()
#remove comment for above line if it needs detelecine for 1080i film.

### NTSC VALUES # (24000 / 1001 = 23.97602397602398)
## 30s=719 | 1min=1438 | 2min=2877 | 3min=4315 | 4min=5754 | 5min=7192 | 10min=14385 ########
SelectRangeEvery(every=1500, length=71, offset=0, audio=false).Trim(0, 7192)

DeRingFiler()

Change "DeRingFilter()" to whatever filter the others have suggested.

If you want to use SetMTMode() with it, you'll have to make sure any dependancy plugins are updated for whatever version of Avisnynth you are using (assuming you are using most current AviSynth 2.6.0 RC1 [Jan 14th, 2015]).

Alternatively, for a speed boost, you can also try out ThreadRequest() and add it to the end most lines after the source line (FFVideoSource, DirectShowSource, DGsource, etc) as ".ThreadRequest()".
But, ThreadRequest is more stable without SetMTMode, but some have experimented with the combination of the two, I have yet to see a difference myself, so it's largely dependent on what filters are used and how...
TMI.

SelectRangeEvery(every=1500, length=71, offset=0, audio=false).Trim(0, 7192).ThreadRequest()

[Batman007]

See how much grains are in the video http://prntscr.com/69n99z... and its BD ..... Please tell me perfect filter so I can encode within less time possible and which will make grains invisible or less ..

[johnmeyer]

Originally Posted by Batman007

See how much grains are in the video http://prntscr.com/69n99z... and its BD ..... Please tell me perfect filter so I can encode within less time possible and which will make grains invisible or less ..

1. There is no "perfect" filter.

2. All filters take a lot of time to do their work.

3. Nothing will make the grain completely invisible without also introducing new artifacts that will be more objectionable than the grain.

Your expectations are completely unrealistic.

[Sparktank]

Originally Posted by Batman007

http://prntscr.com/69n99z

Looks pretty old. Removing a lot of grain would probably destroy the quality/details.

Today, we're spoiled with films switching over to digital cameras for filming movies.

I would leave it as it is.
Depending on your PC specs, you're probably looking at a 48hr+ encode time.

I've got 28 Days Later (the first movie) and it was filmed on a very low budget. The grain and noise in it is severe. I've decided to leave it on the disc and bare with it.

[Batman007]

Originally Posted by Sparktank

Looks pretty old. Removing a lot of grain would probably destroy the quality/details.

Today, we're spoiled with films switching over to digital cameras for filming movies.

I would leave it as it is.
Depending on your PC specs, you're probably looking at a 48hr+ encode time.

I've got 28 Days Later (the first movie) and it was filmed on a very low budget. The grain and noise in it is severe. I've decided to leave it on the disc and bare with it.

Yeah ... Movie is of 1987 .... I would keep the grains as they're ...

[feisty2]

TNLMeans (ax=32, ay=32, az=6, bx=0, by=0, sx=16, sy=16)

[Stereodude]

Originally Posted by Batman007

See how much grains are in the video http://prntscr.com/69n99z... and its BD ..... Please tell me perfect filter so I can encode within less time possible and which will make grains invisible or less ..

MCTD is probably your best bet at minimizing the grain. It's quite slow and is hard to multithread. So, generally if I'm going to process a BD I break the movie up into 4 equal sized pieces with trim() and process all 4 pieces to a lossless intermediate file at the same time with 2 pieces each on 2 different computers. Then I combine the 4 lossless pieces with an AVIsynth script to feed to x264. If you cut it up into pieces you'll probably want to overlap a few frames on the ends of the clips because it's a temporal filter.

So as an example, if I had a 40000 frame clip to MCTD I will do 0-10010, 9990-20010, 19990-30010, 29990-40000 as the 4 pieces. Then I cut 10 frames off the start and end of the 2 middle clips and 10 off the end of the first / 10 off the start of the last when combining them.

[feisty2]

Originally Posted by Stereodude

MCTD is probably your best bet at minimizing the grain.

TNLMeans is even better theoretically, it's not popular only because it's slow like hell, but if you got a fancy computer or whatever, replace MCTD with it.

[feisty2]

theoretical quality compare of some common video denoise filters
BM4D > NLMeans > Motion Compensate > Frequency Filters (FFT/DFT..) > Bilateral > Median Filter > simple linear filters (Gaussian Filter, Wiener Filter...)

[Overdrive80]

I hope that this script you like it:

You can fit this paremeters for decreasing or increasing denoise (thSAD=250, thSAD2=120,sbright=3.5, smedium=0.38, sdark=0,tr = 4)

        original=last

	pre=original.BrightDfttestMod(sbright=3.5, smedium=0.38, sdark=0,th_low=20, th_med=40, th_high=100, tbsize=1,lsb=true,mode=6)

	mascara=original.mt_edge("hprewitt",thY1=8, thY2=255,thc1=0,thc2=255)
	
	prefiltrado=mt_merge(pre, original, mascara,u=2,v=2)

#Temporal degrain

	tr = 4 #Stepping

	super= MSuper(prefiltrado,mt=true)

	multi_vec = MAnalyse (super, multi=true, delta=tr, blksize=8, overlap=4,search=4, searchparam=3,dct=10)

	last=prefiltrado.MDegrainN (super, multi_vec, tr, thSAD=250, thSAD2=120,plane=4,lsb=true,thSCD1=500,thSCD2=160).ditherpost(mode=6)

        gradfun3(mode=6,lsb_in=true, lsb=false)

BrightDfttestMod:

function BrightDfttestMod(clip input, float "sbright", float "smedium", float "sdark", 
 \        int "th_low", int "th_med", int "th_high", int "tbsize", bool "lsb", bool "lsb_in", int "mode")
{


sbright       = default( sbright,  0 ) 
smedium       = default( smedium,  0 )
sdark         = default( sdark,    0 ) 

th_low        = default( th_low,   20 ) 
th_med        = default( th_med,   40 ) 
th_high       = default( th_high, 100 ) 

lsb	      = default( lsb, false )
lsb_in	      = default( lsb_in, false )
mode	      = default( mode, 0    )

#It isn't correct to use lsb_in&&lsb==true at the same time
Assert(!(lsb==true && lsb_in==true), "This function not return lsb clip, if lsb_in=true then set lsb=false")

bright = (sbright > 0 && lsb==true  && lsb_in==false) ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in).ditherpost(mode=mode):\
	 (sbright > 0 && lsb==false && lsb_in==true)  ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):\
	 (sbright > 0 && lsb==false && lsb_in==false) ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):input

medium = (smedium > 0 && lsb==true  && lsb_in==false) ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in).ditherpost(mode=mode):\
	 (smedium > 0 && lsb==false && lsb_in==true)  ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):\
	 (smedium > 0 && lsb==false && lsb_in==false) ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):input

dark =   (sdark   > 0 && lsb==true  && lsb_in==false) ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in).ditherpost(mode=mode):\
	 (sdark   > 0 && lsb==false && lsb_in==true)  ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):\
	 (sdark   > 0 && lsb==false && lsb_in==false) ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):input

mmask = medium.levels(th_med, 1.0, th_high, 255, 0, false)		   
dmask = medium.levels(th_low, 1.0, th_med,  255, 0, false)

output = bright.mt_merge(medium, mmask, U=3, V=3, luma=true) 
         \  .mt_merge(dark, dmask, U=3, V=3, luma=true)

return output
}

[papcom]

Originally Posted by StainlessS

http://avisynth.nl/index.php/External_filters#Denoisers

I like (others will differ)

Spatial: VagueDenoise().

SpatioTemporal: FFT3DFilter(), HqDN3D() [fast], MCDegrain script.

MCDegrain by Didee (requires MVTools v2.0, heavy duty grain removal)

@stainless - I would like to try Your script but I still am not successful. I do not know where to put the Video. I have read on the mvtools guide and on other threads ...but I do not yet understand the principle with the function call.

Where do I put "myavi.avi" in order that it will be processed by Your script?

[StainlessS]

Import("MCDegrain.avs") # assumes that Didee's script saved as MCDegrain.avs
AviSource("myavi.avi")

MCDegrain(frames=3) # 1 to 3, 3 heaviest
#MCDegrainSharp(frames=3)

[papcom]

Originally Posted by StainlessS

Import("MCDegrain.avs") # assumes that Didee's script saved as MCDegrain.avs
AviSource("myavi.avi")

MCDegrain(frames=3) # 1 to 3, 3 heaviest
#MCDegrainSharp(frames=3)

thank You @stainlesss - Your script consists of two functions (MCDegrain and MCDegrainSharp). Is it correct that I save these both functions in one script as MCDegrain.avs?

Is there another way which works without the import avs? ...e.g. to put the videofile at the beginning or the end of the function(s). I saw in the documentation an example where the avi source it inserted after the function...like:

fields.MDegrain2(super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,thSAD=400)
Weave()
}

AVISource("video.avi")
mdegrain2i2(4,0)

[StainlessS]

If you want, the MCDegrainSharp was an additional suggestion (by myself) and changed a little by Didee.
If you dont want that one then just delete from avs.
There is nothing wrong in having multiple functions in a single script.

EDIT: Yes, you can just add the script functions instead of import, import is convenient if you want to keep the
script for use at any time. I use a 'BANK' folder to hold multiple avs scripts that I often use.

And you can put functions anywhere in script, either before or after eg AVISource(), it does not matter where they are.

[papcom]

thanks @stainlesss - I understand the avs Import but still I don't understand the position and manner where to place the avisource command.

In Your example I put it just before the function script or behind... it shows the avi but it is not processed (as far as I see). How do I relate the function to the video? ...how is the function assigned to the video?

In my sample from the documentation of MVTools the avisource is somehow at the end after the function? Why? How does this work?

Sorry for the simplicity of my question, but I must understand something before I can use it. Perhaps You can show me a small example? Thanks a lot.