Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th February 2015, 15:12   #1  |  Link
Batman007
Registered User
 
Join Date: Jan 2015
Posts: 44
How would I remove grains from a video ?

Hi guys

I got a movie of 1987 having grains in it .... How do I remove those grains ?
Please give me plugins and filters for DeGraining ....


Thank you .......
Batman007 is offline   Reply With Quote
Old 24th February 2015, 15:19   #2  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,088
http://avisynth.nl/index.php/External_filters#Denoisers

I like (others will differ)

Spatial: VagueDenoise().

SpatioTemporal: FFT3DFilter(), HqDN3D() [fast], MCDegrain script.

MCDegrain by Didee (requires MVTools v2.0, heavy duty grain removal)

Code:
Function MCDegrain(clip c, int "frames")
{ # By Didee, http://forum.doom9.org/showthread.php?p=1508289#post1508289

    frames = default(frames, 2)
    bs = (c.width>960) ? 16 : 8
    super = c.MSuper(pel=2, sharp=1)
    backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=bs, overlap=bs/2)
    backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=bs, overlap=bs/2)
    backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=bs, overlap=bs/2)
    forward_vec3 = MAnalyse(super, isb = false, delta = 3, blksize=bs, overlap=bs/2)    
    (frames<=0) ? c :\
    (frames==1) ? c.MDegrain1(super, backward_vec1,forward_vec1,thSAD=400) :\
    (frames==2) ? c.MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) :\
              c.MDegrain3(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400)
    return(last)
}

Function MCDegrainSharp(clip c, int "frames", float "bblur", float "csharp", bool "bsrch")
{ # Based on MCDegrain By Didee, http://forum.doom9.org/showthread.php?t=161594
  # Also based on DiDee observations in this thread: http://forum.doom9.org/showthread.php?t=161580 
  # "Denoise with MDegrainX, do slight sharpening where motionmatch is good, do slight blurring where motionmatch is bad"
  # In areas where MAnalyse cannot find good matches, the blur() will be dominant.
  # In areas where good matches are found, the sharpen()'ed pixels will overweight the blur()'ed pixels
  # when the pixel averaging is performed. 
    frames = default(frames, 2)
    bblur  = default(bblur, 0.6)
    csharp = default(csharp, 0.6)
    bsrch  = default(bsrch, true)
    bs = (c.width>960) ? 16 : 8
    
    c2 = c.blur(bblur)
    super = bsrch ? c2.MSuper(pel=2, sharp=1) : c.MSuper(pel=2, sharp=1)
    super_rend  = c.sharpen(csharp).MSuper(pel=2, sharp=1,levels=1)
    backward_vec3 = MAnalyse(super, isb = true, delta = 3, blksize=bs, overlap=bs/2)
    backward_vec2 = MAnalyse(super, isb = true, delta = 2, blksize=bs, overlap=bs/2)
    backward_vec1 = MAnalyse(super, isb = true, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec1 = MAnalyse(super, isb = false, delta = 1, blksize=bs, overlap=bs/2)
    forward_vec2 = MAnalyse(super, isb = false, delta = 2, blksize=bs, overlap=bs/2)
    forward_vec3 = MAnalyse(super, isb = false, delta = 3, blksize=bs, overlap=bs/2)    
    (frames<=0) ? c :\
    (frames==1) ? c2.MDegrain1(super_rend, backward_vec1,forward_vec1,thSAD=400) :\
    (frames==2) ? c2.MDegrain2(super_rend, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) :\
              c2.MDegrain3(super_rend, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=400)
    return(last)
}
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 24th February 2015 at 15:29.
StainlessS is offline   Reply With Quote
Old 24th February 2015, 16:06   #3  |  Link
Witzig
Registered User
 
Join Date: Jan 2014
Posts: 13
I second StainlessS.

On a side note: grain is detail. In my opinion, especially with old movies, the movies look better with the grain preserved. Just some food for thought.
Witzig is offline   Reply With Quote
Old 24th February 2015, 19:52   #4  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,068
Completely removing grain can make the film look artificial and odd. I suggest that your goal should be grain reduction, and not removal.

There are several threads about restoring old film, and several excellent scripts for not only reducing grain, but also removing dirt, removing gate weave, improving color, correcting gamma, etc. Here is a link to one of those threads:

The power of Avisynth: restoring old 8mm films
johnmeyer is offline   Reply With Quote
Old 24th February 2015, 19:58   #5  |  Link
Stormborec
Registered User
 
Join Date: Feb 2013
Posts: 158
Removing grain = removing details.

Fast, but destroying: removegrain(1) / (2) / (3) / (4)

Compromise: fft3dfilter(sigma=x) /Fastdegrain(x) /Edgecleaner(x)

Slow - just experimental: MCtemporaldegrain/ Dfttest
Stormborec is offline   Reply With Quote
Old 25th February 2015, 02:22   #6  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 857
I also agree grain should be touched ever so lightly or not at all.

A prime example, funnily enough, is a 1987 film of Predator.
I read so many reviews when the "Ultimate Hunter Edition" came out only to find out it's heavily DNR'ed and looked like wax to so many people.

I went store to store to look for the first release of the movie, with the grain.
It was significantly cheaper since the UHE was available.

I've personally never dealt with grain removal, but I would go with any of the above for their experience.


EDIT: Assuming the source is progressive:
You can add this right after the source filter so you can trim the video down to 5 minutes or less for test encoding with the various degraining filters:

Code:
### NTSC VALUES # (24000 / 1001 = 23.97602397602398)
## 30s=719 | 1min=1438 | 2min=2877 | 3min=4315 | 4min=5754 | 5min=7192 | 10min=14385 ########
SelectRangeEvery(every=1500, length=71, offset=0, audio=false).Trim(0, 7192)
The number sign (#) is a comment sign that gets ignored in scripts.

71 seconds will give you over a minute of video frames, which should do well to judge a scene long enough.
I luck out during scene changes.

The Trim at the end you can change from 7192 (5min) to any thing else, as long as it's equal to the source frame rate (23.976, 24, 25; all after deinterlacing/detelecining).

You'll get 5min of a test encode using each variation of degraining and you can judge yourself what suits your own needs, as grain is a highly subjective nature rather than an objective nature.

It would look like this really...

degrain.avs
Code:
SetMemoryMax(512)
LoadPlugin(whatever)

FFVideoSource("source.mkv", cachefile="source.mkv.ffindex")
## TFM order | -1 auto | 0 BFF | 1 TFF #
#tfm(order=0, mode=5, slow=2, pp=7, field=0).TDecimate(mode=1, hybrid=1).Vinverse()
#remove comment for above line if it needs detelecine for 1080i film.

### NTSC VALUES # (24000 / 1001 = 23.97602397602398)
## 30s=719 | 1min=1438 | 2min=2877 | 3min=4315 | 4min=5754 | 5min=7192 | 10min=14385 ########
SelectRangeEvery(every=1500, length=71, offset=0, audio=false).Trim(0, 7192)

DeRingFiler()
Change "DeRingFilter()" to whatever filter the others have suggested.

If you want to use SetMTMode() with it, you'll have to make sure any dependancy plugins are updated for whatever version of Avisnynth you are using (assuming you are using most current AviSynth 2.6.0 RC1 [Jan 14th, 2015]).

Alternatively, for a speed boost, you can also try out ThreadRequest() and add it to the end most lines after the source line (FFVideoSource, DirectShowSource, DGsource, etc) as ".ThreadRequest()".
But, ThreadRequest is more stable without SetMTMode, but some have experimented with the combination of the two, I have yet to see a difference myself, so it's largely dependent on what filters are used and how...
TMI.
Code:
SelectRangeEvery(every=1500, length=71, offset=0, audio=false).Trim(0, 7192).ThreadRequest()
__________________
Win10 (x64) build 17134 | GPU Caps Viewer 1.40.1.0
NVIDIA GeForce GT 1030 (GP108) 2047MB/GDDR5 | (R417.22)
NTSC | DVD: R1 | BD: A

Last edited by Sparktank; 25th February 2015 at 05:52.
Sparktank is offline   Reply With Quote
Old 27th February 2015, 17:20   #7  |  Link
Batman007
Registered User
 
Join Date: Jan 2015
Posts: 44
See how much grains are in the video http://prntscr.com/69n99z... and its BD ..... Please tell me perfect filter so I can encode within less time possible and which will make grains invisible or less ..
Batman007 is offline   Reply With Quote
Old 27th February 2015, 20:11   #8  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,068
Quote:
Originally Posted by Batman007 View Post
See how much grains are in the video http://prntscr.com/69n99z... and its BD ..... Please tell me perfect filter so I can encode within less time possible and which will make grains invisible or less ..
1. There is no "perfect" filter.

2. All filters take a lot of time to do their work.

3. Nothing will make the grain completely invisible without also introducing new artifacts that will be more objectionable than the grain.

Your expectations are completely unrealistic.
johnmeyer is offline   Reply With Quote
Old 28th February 2015, 03:07   #9  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 857
Quote:
Originally Posted by Batman007 View Post
Looks pretty old. Removing a lot of grain would probably destroy the quality/details.

Today, we're spoiled with films switching over to digital cameras for filming movies.

I would leave it as it is.
Depending on your PC specs, you're probably looking at a 48hr+ encode time.

I've got 28 Days Later (the first movie) and it was filmed on a very low budget. The grain and noise in it is severe. I've decided to leave it on the disc and bare with it.
__________________
Win10 (x64) build 17134 | GPU Caps Viewer 1.40.1.0
NVIDIA GeForce GT 1030 (GP108) 2047MB/GDDR5 | (R417.22)
NTSC | DVD: R1 | BD: A
Sparktank is offline   Reply With Quote
Old 1st March 2015, 16:20   #10  |  Link
Batman007
Registered User
 
Join Date: Jan 2015
Posts: 44
Quote:
Originally Posted by Sparktank View Post
Looks pretty old. Removing a lot of grain would probably destroy the quality/details.

Today, we're spoiled with films switching over to digital cameras for filming movies.

I would leave it as it is.
Depending on your PC specs, you're probably looking at a 48hr+ encode time.

I've got 28 Days Later (the first movie) and it was filmed on a very low budget. The grain and noise in it is severe. I've decided to leave it on the disc and bare with it.
Yeah ... Movie is of 1987 .... I would keep the grains as they're ...
Batman007 is offline   Reply With Quote
Old 1st March 2015, 16:42   #11  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
TNLMeans (ax=32, ay=32, az=6, bx=0, by=0, sx=16, sy=16)
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 1st March 2015, 16:52   #12  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 981
Quote:
Originally Posted by Batman007 View Post
See how much grains are in the video http://prntscr.com/69n99z... and its BD ..... Please tell me perfect filter so I can encode within less time possible and which will make grains invisible or less ..
MCTD is probably your best bet at minimizing the grain. It's quite slow and is hard to multithread. So, generally if I'm going to process a BD I break the movie up into 4 equal sized pieces with trim() and process all 4 pieces to a lossless intermediate file at the same time with 2 pieces each on 2 different computers. Then I combine the 4 lossless pieces with an AVIsynth script to feed to x264. If you cut it up into pieces you'll probably want to overlap a few frames on the ends of the clips because it's a temporal filter.

So as an example, if I had a 40000 frame clip to MCTD I will do 0-10010, 9990-20010, 19990-30010, 29990-40000 as the 4 pieces. Then I cut 10 frames off the start and end of the 2 middle clips and 10 off the end of the first / 10 off the start of the last when combining them.
Stereodude is offline   Reply With Quote
Old 1st March 2015, 16:59   #13  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
Quote:
Originally Posted by Stereodude View Post
MCTD is probably your best bet at minimizing the grain.
TNLMeans is even better theoretically, it's not popular only because it's slow like hell, but if you got a fancy computer or whatever, replace MCTD with it.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 1st March 2015, 17:45   #14  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Los Angeles, California
Posts: 2,119
theoretical quality compare of some common video denoise filters
BM4D > NLMeans > Motion Compensate > Frequency Filters (FFT/DFT..) > Bilateral > Median Filter > simple linear filters (Gaussian Filter, Wiener Filter...)
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 1st March 2015, 17:52   #15  |  Link
Overdrive80
Anime addict
 
Overdrive80's Avatar
 
Join Date: Feb 2009
Location: Spain
Posts: 625
I hope that this script you like it:

You can fit this paremeters for decreasing or increasing denoise (thSAD=250, thSAD2=120,sbright=3.5, smedium=0.38, sdark=0,tr = 4)

Code:
        original=last

	pre=original.BrightDfttestMod(sbright=3.5, smedium=0.38, sdark=0,th_low=20, th_med=40, th_high=100, tbsize=1,lsb=true,mode=6)

	mascara=original.mt_edge("hprewitt",thY1=8, thY2=255,thc1=0,thc2=255)
	
	prefiltrado=mt_merge(pre, original, mascara,u=2,v=2)

#Temporal degrain

	tr = 4 #Stepping

	super= MSuper(prefiltrado,mt=true)

	multi_vec = MAnalyse (super, multi=true, delta=tr, blksize=8, overlap=4,search=4, searchparam=3,dct=10)

	last=prefiltrado.MDegrainN (super, multi_vec, tr, thSAD=250, thSAD2=120,plane=4,lsb=true,thSCD1=500,thSCD2=160).ditherpost(mode=6)

        gradfun3(mode=6,lsb_in=true, lsb=false)

BrightDfttestMod:

Code:
function BrightDfttestMod(clip input, float "sbright", float "smedium", float "sdark", 
 \        int "th_low", int "th_med", int "th_high", int "tbsize", bool "lsb", bool "lsb_in", int "mode")
{


sbright       = default( sbright,  0 ) 
smedium       = default( smedium,  0 )
sdark         = default( sdark,    0 ) 

th_low        = default( th_low,   20 ) 
th_med        = default( th_med,   40 ) 
th_high       = default( th_high, 100 ) 

lsb	      = default( lsb, false )
lsb_in	      = default( lsb_in, false )
mode	      = default( mode, 0    )

#It isn't correct to use lsb_in&&lsb==true at the same time
Assert(!(lsb==true && lsb_in==true), "This function not return lsb clip, if lsb_in=true then set lsb=false")

bright = (sbright > 0 && lsb==true  && lsb_in==false) ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in).ditherpost(mode=mode):\
	 (sbright > 0 && lsb==false && lsb_in==true)  ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):\
	 (sbright > 0 && lsb==false && lsb_in==false) ? input.dfttest(sigma = sbright,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):input

medium = (smedium > 0 && lsb==true  && lsb_in==false) ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in).ditherpost(mode=mode):\
	 (smedium > 0 && lsb==false && lsb_in==true)  ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):\
	 (smedium > 0 && lsb==false && lsb_in==false) ? input.dfttest(sigma = smedium,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):input

dark =   (sdark   > 0 && lsb==true  && lsb_in==false) ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in).ditherpost(mode=mode):\
	 (sdark   > 0 && lsb==false && lsb_in==true)  ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):\
	 (sdark   > 0 && lsb==false && lsb_in==false) ? input.dfttest(sigma = sdark,tbsize=tbsize,sbsize=18,sosize=9,lsb=lsb,lsb_in=lsb_in):input

mmask = medium.levels(th_med, 1.0, th_high, 255, 0, false)		   
dmask = medium.levels(th_low, 1.0, th_med,  255, 0, false)

output = bright.mt_merge(medium, mmask, U=3, V=3, luma=true) 
         \  .mt_merge(dark, dmask, U=3, V=3, luma=true)

return output
}
__________________
Intel i7-6700K + Noctua NH-D15 + Z170A XPower G. Titanium + Kingston HyperX Savage DDR4 2x8GB + Nvidia GTX750 2GB DDR5 + SSD Vertex 4 256 GB + Antec EDG750 80 Plus Gold Mod + Corsair 780T Graphite
Overdrive80 is offline   Reply With Quote
Old 18th March 2015, 20:16   #16  |  Link
papcom
Registered User
 
Join Date: Feb 2002
Location: Southwest Germany
Posts: 169
Quote:
Originally Posted by StainlessS View Post
http://avisynth.nl/index.php/External_filters#Denoisers

I like (others will differ)

Spatial: VagueDenoise().

SpatioTemporal: FFT3DFilter(), HqDN3D() [fast], MCDegrain script.

MCDegrain by Didee (requires MVTools v2.0, heavy duty grain removal)
@stainless - I would like to try Your script but I still am not successful. I do not know where to put the Video. I have read on the mvtools guide and on other threads ...but I do not yet understand the principle with the function call.

Where do I put "myavi.avi" in order that it will be processed by Your script?
__________________
Lenovo Intel i7Quad/WIN8 - Virtual Dub 1.10.4/Avisinth 2.6MT- MacMini I7 QuadCore
papcom is offline   Reply With Quote
Old 18th March 2015, 20:49   #17  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,088
Import("MCDegrain.avs") # assumes that Didee's script saved as MCDegrain.avs
AviSource("myavi.avi")

MCDegrain(frames=3) # 1 to 3, 3 heaviest
#MCDegrainSharp(frames=3)
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 18th March 2015 at 22:43.
StainlessS is offline   Reply With Quote
Old 18th March 2015, 23:02   #18  |  Link
papcom
Registered User
 
Join Date: Feb 2002
Location: Southwest Germany
Posts: 169
Quote:
Originally Posted by StainlessS View Post
Import("MCDegrain.avs") # assumes that Didee's script saved as MCDegrain.avs
AviSource("myavi.avi")

MCDegrain(frames=3) # 1 to 3, 3 heaviest
#MCDegrainSharp(frames=3)
thank You @stainlesss - Your script consists of two functions (MCDegrain and MCDegrainSharp). Is it correct that I save these both functions in one script as MCDegrain.avs?

Is there another way which works without the import avs? ...e.g. to put the videofile at the beginning or the end of the function(s). I saw in the documentation an example where the avi source it inserted after the function...like:

Quote:
fields.MDegrain2(super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,thSAD=400)
Weave()
}

AVISource("video.avi")
mdegrain2i2(4,0)
__________________
Lenovo Intel i7Quad/WIN8 - Virtual Dub 1.10.4/Avisinth 2.6MT- MacMini I7 QuadCore

Last edited by papcom; 18th March 2015 at 23:08.
papcom is offline   Reply With Quote
Old 18th March 2015, 23:09   #19  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,088
If you want, the MCDegrainSharp was an additional suggestion (by myself) and changed a little by Didee.
If you dont want that one then just delete from avs.
There is nothing wrong in having multiple functions in a single script.

EDIT: Yes, you can just add the script functions instead of import, import is convenient if you want to keep the
script for use at any time. I use a 'BANK' folder to hold multiple avs scripts that I often use.

And you can put functions anywhere in script, either before or after eg AVISource(), it does not matter where they are.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 19th March 2015 at 00:13.
StainlessS is offline   Reply With Quote
Old 19th March 2015, 00:15   #20  |  Link
papcom
Registered User
 
Join Date: Feb 2002
Location: Southwest Germany
Posts: 169
thanks @stainlesss - I understand the avs Import but still I don't understand the position and manner where to place the avisource command.

In Your example I put it just before the function script or behind... it shows the avi but it is not processed (as far as I see). How do I relate the function to the video? ...how is the function assigned to the video?

In my sample from the documentation of MVTools the avisource is somehow at the end after the function? Why? How does this work?

Sorry for the simplicity of my question, but I must understand something before I can use it. Perhaps You can show me a small example? Thanks a lot.
__________________
Lenovo Intel i7Quad/WIN8 - Virtual Dub 1.10.4/Avisinth 2.6MT- MacMini I7 QuadCore

Last edited by papcom; 19th March 2015 at 00:21.
papcom is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.