Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th June 2011, 22:53   #1  |  Link
Alek93j
Registered User
 
Join Date: Apr 2011
Posts: 9
HQDegrain

Hi all, I made this little degrain filter. I hope some of you will find it useful. If you have any kind of feedback, please do let me know what you think about it.
Code:
 
  ######################################################################
#                                                                                                                                                                                                       
#  ~HQDegrain v0.9.2 by Alek93j~  29/06/2011                                                                                                                                          
#                                                                                                                                                                                                        
# Required MVtools2.dll    http://avisynth.org.ru/mvtools/mvtools2.html                                                                                                                            
#       ttempsmooth.dll    http://avisynth.org.ru/docs/english/...tempSmooth.htm                           
#   mt_masktools-25.dll    http://manao4.free.fr/masktools-v2.0a48.zip                                  
#       blockbuster.dll    http://kvcd.net/sansgrip/avisynth/                             
#       removegrain.dll    http://avisynth.org/mediawiki/Removegrain 
#                                               
# ~Syntax~       
 ##################################################################################
#                                                                                                                   
# # It doesn't work perfectely with sudden luma changes such as fades, so it                            
# # is best not to filter those parts.                                                                              
#                                                                                                                  
# -Example of a Backup of a NTSC DvD-                                                                             
#                                                                                                                   
# MPEG2Source("C:\blabla.d2v")                                                                                    
# ivtc() (your prefer ivtc ectr)                                                                                 
# A=hqdegrain() (Clip normally processed)                                                                            
# B=ttempsmoothf(maxr=7,strength=5,lthresh=3,cthresh=4) (or your prefer denoise for clean fade)                 
# B.Trim(0,215)+A.Trim(216,31464)                                                                                
#                                                                                                                  
 ###########################################################################

Function HQDegrain(clip clp,int"Degrain",int"limit",string"preset",bool"HQ",\
                   int"maxr",int"strength",int"lthresh",int"cthresh",\
                   clip"PreNoise",int"BkSize",bool"noise",bool"lmask",\
                   int"Blk",int"PEL",int"DCT",int"Overlap",\
                   int "thSAD", int "thSADC",int "thSCD1", int "thSCD2")
{

#Presets available "Fast" - "Slow"

Preset=Default(Preset,"Slow")
pset=(Preset=="Fast") ? 0 : 1
	
#MDegrain Strength

Degrain = Default( Degrain , Select(pset,1,2)) # Means how many frame should be analyzed for a better result, 3 doesn't help much.
limit   = Default( limit   , 255   ) # limit of mvdegrain. (255 mean no limits)

#ttempsmoothf Strength

HQ      = Default( HQ      , Select(pset,false,true)) # Reduce artefact...
maxr    = Default( maxr    , Select(pset,3,5))
strength= Default( strength, Select(pset,3,4))
lthresh = Default( lthresh , Select(pset,2,3))
cthresh = Default( cthresh , Select(pset,3,4))

#Noise Filter

Noise   = Default( Noise   , true  )
BkSize  = Default( BkSize  , 8     )
lmask   = Default( lmask   , Select(pset,false,true)) #if lmask is true, noise will not be added in the parts of the image with high luma values, because generally there is no need to filter them.

#Motion Vector Analyse

Blk     = Default( Blk     , 8     ) # Use 16/for 1080p - 8/for 720p - 4/for 480p to avoid unnecessary slowdowns.
PEL     = Default( PEL     , 2     )
DCT     = Default( DCT     , 0     )
OPS     = Blk/2
Overlap = Default( Overlap , Select(pset,0,OPS))

#Threshold Setting

thSAD   = Default( thSAD   , 180   ) # Lower values will reduce artefacts for bad motion vectors, but they will also have a lighter denoising effect. For animated sources, 120 seems to be a good setting to start with.
thSADC  = Default( thSADC  , thSAD )
thSCD1  = Default( thSCD1  , 400   )
thSCD2  = Default( thSCD2  , 102   )

#Processing

input = clp

mask = clp.invert().levels(82,1,155,0,255,coring=false).Removegrain(12)

nse  = defined(PreNoise) ? PreNoise 
       \: (Noise==true) ? clp.blockbuster(method="noise",Block_size=BkSize) : clp

clp = (Noise==false) ? clp : (lmask==true) ? mt_merge(clp,nse,mask) : nse

sup = clp.Msuper(hpad=Blk,vpad=Blk,PEL=PEL,levels=0)

BV1 = MAnalyse(sup,isb=true ,truemotion=true,delta=1,blksize=Blk,blksizeV=Blk,DCT=DCT,overlap=Overlap)
FV1 = MAnalyse(sup,isb=false,truemotion=true,delta=1,blksize=Blk,blksizeV=Blk,DCT=DCT,overlap=Overlap)

BV2 = (Degrain>=2) ? MAnalyse(sup,isb=true ,truemotion=true,delta=2,blksize=Blk,blksizeV=Blk,DCT=DCT,overlap=Overlap) : Blackness () 
FV2 = (Degrain>=2) ? MAnalyse(sup,isb=false,truemotion=true,delta=2,blksize=Blk,blksizeV=Blk,DCT=DCT,overlap=Overlap) : Blackness () 

BV3 = (Degrain==3) ? MAnalyse(sup,isb=true ,truemotion=true,delta=3,blksize=Blk,blksizeV=Blk,DCT=DCT,overlap=Overlap) : Blackness () 
FV3 = (Degrain==3) ? MAnalyse(sup,isb=false,truemotion=true,delta=3,blksize=Blk,blksizeV=Blk,DCT=DCT,overlap=Overlap) : Blackness () 

clp = (degrain==1) ? clp.MDegrain1(sup,BV1,FV1,                thSAD=thSAD,thSADC=thSADC,thSCD1=thSCD1,thSCD2=thSCD2,limit=limit): 
 \    (degrain==2) ? clp.MDegrain2(sup,BV1,FV1,BV2,FV2,        thSAD=thSAD,thSADC=thSADC,thSCD1=thSCD1,thSCD2=thSCD2,limit=limit):
 \                   clp.MDegrain3(sup,BV1,FV1,BV2,FV2,BV3,FV3,thSAD=thSAD,thSADC=thSADC,thSCD1=thSCD1,thSCD2=thSCD2,limit=limit)

Diff1 = mt_makediff(input,clp,U=3,V=3)

clp2  = clp.ttempsmoothf(maxr=maxr,strength=strength,lthresh=lthresh,cthresh=cthresh)

Diff2 = mt_makediff(input,clp2,U=3,V=3)

lutxy = mt_lutxy(Diff1,Diff2,"x 128 - abs y 128 - abs < x y ?")

output = (HQ==true) ? Input.mt_makediff(lutxy,U=3,V=3) : clp2

output }
Have a nice day.

Last edited by Alek93j; 29th June 2011 at 22:53. Reason: New Version
Alek93j is offline   Reply With Quote
Old 21st June 2011, 00:38   #2  |  Link
TheProfileth
Leader of Dual-Duality
 
TheProfileth's Avatar
 
Join Date: Aug 2010
Location: America
Posts: 134
Will give this a look
__________________
I'm Mr.Fixit and I feel good, fixin all the sources in the neighborhood
My New filter is in the works, and will be out soon
TheProfileth is offline   Reply With Quote
Old 21st June 2011, 12:13   #3  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Welcome on D9, Alek93j.


Quote:
# Required MVtools2.dll
# ttempsmooth.dll
# mt_masktools-25.dll
You forgot one dependency: BlockBuster.dll


Code:
sup = clp.Msuper(hpad=Blk,vpad=Blk,PEL=PEL,levels=0)
sup2= clp.Msuper(hpad=Blk,vpad=Blk,PEL=PEL,levels=1)
Since no additional pre-filtering is done, there is no need to create a 2nd super clip with levels=1. You can simply use the one "sup" clip for everything.


Also, it seems there is some confusion about chroma processing:

Both the MDegrain's and ttempsmoothF *do* process chroma. But when "HQ=true" (default), then all chroma processing is thrown away because of the mt_makediff's and the mt_lutxy's.

You should decide whether chroma shall be processed or not.
If not, then you can get better speed by setting "plane=0" in MDegrainX, and by setting "cthresh=1" in ttempsmooth.
If chroma should be processed, then you need to add ",U=3,V=3" in all mt_makediff and all mt_lutxy.


Seems to be a nice filter for baby grain. For real men's grain, it's a bit on the weak side.

Funny trivia: It's a "degrain" filter, and the very-first internal operation is to add more grain. (Blockbuster)


Judging visually, the results seem pretty okay.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 21st June 2011, 14:10   #4  |  Link
Gerry62
TV Guy
 
Join Date: Jun 2011
Posts: 46
Am looking for a Nice script to clean some 16mm Kine's!

Quote:
LoadPlugin("TTempSmooth.dll")
Import("HQDegrain.avsi")
AVISource("D:\Raw.avi")
converttoYV12()
FieldDeinterlace(full=true, blend=true)
HQDegrain()
Ive got bloody 'loadplugin' errors.
Will let you know If i get further, as Im keen for a script that can clean 16mm Kine's, even if its only a first pass before something else. Sick of pixilated movement!

Last edited by Gerry62; 21st June 2011 at 14:21.
Gerry62 is offline   Reply With Quote
Old 21st June 2011, 14:19   #5  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
What ttempsmoothf errors? The script is technically correct, there should not occur any errors as long as you respect the valid parameter ranges for ttempsmooth's parameters: maxr=[1...7], strength=[1...8], lthresh/cthresh=[1...256]
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 21st June 2011, 14:24   #6  |  Link
Gerry62
TV Guy
 
Join Date: Jun 2011
Posts: 46
Quote:
Originally Posted by Didée View Post
What ttempsmoothf errors? The script is technically correct, there should not occur any errors as long as you respect the valid parameter ranges for ttempsmooth's parameters: maxr=[1...7], strength=[1...8], lthresh/cthresh=[1...256]
Yes the script is fine, Very fine!
But the user (IE Me) is a little green, but as keen as mustard to get it to work!
Spent all day on Killer() and failed!

Must be where Im placing the ttempsmooth.dll.
And over the last few weeks ive been reading some of your posts Didee, If I could create a , 'I Am not worthy' script, it would be now, you are a god of the synth~! and please disregard my amuter efforts in getting things to work.

Your a champ alek93j, love your efforts, will love it more when Ive seen it work!

Last edited by Gerry62; 21st June 2011 at 14:28.
Gerry62 is offline   Reply With Quote
Old 21st June 2011, 14:35   #7  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
You still didn't state the exact error. If it is "There is no function named 'ttempsmoothf' ", then you need to LoadPlugin() the ttempsmooth.dll ...
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 21st June 2011, 15:34   #8  |  Link
Gerry62
TV Guy
 
Join Date: Jun 2011
Posts: 46
Wow, how stupid am I!
I had the x64 DLL in my plugins directory, hence the error's, Have now put the correct 32bit dll and it's working fine, the defult setup is not as 'strong' as id like the 16mm kino's are 'grain city'. which makes any restoration very difficult as when you clean the grains all movement looks very 'gritty' and pixelated to hell. Will spend some time mucking about with the settings and see how this new function works with my dire material!
Gerry62 is offline   Reply With Quote
Old 21st June 2011, 22:46   #9  |  Link
Alek93j
Registered User
 
Join Date: Apr 2011
Posts: 9
Quote:
Originally Posted by Didée View Post
Welcome on D9, Alek93j.
thanks for welcome

Quote:
Originally Posted by Didée View Post
You forgot one dependency: BlockBuster.dll
thanks for reminding me, I had forgotten about it.

Quote:
Originally Posted by Didée View Post
Code:
sup = clp.Msuper(hpad=Blk,vpad=Blk,PEL=PEL,levels=0)
sup2= clp.Msuper(hpad=Blk,vpad=Blk,PEL=PEL,levels=1)
Since no additional pre-filtering is done, there is no need to create a 2nd super clip with levels=1. You can simply use the one "sup"

clip for everything.
I put 2 sup clips because I read in the mvtools documentation that only manalyse requires access to all levels, whereas for the other functions 1 level is enough, thus having the other functions work on just 1 level would give a speedup without hinder quality. Or did I just not understand the documentation (I admittedly am not too sure about this)?

Quote:
Originally Posted by Didée View Post
Also, it seems there is some confusion about chroma processing:

Both the MDegrain's and ttempsmoothF *do* process chroma. But when "HQ=true" (default), then all chroma processing is thrown away because

of the mt_makediff's and the mt_lutxy's.

You should decide whether chroma shall be processed or not.
If not, then you can get better speed by setting "plane=0" in MDegrainX, and by setting "cthresh=1" in ttempsmooth.
If chroma should be processed, then you need to add ",U=3,V=3" in all mt_makediff and all mt_lutxy.
I'm not very good with the masktool, I added the HQ function for repair the ttempsmooth artifacts, I mean these:

original

hq false

hq true


I admit that I copied the masktool part from your script "temporal degrain", beacuse I got the exact results which I wanted, and I was stupid to not specify it from the beginning, and the main reason I posted the filter is to get help for that part, so, can you help me to get those results in the right way, or, if is it possible, also better? I would be grateful to you.

Quote:
Originally Posted by Didée View Post
Funny trivia: It's a "degrain" filter, and the very-first internal operation is to add more grain. (Blockbuster)
"This method adds normally distributed -- also known as Gaussian -- noise to the clip. Testing has shown that Gaussian noise is far more suitable for this filter's purposes than uniformly distributed noise."
Blockbuster adds noise, doesn't it? I created this function for "restore" the grain affected by macroblocking etc. caused by a bad compression, they aren't really pleasant during the reproduction. This function also removes a little part of grain extra (for example, in the bright parts of the image).My idea was to fill these empty parts with blockbuster and then to work with MDegrain: it would have switched a part of noise for the grain e it would have removed it, but not completely. And, to finish, make everything homogeneous with ttempsmooth, which is my favourite smoother, as you can see in the screens that I posted first.

Quote:
Your a champ alek93j, love your efforts, will love it more when Ive seen it work!
thanks, I hope to learn more and to be more helpful
Alek93j is offline   Reply With Quote
Old 23rd June 2011, 10:18   #10  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
Quote:
Originally Posted by Alek93j View Post
Hi all, I made this little degrain filter. I hope some of you will find it useful. If you have any kind of feedback, please do let me know what you think about it.
Thanks. Do you plan to release a x64 aware version? Some filters don't even exist in x64 world
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 23rd June 2011, 10:48   #11  |  Link
Nephilis
--preset WTF!
 
Join Date: Feb 2009
Posts: 86
Quote:
Originally Posted by Alek93j View Post
I put 2 sup clips because I read in the mvtools documentation that only manalyse requires access to all levels, whereas for the other functions 1 level is enough, thus having the other functions work on just 1 level would give a speedup without hinder quality. Or did I just not understand the documentation (I admittedly am not too sure about this)?
Yes only manalyse needs all levels. But when you prepare 2 super clip, the process will need more memory and spend more effort. Also as Didée said you should decide if the filter process the Chroma or not..

Quote:
Originally Posted by tormento View Post
Thanks. Do you plan to release a x64 aware version? Some filters don't even exist in x64 world
Everybody is boring for your x64 requests. Try to share something useful posts here not only x64 aware filters/plugins requests..

Last edited by Nephilis; 23rd June 2011 at 10:54.
Nephilis is offline   Reply With Quote
Old 23rd June 2011, 19:04   #12  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
Quote:
Originally Posted by Nephilis View Post
Everybody is boring for your x64 requests. Try to share something useful posts here not only x64 aware filters/plugins requests..
Your 79 posts and 2009 subscription are enough to judge my behaviour and address your brain to such a democratic answer?

Welcome, my new forum moderator.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 23rd June 2011, 20:35   #13  |  Link
Nephilis
--preset WTF!
 
Join Date: Feb 2009
Posts: 86
Quote:
Originally Posted by tormento View Post
Your 79 posts and 2009 subscription are enough to judge my behaviour and address your brain to such a democratic answer?
Welcome, my new forum moderator.
Only in last one year (between 23.06.2010-23.06.2011)

http://forum.doom9.org/showthread.ph...18#post1509718
http://forum.doom9.org/showthread.ph...17#post1509717
http://forum.doom9.org/showthread.ph...48#post1505148
http://forum.doom9.org/showthread.ph...70#post1494770
http://forum.doom9.org/showthread.ph...26#post1473026
http://forum.doom9.org/showthread.ph...66#post1472166
http://forum.doom9.org/showthread.ph...63#post1472163
http://forum.doom9.org/showthread.ph...61#post1460161
http://forum.doom9.org/showthread.ph...85#post1459685
http://forum.doom9.org/showthread.ph...39#post1432839
http://forum.doom9.org/showthread.ph...42#post1431042
http://forum.doom9.org/showthread.ph...82#post1423182
http://forum.doom9.org/showthread.ph...75#post1416675
http://forum.doom9.org/showthread.ph...81#post1415381
http://forum.doom9.org/showthread.ph...50#post1414650
http://forum.doom9.org/showthread.ph...69#post1412869
http://forum.doom9.org/showthread.ph...57#post1411457
http://forum.doom9.org/showthread.ph...07#post1411207
etc.
etc.
etc.

There are more of them..

No comment...

Edit: I think 4763 posts and 2002 subscription are enough

Last edited by Nephilis; 23rd June 2011 at 20:53.
Nephilis is offline   Reply With Quote
Old 23rd June 2011, 20:35   #14  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
But, speaking frankly, it IS like that ...

Edit: Haha, Nephilis - this search I did a few moments ago, too.


Edit#2: @tormento:

Funnily, this filter here can be run under Avisynth-x64 practically without any problem. The builds of MVTools-x64 and ttempsmooth-x64 are out there. You have to ditch Blockbuster, but that's not essential for "the method" anyway. Just cancel it, or replace with AddGrain or Gradfun2db. And ba-DANG - another filter you can use with Avisynth-x64 until the next script crash.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)

Last edited by Didée; 23rd June 2011 at 20:42.
Didée is offline   Reply With Quote
Old 23rd June 2011, 20:42   #15  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 1,034
No problem. I'll shut up, at last.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 24th June 2011, 16:43   #16  |  Link
Alek93j
Registered User
 
Join Date: Apr 2011
Posts: 9
Ok, I updated the filter with the changes that Didée suggested, thanks for your help.


Edit: up new version

Last edited by Alek93j; 29th June 2011 at 22:40.
Alek93j is offline   Reply With Quote
Old 16th April 2013, 12:08   #17  |  Link
steptoe
Registered User
 
steptoe's Avatar
 
Join Date: Mar 2003
Location: UK
Posts: 360
Just changed your code very slightly to point directly to the extra filters needed so you don't have to hunt them down

#######################################################################################################
#
# ~HQDegrain v0.9.2 by Alek93j~ 29/06/2011
#
# Required MVtools2.dll http://avisynth.org.ru/mvtools/mvtools-v2.5.11.3.zip
# ttempsmooth.dll http://www.missouri.edu/~kes25c/TTempSmoothv094.zip
# mt_masktools-25.dll http://manao4.free.fr/masktools-v2.0a48.zip
# blockbuster.dll http://avisynth.org/warpenterprises/files/blockbuster_20_dll_20021229.zip
# removegrain.dll http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar
#
# ~Syntax~
#######################################################################################################
steptoe is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:39.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.