Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 23rd July 2012, 16:16   #61  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Source Filter means, a way to open your video file.
For example, va=DirectShowSource("music video.mpg")
You don't actually need all of his script. For example, I believe the purpose of the SmoothFilter line was just to make the video look better.
The main line to remove dot crawl could just be a quiver command.
I don't know what he meant by EmuFFT.

Perhaps you can work on another project until you get a reply in some days.
jmac698 is offline   Reply With Quote
Old 23rd July 2012, 20:35   #62  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
How do you make a fishing net? -- Take ninety-thousand holes, and knot them together.

Code:
# Required plugins:
# -----------------
# dgdecode.dll
# TIVTC.dll
# mt_masktools.dll
# removegrain.dll
# repair.dll
# frfun7.dll

mpeg2source("G:\Video\_Work\_dotcrawl\dot_crawled.d2v")

borders = bicubicresize(width()/2,height()/2).bicubicresize(width()+8,height())
stackhorizontal(borders.crop(0,0,4,0),last,borders.crop(width()+4,0,0,0))

o=last ox=o.width() oy=o.height()

drr     = o.separatefields().vinverse2().weave()
both    = stackhorizontal(drr,o)
IVTC    = both.tfm(pp=0,clip2=stackhorizontal(drr,drr)).tdecimate()
raw     = IVTC.crop(ox,0,ox,oy)
clean1  = IVTC.crop(0,0,ox,oy)

D1      = mt_makediff(raw,clean1)
D8      = D1.bicubicresize(ox/2-72,oy).blur(1,0).bicubicresize(ox,oy,1,0)
D9      = mt_lutxy(D1,D8,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
clean1a = clean1.mt_adddiff(D9,U=2,V=2)
clean1b = clean1a.frfun7(1.01,8,1)

allD    = mt_makediff(raw,clean1b).greyscale()
shrpD   = mt_makediff(clean1b,clean1b.removegrain(20))
DD      = shrpD.repair(allD,13,0).mt_lutxy(shrpD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
clean1c = clean1b.mt_lutxy(clean1a,"x 3 + y < x 2 + x y < x 1 + x 3 - y > x 2 - x y > x 1 - x ? ? ? ?",U=2,V=2)
                \.mt_adddiff(DD.sbr(),U=2,V=2)

raw     = raw.crop(4,0,-4,0,true)
clean1c = clean1c.crop(4,0,-4,0,true)

interleave(raw,clean1c)   # interleave "only-IVTC" with result of full filterchain, for frame comparison
# clean1c                 # just result
return(last)

#--- end of script ---#

#######################################################

#--- Helper functions below ---#

function Vinverse2(clip clp, float "sstr", int "amnt", int "uv")
{
uv   = default(uv,3)
sstr = default(sstr,2.7)
amnt = default(amnt,255)
uv2  = (uv==2) ? 1 : uv
STR  = string(sstr)
AMN  = string(amnt)
vblur  = clp.sbrV()
vblurD = mt_MakeDiff(clp,vblur,U=uv2,V=uv2)
Vshrp  = mt_LutXY(vblur,vblur.mt_convolution("1","1 2 1"),expr="x x y - "+STR+" * +",U=uv2,V=uv2)
VshrpD = mt_MakeDiff(Vshrp,vblur,U=uv2,V=uv2)
VlimD  = mt_LutXY(VshrpD,VblurD,expr="x 128 - y 128 - * 0 < x 128 - abs y 128 - abs < x y ? 128 - 0.25 * 128 + x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
mt_AddDiff(Vblur,VlimD,U=uv,V=uv)
(amnt>254) ? last : (amnt==0) ? clp : mt_LutXY(clp,last,expr="x "+AMN+" + y < x "+AMN+" + x "+AMN+" - y > x "+AMN+" - y ? ?",U=uv,V=uv) 
return(last)
}

function sbr(clip o) {
rg11=o.removegrain(11)
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.removegrain(11)).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2)
}

function sbrV(clip o) {
rg11=o.mt_convolution("1","1 2 1")
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.mt_convolution("1","1 2 1")).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2)
}
Apart from IVTC, all filtering in fact is purely spatial. Not any temporal filter is involved.

<-->

<-->

<-->

Sorry, for the screenshots I forgot to crop the 4 pixels left/right that are added in the script.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 23rd July 2012, 20:46   #63  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Oops Didée I pmed you beofre seing this. I thank you, I can't wait to test this tomorrow !
sirt is offline   Reply With Quote
Old 24th July 2012, 01:37   #64  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
Good job Didée! Can I add it to the composite artefacts package?
jmac698 is offline   Reply With Quote
Old 24th July 2012, 04:24   #65  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,173
sirt:
Now all you need is some derainbowing and you'll be good to go!

Didée:
The results look great. I tried it with a few different samples (anime and film) and this script seems to do a excellent job on all of them.
I'm truly amazed at what Avisynth can accomplish under the hands of a skilled operator. Thanks Didée!

I have a few questions, if you don't mind?
What are some of the key parameters that dictate the strength of the dot crawl detection and or removal?
Also, what does one have to do in order to be able to process progressive/interlaced NTSC and PAL? Removing TFM and TDecimate seems too easy.

Last edited by Reel.Deel; 24th July 2012 at 04:27.
Reel.Deel is offline   Reply With Quote
Old 24th July 2012, 08:54   #66  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Reel.Deel

Yes, In fact I've filtered that separetly in an adaptative way according to the scenes of the video.

Didée

Fantastic ! It's quite good. Thanks for your time and efforts spent on this.

I would like to thank everybody that helped me on this subject.

One question more remains pending...here Didée iVTCed my source and based his work on raw and clean1. But let's say I would like to apply the code on an AVI source as Input that has been ivtced, which means I would also use the script after iVTC like there, how could you do ? I've reflected about this but I'm lost, it looks like I will have to modify too many things. Do you have a suggestion ?
sirt is offline   Reply With Quote
Old 24th July 2012, 09:05   #67  |  Link
Ghitulescu
Registered User
 
Ghitulescu's Avatar
 
Join Date: Mar 2009
Location: Germany
Posts: 5,717
Quote:
Originally Posted by Didée View Post
Is it only me the one that doesn't see the full image?
__________________
Born in the USB (not USA)
Ghitulescu is offline   Reply With Quote
Old 24th July 2012, 09:57   #68  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
It was a link a image host
http://www.imagebam.com/image/303a44202846639
jmac698 is offline   Reply With Quote
Old 24th July 2012, 10:30   #69  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
@Ghitulescu
Don't know how it looks on your side, but for me at least, ImageBam surely shows the full images.
Just in case - those pics zipped.

@sirt
If a video already has been IVTC'ed, then everything is different. Remember
Quote:
Originally Posted by Didée
[from PM]
dotcrawl is not always the same, and very small differences in the characteristics can make the difference between "filter works" and "filter fails".
The posted script relies on a very certain symmetry being present in the dotcrawl's pixel-pattern. If IVTC already has been done beforehand, that symmetry has been destroyed. Those things that "naturally cancel-out each other" prior to IVTC, will not cancel-out anymore after IVTC.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 24th July 2012, 10:51   #70  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Quote:
Originally Posted by Didée View Post

@sirt
If a video already has been IVTC'ed, then everything is different. Remember


The posted script relies on a very certain symmetry being present in the dotcrawl's pixel-pattern. If IVTC already has been done beforehand, that symmetry has been destroyed. Those things that "naturally cancel-out each other" prior to IVTC, will not cancel-out anymore after IVTC.
Didée then you won't like what I will ask for...obviously I will use your script if I plan on using tfm. But, in the case I have an AVI source which is progressive (that has been ivtced upstream) or whatever the source, consider it had previously been iVTCEd. In this situation, do you think you can write a spatial script similar to the last one where you used my sample MPG source or is it totally irrelevant and requires too much time ? I know it is different, but yesterday you made some temporal script that applies on the iVTCed video, that's why I'm asking for a spatial script now.

I know it is somewhat twisted but I am also corecting patterns manually and I don't want to use an automatic tfm as you did.
sirt is offline   Reply With Quote
Old 24th July 2012, 12:18   #71  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,394
Whatever. If IVTC has been done already, then you left the field of certainty, and are in the swamp of vague possibilities.

Try FRFun7 on its own, this alone often snaps surprisingly nice onto dotcrawl. But, it always will also soften the picture ... propably it is indicated to shoot back with "shift-values-two-notches-back", or contrasharpen, or both together.

You can teach how to swing the racket, but not how to get the feeling for the ball.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 24th July 2012, 13:27   #72  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Thanks Didée, I'm giving it a try. It is really not bad when taking your time to adjust it !
sirt is offline   Reply With Quote
Old 24th July 2012, 13:32   #73  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,865
In your case, maybe you don't need automated IVTC at all. There could be one constant pattern throughout the whole video, so you can use that pattern and there will never be a mistake.
If the pattern changes, it's likely at scene changes when it was edited. A check could be made at those points.
jmac698 is offline   Reply With Quote
Old 24th July 2012, 13:39   #74  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Quote:
Originally Posted by jmac698 View Post
In your case, maybe you don't need automated IVTC at all. There could be one constant pattern throughout the whole video, so you can use that pattern and there will never be a mistake.
If the pattern changes, it's likely at scene changes when it was edited. A check could be made at those points.
jmac698, thanks for your advise. As part of my daily task, I am not neccessarly required to "manually ivtc" the input. The idea is to iVTC the video then test such and such filters. In this case, it turns out the pattern is indeed constant but even if it would not be I would not be compelled to solve it perfectly. I was just asking in case in fact, but using tfm is ample !
sirt is offline   Reply With Quote
Old 24th July 2012, 14:06   #75  |  Link
Ghitulescu
Registered User
 
Ghitulescu's Avatar
 
Join Date: Mar 2009
Location: Germany
Posts: 5,717
Quote:
Originally Posted by jmac698 View Post
I am talking about this


It happens to all images posted on imagebam.com.

Quote:
Originally Posted by Didée View Post
@Ghitulescu
Don't know how it looks on your side, but for me at least, ImageBam surely shows the full images.
Just in case - those pics zipped.
Thanks.
__________________
Born in the USB (not USA)
Ghitulescu is offline   Reply With Quote
Old 24th July 2012, 14:37   #76  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,040
Quote:
Originally Posted by Ghitulescu View Post
It happens to all images posted on imagebam.com.
The problem must be on your side. It seems everyone else can view the images.
Groucho2004 is offline   Reply With Quote
Old 24th July 2012, 16:29   #77  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,173
Hello, I have some examples of dot crawl in NTSC interlaced footage. QTGMC does a good job at removing most of it, but it still leaves subtle hints of the existing dot crawl. Using Didée's solution before QTGMC helps almost completely eliminate the remaining dot crawl. Here is the change I made to dot crawl script to make it work (at least I think) with the interlaced footage.

Code:
........
drr     = o.separatefields().vinverse2().weave()
both    = stackhorizontal(drr,o)
IVTC    = both # Removed TFM and TDecimate.
raw     = IVTC.crop(ox,0,ox,oy)
clean1  = IVTC.crop(0,0,ox,oy)
........
The rest of the script is the same and its followed by

QTGMC(preset="slower",EdiMode="NNEDI3", EdiThreads=2)
The results look fine to me but I was wondering if there's a better way to go about this?
Reel.Deel is offline   Reply With Quote
Old 24th July 2012, 17:14   #78  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Great. Would you like showing some frames comparison then ?
sirt is offline   Reply With Quote
Old 25th July 2012, 06:48   #79  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,173
So doing a little bit more testing I found out that simply just removing TFM and TDecimate does not work.
It seems as Vinverse2() does a partial vertical pixel shift. Processing the interlaced content as individual frames produced better results. Here's the comparison.
Code:
........
o=last.separatefields() ox=o.width() oy=o.height()

drr     = o.vinverse2() # Removed separatefields and weave
both    = stackhorizontal(drr,o)
IVTC    = both # Removed TFM and TDecimate.
........
The rest of the script is the same and it's followed by

QTGMC(preset="slower",EdiMode="NNEDI3", EdiThreads=2)
Reel.Deel is offline   Reply With Quote
Old 25th July 2012, 08:03   #80  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Even though Dot crawl + QTGMC solve much more things such as the baloons, I don't think using QTGMC is a good idea given that it is a deinterlacer. It echoes badly on man's shirt for example and on the bricks of the wall behind.
sirt is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:48.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.