Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 29th August 2012, 10:11   #1  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Moving dots I cant seem to get rid of

Ive seen and fixed plenty of dot crawl in sources but this really takes the cake at being a pain in the ass.

Perhaps its not dot crawl or something Ive not yet come across?

These lines during playback seem to have plenty of moving dots running along the lines. It looks pretty much like dot crawl to me but seems a little different.

I used the dot crawl filter checkmate() and an antialiaser and they still wont go away. They seem to be around a bit less afterwards though.

Its several times more noticeable in the DVD video source.

any suggestion?

Heres a sample vob clip unaltered video. I used DGIndex and saved project + demuxed video. Its 1 minute 28 seconds long.

They show up alot once it gets to around the 1 minute point and you see the guy wearing the red jacket. They show up like that in various places all through the entire video and DVD.

http://glumbouploads.com/0qoh16vn3j9p.html

Here are a few screenshots of the source after I have filtered it with checkmate() and an antialiaser.








Last edited by DarkDream; 29th August 2012 at 12:36.
DarkDream is offline  
Old 29th August 2012, 11:02   #2  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
Screenshots of a filtered video are almost useless to make a diagnostic. Please post an untouched sample of the DVD VOB.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline  
Old 29th August 2012, 11:09   #3  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Quote:
Originally Posted by cretindesalpes View Post
Screenshots of a filtered video are almost useless to make a diagnostic. Please post an untouched sample of the DVD VOB.
Its been so long since I needed to post an unaltered sample that I forget what I used to use.

was it dgindex?

how can I get you a vob file from the dvd without it being over 1GB?
DarkDream is offline  
Old 29th August 2012, 11:14   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,742
Yes, just mark with the sliders in DGIndex.

EDIT: Then "Save Project".
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 29th August 2012 at 11:16.
StainlessS is offline  
Old 29th August 2012, 11:19   #5  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Quote:
Originally Posted by StainlessS View Post
Yes, just mark with the sliders in DGIndex.

EDIT: Then "Save Project".
Alrighty, Ill work on getting the sample up asap if I can figure it out again. In the meantime heres a couple screenshots of the video unaltered coming from the original DVD. I hope these can help some. The moving dots are way worse before I filtered it.





DarkDream is offline  
Old 29th August 2012, 11:40   #6  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Ok heres the sample clip requested. I used DGIndex and saved project + demuxed video. Its 1 minute 28 seconds long.

They show up alot once it gets to around the 1 minute point and you see the guy wearing the red jacket. They show up like that in various places all through the entire video and DVD.

http://glumbouploads.com/0qoh16vn3j9p.html

Last edited by DarkDream; 29th August 2012 at 11:50.
DarkDream is offline  
Old 29th August 2012, 14:26   #7  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
A shorter 30 second clip can be found here

http://glumbouploads.com/jq5qwkb0oplb.html
DarkDream is offline  
Old 29th August 2012, 15:25   #8  |  Link
librarian
Registered User
 
Join Date: Nov 2011
Posts: 63
Try this script:
Code:
Mpeg2Source("C:\Script Avisynth\Sample\Sample\VTS_01_1.demuxed.d2v")

telecide()
decimate()
o=last
spline36resize(640,480)
removegrain(19,22)
bicubicresize(720,480)
awarpsharp2(128,chroma=3)
contrasharpening(last,o)
return last
 


function ContraSharpening (clip denoised, clip original)
{
s = denoised.minblur(1,1) # Damp down remaining spots of the denoised clip.
allD = mt_makediff(original,denoised) # The difference achieved by the denoising.
ssD = mt_makediff(s,s.removegrain(11,-1)) # The difference of a simple kernel blur.
ssDD = ssD.repair(allD,1) # Limit the difference to the max of what the denoising removed locally.
ssDD = SSDD.mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?") # abs(diff) after limiting may not be bigger than before.

denoised.mt_adddiff(ssDD,U=2,V=2) # Apply the limited difference. (Sharpening is just inverse blurring.)

return( last )
}
# Nifty Gauss/Median combination
function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200


RG11D = (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}
librarian is offline  
Old 29th August 2012, 15:31   #9  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Thanks for the script, Ill give it a try later today and post about how it worked.

Judging by the script you've given, it has something to do with grain and removing grain to get rid of those dots?

Is there a name or term for what that is in the video?
DarkDream is offline  
Old 29th August 2012, 15:39   #10  |  Link
librarian
Registered User
 
Join Date: Nov 2011
Posts: 63
Quote:
Originally Posted by DarkDream View Post
Thanks for the script, Ill give it a try later today and post about how it worked.

Judging by the script you've given, it has something to do with grain and removing grain to get rid of those dots?

Is there a name or term for what that is in the video?
Removegrain doesn't remove grain only (with mode from 13 to 16 it is a deinterlacer!). I used RG mode 19 and 22 (blur and median blur), then contrasharpening to limit the blurring.
Sorry I dont know the name of this particular artifact.
librarian is offline  
Old 29th August 2012, 15:54   #11  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Quote:
Originally Posted by librarian View Post
Try this script:
Code:
Mpeg2Source("C:\Script Avisynth\Sample\Sample\VTS_01_1.demuxed.d2v")

telecide()
decimate()
o=last
spline36resize(640,480)
removegrain(19,22)
bicubicresize(720,480)
awarpsharp2(128,chroma=3)
contrasharpening(last,o)
return last
 


function ContraSharpening (clip denoised, clip original)
{
s = denoised.minblur(1,1) # Damp down remaining spots of the denoised clip.
allD = mt_makediff(original,denoised) # The difference achieved by the denoising.
ssD = mt_makediff(s,s.removegrain(11,-1)) # The difference of a simple kernel blur.
ssDD = ssD.repair(allD,1) # Limit the difference to the max of what the denoising removed locally.
ssDD = SSDD.mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?") # abs(diff) after limiting may not be bigger than before.

denoised.mt_adddiff(ssDD,U=2,V=2) # Apply the limited difference. (Sharpening is just inverse blurring.)

return( last )
}
# Nifty Gauss/Median combination
function MinBlur(clip clp, int r, int "uv")
{
uv = default(uv,3)
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200


RG11D = (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
clp.mt_makediff(DD,U=uv,V=uv)
return(last)
}




It blurred the video terribly and made alot of the black lines very thin making it look terrible in comparison.

The ending result makes the video look like shit to where its far from acceptable in quality but it did get rid of the dots.
DarkDream is offline  
Old 29th August 2012, 16:45   #12  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,106
Hello DarkDream, this maybe overkill but you may want to try this. It's slow but is does seem to get rid of *almost* all artifacts. The orginal post of this script can be found here.

Code:
# Required plugins:
# -----------------
# dgdecode.dll
# TIVTC.dll
# mt_masktools.dll
# removegrain.dll
# repair.dll
# frfun7.dll
# checkmate.dll

mpeg2source("VTS_01_1.demuxed.d2v")

checkmate()

borders = bicubicresize(width()/2,height()/2).bicubicresize(width()+8,height())
stackhorizontal(borders.crop(0,0,4,0),last,borders.crop(width()+4,0,0,0))

o=last ox=o.width() oy=o.height()

drr     = o.separatefields().vinverse2().weave()
both    = stackhorizontal(drr,o)
IVTC    = both.tfm(pp=0,clip2=stackhorizontal(drr,drr)).tdecimate(mode=1)
raw     = IVTC.crop(ox,0,ox,oy)
clean1  = IVTC.crop(0,0,ox,oy)

D1      = mt_makediff(raw,clean1)
D8      = D1.bicubicresize(ox/2-72,oy).blur(1,0).bicubicresize(ox,oy,1,0)
D9      = mt_lutxy(D1,D8,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
clean1a = clean1.mt_adddiff(D9,U=2,V=2)
clean1b = clean1a.frfun7(1.01,8,1)

allD    = mt_makediff(raw,clean1b).greyscale()
shrpD   = mt_makediff(clean1b,clean1b.removegrain(20))
DD      = shrpD.repair(allD,13,0).mt_lutxy(shrpD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
clean1c = clean1b.mt_lutxy(clean1a,"x 3 + y < x 2 + x y < x 1 + x 3 - y > x 2 - x y > x 1 - x ? ? ? ?",U=2,V=2)
                \.mt_adddiff(DD.sbr(),U=2,V=2)

raw     = raw.crop(4,0,-4,0,true)
clean1c = clean1c.crop(4,0,-4,0,true)

#interleave(raw,clean1c)   # interleave "only-IVTC" with result of full filterchain, for frame comparison
# clean1c                 # just result
return(clean1c)

#--- end of script ---#

#######################################################

#--- Helper functions below ---#

function Vinverse2(clip clp, float "sstr", int "amnt", int "uv")
{
uv   = default(uv,3)
sstr = default(sstr,2.7)
amnt = default(amnt,255)
uv2  = (uv==2) ? 1 : uv
STR  = string(sstr)
AMN  = string(amnt)
vblur  = clp.sbrV()
vblurD = mt_MakeDiff(clp,vblur,U=uv2,V=uv2)
Vshrp  = mt_LutXY(vblur,vblur.mt_convolution("1","1 2 1"),expr="x x y - "+STR+" * +",U=uv2,V=uv2)
VshrpD = mt_MakeDiff(Vshrp,vblur,U=uv2,V=uv2)
VlimD  = mt_LutXY(VshrpD,VblurD,expr="x 128 - y 128 - * 0 < x 128 - abs y 128 - abs < x y ? 128 - 0.25 * 128 + x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
mt_AddDiff(Vblur,VlimD,U=uv,V=uv)
(amnt>254) ? last : (amnt==0) ? clp : mt_LutXY(clp,last,expr="x "+AMN+" + y < x "+AMN+" + x "+AMN+" - y > x "+AMN+" - y ? ?",U=uv,V=uv) 
return(last)
}

function sbr(clip o) {
rg11=o.removegrain(11)
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.removegrain(11)).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2)
}

function sbrV(clip o) {
rg11=o.mt_convolution("1","1 2 1")
rg11D=mt_makediff(o,rg11)
rg11DD=mt_makediff(rg11D,rg11D.mt_convolution("1","1 2 1")).mt_lutxy(rg11D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
o.mt_makediff(rg11DD,U=2,V=2)
}
Reel.Deel is offline  
Old 30th August 2012, 01:03   #13  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,124
well the basic concept of removing the dots is, mask out all the lines(edges) in the image, and then apply blur to the lines to a point where the dots blend into a line, and then merge the mask back to the image.
Unfortunately, the edge masking filter is not smart enough to filter only the line with dots, so you'll end up with a blurrier image overall, with the "good" areas also blurred (smoothed) out.
lansing is offline  
Old 30th August 2012, 08:48   #14  |  Link
sirt
x264 fan
 
sirt's Avatar
 
Join Date: Feb 2011
Location: In the trap
Posts: 458
Do you think you can make it "smarter" in this case ?
sirt is offline  
Old 30th August 2012, 12:33   #15  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
None of the above scripts worked well enough. They got rid of the dots but it made the video look like shit so I played around with my own combination of things and didnt even bother with the above submissions.

Actually, I finally got it quite well and I didnt even have to ruin much if anything that I can see. I dont notice anything going bad about it really. I didnt need any masking that I know of and I didnt really blur the image.

Heres some comparison screenshots.

(This will be much easier for you to see changes if you save all the pictures and then flip back and forth between them on your computer one right after another. Then you can see the changes easily because the picture wont be moving and will be switching back and forth between them)


TOP = My original encode that I posted shots of with dots
BOTTOM = The one I just did last night that I was going to see how it turned out


Ok in this first example, 99% of the dots on the shoulders are gone and I didnt really notice anything ruined about it. The new one even looks a bit sharper I think. If you look around the eyes, it looks like the new one might have darkened/thickened black lines in some areas of the video a tiny bit but no one will really notice during playback unless they had the original footage to compare to it. I could barely tell in the screenshots.






Thats much better than the original footage as you can see





In this next example again 99% of the dots are gone and I dont really see anything gone bad yet. Again I think I see black lines barely getting thicker/darker in some spots but without footage to compare and being picky I dont really think it means too much or is a bad thing. The only real change I notice between the two besides the dots gone is the bottom left corner of that opened door when I flip back and forth between the two quickly.




Last edited by DarkDream; 30th August 2012 at 12:37.
DarkDream is offline  
Old 30th August 2012, 13:02   #16  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
So what was your script that produced these results?
Guest is offline  
Old 30th August 2012, 13:06   #17  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Quote:
Originally Posted by neuron2 View Post
So what was your script that produced these results?
I added in

checkmate(tthr2=0) ------- [[[did (tthr2=0) so artifacts wouldnt show up during rapid movement]]]

QTGMC(preset="Slow", InputType=1) ------- [[[This is what I added in last night that took away the rest of the dots with ease]]]

and I added an antialiaser of my choice, a denoiser of my choice, and a sharpener of my choice at the end to get rid of the rest of the dots and to sharpen the image.

The antialiaser and QTGMC are what got rid of them completely. In my first encode I only had the antialiaser and checkmate without QTGMC. They reduced them alot but didnt fully get rid of them alone.
DarkDream is offline  
Old 30th August 2012, 14:17   #18  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Can you please post the whole script for us?
Guest is offline  
Old 30th August 2012, 15:07   #19  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Quote:
Originally Posted by neuron2 View Post
Can you please post the whole script for us?

LoadPlugin("C:\Program Files (x86)\megui\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("C:\Documents and Settings\Administrator\Desktop\Zoids New Century Zero\Disc 1\Episode 1\VTS_01_1.d2v", info=3)
LoadPlugin("C:\Program Files (x86)\megui\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, interlaced=true, threads=0)
checkmate(tthr2=0)
LoadPlugin("C:\Program Files (x86)\megui\tools\avisynth_plugin\TIVTC.dll")
tfm(order=-1).tdecimate(mode=1,hybrid=1)
QTGMC(preset="Slow", InputType=1)
crop( 8, 0, -8, 0)
LanczosResize(640,480) # Lanczos (Sharp)
LoadPlugin("C:\Program Files (x86)\megui\tools\avisynth_plugin\UnDot.dll")
Undot() # Minimal Noise
[[[denoiser of my choice]]]
[[[antialiaser of my choice]]]
[[[sharpener of my choice]]]


thats the entire thing
DarkDream is offline  
Old 30th August 2012, 15:48   #20  |  Link
DarkDream
Banned
 
Join Date: Jul 2011
Posts: 27
Quote:
Originally Posted by neuron2 View Post
Can you please post the whole script for us?


DGDecode_mpeg2source("C:\Documents and Settings\Administrator\Desktop\Zoids New Century Zero\Disc 1\Episode 1\VTS_01_1.d2v", info=3)
ColorMatrix(hints=true, interlaced=true, threads=0)
checkmate(tthr2=0)
tfm(order=-1).tdecimate(mode=1,hybrid=1)
QTGMC(preset="Slow", InputType=1)
crop( 8, 0, -8, 0)
LanczosResize(640,480) # Lanczos (Sharp)
Undot() # Minimal Noise
[[[denoiser of my choice]]]
[[[antialiaser of my choice]]]
[[[sharpener of my choice]]]


thats the entire thing
DarkDream is offline  
Closed Thread

Tags
aliasing, checkmate, crawl, dot, moving

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 01:25.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.