Anyway to get rid of what appears to be cut & paste outlines?

[Socio]

I am trying to clean up a DVD and I get what appears to be cut and paste outlines through out the movie the studio did not bother to remove completely.

The are not a result of filter misconfiguration as they are in the before and after images below, they just become more pronounced after filtering, look at the ridges of the hills.

There is probably no way to completely correct it short of hand editing every single frame which would be nuts but perhaps someone has a trick to diminish them some what.

Original:
http://img253.imageshack.us/img253/7080/mountian1tw3.jpg

Filtered:
http://img253.imageshack.us/img253/5643/mountian2xa7.jpg

See Post #6
http://img442.imageshack.us/img442/7577/mountian3mh0.jpg

See Post #7
http://img509.imageshack.us/img509/6176/mountian5te7.jpg

This is the script I am using for image 2:

SetMemoryMax(96)
Import("C:\Program Files\AviSynth 2.5\plugins\Limitedsharpenfaster.avs")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT_MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMatrix.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\RepairSSE2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\FFT3DFilter.dll")
SetMTMode(2)
MT("FFT3DFILTER(sigma=1.5, ncpu=4, bw=14, bh=14, ow=7, oh=7, bt=2, dehalo=1, hr=1, sharpen=0.7)
LimitedSharpenFaster(ss_x=2.0,ss_y=2.0,Smode=3,lmode=2,strength=60,wide=true, overshoot=0)")
ColorMatrix()

[TheRyuu]

Try this:
MT("
FFT3DFILTER(sigma=1.5, ncpu=4, bw=32, bh=32, ow=16, oh=16, bt=3)
LimitedSharpenFaster(ss_x=1.5,ss_y=1.5,Smode=3,strength=100)
",4,8) #assuming you have a quad core

Your "halos" are being caused by oversharpening. Try the above code. You can also try seesaw. If the halos are in the original as well, give dehalo_alpha a shot.

[Chainmax]

That looks like plain haloing. I can hardly see any in the source pic, but what appears on the filtered one seems like classic oversharpening. Which makes sense, since you are using two sharpening stages on your script. I'd recommend setting FFT3DFilter's sharpening to 0 (which would have the added benefit of letting its mild built-in dehaloing do its work) and then adjust LSF's strenght if you want. Also, ColorMatrix is supposed to the the very first filter in a filterchain, right after source loading. Also, I'm not sure about it, but I think it doesn't play well with MT.


[edit]beaten to the punch by wizboy11, his post wasn't there when I started writing.

[Socio]

Try this:
MT("
FFT3DFILTER(sigma=1.5, ncpu=4, bw=32, bh=32, ow=16, oh=16, bt=3)
LimitedSharpenFaster(ss_x=1.5,ss_y=1.5,Smode=3,strength=100)
",4,8) #assuming you have a quad core

Your "halos" are being caused by oversharpening. Try the above code. You can also try seesaw. If the halos are in the original as well, give dehalo_alpha a shot.

Thanks I will give both the script and Dehalo_Alpha a shot because if you look close they are in the original just faint compared to the filtered one.

I have a duel processor with HT so it is a virtual quad.

[Socio]

That looks like plain haloing. I can hardly see any in the source pic, but what appears on the filtered one seems like classic oversharpening. Which makes sense, since you are using two sharpening stages on your script. I'd recommend setting FFT3DFilter's sharpening to 0 (which would have the added benefit of letting its mild built-in dehaloing do its work) and then adjust LSF's strenght if you want. Also, ColorMatrix is supposed to the the very first filter in a filterchain, right after source loading. Also, I'm not sure about it, but I think it doesn't play well with MT.


[edit]beaten to the punch by wizboy11, his post wasn't there when I started writing.

I put ColorMatrix at the end to fix the color as a last step or else color tends to be a little off and yes I believe you are right about MT that is why I have ColorMatrix separate from the MT call.

[Socio]

I added a third pic in the first post, I had to switch to SeeSaw and use DeHalo_Alpha to get rid of some of the halos, unfortunately Dehalo_Alpha does cost some image detail to use.

Here is the script I am using for image 3:

SetMemoryMax(96)
Import("C:\Program Files\AviSynth 2.5\plugins\SeeSaw.avs")
Import("C:\Program Files\AviSynth 2.5\plugins\DeHalo_Alpha.avs")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT_MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMatrix.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\RepairSSE2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\FFT3DFilter.dll")
SetMTMode(2)
MT("DeHalo_alpha()
FFT3DFILTER(sigma=1.5, ncpu=4, bw=32, bh=32, ow=16, oh=16, bt=3, dehalo=1, hr=1,sharpen=0)
SeeSaw(NRlimit=0, NRlimit2=0, Sstr=2.5, Szp=15, Slimit=16, Spower=2, bias=80, sootheT=0, Sdamplo=10, SdampHi=10)")
ColorMatrix()

Thanks Chain and Wiz

FYI: The script in first post took 613 minutes to process a 2:07 hr movie this script to 1262 minutes for the same movie however turned out to be one of the best processed movies I have done to date. It is on par with IIP but my last one with IIP took 7 days to process so I am going to stick with SeeSaw/FFTD/DeHalo for now and tweak as needed.

[Socio]

After seeing videoFred's results with MVDegrain in this thread;

http://forum.doom9.org/showthread.php?t=135849

And after close scrutiny of the processed movie, seeing some over sharpening in some areas and a tad to much denoising in some others I decided to experiment with MVDegrain myself and see what affect it has and this is what I came up with.

SetMemoryMax(96)
Import("C:\Program Files\AviSynth 2.5\plugins\SeeSaw.avs")
Import("C:\Program Files\AviSynth 2.5\plugins\DeHalo_Alpha.avs")
Import("C:\Program Files\AviSynth 2.5\plugins\antialiasing.avs")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT_MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MaskTools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\ColorMatrix.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\RepairSSE2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\SangNom.dll"
global idx1 = 10
SetMTMode(2)
MT("DeHalo_alpha()
idx1 = idx1 + 2
backward_vec3 = MVAnalyse(isb = true, delta = 3, pel = 2, overlap=4, sharp=1, idx = idx1)
backward_vec2 = MVAnalyse(isb = true, delta = 2, pel = 2, overlap=4, sharp=1, idx = idx1)
backward_vec1 = MVAnalyse(isb = true, delta = 1, pel = 2, overlap=4, sharp=1, idx = idx1)
forward_vec1 = MVAnalyse(isb = false, delta = 1, pel = 2, overlap=4, sharp=1, idx = idx1)
forward_vec2 = MVAnalyse(isb = false, delta = 2, pel = 2, overlap=4, sharp=1, idx = idx1)
forward_vec3 = MVAnalyse(isb = false, delta = 3, pel = 2, overlap=4, sharp=1, idx = idx1)
last.MVDegrain3(backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3, thSAD=400,idx=idx1)
SeeSaw(NRlimit=0, NRlimit2=0, Sstr=2.2, Szp=15, Slimit=16, Spower=1, bias=60, sootheT=0, Sdamplo=8, SdampHi=16)",2)
antialiasing()
Colormatrix()

I was able to turn down SeeSaw's sharpening down considerably and achieve cleaner denoising while attaining crisp sharpened detail like that of LSF with halos so minuscule you would need a magnifying glass to see any.

I am using the MT example in the MVtools documentation for which I modified for MVDegrain3 however I am not sure it is a 100% correct.

I assumed a setting of idx1 = idx1 + 2 would give me three levels of saved data for faster searches and setting delta = 3 would give me 3 levels to search thus make use of the three levels of saved data. I could be wrong, so if I am wrong anyone wishing to try script might not be using the most accurate or fastest way of doing this.

Even if I am wrong the results are phenomenal as seen in the fourth picture added in the first post.

Note: It does take longer to process, it took 1774 minutes to process a 2:07 movie with this script, still it is a lot faster than IIP and like I said the results rock!