Moire effect + scratches

[franketto]

I have an NTSC DVD source with a Moirè effect - I think it's called so - and some film scratches. I must make a PAL one.

I tried:

DGDecode_mpeg2source("VTS_01_1.d2v")
tfm(mode=5).tdecimate().assumefps(25)

and it works, but sometimes I get some sparse points like this:
http://i.imgur.com/ps39gHz.png
frame 46 ok, 47--> sparse points

The moirè (48) and scratches (49):
http://i.imgur.com/BG01dl1.png


Could you help me to make it better?


Original vob:
http://www.sendspace.com/file/77j9vo

[feisty2]

its an artifacts caused by a low quality deinterlacer, and it's called aliasing
u can weaken the artifacts by using a higher quality deinterlacer but no spacial deinterlacer is free from this artifacts

[franketto]

I tried also using tdeint with:
tfm(mode=5, clip2=tdeint(mode=0) )

but it's worse than the only tfm.

EDIT
Using Telecide(guide=1) alone, a lot worse.

So, what other deinterlacers do you suggest?

[feisty2]

I tried also using tdeint with:

but it's worse than the only tfm.

EDIT
Using Telecide(guide=1) alone, a lot worse.

So, what other deinterlacers do you suggest?

try tfm (mode=3,slow=2,pp=0) first
tfm tends to wrongly recognize high frequency patterns like this as interlaced signs

[franketto]

I tried.
It's the same.

[colours]

Merely specifying pp=0 in TFM fixes everything other than the film defects. Postprocessing turns out to be more harmful than useful in your source because TFM misdetects the striped shirt as being combing artifacts (which is what feisty2 already said) and flagging it for postprocessing.

TFM won't fix film defects; the Avisynth wiki lists a couple of plugins that'll handle those which you can try out.

[franketto]

Just to clarify:
In this clip we have 3 problems:
1) the IVTC gives some sparse spots due to high freq details wrongly interpreted as combing (frame 47)
2) the lines sometimes give moirè effect (frame 48)
3) there are some scratches, dots and so on (frame 49)

We started to talk about 1).
I used:

telecide
tfm
tfm + tdeint
I also used Srestore with NNEDI3 (with QTGMC) and it's great respect tdeint.
But comparing NNEDI3+srestore with tfm there's half resolution and in these high freq frames is very visible. EDIT: Even if with Srestore I get no sparse pixels...

So the better IVTC I found on this clip is tfm.
But sometimes it creates those sparse points.

So the question about 1) is:
is there some other deinterlacer to use in combo with tfm to perform a better IVTC? Maybe a temporal one (I never tried one, so I'm open to suggestions)?
Or also a full frame bob (instead of NNEDI3 that's based on only 1 field) with Srestore (I tried yadif and tdeint, neither are good respect tfm).

After solving 1) I will pass on 2-3.

[franketto]

I'm sorry, you're right!
I had 2 avs with the same name but different folders opened in the editor and I changed the line in that not loaded in VD...

tfm( pp=0 )


solves the sparse points, perfect. Thank you feisty2.


So, now the point 2), the moirè.
I get that flickering effect on some frames with that dress. Sometimes there is also chroma rainbow.

What could I use?

[franketto]

I uploaded an example with some scenes of what I get after

tfm(pp=0).tdecimate().assumeFps(25)

Chroma Raimbow and moirè flicker:

http://www.sendspace.com/file/5xa3hy

[bxyhxyh]

TFM(cthresh=12) works ok for your original sample. For whole video, you'll maybe want different values.

So try tweak blocky, blockx and cthresh arguments.

[franketto]

Thank you bxyhxyh.

It's true that you can keep pp to something else from 0 adjusting cthresh. Good catch.
But the results are similar/equal to pp=0.
In the original vob I posted, they work ok, but in other scenes, such as those I posted in the previous post, both produce moirè and chroma rainbow. Anyway the sparse points nomore.

This is a vob clip with moirè [it remains after tfm( pp=0) ]:
http://www.sendspace.com/file/on685f

The problem is that the fields have moire pattern, so maybe it's impossible to repair that...


I read this thread:
http://forum.doom9.org/showthread.php?t=166363

and cretindesalpes made an excellent script: see the result!
But the original is no more available, so we can't know if all fields were moirè patterned or only the resulting frames.

Anyway I tried his script:

MPEG2Source ("moire.d2v")

Crop (0, 104, 0, 364)
o = last

AssumeFrameBased ().AssumeTFF ()
SeparateFields ().SelectOdd ()

a = Blur (0, 1).Blur (0, 1)
a = a.TemporalSoften (4, 30, 30, 15, 2)
a = a.BicubicResize (o.Width (), o.Height (), 1, 0)
sup_a = a.MSuper (chroma=false)
vec = sup_a.MAnalyse (chroma=false, multi=true, delta=3, blksize=16, overlap=8, search=5, searchparam=4, DCT=5)

nnedi3 (field=0, dh=true, nns=4, U=false, V=false)
sup = MSuper (levels=1, chroma=false)
premdg = last
MDegrainN (sup, vec, 3, thSAD=1000)

rst = 1 # Increase this if you want to restore more grain
dif = mt_makediff (last, premdg)
difr = dif.mt_lut ("x 128 - abs "+String(rst)+" - 0 max x 128 < -1 1 ? * 128 +")
premdg.mt_adddiff (difr, u=-128, v=-128)

# Display and comparison
Interleave (o, last)
PointResize (o.Width (), 358, src_top=3, src_height=358)
StackVertical (SelectEven ().Subtitle ("original"), SelectOdd ().Subtitle ("filtered"))

But I can't find MDegrainN....
I have Mdegrain1-3 but not that.
I saw it could be in SMDegrain, but in that I downloaded there isn't. Do you know where?

[foxyshadis]

MDegrainN is in cretindesalpes's custom mvtools2 (http://forum.doom9.org/showthread.php?p=1386559#post1386559). It might be dependent on Dither, I'm not sure.

Since the moire is already burned into the image, the only way you can get the detail back is the way you're doing it, using parts of surrounding good frames to replace the bad. You have to watch out that you don't end up wrecking good frames, though.

[franketto]

I had no time to test, now I tried (I had to slightly modify it because it was for BW, even if I don't know if I made it correct):


source = DGDecode_mpeg2source("....d2v")
last = source
o = source

SeparateFields ().SelectOdd ()

a = Blur (0, 1).Blur (0, 1)
a = a.TemporalSoften (4, 30, 30, 15, 2)
a = a.BicubicResize (o.Width (), o.Height (), 1, 0)
sup_a = a.MSuper (chroma=false)
vec = sup_a.MAnalyse (chroma=false, multi=true, delta=3, blksize=16, overlap=8, search=5, searchparam=4, DCT=5)

nnedi3 (field=0, dh=true, nns=4) # -------Modified: , U=false, V=false)
sup = MSuper (levels=1, chroma=false)
premdg = last
# return last
MDegrainN (sup, vec, 3, thSAD=1000)


rst = 1 # Increase this if you want to restore more grain
dif = mt_makediff (last, premdg)

difr = dif.mt_lut ("x 128 - abs "+String(rst)+" - 0 max x 128 < -1 1 ? * 128 +")

premdg.mt_adddiff (difr) # -------Modified: , u=-128, v=-128)
# Display and comparison
Interleave (o, last)
PointResize (o.Width (), 358, src_top=3, src_height=358)
return StackVertical (SelectEven ().Subtitle ("original"), SelectOdd ().Subtitle ("filtered"))


http://i.imgur.com/cUcLm6N.png

So, nnedi3 seems not to work well with this source.
It's a pity because it was astonishing result in the original post: maybe there was better fields, but it doesn't seem... (even if these are taken from the final result, not the original one) :
http://i.imgur.com/HugOmbX.png

[ChiDragon]

Maybe just me, but I don't understand what you're trying to fix in the most recent post. The jacket in "original" looks as good as I would expect any NTSC representation of it to look.

The second VOB, "moire.VOB", is too much of a challenge for TFM. It matches the wrong fields. You can try a manual IVTC (http://avisynth.org.ru/docs/english/corefilters/showfive.htm) and hope that the pattern remains consistent for the entire film.

But that still leaves random moiré that pops out intermittently to annoy our eyes.
http://s13.postimg.org/67vmcpcuf/glitchyjacket.png

[franketto]

Yes, you're right, the jacket is ok in that frame, but the problem using that script is that ruins also a good frame! Imagine a moirè one... See the loss of resolution and moirè in the filtered pic.
And, more, doesn't correct the real moirè frames.
So it doesn't work at all... with this source, because as I posted, with the other BW source was so good.

I suspect nnedi3 can't reproduce correctly those lines because they are too much thin and near each other: infact it uses only a field to reconstruct, so it halves the resolution. Just a supposition.
I tried also nnedi3 with Srestore and the result is very similar: it's very visible the loss of resolution when there are these frames with thin lines (and it doesn't solve moirè).

Here I was asking experts if there could be a solution, otherwise I left it as is and peace.
I doubt a manual IVTC will work because in other scenes is real 3:2, so it's not consistent. Infact the problem is it confuses interlaced frames with the thin lines moirè.

BTW:
I solved the chroma rainbow and dirt removal. The only and latest thing is this moirè...
Maybe we can consider it a feature (of old films), not a bug ;)

[Reel.Deel]

I read this thread:
http://forum.doom9.org/showthread.php?t=166363

and cretindesalpes made an excellent script: see the result!
But the original is no more available, so we can't know if all fields were moirè patterned or only the resulting frames.


Here you go: moire.vob (https://dl.dropbox.com/s/ce1pnxkczj1usdw/moire.vob)
The original sample clip from that thread. I save sample clips from time to time. :)

[franketto]

Thank you Reel.Deel!
In every case the original vob had the same fields as the filtered file: bad fields. Here nnedi3 maked a good reconstruction demoirè.

In my source instead, no :(

[feisty2]

Thank you Reel.Deel!
In every case the original vob had the same fields as the filtered file: bad fields. Here nnedi3 maked a good reconstruction demoirè.

In my source instead, no :(

u need eedi3 (sclip=nnedi3 ())

[franketto]

I tried this inside the previous script:
eedi3 (dh=true, sclip= nnedi3 (nns=4, field=0,dh=true) )


http://i.imgur.com/nrj1i10.png


nnedi3 changes also the angle of lines, that's not good:
http://i.imgur.com/aXjzSFx.png