Deinterlacing Hard Telecine Animation

[Dogway]

Once in a while I stumble upon one of these sources (http://www.mediafire.com/?2z2m562cyq2w837)(11mb); they look like blended telecine. By following the guidelines of manono's post (http://forum.doom9.org/showthread.php?p=1116168#post1116168) I found that it is in fact hard telecine, as dgindex flags it as interlaced and there is a clear 3:2 pulldown, so it can be ivtc'ed. But I couldn't ivtc it, output has lots of combing and blending.

What is rare is when fields are inspected, each one is different (field,blendedfield,...) in this pattern f,f,Bf,f,Bf
what I normally do is bob deinterlace and srestore, and it seems to work, but I would like to know if this is correct because by deinterlacing I'd be harming the source too much in case it's not necessary. And related to deinterlacing I'm very interested on what would be your take on this since QTGMC seems not to be appropiate (http://forum.doom9.org/showthread.php?p=1478442#post1478442) for not pure interlaced content, and much less anime, maybe MCBob or is it in the same league? tdeint? Or can this hard telecine really be fixed by using tfm? (I never got to understand its parameters beyond "order=")

what I can do is take the unblended fields:
SeparateFields()
selectevery(5,0,2,3)
by 2 being a dup of 3 I can drop it and I get 23.976fps output:
selectevery(5,0,3)
But these are fields so... now what? (they shimmer a lot)

[Guest]

Once in a while I stumble upon one of these sources (http://www.mediafire.com/?2z2m562cyq2w837)(11mb) Where did you stumble upon that source?

[Dogway]

Where did you stumble upon that source?
is:
bob deinterlace
srestore(23.976)
OK?

and if so, which bob deinterlacer? QTGMC is not suited for not pure interlaced content, and much less animation. I'm not sure if this is also true for MCBob and tdeint. Yadif gives me horrible results.

[Overdrive80]

Maybe you could use animeivtc.

[Guest]

[bump after re-open]

[Dogway]

http://img41.imageshack.us/img41/9697/neuron2.png

Ok now?

[librarian]

Deblending isn't too difficult:

MPEG2SOURCE("VTS_05rrr.demuxed.D2V")
blur(0,1)
Srestore(omode="pp2")
Tdecimate()

But motion is really jerky.

[Overdrive80]

animeivtc(mode=1,aa=0,killcomb=2, blend=true)

[Dogway]

Thanks overdrive80.
That code is not working for me. Lots of blending and artifacts.

librarian's code kinda works, but I used vinverse instead of blur(0,1). Still it has a fade of blending.

animeivtc (http://www.mediafire.com/?x5ezlwur1aghvid)(50mb) | srestore(omode="pp2") (http://www.mediafire.com/?dcf5d9ds4uyv7p4)(50mb) | qtgmc (http://www.mediafire.com/?eca8yd8wriv6yda)(50mb)

Deblending wise, the best I get is the qtgmc route:
qtgmc(thsad1=300,blocksize=8,TR0=1,TR1=1,TR2=0,lsb=true,chromamotion=false,edithreads=2)
Srestore(23.976)

But I might be doing it wrong by deinterlacing a source that doesn't need to be deinterlaced, besides QTGMC is not suited for animation nor not pure interlaced content (http://forum.doom9.org/showthread.php?p=1478442#post1478442). MCBob is a "dinosaur (http://forum.doom9.org/showthread.php?p=1428634#post1428634)", and tdeint... I don't know, it's too old and nobody have stated whether it works for animation.

[librarian]

Yes, QTGMC is not suited for IVTC or blending(see also this specific thread (http://forum.doom9.org/showthread.php?t=161991)). Tdeint is a good (and very tweakable) deinterlacer, for "normal" animation (it is usually telecined, not interlaced) a field matcher (Telecide, TFM) is the right tool. Srestore, at the end of the day, is a fieldmatcher.
As to blur(0,1), in my script, it is used to cause a damage (double blending) that Srestore(omode="pp2") can recover (with bobber+Srestore I didn't get a good output). For this specific use - IMHO - a vertical blur is a better choice than Vinverse (I think there is a post of Didée about this blur(), but I cannot find it now).
As to doing wrong, set aside the technicalities, there is only one final judge: our taste. What we like better is the perfect solution. Moreover there is a Murphy's Law: as soon as a person finishes a long and difficult encode this person will find a better solution:-)

[Sapo84]

Loaded in YATTA.
Patterne Guidance actually finds a pattern, but that doesn't solve the real problem.
Still, I think the blendings have somewhat of a pattern.

This
DGDecode_Mpeg2Source("D:\VTS_05rrr.demuxed.d2v")
FieldHint(ovr="D:\VTS_05rrr.demuxed.d2v.fh.txt")
TelecideHints(NNEDI3())
#pattern is NccnnNcc...

range(2,2,nnedi3(0))
range(7,7,nnedi3(0))
range(12,12,nnedi3(0))
range(4,4,nnedi3(1))
range(9,9,nnedi3(1))
range(14,14,nnedi3(1))
...
TDecimate(mode=1,cycle=5,ovr="D:\VTS_05rrr.demuxed.d2v.dec.txt")

Basically: decimated frame, ivtc (progressive frame), bob(0), ivtc (next frame), bob(1)(of the next frame), then repeat every 5 frames.

Edit:
Something like this

DGDecode_Mpeg2Source("D:\VTS_05rrr.demuxed.d2v")
fields=SeparateFields()
prog=fields.selectevery(10,2,3).Weave()
n=fields.selectevery(10,7,8).Weave()
nnedi0=Selectevery(5,2).nnedi3(0)
nnedi1=Selectevery(5,0).nnedi3(1)
return Interleave(nnedi1,prog,nnedi0,n).Trim(2,0)

The nnedied frames look like crap, a good bobber may give better results.

[cretindesalpes]

My attempt :

al = 0.15 # Luma blend
ac = al # Chroma blend
g = 3

# Deblending with pseudo-gamma correction
edi = nnedi3 (field=-2)
s1 = " 16 - 219 / 0 max "+String(1.0/g)+" ^ "
s2 = " 0 max "+String(g)+" ^ 219 * 16 + "
fix = mt_lutxy (edi, edi.SelectEvery (1, -1),
\ yexpr="x "+s1+String(1+al)+" * y "+s1+String(al)+" * -"+s2,
\ expr="x "+String(1+ac)+" * y "+String(ac)+" * -",
\ y=3, u=3, v=3)
fix = fix.SeparateFields ().SelectEvery (4, 0, 3)

# Replaces only the blended fields with their fixed counterparts
Interleave (last.SeparateFields (), fix)
SelectEvery (10, 0, 2+1, 4, 6, 8+1) # Comb pattern here
Weave ()

# Usual IVTC + light postprocessing for residual combing
TFM (PP=0)
Vinverse ()
TDecimate (mode=1)

It may exhibits occasional artifacts. I'm not sure about the values or the formulas, but it's a decent starting point.

[Dogway]

Thanks librarian, I know it is interlaced by 2 facts, combing, and field blending. And a confirmation by dgindex. That's ok but, don't you see a 3:2 pulldown as I do? how can they interlace a telecine source (this is what fooled me)? how can that be done and why do they do it?

It's strange because I got less blending with bob+srestore(23.976) than with your code, even using blendbob has less blending (but more than bob+srestore):
nnedi3(field=-2)
blendbob()
selectevery(5,0,1,2,4)

@Sapo84
I rearranged the code to understand it better.

fields=SeparateFields()

nnedi1 =Selectevery(5,0).nnedi3(1) # fields 0,1 keep 0 drop 1
prog =Selectevery(5,1) # fields 2,3
nnedi0 =Selectevery(5,2).nnedi3(0) # fields 4,5, drop 4 keep 5
w=fields.Selectevery(10,7,8).Weave() # fields 7,8
# 0, 2,3, 5, 7,8
return Interleave(nnedi1,prog,nnedi0,w).Trim(2,0)

What this means is that not blended hard telecine shouldn't be deinterlaced? I suppose that after this a decomber is necessary in my case at least. And as you say a better deinterlacer for the nnedi1 and nnedi0 lines. Actually this is my main question, what is a good bobber for animation? in the 2 nnedi lines one field is clean and the other is blended, isn't there a smartdeinterlacer that only deinterlace spatially where there is blending?

PD: Thanks cretindesalpes, had to post this after a few hours building the post. But I'm gonna test and analyse/understand your code

[librarian]

Yes the source is definitely telecined. I am not so experienced to imagine what they did to this poor footage, why they did so is simple: to save time and money.
When I tried bob+srestore I got an unsatisfactory output, therefore I used the script posted above (pointing out the jerky motion).
I like your solution, I often use nnedi3(field=2)+blendbob but never on this type of problem, well something to learn and remember, thank you.
I quickly tried the script of cretindesalpes and it seems really effective.
Now choose your poison and good enconding.

[Dogway]

Thanks for all the help.
Yes after some more hours analysing cretindesalpes code I definitely don't understand it lol
Well, I understand the structure, it kinds of deconstruct-->deblend-->reconstruct back to unblended hard telecine-->IVTC

easy to imagine, hard to get done, so thanks a lot cretindesalpes. I wondered about the sporadic black artifact but you corrected it with the 219 value. Other than that more than enough really, since this needs a denoiser on top of it as well. No shimmer, preserves full resolution (practically) and deblends more than decent.
I learned a lot, still I would like to know if there exists a smartbobber that is OK (technically correct) for animation when source is interlaced (not as hard telecine)

[mandarinka]

My way of dealing with these telecine+blending sources is to ivtc with tfm (pp off and you can get rid of some blends in static parts by using mmsco=false, but take care, I think it can cause a wrong decision sometimes) and decimating, then finally get rid of the annoying combs by spline36resize(width,height*3/2).daa().spline36resize(width,height). The blending will stay but it isn't that annoying if it isn't combed/aliased. Make sure you are using nnedi3 in daa btw, that makes a lot of difference.

The other option is qtgmc and srestore, which will eliminate a lot of the blending, but isn't as friendly to detail as the above (based on my opinion/experience/preference).
Using other bobbers I wouldn't even consider, for the same reason.