Deinterlacing hybrid anime

[RoKR]

Hello everyone, sorry for my bad english
I have a problem deinterlacing Gurren Lugann DVD.
First of all - what DGIndex says
http://pic.ipicture.ru/uploads/081126/vycZkdYTh8.jpg
The problem is that i tried many various plugins to deinterlace hybrid video (Decomb, AnimeIVTC) - but i can't make normal deintarlacing of the ending - the video always freezes for a moment in different places.
I also tried to make VFR mkv -
At first
TFM(mode=1,output="tfm.txt")
TDecimate(mode=4,output="stats.txt")
then
TFM(mode=1)
TDecimate(mode=5,hybrid=2,dupthresh=1.0,input="stats.txt",tfmin="tfm.txt",mkvout="timecodes.txt")
then encoded the video in Vdub, then muxed it in mkvmerge adding the timecodes - but the result was not good.

Here is vob-sample from dvd
http://www.sendspace.com/file/2g3941

Maybe someone can advice me what to do

[thetoof]

Can you upload a smaller m2v sample (10-20mb) where we can see some of the "the video always freezes for a moment in different places" effect?
DGIndex-select the section-save and demux video

[RoKR]

Here is one of the problem parts, taken from the ending
http://www.sendspace.com/file/q8fcty
And here is a part from the mid of the episode - just to compare, no problem with it
http://www.sendspace.com/file/6jvzqa

[Nightshiver]

Here's what worked for me (in the credits). It seems you have purely interlaced credits.

Use mode=3, omode=1, and pick the mix that best matches your episode (i.e., is the beginning of the episode purely interlaced or not). And there you go.

[RoKR]

How can it be pure interlaced, if there is 3:2 pattern - it's telecined, i am right?
Nightshiver - can you post the script you have used for the current part from ending?
Thanks in advance

[Nightshiver]

There was no 3:2 pattern in the first sample. What is used for the above was mix=5 but you only had the credits, and mix 5 is:

e1 + i1

e1 representing the non pure interlaced material, while i1 representing the pure interlaced material. Give me a sample that includes the credits and a few seconds before the credits start.

[RoKR]

Here it is
http://www.sendspace.com/file/2r6pkl

[Nightshiver]

animeivtc(mode=3, omode=1, mix=5, aa=0, estart1=0, eend1=543, istart1=544, iend1=1208)

There you go. Now, to find actual frame numbers (like I did) you need to add check=3 at the end of your script and go to where the credits start and note that the frame numbers change because the frame rate changes.
As a side note, is that Tengen Toppa Gurren Lagann?

[thetoof]

The OP had the intention of doing a VFR clip, so omode=2 is more appropriate imo. Also, it's faster and you don't have to apply a framerate conversion that will most likely introduce artifacts.
Also, make sure that your 51% film is only due to interlaced credits and not that + hybrid video. You can confirm it by checking a few high motion scene. If there is no interlacing whatsoever in those scenes, your source is hybrid (mode=6 since you have interlaced creds)... another way of finding it is encoding as telecined... and realize it made some sections jerky.
You can encode telecined as hybrid and it'll do you no harm... (ok, maybe processing time... but heh) it'll be decimated anyways, but you'll be sure to catch (and keep) all the 30p sections.

[Nightshiver]

Oh well. Both work. He can choose either omode 1 or 2, it doesn't really matter.

[RoKR]

Nightshiver - yes, it is Gurren Lagann.
So i used your script with the sample i had uploaded - and here is what i got
http://img374.imageshack.us/img374/1789/snapshot20081207231812my8.th.jpg (http://img374.imageshack.us/my.php?image=snapshot20081207231812my8.jpg)
thetoof - i checked a few high motion scenes - and i see interlacing - a few normal frames and then a frame like this:
http://img353.imageshack.us/img353/5512/50974133wh6.th.jpg (http://img353.imageshack.us/my.php?image=50974133wh6.jpg)

[Nightshiver]

You'd have to ask toof about why that happened on the first pic.

[RoKR]

Nightshiver -have you tried to encode the sample yourself or just made the script?

[thetoof]

You'd have to ask toof about why that happened on the first pic.
That's what I meant by "you don't have to apply a framerate conversion that will most likely introduce artifacts" ;) omode=1 at its current state should only be used as a last resort when you absolutely need CFR (or when the background's motion is so low that the artifacts can barely be seen, like it was with the sample that is actually the origins of omode=1 : Evangelion (http://forum.doom9.org/showthread.php?p=1139170#post1139170)) Some of it may be removed by a denoising pass of some sort (preferabely mo-comped)
Next version's omode will include credits masking, which will be slower, but will greatly reduce the occurence of such problems.

RoKR, I just had a look at your credits sample and it's actually one of the best double hard telecined sample I've seen yet: you can clearly see that the front (changing from 8 to 24 fps) and back (24 fps) both have a 3:2 pattern, but it looks like it was not applied at the same time (ok, there might be a more "technical" name and explanation for that, cuz here I'm just describing what it looks like judging from what I see)
Here's some kind of visual explanation:
p means progressive
i means interlaced

Starting from frame # 1325 of VTS_01_2.demuxed.m2v
Back: p p i i p p p i i p (and so on...)
Front: p p p i i p p p i (but there's no motion so we can't see it) i (and so on...)
You are kinda "lucky" because the credits are not moving... now this could have been a much bigger mess if they were + progressive or interlaced :p

mode=2 is your dumb answer, but I'm looking into something that'd not include bobbing.

[RoKR]

thetoof - would you be so kind and post a script with mode=2, which you used to reach good results with that sample?

[Nightshiver]

Not trying to be rude, but have you attempted to read the documentation on animeivtc? Mode=2 is all that you would put in.... ie animeivtc(mode=2, aa=whatever you want)

[RoKR]

Nightshiver - of course i did! I asked because I tried this script - it gives me the same problem which i described in the first post - jerking. Thats why i thought, that i did something wrong

[thetoof]

You did nothing wrong. I had a closer look at your source to use something better than a "dumb" method.
The problem with your clip is that when bobbed (first step of DHT), you have motion for the walking guy when the background is not moving. So, upon decimation, the motion will be kept, resulting in jerks for the background:
Visual explanation and pattern finding:
Starting with frame # 1325 of VTS_01_2.demuxed.m2v (2650 when bobbed) as the reference point (meaning first entry if for frame 2651)
f means front motion
b means background motion
m means motion for both
D means duplicate

D b D D b D m D f b D b D f b D b D f b D m D D b D b D f b D b D b D m D D b D b D f b D b D (2698)

Base for decimation : we must keep all m (or m's duplicate) and if we remove a f before a b, it's fine since there is no motion loss for the front and we won't introduce jerks in the background motion.

x k x x k x k x x k x k x x k x k x x k
ohoho! what am I seeing here, a pattern!! if we keep 2 frames out of 5 in the 59.94fps stream, we're fine!
wait... 2/5*59.94 = ... 23.976!!! here goes the solution.
We could also have kept the duplicates (test for yourself to see if they are of higher quality)
k x k x x k x k x x k x k x x k x k x x


To automate it, the function should do the following:
- Keep frames with motion (compared to previous) only if the next frame is a duplicate
I don't know any function that can do that atm (please correct me if I'm wrong), so I'll include it in the next release of animeivtc.

For now:
s=mpeg2source("VTS_01_2.demuxed.d2v")
a=s.animeivtc(mode=1, other settings).trim(0,897) # (change last frame of trim)
b=s.trim(898,0) #Change the numbers to match the frame in the full stream and the last frame of the double hard telecine section)
b=b.tempgaussmc_beta1mod(edimode="nnedi",tr2=3, other settings) # high quality bobbing that prevents flickering
b=b.selectevery(5,1,4) # or Selectevery(5,0,2), compare by yourself to see using which duplicate leads to better results (a lossless pass of both would be recommended to see what it actually looks like moving)
#c=s.animeivtc(mode=1,other settings).trim(last frame of the DHT section, last frame of the clip)

a+b #+c if you have a normal 3:2 pulldown section after DHT
You could use the following script (lossless pass) to see which looks better moving:
mpeg2source("VTS_01_2.demuxed.d2v")
trim(898,0) #Change the numbers to match the frame in the full stream and the last frame of the double hard telecine section)
tempgaussmc_beta1mod(edimode="nnedi",tr2=3, other settings)
a=selectevery(5,1,4)
b=selectevery(5,0,2)
stackhorizontal(a,b)

[RoKR]

thetoof - thank you for great explanations - your method helped - everything seems to be ok now