Cartoon IVTC Problem

[Nealhon]

Hello everyone, I am trying to convert some cartoons from 29.970 FPS -> 23.976 FPS. The settings I normally use on DVD cartoon sources work fine, however the same settings don't seem to work so great with some TV sources of some cartoons I have. What happens is part of the image appears to be left behind or tries to appear too soon. I guess ghosting would be an appropriate term? Anyway here are 3 frames in sequential order with the settings I currently use:

http://i5.photobucket.com/albums/y163/Nealywoowoo/Picture1-GoodFrame.jpg

http://i5.photobucket.com/albums/y163/Nealywoowoo/Picture2-BadFrame.jpg

http://i5.photobucket.com/albums/y163/Nealywoowoo/Picture3-GoodFrame.jpg

I have posted a small raw (Huffyuv, .avi) sample that has a lot of motion. Can someone please advise me on some good settings to use to properly convert this to 23.976 FPS?

http://www.megaupload.com/?d=KOYFP58J

[thetoof]

What happens is part of the image appears to be left behind or tries to appear too soon.

afaik, this is called blending, so you could try using cdeblend() (part of the R-pack)

But, the screenshots you included show some kind of weird ''temporal bleeding'' or whatever it may be called and I don't have an idea of how to deal with it (not on my comp atm, so I can't test any plugins)

[Nealhon]

Thanks for the reply. Do you have a link for R-pack? All of the download links I found for it are dead.

EDIT: Never mind I found a good link. I am testing it out now.

EDIT 2: It seemed to have helped, but panning scenes are incredibly choppy (dup frames). I'll try adjusting some settings or something.

[45tripp]

looks simpler to me.

also looks horrible to me,
is this what passes for kids entertainment.

thought i'd give it a little try.


import("path\FastLineDarken_1.3.avs")
AVISource("Clip.avi")
selectevery(5, 1,4)
chromashift(c=-4)
crop(2,4,-20,-6)
lanczos4resize(640,480)
dfttest(sigma=4)
converttoyv12()
fastlinedarken(strength=128,thinning=0)
asharp(1,3)
changefps(24000,1001)


http://www.mediafire.com/?mziimkf4mxw

[thetoof]

Yeah, that's cuz you have to decimate it after to reach the 23,976 fps you want.

For interlaced anime sources, I usually use this:
mpeg2source("yourfile.d2v")
tdeint(mode=2)
cdeblend(mthresh=0.3)
tdecimate(mode=1,cycleR=2,cycle=10)

and for telecined ones:
mpeg2source("yourfile.d2v")
tfm(d2v="Honor pulldown VTS_01_1.d2v", slow=2)
cdeblend(mthresh=0.3)
tdecimate(mode=1,cycleR=2,cycle=10)

[thetoof]

I just checked the file you uploaded (couldn't before)... it's kinda weird, you have a 4:1 pattern instead of the usual 3:1...

Anyways, I'd use the script with tdeint and I'd add dup(threshold=7.5) to it because the second frames of the duplicates you have are better (at least on the sample you uploaded).

Some ideas of Immersion's script are intersting, but there is noticeable detail loss (the wall becomes flat when the light bulb is in front of it).

Also, you could try using a logo removal function. (the one with inpainting by Spuds is great) (http://forum.doom9.org/showthread.php?p=1124403)

[Nealhon]

Thanks for the help guys. I am trying out everyone's settings, and I'll report back with how they turn out on a bigger scale. With Immersion's settings, I cannot get dfttest to work. I've read the readme file, but I don't get it (something about compiling libfftw3f-3.dll with Visual C++). I have no idea how to get it to work. Can anyone shed some light on me with this one?

[45tripp]

http://web.missouri.edu/~kes25c

the dll is part of the package
just put it in /windows/system32

[poisondeathray]

How do you plan on keeping audio in-sync?

EDIT: I meant without pitch shift or "slo-mo" voices?

[Nealhon]

http://web.missouri.edu/~kes25c

the dll is part of the package
just put it in /windows/system32

Alright, that did the trick... I just needed to put it in the /windows/system32 directory. However, with using your settings now I'm getting the following error:

"Avisynth open failure:

Script error: there is no function named "expand"
("path/FastLineDarken_1.3.avs, line 49"

I used the FastLineDarken_1.3.avs that I found here on the forums... any idea what do to about this?

[Nealhon]

I just checked the file you uploaded (couldn't before)... it's kinda weird, you have a 4:1 pattern instead of the usual 3:1...

Anyways, I'd use the script with tdeint and I'd add dup(threshold=7.5) to it because the second frames of the duplicates you have are better (at least on the sample you uploaded).

Some ideas of Immersion's script are intersting, but there is noticeable detail loss (the wall becomes flat when the light bulb is in front of it).

Also, you could try using a logo removal function. (the one with inpainting by Spuds is great) (http://forum.doom9.org/showthread.php?p=1124403)

I tried using your settings, but when I scanned through the file, it was full of interlace. Maybe I need to provide a longer sample?

[thetoof]

FastLineDarken is old... so it uses masktools v1.x (different syntax than 2.x)

Could you send me about 20-30 frames of the original clip that remains interlaced after using tdeint? (no need for a longer clip... would be a pain to transfer, only the buggy parts are needed)

[45tripp]

masktools 2
http://manao4.free.fr/mt_masktools.html

removegrain
http://avisynth.org/mediawiki/Removegrain

import
fastlinedarken.avs

function FastLineDarken( clip c, int "strength", int "luma_cap", int "threshold", int "thinning")
{
## parameters ##
str = string(default(strength, 48) /128.)
lum = string(default(luma_cap, 191))
thr = string(default(threshold, 4))
thinning = default(thinning,24)
thn = string(thinning /16.)

## filtering ##
exin = c.mt_expand().mt_inpand()
diff = mt_lutxy(c,exin,yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "+lum+" < y "+lum+" ? - 0 ? 127 +",uexpr="x",vexpr="x",u=2, v=2)
linemask= mt_lut(diff.mt_inpand(),"x 127 - "+thn+" * 255 +").removegrain(20,-1)
thick = mt_lutxy(c, exin, yexpr="y "+lum+" < y "+lum+" ? x "+thr+" + > x y "+lum+" < y "+lum+" ? - 0 ? "+str+" * x +",uexpr="x",vexpr="x",u=2, v=2)
thin = mt_lutxy(c.mt_expand(U=2,V=2),diff,yexpr="x y 127 - "+str+" 1 + * +",u=2, v=2)

return (thinning == 0) ? thick : mt_merge(thin,thick,linemask,y=3,u=2,v=2)


(and in case it needs mention,
selectevery(5, 1,4)
was specific to the section provided,
depending on where your start frame is in the sequence it'll need changing,
and if there's a break in sequence you'd have to cut into pieces and do each sequence seperately before joining.)

[Nealhon]

Sorry about not responding lately, I had family over for a few days. Immersion, I'm not exactly sure how to configure selectevery.

I have uploaded a much longer sample to MU. This one is about a 1 minute long and it is in .TS format. I use DGIndex 1.5.0 RC 5 to save it as a .d2v file that avisynth/virtualdub can handle.

http://www.megaupload.com/?d=V1DHIODX

With the settings I've tried on this, I get bad frames on some scene changes. However, one of the major issues is scenes like the one starting at 2 1/2 seconds (a lady and man are walking). I get very bad blended results. Some scenes look just fine, others are blended.

I tried using thetoof's settings, and it got rid of almost all of the blending and interlacing... however some scenes (panning oens in particular) are a bit choppy (duplicate frames), and sometimes with these settings it seems like the people's mouths don't move when they talk.

With Immersion's settings on this file, it's mostly interlace. I'm sure it has to do with selectevery, but I just don't know how to configure it.

I am just somewhat puzzled at how to properly get these things (tv captured cartoons) looking good interlacing/blending/choppyness wise. Do you all care to take a look at this and give me some more tips?

:thanks:

[Nealhon]

I've been messing and messing around with these settings trying to come up with something that works well. Using cdeblend(mthresh=0.3) seems to work quite well to get rid of the blended frames, but it is killing the panning scenes--It's making them very choppy (I have quite a few cartoon samples I've been messing around with these settings on). I've tried altering mthresh by +/o 0.1 but it doesn't help with the panning scenes. Does anyone know of some settings to use that'll make it smooth?

[thetoof]

Are you decimating your clip to 23,976 using tdecimate(mode=1) after cdeblend?

[Nealhon]

Yup, I've tried using your exact settings (along with dup).

[45tripp]

With Immersion's settings on this file, it's mostly interlace. I'm sure it has to do with selectevery, but I just don't know how to configure it.


it would only work if the pattern was constant.
it was for the small first sample,
and one was hopeful it might only have a break or two.

but the second sample clip is quite shocking.
if it was the same type of pattern,
with single interlaced useless frames,
even with breaks,
it would be easy enough to work with.
but the pattern changes and there's some weird ugly interlacing,
with two consecutive intl frames as well as the single int frame seq.
and the pattern alteration is frequent.
i worked it manually, just to see:
http://www.mediafire.com/?4vcnjlv9dit

good thetoof's suggestion is working for you

[thetoof]

Yup, I've tried using your exact settings (along with dup).
Oh now I get it... the settings I gave for dup() were only good for the first clip you uploaded (it needed settings that high to properly duplicate the frames that needed it), but it'll make pans ugly and small movement will disappear.

You could simply remove dup() and use some other filters (like a denoiser) to help enhancing your source.

[Nealhon]

Oh now I get it... the settings I gave for dup() were only good for the first clip you uploaded (it needed settings that high to properly duplicate the frames that needed it), but it'll make pans ugly and small movement will disappear.

You could simply remove dup() and use some other filters (like a denoiser) to help enhancing your source.

Yeah, I've tried removing "dup" too. It's producing pretty much the same thing--jerky/choppy panning scenes.

The settings do work great on about half of the toon samples I've tried them on, the other half don't. I'm guessing it just has something to do with the way the toon was originally created.

[45tripp]

might as well post a sample clip containing such a panning scene.

[Nealhon]

Alright, I appreciate you guys working with me and being patient.

Here is another 1 minute sample of a different toon in .TS format. A panning scene starts at about 4 1/2 seconds into the clip.

http://www.megaupload.com/?d=6W7CMMAK

[thetoof]

Last edit: This post is now totally obsolete. Please refer to AnimeIVTC()'s documentation (will be released around May 27th 2008).

This is a serious issue as I don't recall seeing any filter that can solve the problem mentioned in bold below. Pluggers and scripters, would there be a way of correcting this?

Man, I'm SO grateful that you posted on this issue! I just realized my usual method for IVTCing anime wasn't good.

This is why: Many anime/cartoons are at 12fps (+ a duplicate of every frame to reach the normal film rate of 24fps), but to create the illusion of movement, they alternate the movement of the 2 fields ("cinema language" meaning... front, middle and back field (translated from french: avant champ, millieu, arrière champ, so I'm not sure if it's the right expression in english), i.e. background and character) and the problem appears when there is a different telecining pattern for each part of the image. From what I've seen, this is VERY common.

To recover the original 24 fps and avoid blends or jerkiness, you must make your source progressive by doing a 30i to 60p conversion and then select the good frames. When your clip is at 60p, you'll have 2 duplicates, a blend, a good frame and a blend (5 frames). The frames you want to keep are: the second of the duplicate (for some reason, 2nd looks better than 1st on the 3 sources on which I did the tests) and the middle frame of the 3 others. Select them with a selectevery function. By decimating 3 frames out of 5 on a 59,94fps clip, you'll have a 23,976fps one.

For the last clip you uploaded, those are the good settings:
mpeg2source("Lilo & Stich Sample.d2v")
tdeint(mode=1) # you can use something else like mcbob(), but this is good enough imo
selectevery(5,1,3) #for each 5 frames sequence, the 2nd and 4th ones are kept (frame 0 is the 1st one, so for this call, select every will ditch frame 0 (duplicate),2(blend) and 4(blend))

This only works on "double 12fps" sources. It will not work when there is genuine 24fps movement

Situation where every other method of IVTCing or deinterlacing fail, but my solution works:
Double 12fps m2v (http://www.mediafire.com/?rzm3j4dm3gk) // mpeg2source("Double 12 fps.d2v").tdeint(mode=1).selectevery(5,2,4) (http://www.mediafire.com/?hyqt9tyeemd)

Situation where every other method of IVTCing or deinterlacing fail, but my solution works... and introduces a "jaggy edges" problem:
24 fps m2v (http://www.mediafire.com/?mm4yz1rogvn) // mpeg2source("24fps.d2v").tdeint(mode=1).selectevery(5,1,3) (http://www.mediafire.com/?i0mgmuuziv2)

Anything else = blends or jerkiness... my solution = none, but jaggy edge (I tried aaa() and it gave somewhat good results... but it could be a lot better)

What should look like this (http://img187.imageshack.us/img187/7835/blendypjt9.jpg) will look like this (http://img387.imageshack.us/img387/8516/jaggyll9.jpg) after "my" IVTC if there is 24 fps movement.

Avisynth pros, to your keyboards! :p

Edit : Because of this (http://forum.doom9.org/showthread.php?p=1134306#post1134306), selectevery isn't needed anymore.
Edit 2 : But, because of this (http://forum.doom9.org/showthread.php?p=1134323#post1134323), you may still need it...
Edit 3 : Use this (http://forum.doom9.org/showthread.php?p=1134670#post1134670) to identify what type of source you have (and therefore understand which correction is appropriate)

[thetoof]

How do you plan on keeping audio in-sync?

EDIT: I meant without pitch shift or "slo-mo" voices?

Hmmm... seems like we ignored you and continued our discussion :rolleyes:

Sry 'bout that :p, here's the answer: when you're IVTCing + decimating, the objective is to restore the clip like it was before conversion to the NTSC standard, so only the framerate changes (the amount of frames that appear in 1 second), not the length of the video, so the audio needs no ajustment. It's not like a 25 to 24 fps conversion where the lips will move slighly faster (4%) because the number of frames to display the movement will have been reduced.

[Nealhon]

Wow those settings worked awesome on the latest sample! Thanks a lot thetoof! I have to head out of town in a few minutes, but I'm going to try these out on some of the other problem toons I've messed with.

:thanks::thanks::thanks:

[thetoof]

Yeah, it works better than pretty much everything... but there are a few issues regarding the "jaggy edges" problem. I've spotted 2 "genuine 24fps" sequences (maybe there's more, doesn't really matter) where you can see this on your source: the zoom in on the house after the pan and when the weird creature duplicates the candy. It's more noticeable on the house since the motion is higher.

Maybe some edge mask + correction applied only on those edges could work.

Or, get the "good" edges from a reference clip created with tdeint(mode=2) or something similar and overlay them on the output clip

Dunno how this could work, as I'm no expert of masktools. Didée (or any other person who could help), I know you're good with that stuff, do you have any ideas to solve this problem?

[45tripp]

This is why: Many anime/cartoons are at 12fps (+ a duplicate of every frame to reach the normal film rate of 24fps), but to create the illusion of movement, they alternate the movement of the 2 fields ("cinema language" meaning... front, middle and back field (translated from french: avant champ, millieu, arrière champ, so I'm not sure if it's the right expression in english), i.e. background and character) and the problem appears when there is a different telecining pattern for each part of the image. From what I've seen, this is VERY common.

yeah,
i noticed on the second clip posted here,
very evident in the scene where the boy and girl are tracked across the corridor.

as for the suggestion,
i'd say a plain ivtc would work better,
at least on the last two samples.
just
tfm(slow=2)
tdecimate()

it's better to use the second clip as reference as it mixes things up.

i didn't see a pattern like thetoof's clip here,
my last suggestion for that clip, btw, returns the same
as the selectevery you suggest.
don't know how the clip continues,
and what's mixed up in it,
i'm sure you guys know animation better.


btw,
panning is camera movement from side to side,
what you have stuttering in lilo is a zoom.
if you use a straight ivtc, you'll get no stutter.

you can use cdeblend,
with the risk of some slight stutter:
tfm(slow=0)
cdeblend(mthresh=0.3)
tdecimate()
it'll kill some blends,
but you'll have the risk of stutter.

i'd liked the approach i used to do clip2,
what did you think of the result?
breaking the clip into sections according to how many
interlaced frames per sequence.
if one could automate it that is...
too laborious to work manually.

[thetoof]

as for the suggestion,
i'd say a plain ivtc would work better,
at least on the last two samples.
just
tfm(slow=2)
tdecimate()


Ahem...
1 - My solution (http://img220.imageshack.us/img220/950/1mysolutionex2.jpg) / TFM+TDecimate (http://img80.imageshack.us/img80/7895/1tfmtdecimatecu3.jpg)
2 - My solution (http://img149.imageshack.us/img149/70/2mysolutionzg4.jpg) / TFM+TDecimate (http://img357.imageshack.us/img357/2995/2tfmtdecimatezg7.jpg)
3 - My solution (http://img357.imageshack.us/img357/3186/3mysolutionip4.jpg) / TFM+TDecimate (http://img357.imageshack.us/img357/4214/3tfmtdecimategi5.jpg)
(From the "double 12 fps" m2v)

Because the pattern changes a LOT during an episode/a movie, it's very annoying to manually change the selectevery for every part of the clip, so used this to make it automatic:
tdeint(mode=1)
cdeblend(mthresh=0.3,omode=1)
vinverse()
dup(threshold=3.4) # reduce if slow pans or zooms are screwed up
tdecimate(1,3,5)
Explanation:
Tdeint(mode=1) gives: 1 (bad), 2 (good), 3 (blend), 4(good), 5 (blend), 6 (bad), 7 (good), etc etc etc
With Cdeblend(omode=1), the order becomes: 1,2,4,4,6,6,7, etc etc etc
We want 2 and 7 (and 4), because they are slighly better than 1 and 6, so since the difference is not that big, a simple dup() will be enough. To further decrease the difference in those frames (lower threshold is needed for dup, so it won't make slow pans and zooms choppy) by removing lacing artifacts that may have been left behind, we use vinverse()
So, after all this, 1,2,3,4,5,6,7 becomes 2,2,4,4,7,7,7
Then, you remove the dups with tdecimate to get to 2,4,7 and you've got a progressive 23,976 fps clip with no blending or jerkiness!

[thetoof]

Just ran a few more tests... The cdeblend+vinverse+dup+tdecimate combo works on some source with multiple pattern changes AND easy to detect blends.

For sources with the same pattern (like the "24 fps" clip I posted before... I checked and it's doesn't change in 2h30), selectevery is the way to go, especially since it has some undetectable blends (http://forum.doom9.org/showthread.php?t=137334).

If your source is always at 12 fps, you can even decimate it to that framerate (but it's quite rare). Selectevery(5,2) (2 should be replaced by the best looking frame of the 5)

If you have a "hybrid" source (changing pattern + hard to detect blends), I suggest that you use the cdeblend solution, spot the sequences where blend detection fails and override the ivtc with selectevery, like this:
source=last.tdeint(mode=1)
a=source.trim(0,3214).cdeblend(mthresh=0.3,omode=1).vinverse().dup().tdecimate(1,3,5)
b=source.trim(3215,9869).selectevery(5,1,3)
c=source.trim(9870,89899).cdeblend(mthresh=0.3,omode=1).vinverse().dup().tdecimate(1,3,5)
a++b++c
When this method will have been improved (especially regarding the edge issue), I'll probably make an easy to use function out of it.

[45tripp]

Ahem...
1 - My solution (http://img220.imageshack.us/img220/950/1mysolutionex2.jpg) / TFM+TDecimate (http://img80.imageshack.us/img80/7895/1tfmtdecimatecu3.jpg)


not a clip i have access to,
i was referring to sample2 from nealhon
and could have posted analogous pics too.


tdeint(mode=1)
cdeblend(mthresh=0.3,omode=1)
vinverse()
dup(threshold=3.4) # reduce if slow pans or zooms are screwed up
tdecimate(1,3,5)


doesn't work with sample2.


yeah,
difficult sources.

[thetoof]

Yeah, it doesn't work cuz the blends are undetectable. So, for this source, pray that the pattern is constant and use selectevery :p

If it's not... multiple trim+selectevery, but it's very annoying to do.

edit: and could have posted analogous pics too.
What do you mean? Those 3 frames appear in that sequence... 1,2,3 and I showed the difference between the two methods; 1=no diff, 2=blend with tfm+tdecimate that is removed by my solution, 3 = no diff

[45tripp]

If it's not... multiple trim+selectevery, but it's very annoying to do.


no doubt!


edit:
What do you mean? Those 3 frames appear in that sequence... 1,2,3 and I showed the difference between the two methods; 1=no diff, 2=blend with tfm+tdecimate that is removed by my solution, 3 = no diff

simply that i was talking about what was in front of us, and you brought an example of something i didn;t have.
this goes back to tdeint(1).selectevery(5,1,3) vs tfm(slow=2).tdecimate
frame seq 1,2,3 where ivtc is better:
http://www.mediafire.com/?l1e0ecgyhnd

just saying...

[thetoof]

Heh, didn't work on the only sample I hadn't tried it on :p
So yeah, in clip#2 by Nealhon, your approach was indeed better.

simply that i was talking about what was in front of us, and you brought an example of something i didn;t have.
I said this in post#28:
1 - My solution (http://img220.imageshack.us/img220/950/1mysolutionex2.jpg) / TFM+TDecimate (http://img80.imageshack.us/img80/7895/1tfmtdecimatecu3.jpg)
2 - My solution (http://img149.imageshack.us/img149/70/2mysolutionzg4.jpg) / TFM+TDecimate (http://img357.imageshack.us/img357/2995/2tfmtdecimatezg7.jpg)
3 - My solution (http://img357.imageshack.us/img357/3186/3mysolutionip4.jpg) / TFM+TDecimate (http://img357.imageshack.us/img357/4214/3tfmtdecimategi5.jpg)
(From the "double 12 fps" m2v)
Which refered to the "double 12 fps" m2v in post#23 Situation where every other method of IVTCing or deinterlacing fail, but my solution works:
Double 12fps m2v (http://www.mediafire.com/?rzm3j4dm3gk) // mpeg2source("Double 12 fps.d2v").tdeint(mode=1).selectevery(5,2,4) (http://www.mediafire.com/?hyqt9tyeemd)
Have fun trying some stuff now that you know which clip I was talking about ;)

[thetoof]

By checking Nealhon's 2nd clip a bit more, I understood why regular IVTC works and not on others.

- Clip 2 by Nealhon = genuine 24fps with the same telecining pattern for everything, so normal IVTC works (with something as simple as telecide().decimate() from the Decomb package), but not perfectly (http://img517.imageshack.us/img517/495/hardtelecinevy6.jpg) since it was hard telecined (telecined before being written on the DVD). Maybe the result could be improved by tweaking some settings of telecide() or by using some other plugin...
- My "double 12 fps" clip (from post#23) = 12fps "background motion" + 12fps "front motion" with 2 different telecining patterns (hard telecine, but done separately on each part of the frame), which is the intended type of clip of my solution.
- My "24 fps" clip (from post#23) = mix of "double 12 fps" concept and "genuine 24 fps"; the starfield and the ship both move at 24 fps (motion on every frame), but with a different telecining pattern (again, hard telecined separately), which messes everything up because one will have movement when the other's telecine is applied (which is not the case with "double 12 fps"). This will result in undetectable blends, because one of the telecine is a lot less visible than the other (http://img238.imageshack.us/img238/2776/doublehardtelecinest9.jpg).

Conclusion : anime with proper telecining can be IVTCed normally, but scewed up sources will need my solution with cdeblend (double 12 fps) or with selectevery (double 24 fps)

Edit: The solution with cdeblend will also work if one moves at 12 fps and the other moves at 24 fps... the undetectable blends problem occur only with double 24 fps sequences (back + front = motion on every frame and each has its own hard telecine where the pulldown is not applied at the same time)

I'll post a chart in a short while to give a visual explanation.

[Nealhon]

I've got 1 more clip for you guys to look at if you don't mind. This one is a real doozy for me to get looking good. I recorded the whole episode with my DVR in .TS format, but I don't know how to crop certain sections of it, so I just saved a 1 minute sample of it in XviD format. I didn't make any changes to the source, so I hope this works out okay to mess around with.

AGU Sample (http://www.megaupload.com/?d=T14PERXT)

Thanks again for all of the help!

[45tripp]

- Clip 2 by Nealhon = genuine 24fps with the same telecining pattern for everything, so normal IVTC works (with something as simple as telecide().decimate() from the Decomb package), but not perfectly (http://img517.imageshack.us/img517/495/hardtelecinevy6.jpg) since it was hard telecined (telecined before being written on the DVD). Maybe the result could be improved by tweaking some settings of telecide() or by using some other plugin...

it's not plain,
look closer.


Have fun trying some stuff now that you know which clip I was talking about ;)

missed that.
i'll get to it.


I'll post a chart in a short while to give a visual explanation.

nice idea.
when you do,
post a nice sample clip of anything we've not collected here
if you like.
i might want to have some more fun...
;)

I didn't make any changes to the source, so I hope this works out okay to mess around with.

AGU Sample (http://www.megaupload.com/?d=T14PERXT)

not seeing breaks in the pattern,
so
selectevery(5,1,4)


gl

[thetoof]

it's not plain, look closer.
Yeah, I know... that's why I said "not perfectly". Actually, it looks exactly the same as when you used tfm. I used telecide+decimate only to demonstrate that this could also work... and I think some settings of the Decomb package can be tweaked to enhance those buggy frames.

not seeing breaks in the pattern,so selectevery(5,1,4)
Ehhh.... no? Why are you decimating to 12fps when there is 24 fps movement?? This is a "proper telecine" case, but again, hard telecined. Frame/Fields : 1481/AA 1482/AB 1483/BC 1484/CC 1485/DD (motion on every frame... genuine 24 fps movement) We need to restore the pattern to AA BB CC DD, which is the very purpose of normal IVTC.
Nealhon, could you upload a m2v demuxed from the .ts? Drag and drop your .ts into DGIndex, select "honor pulldown flag" & "IEEE-1180 reference" in the video options, select a few frames of your video and "save and demux video". I'd like to check a few things.

[Nealhon]

not seeing breaks in the pattern,
so
selectevery(5,1,4)


gl

That seems to work good for the provided clip, but when I try it on the whole episode there are quite a few pattern changes, because there's tons of interlace in some scenes.


thetoof:

Here's a sample in .m2v. (http://www.megaupload.com/?d=V83HWV9Z)

[thetoof]

As I thought, this is the same thing as the "Kim Possible" case: regular hard telecided 24 fps material.
So, again, this is what does the trick:
mpeg2source("AGU Sample #2.demuxed.d2v")
telecide()
decimate()
I haven't found a way to improve hard telecine IVTC yet, but this is already quite good. The reason I asked a m2v was that the results were ugly because of the conversion to XviD. Make sure to apply IVTC or deinterlacing only on your unprocessed source.

[45tripp]

Ehhh.... no? Why are you decimating to 12fps when there is 24 fps movement??

:D


anyway,
seeing the original pattern in sample1 of AA, A+A+, AB, BB, B+B+,
i liked the idea of cleanly picking A+A+, B+B+,
but i see all the mix of double 12fps, breaks in patterns at cuts,
24fps with what sometimes looks like straight telecine but has traces of field blends, to sometimes awful mixes, to foreground vs background with diff patterns...

so,
although i'd have liked to be able to break a clip into it's 12fps parts conditionally and pick clean frames without bob hassle, and deal with the rest separately
it seems very difficult if possible, and the majority would have to be 12fps to justify i guess.

so bite the bullet,
take it to fieldblends and use mrestore.
like i'd suggested last in thetoof's 'blends'

something like:

tdeint(1)
a=last.crop(0,8,-8,0).dfttest(sigma=2)
mrestore(numr=40,denm=100,chroma=true,mthr=-8,dclip=a)
dfttest(sigma=0.5)
crop(0,8,0,-8)


which will do better than anything on all the samples.

in a sort of basic form,
i'm sure one can add de-halo for one, darken lines.....
and slow it down even more :)

kim:
http://www.mediafire.com/?xzd1zgkc4xs

AUG (from avi):
http://www.mediafire.com/?n5nj2xnym8m

gl


p.s
megaupload sucks cock, to put it politely

[Nealhon]

Hi guys, sorry for not replying lately. I've been extremely busy with finals and graduation. Anyhow I tried both thetoof's and Immersion's recent settings below.

thetoof:
mpeg2source("AGU Sample #2.demuxed.d2v")
telecide()
decimate()


Immersion:
tdeint(1)
a=last.crop(0,8,-8,0).dfttest(sigma=2)
mrestore(numr=40,denm=100,chroma=true,mthr=-8,dclip=a)
dfttest(sigma=0.5)
crop(0,8,0,-8)


They seem to both produce jagged edges quite a bit. Immersion's look slightly better though. I can live with that, but I'd rather not. :)

I've seen TV encodes of this series going around that are properly converted to 23.976 FPS, so there must be some way.

[thetoof]

Yeah, the results are still not very good. I won't have the time to do that in the next days... maybe weeks, so I'd recommend that you read Decomb's documentation and try to tweak some settings to have more agressive post processing, or use another filter.

edit: also read everything you can on hard telecine... might help a bit.

[Nealhon]

Alright, thanks thetoof. I appreciate all of the help you've given me so far!

I will just try to continue reading/tweaking more. I'll post any good results I come up with. If anyone finds/knows of some other good settings, please let me know too.

[thetoof]

Oh, and something I mentioned quickly some time ago that you can add after decimate...
aaa() (http://avisynth.org/mediawiki/AAA)
It's kinda old and far from perfect, but it helps for the jaggy edges. There are some other issues, but still, it helps...

[45tripp]

They seem to both produce jagged edges quite a bit. Immersion's look slightly better though. I can live with that, but I'd rather not. :)

mrestore is better because it works with all the sources provided.

simple ivtc is returning 3 of the original 5 frames,
so,
you should be seeing the jagged edges that are already there...

anyway there's not been focus on filtering,
but rather on retrieval of the best 23.976 clip.

so I'd recommend that you read Decomb's documentation and try to tweak some settings to have more agressive post processing, or use another filter.


decomb can't help with field blending.

[Nealhon]

Well yeah Immersion those settings you provided seem to look good on some toons (no jagged edges), but on the AGU sample there still seems to be some that I don't believe are in the source.

I have been messing with some settings and have found some that produce some pretty good results for AGU. I'm sure the settings could be tweaked (let me know if you figure something out that's better), but here's what seems to be working just about the best so far:


Telecide(order=1,vthresh=150)
AssumeTFF()
interp = separatefields().selecteven().EEDI2(field=1)
deinted=tdeint(order=1,field=1,edeint=interp)
TFM(mode=1,order=1,clip2=deinted)
Decimate()

[thetoof]

TDeint(1) will always introduce jagged edges when there's 24fps progressive motion because, by definition, it doesn't know what "progressive" means.

mrestore "works", but did not give the best results...

Screenshots time!
Sorted from what looks like "worst to best" to my eyes.

Immersion's Best So Far (http://img262.imageshack.us/img262/1900/immersionkm4.jpg) 4,5 fps
tdeint(1)
a=last.crop(0,8,-8,0).dfttest(sigma=2)
mrestore(numr=40,denm=100,chroma=true,mthr=-8,dclip=a)
dfttest(sigma=0.5)
crop(0,8,0,-8)
My BSF (http://img501.imageshack.us/img501/3763/thetoofvk0.jpg) 8 fps
telecide()
decimate()
aaa()
Nealhon's BSF (http://img501.imageshack.us/img501/4781/nealhonab1.jpg) 49 fps
Telecide(guide=1,vthresh=150)
AssumeTFF()
interp = separatefields().selecteven().EEDI2(field=1)
deinted=tdeint(order=1,field=1,edeint=interp)
TFM(mode=1,order=1,clip2=deinted)
Decimate()
Mix of Nealhon's BSF and mine (http://img508.imageshack.us/img508/4815/nealhonaaauz9.jpg) 9 fps
Telecide(guide=1,vthresh=150)
AssumeTFF()
interp = separatefields().selecteven().EEDI2(field=1)
deinted=tdeint(order=1,field=1,edeint=interp)
TFM(mode=1,order=1,clip2=deinted)
Decimate()
aaa()

There could still be some slight improvement (maybe by using nnedi or something), but our "best best-so-far" is already quite good!

[45tripp]

as for the suggestion,
i'd say a plain ivtc would work better,
at least on the last two samples.
just
tfm(slow=2)
tdecimate()



TDeint(1) will always introduce jagged edges when there's 24fps progressive motion because, by definition, it doesn't know what "progressive" means.

yeah i'm aware of the problems,
still, it works with all samples provided,
as a blanket solution, it's easily no1.


nnedi or something), but our "best best-so-far" is already quite good!

yes,
it needs a vinverse() call after decimate()
then it's better

nice work

[thetoof]

Yup, vinverse is needed to remove the residual interlacing artifacts.

Some additional post-processing is also needed (denoise, deblock, sharpen, etc, etc), but discussing this would be going off topic, since this thread is only about ivtc.

[Comatose]

I only read your large post on page 2, thetoof, but you said that this will only work on double 12fps sources.
The problem I see with this is that the animators resort to 24fps (no duplicate frames) when there's quick motion, and if there's CG or real life video or something in the frame, the double 12fps animation won't move, but the CG/whatever will.

These kind of things are in most anime, even anime that doesn't have combat or anything like that. Switching to 24fps is pretty common.
I might upload a few samples later - one double 12fps, one quick motion 24fps, and one double 12fps with CG in the background.

[thetoof]

The solution with cdeblend will also work if one moves at 12 fps and the other moves at 24 fps... the undetectable blends problem occur only with double 24 fps sequences (back + front = motion on every frame and each has its own hard telecine where the pulldown is not applied at the same time)

I also found a way to make my approach work on double 24 fps sequences and I'll post an easy to use function when I'm done with all my finals.

In other words, I think I found something that'll work on every type of anime. *waiting for the sample to do some tests*

[Nealhon]

thetoof: Where can I get aaa.avs ?? The one I just recently downloaded gives errors in VirtualDub that I don't know how to fix.

And yes, vinverse() helps out quite a bit. Thanks Immersion!

[thetoof]

Put this (http://www.sendspace.com/file/nq6ux3) in the plugins folder of avisynth and you shouldn't have any problems. (Yeah, I know sendspace sucks, but I plan to upload a package of all the necessary plugins when I post my function, so this is only for you)

btw, I had also been using vinverse() on your samples, but it wasn't needed on the screenshots I posted, so I forgot to mention it :p

[thetoof]

I just found a way to get even better results than Nealhon's BSF + aaa() & vinverse()

I'll post screenshots later today or tomorrow to show where you can see a big difference.

My new BSF 4,66666 fps
tfm(clip2=tdeint(edeint=nnedi(),emask=tmm(0)))
vinverse()
aaa()
decimate()

[Nealhon]

Hmm yeah those settings seem to work pretty well. However there are quite a few blended frames compared to the settings I posted. Both settings produce a good number of blended scene glitches though. I am working on using a combination of your settings + mine right now.

I'd like to see some screen shots you have though. I'll prolly post some once I tweak around some more.

[thetoof]

Screenshots: You (http://img337.imageshack.us/img337/218/nealhon2ul6.jpg) // Me (http://img218.imageshack.us/img218/593/thetoof2st6.jpg)

I noticed after posting that it introduced, as you said, weird blending in other scenes. At this point, I am unable to find a good way to combine our solutions in order to avoid such blends.
The blends introduced by your bsf look like "temporal chroma bleeding" and mine look like "temporal luma bleeding". This is the first time I've ever tried to name that kind of problem... sorry if it's not the most accurate descripton :p.
*waiting for your tweaks*

[thetoof]

In the end, I was able to solve most of the blending issues, but the edges are jaggier...
telecide()
tdeint(edeint=nnedi(1),emask=tmm(0))
vinverse()
aaa()
decimate()

[thetoof]

A while after I said that I'd write a function that would do IVTC for all the possible situations with anime, here it is!
For the tricky AGU sample, you can achieve very good results with animeivtc(1, aa=3)
Additional post-processing is needed... msmooth would be a good start.