How to Perform an effective IVTC

[TraXeN]

// extra - since this first post - the new addition of a logfunction
has been added to areabased deinterlacer. This simplifies this task quite alot
---------------------------------------------------------------------

Remember, this is a suggestion .. in the end its up to u if u dont like the way its being handled.

1. First perform the normal automatic ivtc in tmpeg. From 30fps down
to 24fps (remember to use insert/delete to alter the frames the ivtc
will process, if u need it that is)

2. Take the projectfile and use VFAPI on it to make it into a fake avi file.

3. Load the fake avi file up in virtualdub or nandub.

4. Get a filter called "areabased deinterlacing" Load it in and set
the option "show deinterlaced areas only" to ON.

You might wanna alter the "edge detect" for a bit depending on the
initial quality on ur file. 75 seems to be a good number for sharp
animes for instance.

5. Run the movie with the filter and watch how combed areas get
caught in. You can directly see a flash when something suspicious
that might be a badly IVTC:ed frame appears.

6. When you have one of those frames go into tmpeg with the loaded
projectfile that u created for the original file. Go into the ivtc
section and rightclick in the big frame.

7. Use the option: >> move to specified frame >> after ivtc - and
enter the framenumber that you found in ndub/vdub.

8. Use the arrows UP/DOWN and alter with holding in ctrl+UP or DOWN.
That gives you easy access to all the ways you can deinterlace a frame.

When achieved result is performed and the frame looks good.. just keep going for the next one in the same manner.

play >> spot a bad frame >> take the nr >> go into tmpeg >> jump to the correct frame >> deinterlace it

[TraXeN]

Hmmm.. something that I forgot to mention last time.

If ur pc can take it run the input screen in ndub/vdub on 2x the size. It will be extremly easy to spot if its a interlaced frame.

Just rightclick on the frame and choose the prefered size =)

[poopity poop]

Why is this here? I think you meant to post reply under the other dude who had the ivtc question.

And Decomb is better dude, tmpeg is fine, but not as good, look into it

[TraXeN]

I made this post so it wouldnt dissapear in the whole heap of IVTC
questions that is out there since this is one way that really
(REALLY) works and really gives back an astounding perfection in most cases.

There are to many arguments about if GREEDYHMA or DECOMB and so on is
better, and that the process of tmpegs ivtc isnt perfected.

I know tmpeg isnt the best in the ivtc process.. but that is why Im
using "areabased deinterlacer"

--------------------------------------------------------------------
A new addition: "areabased deinterlacer" has been upgraded after a
request about making a log function for bad frames.

ver is now 1.3

It depends on a % threshold that u set and then it put all of the
frames that goes equal or above it into a file.

For Ex (this was made from a sample on Jubei ep01):
I set the threshold to 1% and then ran through the ep with the filter on.
Reason for setting it to 1% is that I want to catch small occurances with mouthinterlacing.

Frame Interlaced area (%)
09545_______________1.0%
12007_______________1.4%
12009_______________2.4%
12018_______________1.7%
12086_______________2.2%
13617_______________1.1%
13981_______________1.8%
15571_______________1.3%
15573_______________28.6% << definetly a bad frame(but I recommend a check on the others also)
15830_______________1.2%
17047_______________2.5%
17048_______________2.3%


This is also something that can easily be saved for future reference for new encoders.
Since its hard to save tpr files and use it again in a different location

[Hanty]

poop :

I think you misunderstand, he's not using TMPGEnc by itself, but combining methods. Personally when It comes to IVTC I question all methods operating solely by themselves, decomb included.

[midiguy]

ok, stupid question, but I searched for Area Based Deinterlacer, and I could only find ver 1.2, so if you can, can you send a link to the newest version? thanks. this is my last deperation at an IVTC for this anime.. and if it doesn't work out, I will either not encode it or just do a smart deinterlace..

[TraXeN]

actaully... I just mailed the creator of the filter so he can update
his homepage with it.

Btw.. a small idea that I have now is to get in touch with the
creator/creators of tmpeg and see if they can import this log into
their ivtc process and simplify it even further.

Cause in the end its here all the work should be done. Just using the
log as a base for what frames to step between and correcting. That
would save a whole lot of time and I guess this isnt a very hard
thing to code.

[OUTPinged_]

traxen,

late decomb versions are recreating fileds well (at least 97% of frames are recreated, all others have smart deinterlacing applyed over them, none look interlaced).

the only scenario where you could prefer tmpeg over decomb is where you just absolutely have to perform a field drop on some frames instead of smart deinterlacing to get a smooth movement.

and one more, you should not apply area deinterlacing filters over each frame. by doing this you simply ruin the point of clean ivtc.

here is how much frames were deinterlaced approx (imposible to recreate)

"hard ivtc" hollywood movie dvd = under1%
into this category fall movies which couldnt be force-filmed.

"properly mastered" anime dvd = 1-3% (excluding CG scenes, these could last up to 5% of full length)
most r2 dvds, most late r1 dvds with clean source, some hk or r1 dvds with early content. if simple field matching recreates most frames, then a dvd is mastered ok.


"improperly mastered" anime dvd = 5-20%(!).
here we have a telecine->over cheap ivtc->pure interlaced hvs source.

20% we have with worst cases, where we have to deinterlace each 5th frame because of improper mastering.

this is the only case where you can apply deinterlacing over a whole movie.

the best new feature the area based deinterlace could have is a frame range(s) tab imo.



midiguy: and what are you encoding that it is so bad man?

[TraXeN]

and one more, you should not apply area deinterlacing filters over
each frame. by doing this you simply ruin the point of clean ivtc. <<

Let me again say that Im not applying this filter at all :(

Im just using it as a way to find the frames that needs to be fixed

I strongly advise the next person that will reply on this thread to
read carefully about the technique that is elaborated and not rush
into the conversation before its fully understood.

[OUTPinged_]

for some people time matters.

50% of anime dvds look near perfect with default decomb settings and 45% more look excellent with threshold tweaking.

other 5% look better with heavily tmpeg tweaking.

the problems with your method are:

1. "mouth problems".
2. long pure-30fps parts (200+ frames) problem.

that is if you dont want to split an encode into several parts and process each separately.

that isnt that simple. you have transparent ivtc'ing with decomb and results are usually same as with tmpeg. that is why busy dvd owners dont use tmpeg anymore.

tmpeg doesnt have "smart deinterlacing" routines for its ivtc, i have mailed developers about that long ago but got no response.

[Hanty]

you have transparent ivtc'ing with decomb and results are usually same as with tmpeg.

With this method applied I have experienced results to be radically different from TMPG by itself. Generalising like that is doomed to prove you wrong. Since this filter isn't public and you can't possibly have tried it you seem hellbent on disqualifying the method before you have test driven it.

And why are you so hung up on anime? I see TraXen giving an example value at one point for anime that is all. What's this about comperative numbers for different type of anime DVDs? I understand that you are an anime ripper but that is only a small field and a very large market.

[midiguy]

the anime I am encoding is the street fighter alpha movie..
and about IVTC and anime.. in some anies, there are sequences that are pure 29.97 fps! for instance: some sequences in Love Hina. So how can you possibly IVTC this without losing some good progressive frames??

[midiguy]

ok, I got the latest version of area based deinterlacer,.. but how do you produce the log file? just encode the movie? how will this work? thanks.

[Guest]

@midiguy

Telecide()
Decimate(mode=1,threshold=50)

[midiguy]

@neuron2
thanks for the decomb suggestion but decomb still had to deinterlace a lot of frames.. it didn't get all the interlaced frames out in its IVTC, so it is just deinterlacing a whole bunch.. it looks like the IVTC almost totally failed... thanks though... but why those settings impaticular?

[midiguy]

I just relized that it wasn't suppose to be an IVTC.. the results look ok, but I would rather IVTC it, but thanks though.. hrmm.. any other suggestions?

[OUTPinged_]

2hanty, i had a working tmpeg version since i have begun ripping. i used tmpeg on almost every rip i did before switching to decomb. i do not use it for filtering or transcoding, just as ivtc tool.

warez lives :P

as for anime dvds, first, i am encoding more anime dvds than movie dvds, second, when i have applyed my anime-tweaked setup to encode a movie, i have got excellent results every time. but i dont rip too long movies as a 1cd encodes (usually somebody has these in crappy quality on my lan already), so i cant state (and i am not doing that) on that.

anyway, the point is, for beginners its better to stay with decomb.

i still cant see how traxen is going to apply area based deinterlacer to a specific frames he will select as being interlaced.

if he is going to just manually set these frame-by-frame in tmpeg, then i claim this method to be time-consuming and uneffective (cause i have lived thru that myself), tho giving a good result sometimes.

if he is going to split every encode for each pure-interlaced sequence, it is still very time consuming.


decomb is a fire-and-forget type. you cant argue over that. and it gives excellent output from a clean source. it even gives good output on a badly mastered dvds(eva). it gives tolerable results from a completely broken dvds(honneamise r1) assuming you will use very low threshs, at which is better than dumb area-based deinterlacing.


2mamono: i guess there should be alot of testing done before there will be safe to claim best gthresh for a starter.

this setting (gthresh) is a double edged sword and will hit back hard.

here is an egzample:

as tmpeg does have some amount of gthresh hardcoded (you do know that hanty, ne? :P) it has failed on one scene of my encode of lain ep1 a while back. it had some pure interlaced stuff going on a small computer screen(static scene), and that was slipping thru tmpeg and was clearly visible after encoding. i dont have that dvd already but i would really like to know what gthresh was enough to get rid of it.

[Guest]

@midiguy

I just relized that it wasn't suppose to be an IVTC.. the results look ok, but I would rather IVTC it, but thanks though.. hrmm.. any other suggestions?
If a movie is pure interlaced and therefore not "supposed" to be IVTC'ed, then you can't do it, whether you "would rather" or not! That is like saying, I can't run my car on water, but I would rather do it, any suggestions? :)

The suggestion I gave about Decimate() was intended for the case of mixed film/nonfilm content. There you can argue about whether it is supposed to be IVTC'ed or not, but rather than engage in video metaphysics, you can use the method I suggested and get reasonable results. If you apply it to a source that is all or primarily interlaced, then of course you're going to get most of the frames having the deinterlace applied.

As to settings, RTFM!

@OUTPinged_

Yes, you *can* get too high a gthresh, which will result in failure to rapidly adjust to pattern changes. There should be an optimum value for a given source that will minimize unnecessary deinterlacing.

[OUTPinged_]

2dg:

in my case it was that tmpeg has missed a pattern change thanks to its internal thresh.

[manono]

Hi OUTPinged_

Interesting you should mention Lain. DG himself was working on that one recently (his results are in the AviSynth Forum-maybe you saw it) and decided to try it in his new Mode=1 because of the 30fps stuff in it, and was pleased with the results. Of course, for the sake of file size savings I'd rather have it at 24fps whenever possible when there's only a small amount of 30fps stuff.

As for Gthresh=75 just being a starting point and just to back up neuron2's previous point, I started working with Trigun a couple of days ago. I noticed that there was interlacing getting through with Gthresh=75, so I throttled it back to 40, and it was better. That's the first time I've noticed Gthresh being too high. But the DVDs of that series have some serious problems, and I can't get it all cleaned up, no matter what I do.

[OUTPinged_]

mamono:

the opening of lain was recovered excellently for me by decomb 1.91 (!). the anime itself has pretty little of 30fps scenes (approx 1 minute for all 13eps), so i guess i cant kill other 300minutes for the sake of this one.

the 60fps CG sequences are recovered pretty well, tho look a bit jerky(3x5seconds in each eps). the good thing is that 60fps is downsampled to 30NTSC, and only each 5th frame has minor (60fps) interlacing after no-postprocess ivtc. that is on r1 dvds.

one scene in each ep definitely needed gthresh tweaking but decomb didnt do guiding at that point(was jerky).

as for trigun: you have to get all the opening running smoothly, i did that myself and it wasnt hard :P

[Hanty]

decomb is a fire-and-forget type. you cant argue over that

Does that mean you don't QC? You just stick your encode through a number of filters/tools? Sorry, I will never trust an automatic process more than my own eyes.

And no, I'd definitely say decomb isn't for beginners, Avisynth would seriously confuse those that doesn't have a good understanding and experience of how filters/telecine works and in what order the process goes and why.

You advocate a process where you don't have to worry over the extra work, a "fire and forget" approach. I advocate intense QC and similar manual control, which is why I can relate to this method. Both are fine but there is no single "best" method. Those kinds of generalizations serve only to confuse those that are not experienced or open minded aboutabout the processes of encoding.

[OUTPinged_]

>Does that mean you don't QC?

no. fire-and-forget means only this:

1. "dumb" tmpeg encoding:

ep1:open tmpeg, fire up the evaluating, [wait 1 hour], check thru fields, think some, set field order, set a deinterlacing mode/thresh you think will be best, check some places, make1&2 pass, [QC 25min], write down if any places were interlaced, tweak stuff in tmpeg, redo 2pass, QC again.

2. decomb encode:

open editor, change imput filename in avs (set decomb's thresh), 1&2 pass, qc, if any places are jerky, tweak decomb and redo 2pass.

if you are not tweaking threshs (newbie/lazy/busy person case), you will end with some interlaced frames left with tmpeg and some jerky frames with decomb. now if there will be interlacing left everyone will point a finger at you and laugh madly, and if there will be any jerky frames everyone will think "its a poor dvd mastering" (ripper is never guilty, its a dvd mastering guys are the worst evil who make a dvds which are not so easy to rip)

ok, everything above is bullshit, here are some lame statements:

1. use the damn tool you want.
2. if both tools work ok, i use decomb.
3. if i cant set decomb right (cause i suck) i use tmpeg.
4. it tooks me 10 seconds to set up decomb and a 1minute to set up a tmpeg (plus 1hour of wait/ep)
5. if i am asking a newbie to rip a dvd i am recommending him to use decomb (assuming i am giving him avs script. besides, avs isnt that hard to learn for a simple use. anyway you have to use vfapi tool so avs kicks ass.)
6. i rule.
7. being hungry sucks.
8. DG is your god.
9. my cat wants me to feed it.

[Hanty]

Well, whatever floats your boat, but blaming bad mastering when poor encoding is actually the case is a good way shoot down any Quality assurance you might have. Might as well bite the bullet and sit through and watch the thing a few times.

PS. Of course you can hand a newbie a script and let him run it. anyone can sucedd if they have an exact step-bystep formula to follow But if it doesn't work out the way it should how far would I trust a beginner to adapt avisynth to the circumstances? About as far as I can throw him. (and I'm not a small guy)

I wouldnt recommend manual QC to beginners either, they are usually far to impatient.

[TraXeN]

-------------------------------------------------------------------
OUTPinged_

anyway, the point is, for beginners its better to stay with decomb.

i still cant see how traxen is going to apply area based deinterlacer
to a specific frames he will select as being interlaced.

if he is going to just manually set these frame-by-frame in tmpeg,
then i claim this method to be time-consuming and uneffective (cause
i have lived thru that myself), tho giving a good result sometimes.

if he is going to split every encode for each pure-interlaced
sequence, it is still very time consuming.
-------------------------------------------------------------------

You are still going over how to apply "area based deinterlacer"
I am not applying it. Im just using its testmode :(

And working with this tech:

Set Edge Threshold: XX
Set Log %: 1%

And then u will end up with a fairly long log depending on frames that were caugt.

I then use that log to catch frames that usually are stronger than 3%

Then when those frames are fixed I just run the movie through the areabased testmode.
That way I can catch renegade frames that was sneaky enough to avoid the log :)

This will basicly take away the worst problems right away and the QC
afterwards is pretty quick.

So try this method and eval it and pls dont rant about people who dont
have the time or that they should use decomb because they are newbies.

Cause there will always be those people and usually they wouldnt even
care about to read through the first message completly.

Im aiming at people who are patient and want to encode a great movie
that will hold in all weathers.

[OUTPinged_]

traxen, i can now see how that technique may be usefull. just didnt get it from your earlyer posts.

afaik it is still a long process :/ what is your default tool/threshs you begin with?

2hanty:

>(ripper is never guilty, its a dvd mastering guys are the
>worst evil who make a dvds which are not so easy to rip)

heh, that was just a lame joke man. :-) there are alot less dvds with bad mastering than one may think reading all the anime ripping related threads :/


>I wouldnt recommend manual QC to beginners either, they are
>usually far to impatient.

thats where the problem is. i can guess that decomb with blend deinterlace at default threshs will produce an ok-looking rip, much better than anything that guy can come up with after he will read tmpeg guide 1 time. even if some frames will be left with minor interlacing areas, i can live with that.

as for long-term backups, i dont think current divx rips (even 2cd ones) will count as "high qiality" in, say, 2 years. 5mbit encode with HDTV level source looks so much better that it is evil.

[OUTPinged_]

mmm...


just got 20megs of hanty's kiki 2cd encode. looks pretty good. :-) too bad i couldnt get 1280mb left.

hanty, is it me or did you apply some minor filtering over a source?

[Hanty]

Some smart smoother, warp sharp, ghost remover. I try to keep filters at a minimum but the source for Kiki was a HK DVD and whoever mastered it should be shot. But I digress...getting off topic.
I covered that release with all applications in full on this thread :
http://forums.psychlo-anime.net/showthread.php?s=&threadid=6