IVTC and telecine

[opp]

I just started ripping svcds 3 or 4 days ago with all that bad stuff like dvdx and vfapi, but then went into avisynth yesterday. I noticed that ripping the same movie using each method took almost the same amounts of time. I talked to some poeple and they said that avisynth should be much faster because it does not convert from yuv->rgb and back. In the end avisynth didn't have to do the 2 hour automatic ivtc that tmpeg did.

Before I get to the point, I wanna say that I don't know how avisynth works exactly, and don't know how the decomb plug-in works exactly, or how DVD2AVI works exactly, but when I ripped jurassic park, using forced film, there were a couple frames that repeated. Because of that I knew I had to use ivtc that use other ways, and I know that tmpeg with vfapi only does it once and selects what frames to use.

Does the decomb plugin do everything on the fly? Does this mean that it has to decomb for each pass? Is there a way to integrate the decomb filter or use a smart ivtc plugin or something for dvd2avi or any other way to do it so it only works once?

Here is the script i used:
LoadPlugin("D:\VbDub\mpeg2\MPEG2DEC\MPEG2DEC.dll")
LoadPlugin("D:\VbDub\ivtc\Donald~1\decomb.dll")
mpeg2source("D:\jurassic\jpark.d2v")
Telecide
Decimate(5)
Crop(0,9,711,457)
BilinearResize(480,360)
AddBorders(0,60,0,60)

*I just noticed convert2yuv function in another thread, does this mean the decomb converts to rgb?

[Guest]

@opp

>Does the decomb plugin do everything on the fly?

I don't understand your question. What is "on the fly"?

>Does this mean that it has to decomb for each pass?

I don't understand your question. What does "this" refer to?
What does "each pass" refer to?

>Is there a way to integrate the decomb filter or use a smart ivtc
>plugin or something for dvd2avi or any other way to do it so it only
>works once?

I don't understand what you think is working twice. No, you cannot integrate Decomb into DVD2AVI.

Decomb works in the YUY2 space; you do not need a convert call.

[opp]

on the fly and each pass is really the same question. When I do a multipass vbr svcd, does actually figure out what to let avisynth serve everytime i do a pass? or does it just decomb it once, and remember to serve only those frames?

[Guest]

>When I do a multipass vbr svcd

Using what software?

[Kedirekin]

You gut instinct is correct.

Doing IVTC in TMpg, you're sort-of pre-processing. When it comes time to do multipass frame serve via vfapi, you save the time of re-analyzing all the fields, because TMpg saved the results of the analysis you got during you IVTC scan. However, you loose the advantage of this stored analysis because vfapi is slow.

Using AviSynth with mpeg2dec.dll, you get much faster speed than vfapi. However, when you do IVTC in AviSynth (regardless of which IVTC filter you use - this isn't a decomb problem), all the analysis has to happen on each pass, so again you loose the speed advantage.

However, the two approaches (with IVTC in the mix) are roughly the same speed, and the AviSynth approach saves you the effort of having to do an IVTC scan (and with decomb, produces arguably better results).

[Kedirekin]

Oh, BTW: there is a way to have decomb only do it's analysis once, but it requires a large hard drive - you create an intermediate (preferrably lossless) avi file.

This is especially useful if you're applying lots of filters (BiCubicResize, TemporalSmoother, SmartSmoother, etc.). The hidden advantage is the final multipass encoding happens very fast. I've gotten speeds as high as 3 times realtime using an intermediate avi file.

[madoka]

Originally posted by Kedirekin
Oh, BTW: there is a way to have decomb only do it's analysis once, but it requires a large hard drive - you create an intermediate (preferrably lossless) avi file.

This is especially useful if you're applying lots of filters (BiCubicResize, TemporalSmoother, SmartSmoother, etc.). The hidden advantage is the final multipass encoding happens very fast. I've gotten speeds as high as 3 times realtime using an intermediate avi file.

But how long does it take to create such uncompressed AVI? Does this time (and space) investment really pay off in the end?

[Guest]

Whew. I'm glad somebody knew what he was talking about. :)

I'll just slink away then.

[Kedirekin]

The answer to both questions depends on how many filters you're using and how slow they are.

The short answer is, the time required to create the intermediate avi is the time it would take to do one pass. Using Huffy (at least on my PC), the avi requires about 1 GB for each 3 minutes of video, so a two hour movie would take about 40 GB (a good argument for splitting the encoding into sections).

The amount of time saved is harder to quantify. If encoding directly proceeds at 0.1 rt, encoding from the intermediate avi proceeds at 2.0 rt, and you're doing 5 pass encoding in CCE of a 23 minute anime episode, then you'll save:
(23/0.1 * 6) - (23/0.1 + 23/2 * 6) = 1081 minutes.
Multiply that by 4 anime episode per DVD, and you're talking some pretty serious time savings.

[meleth]

Originally posted by Kedirekin

The answer to both questions depends on how many filters you're using and how slow they are.





The short answer is, the time required to create the intermediate avi is the time it would take to do one pass. Using Huffy (at least on my PC), the avi requires about 1 GB for each 3 minutes of video, so a two hour movie would take about 40 GB (a good argument for splitting the encoding into sections).





The amount of time saved is harder to quantify. If encoding directly proceeds at 0.1 rt, encoding from the intermediate avi proceeds at 2.0 rt, and you're doing 5 pass encoding in CCE of a 23 minute anime episode, then you'll save:


(23/0.1 * 6) - (23/0.1 + 23/2 * 6) = 1081 minutes.


Multiply that by 4 anime episode per DVD, and you're talking some pretty serious time savings.



Divx is ofcourse far superior to svcd. Only thing svcd is good for is wasting hd space. :P

[Blight]

Don't start this DivX vs SVCD stuff.

DivX is better for computer based playback and SVCD is better for set-top DVD players.

Personally, I have an HTPC and can play anything on everything.

[pandv]

However, when you do IVTC in AviSynth (regardless of which IVTC filter you use - this isn't a decomb problem), all the analysis has to happen on each pass, so again you loose the speed advantage.

I programmed only a not useful avisynth filter (to get a random % of a movie), but I think is possible to do the IVTC (the analysis) only one time.

A possible method:

- Add another parameter with a file path/name.

- If this file doesn't exists, Decomb creates it, and saves the analysis results in the file.

- If this file exists, Decomb reads the analysis results from it.

The user needs to remember to delete the file (or use another file name) for diferent films, but the avisynth script is the same for the two passes.

If the file parameter doesn't exists, decomb works normally.

What do you think neuron2? It's possible?

pandv

Edited to obtain a language more similar to english.

[Guest]

@pandv

Consider Telecide. The actual savings could be in storing the decisions about which fields match up. We'd still have to actually do the field copying to assemble the recreated frames on each pass. So maybe we can save 1/2 to 2/3 of the work for that. But Telecide also has a postprocessing phase and if that is enabled we'd have to save a motion map for each frame. And of course, the actual deinterlacing would have to be repeated on each pass. So maybe we are talking about a 50% saving in time for Decomb on the second pass but at the expense of a massive store. That store might as well be an intermediate AVI.

And Decomb is only part of the processing during a pass. If it's (say) a third of the total processing, then we'd only net a saving in time of about 1/6.

Daxab has an easier time with this because his IVTC4 does no postprocessing. Even there, he can't avoid all the rework in the second pass.

Finally, since I do not use 2-pass, I simply am not motivated to do anything about it. :)

[meleth]

Originally posted by neuron2
@pandv



Finally, since I do not use 2-pass, I simply am not motivated to do anything about it. :)

You mean to say that there's still people who only do single pass encodes??? :P

[Guest]

Hey, I'm a newbie; I wouldn't know how to do a 2-pass if my life depended on it. I know, read the guides?

[meleth]

Originally posted by neuron2
Hey, I'm a newbie; I wouldn't know how to do a 2-pass if my life depended on it. I know, read the guides?

hehe yeah, with nandub or gknot it's fairly easy to make 2pass encodes though.

[daxab]

@pandv

As far as IVTC4 goes, most (but not all) of the work is saved in the cache file. It is of course possible to save all of the state, but it is intentionally not done this way. The reason is that if you change parameters, you should still be able to use the same cache file. For instance, you don't want to reject the cache just because you've bumped the motion threshold from 1.10 up to 1.50 to improve your matching. (Edit: That doesn't always improve matching. Just an example.)

It's quite possible to have the best of both worlds, since in a multi-pass encode, you won't change any parameters from pass to pass. The cache can keep all the frame data, and just recompute the decisions affected by changed parameters. However, the amount of recomputation left when the cache is fully populated is little enough that it isn't really worth doing this.

[dsmith]

How about a slightly different take? I don't care about multiple passes because I'm not using it do an encode. Rather, I'm serving the video straight into Premiere for editing. 1) Can I be sure that every time I scrub through that section of video that I'm going to get the same selection of frames? 2) Is there going to be a lot of overhead from recalculating this information each time?

This is probably more difficult to deal with since I'm not going through the entire video sequentially. I guess I'd be fine as long as I could be assured of #1 always being true. IVTC4 looks like it might be better in that regard due to the caching, but I'll need to read up on exactly how it works.

--
David

[daxab]

@dsmith

1) IVTC4 guarantees* that the same frames are served with each pass, with random access, front to back playing, or back to front, etc. No matter what you do the same frames are served.

2) There is essentially no additional** overhead if you are in the same "session". That is, once you open the .avs file, each frame's decision is only made once. If you close the .avs file, the decision data is lost -- unless you use a cache file. Then the most costly decision data is saved in the cache file, and not recomputed. This makes operations during a second opening of the .avs file much faster (for frames that were already visited during the first session).

*If this isn't the case then it's a bug.

**Beyond that needed by the codec to decompress the frame again.

[dsmith]

@daxab

Then the most costly decision data is saved in the cache file, and not recomputed.


I might be misunderstanding here, but why not simply save the final decision itself? It seems like that would only require a few megabytes of data to save, instead of the (from web page) .5 to 1.5 gigabytes. Or is it because it's support information for non-computed frames? If the entire video is complete, could it drop down to only final decision info?

Edit: played around with it some, and it seems to only generate a 300K cache file, *much* more managable than what I was afraid was going to appear.


--
David

[daxab]

@dsmith

It's intentional not to save the final decision list. I talk about this on another thread. The idea is that if you change, say, the motion threshold parameter, you can still use the cache file. It saves only low-level, expensive computations.

Yes, :) that's a typo. The cache file is 0.5 to 1.0 MB, not GB.

I need to fix that.

[opp]

@daxab

do you know how program in c++ or something? ever considered making a little app that will analyize the video or avs file and actually process it before you pass it though, similar in a way to what tmpeg does?

[daxab]

@opp

Yup -- C++ programming is my day job :). I have been thinking of what you suggest, just today in fact. It would certainly make certain kinds of analysis possible that are not possible the way IVTC4 is written now. But I'm shying away from it because I like to be able to process on the fly. It's nice to quickly load a .avs file into VirtualDub, scrub around, and see what's happening -- as opposed to initiating a long analysis phase, going away for an hour, and then seeing the results.

[Roginator]

What *I* would like to see is a program that runs through a video and analyzes the heck out of it.

It would return useful info like histograms and would actually suggest settings for brightness, contrast, gamma, hue, saturation, intensity, IVTC method (if any), whether temporal noise reduction would be advised, whether spatial noise reduction would be advised.

You get the idea.

[daxab]

@Roginator

That's a good idea. Sort of an "auto levels" for video. It could find scene changes too, and maybe distinguish between interlaced and telecined segements of video. Mabye it could dump out a "master processing list" which you could read and edit. Then another tool would take the master processing list and perform all specified adjustments.