How to frameserve variable-framerate MPEG2

[temporance]

Suppose I have a NTSC MPEG-2 file which contains a mixture of 24fps telecine (3:2) and 30fps interlaced material.

Here is an example with the MPEG-2 pictures numbered

<------24fps-----><-30fps->
[ 0 ] [1] [ 2 ] [3] [4] [5]
60Hz fields ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^

I want AviSynth to frameserve a 60fps AVS like this (* represents a null frame):


0 * * 1 * 2 * * 3 * 4 * 5 *
60Hz AVI frames ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^

Is this possible (with or without small modifications to the code)?

[sh0dan]

Dropped frames currently cannot be sent by AviSynth.

Could you post a small section of the MPEG2 file containing this structure - I have never seen one like this.

[bilu]

Wouldn't it be better to use the hybrid handling modes from Decomb?

Bilu

[temporance]

@sh0dan: An awful lot of NTSC DVD extras have sections like I've described - they have clips from the movie @24fps and extra material (CG or video) @30fps.

@bilu: I could but ideally I'd like to preserve the exact timing of the video frames as they exist in MPEG-2. I presume decomb always produces a fixed frame rate AVI @24fps, for example.

Thanks anyway, I guess it's harder than I first thought.

[digitize]

Hmmmm, 60 fps.... I've never heard of one doing that. Now many people use 120 fps for hybrid material (both 24 & 30 go into 120 evenly) but i've never heard of 60.

[temporance]

Originally posted by digitize
Hmmmm, 60 fps.... I've never heard of one doing that. Now many people use 120 fps for hybrid material (both 24 & 30 go into 120 evenly) but i've never heard of 60. 120fps would work just as well, if not better. With 120fps you can restore the original 24fps without any of the 3:2 jitter that you get at 60fps. Although I want to restore the original NTSC MPEG-2 timings, warts and all - and this means keeping to 60fps.

digitize, how do the people who use 120fps for hybrid material make their AVIs? Not with Avisynth, presumably?

[Wilbert]

http://forum.doom9.org/showthread.php?s=&threadid=49561&pagenumber=2

[temporance]

Cheers, Wilbert - seems like this is quite a struggle!

I'd like to bring 60 / 120 fps hybrid support to AviSynth (such that MPEG-2 is decoded whilst respecting the timing information carried by repeat_first_field). But I don't know where to start. I'd need to do the following things:

1. Introduce the concept of a null frame (AVI frame length = 0) to AviSynth.

2. Modify the MPEG-2 decoder so that it can decode NTSC and produce a 60 or 120 fps output so that repeat_first_field is respected.

I am not at all up to speed with AviSynth code so I really want to know if this is possible and desireable before investing my time.

[AS]

Excellent news. the new vdub 1.53 can now add in drop-frames in direct stream mode.

So no more of the 60fps/120fps, you can just raise the frame rate to 30 fps(or 29.97) with accuracy and with far less overhead :)

I tested on some anime content and I get approx 11 bytes/frame (drop-frame) overheads

Grab it over virtualdub.org :)

So basically you can encode the film and video part separately as you normally do and add in drop-frames at a later stage.

[Bidoche]

@temperance

Shouldn't be null frames more the concern of the encoding application than of the video processor.

Null frames arises difficult issues for video procession and avisynth in particular :
Avisynth handles frames out of context, ie it can recombine frames without knowledge of previous ordering or ...
Obviously that's not the case for null frames.
The point is : How filters will process these null frames ?


Besides we already have a kind of null frame : a copy, since frames are handled by pointers, it costs nothing

[temporance]

@AS: I'm not sure what you mean - what does vdub now do to the video?

> So basically you can encode the film and video part separately as you normally do and add in drop-frames at a later stage.

But I don't want to encode film and video parts separately!! I want to run the whole hybridclip through the same process and have it handle everything perfectly!!


@Bidoche: Well, a null frame just means "this frame is identical to the previous frame". Within Avisynth I guess this could be represented using pointers.

Let's say I have frame n+1 identical to frame n. This would be pretty common in a 60 or 120fps AVI. As a user of the .AVS I don't want to have to render or encode the same frame twice. It's just a waste of time.

So, internally to AviSynth, frame n+1 and frame n would comprise pointers to the same image buffer. But externally (in "AVI" or "VfW" terms), it's more efficient to represent frame n+1 as a null (zero length) frame. All Windows apps understand this - it means just repeat the previous frame.

My question now is: when a frame is output from AviSynth, is it possible to say:

// dodgy pseudocode....
void frameOutput(frame) {
if (framesAreIdentical(frame, previousFrame)) {
send(0, 0);
} else {
send(frame, sizeof(frame));
}
}

?

[sh0dan]

... but your pseudocode does not make anything faster!?!

Every frame still have to go through the entire filter chain before it can be compared to the previous. Furthermore filters that use temporal information (many come to mind) will make differences between duplicated frames appear.

A drop-frame mehtod could be implemented, but I'm not sure if this is the job of AviSynth. IMO codecs should be the ones dropping frames. I'm not even sure if VirtualDub passes through dropped frames - meaning: If the input is a dropped frame, will output also contain a dropped frame, or will it be encoded as all other frames?

[Bidoche]

If there is a way to pass the info, that identical frames are being served in a row, to VFW; Then maybe we can consider using it.

At our point, a frame comparison (pointer comparison) is nothing, which is not the case for codecs looking for nul frames, so it may be worth...

Supposed everyhting works right for the client application, but I guess that if it's part of VFW, they should handle it.


Edit: if someone familiar with VFW could tell us if it handles null frames, it would help. (I have no idea)