help with field operation

[graysky]

I'm using dgindex 1.4.0rc6 now that dvd2avi 1.77.3 failed to give me an avi (after vdub encoding) that would sync with the audio it extracted for me. I'm noticing there are some new (to me) field operations, namely, honor pulldown flags and ignore pulldown flags.

When previewing my source, my frame type bounces from progressive to interlaced and the video type from film 70% or so to ntsc 30% as the thing previews. I read the Essential reading: Force Film, IVTC, and Deinterlacing and more (http://forum.doom9.org/showthread.php?t=34696) thread but it was unclear to me what to use in this case.

Can someone point me to a guide or help me out with this? Thanks!

[Guest]

Following is what Cyberia and I wrote in the soon-to-be-released users manual for DGIndex. After that I address your specific problem.

-----
MPEG2 video provides a syntax element (RFF flags) that allows a single field of any frame to be repeated automatically during display. The extra fields do not occupy space in the MPEG2 stream but are simply created by copying when the stream is decoded and displayed. This facility is most often used to display film rate clips (23.976 fps) at the NTSC rate (29.97 fps), a process called 3:2 pulldown. In practice, however, pulldown in various different patterns is commonly encountered. It can be used, for example, to display 25 fps PAL at the NTSC rate.

Understanding the theory and practice required to properly set this option for a given application is nontrivial. Here we can only define the operations and give some general guidance. There is ample additional material available in existing guides.

The Field Operation setting allows the user to specify how the pulldown (RFF) flags are to be handled.

Honor Pulldown Flags - The pulldown flags (if any) are obeyed and the fields are repeated. This means that the frame-served video will appear exactly as it is intended to appear on the final display device. Therefore, if you have a 3:2 pulled-down clip, you'll get the standard repeating pattern of 3 progressive frames followed by 2 interlaced frames. If you have PAL or MPEG1, you'll just get the encoded pictures with no repeated fields (because PAL and MPEG1 have no RFF flags to honor). The frame rate will always be the same as the display frame rate of the source.

Ignore Pulldown Flags - The pulldown flags are ignored. This allows one to obtain the raw encoded MPEG pictures, with no repeated fields. However, because repeated fields intended for display are ignored and not displayed, the resulting frame rate may differ from the source frame rate. It may even vary throughout the clip, due to irregular patterns of pulldown flags. If the pulldown is irregular, use of this option will cause the audio-video sync to change at different parts of the clip, and most likely sync will not be acceptable. This option is mostly intended for power users, who would use it as a diagnostic aid for inspecting the encoded MPEG pictures. Although this option ignores the flags, they are still stored in the D2V file although DGDecode will also ignore them.

Force Film - This option is intended for the special case of film content encoded at 23.976 fps but pulled down for display at 29.97 fps, i.e., 3:2 pulldown. This option will restore the 23.976 fps film rate while keeping audio and video in sync throughout the clip. This option works by ignoring pulldown flags and inserting/removing frames to maintain a constant and sync'ed output stream. Do not use this option on non-3:2 pulled-down material, and, therefore, never use it on PAL or MPEG1.

The Field Operation is perhaps the most important option to understand when using DGIndex. Most users should first preview the project using the Honor Pulldown Flags option. If the clip is not PAL or MPEG1, and the Video Type field in the Information window indicates FILM 95% or higher, it is likely that the clip can be treated as 3:2 pulldown material, and so the Force Film option should be selected for generation of the D2V file.

If the percentage is much lower than 95% or the Video Type indicates an NTSC percentage, then you should leave this option set to Honor Pulldown Flags for generation of the D2V file. If the result is combed (because the source is interlaced, hybrid interlaced/3:2, field blended, etc.), you may want to apply a deinterlacer or inverse telecine (IVTC) filter in your Avisynth script.

If the film percentage is low but still mostly film, you can try using Force Film and see what you get. You may find some stray combed frames in the output. You can fix those by post-processing with FieldDeinterlace(full=false). If the result is satisfactory to you, then fine. If not, there are more advanced ways to handle hybrid, hard-telecined, field-blended, and other pathological video. These more advanced methods almost always require the video to be served with the Honor Pulldown Flags option.

When assessing the FILM percentage using a preview operation, it is important to use a lengthy portion of the film itself, because introductions and credits are sometimes rendered differently than the main movie content.

Almost invariably, if you know the source is not mostly 3:2 pulled-down NTSC, you will want to select Honor Pulldown Flags and then post-process as required. Keep in mind that if there are no pulldown flags, there is nothing to honor, so this option just delivers the encoded frames.

The Ignore Pulldown Flags option should be necessary only for advanced users who want to see just the exact coded pictures in the MPEG stream.

Note that the Honor Pulldown Flags option was formerly called None, and the Ignore Pulldown Flags option was formerly called Raw Encoded Frames.
-----

Please report the final film percentage from the bottom of the D2V file. I suspect based on the range you gave, that your clip is hybrid material. You can try Force Film, postprocess if necessary to remove stray combs, and see if you like the result. If not, then you can use Decomb in one of its hybrid modes. Also check whether you have field-blended video.

[graysky]

Thanks for the info, man. Can't wait to read the full version of document you've clearly put a lot of time into. Seems as though my biggest problem is getting the audio and video to sync. As to your questions:

FINISHED 76.35% FILM (using the honor pulldown flags). I'd be interested in reading a more basic guide that would detail how one needs to preform these postprocessing functions you mentioned. Can you point me to a good one or two?

[Guest]

Seems as though my biggest problem is getting the audio and video to sync. Save the project using Honor Pulldown Flags and then serve the D2V into VirtualDub and use File Information to determine the time length of the video. Similarly, determine the time length of the demuxed audio by loading it into a player that can play it. Are they the same to under a second? You should not have any audio sync issue other than a small offset that can be corrected when you encode. Please describe what kind of problem you are having. For exanmple, is the desync constant throughout the file or does it grow in size?

FINISHED 76.35% FILM (using the honor pulldown flags). That's a bit low for force film but it should still work and give sync'ed audio. Did you try it?

I'd be interested in reading a more basic guide that would detail how one needs to preform these postprocessing functions you mentioned. Can you point me to a good one or two? You can look at the Decomb documents.

[graysky]

Save the project using Honor Pulldown Flags and then serve the D2V into VirtualDub and use File Information to determine the time length of the video. Similarly, determine the time length of the demuxed audio by loading it into a player that can play it. Are they the same to under a second?

Did it with honor pulldown flags. Then used this simple script:

LoadPlugin("E:\Proggies\Video\DGIndex\DGDecode.dll")
MPEG2Source("d:\van.d2v")

VDMOD gives 1:34:05.984 for audio and 1:34:06.480 for video.

You should not have any audio sync issue other than a small offset that can be corrected when you encode. Please describe what kind of problem you are having. For exanmple, is the desync constant throughout the file or does it grow in size?

The last time I did the encode job, then muxed the audio, the desync was indeed constant throughout... that was however with a d2v file generated from dvd2avi 1.77.4 using forced film. I will try it with RC6 of DGINDEX using

That's a bit low for force film but it should still work and give sync'ed audio. Did you try it? You can look at the Decomb documents.

I can try FF using DGINDEX 1.4.0RC6 which I haven't done yet. Do you think I'd be better off using FF or Honor Pulldowns? I will check out the decomb docs. Thanks again for the time :)

[Guest]

Regarding your sync issue, it looks like you just have a constant offset that is easily corrected during encoding.

I can't really guess about what would be better for your movie.

You most likely have hybrid material, i.e., there are both 3:2 pulldown sections and full rate video sections mixed in your movie. You therefore have a Hobson's choice between rendering the final output at film rate or video rate. If you choose film rate, the video sections will be a bit jerky. If you choose video rate, the film sections can be jerky on pans.

The easy way out is just to use Force Film. It may be good enough.

IIRC, the combination of Decomb + Yatta handles hybrid material pretty well.

Hybrids are challenging but will teach you a lot about video. :)

[graysky]

I'll try the decomb plugin when I get a bit more free time.

Is there a way one can preview the d2v file (through the avs script) with the audio track (to check for sync issues) before spending the 4+ hours to do a 2-pass xvid encode? Thanks again for the help.