Interlaced video tff/bff...hangon...tff again WTF?.

[Arkay]

Greetings guys. First post, nice forum!!!

I need to ask some questions relating to pal interlaced content and encoding to divx format.

I've read absolutely every piece of information I can find scattered throughout this and other forums and I still am unable to come to a descision. I am trying to encode STTNG and I have found that not only is the content interlaced but the field order swaps here there and everywhere, like they used 1 camera with tff and another with bff when filming and just slammed it all together in the cutting room.

I've been through the process of reduceby2, greedyHMA, smoothdeinterlace (to 50fps progressive) and none of these provide what I am looking for and that is, divx playback as good as the original.

Smoothdeinterlace does the best job but hates the reversing of field ordering as you'd expect and I'm a quality junky, can't have even one scene that jerks all over the place.

What I need to know is, is there any plugin that exists or is it even possible for a plugin to detect a field order shift from one scene to the next and auto compensate?

If that is not an option I have also considered keeping the video interlaced but am yet to find enough evidence to suggest that I can play it back in a truly interlaced form (via TV out). Some say avi cannot handle true interlaced content, others say it's divx that can't handle it. The option of "All video is interlaced" in the divx codec remains unexplained and even if all the above can be done there's no telling what field order your tv-out will use so it still may not work. (Not to mention you have to encode at the original vertical resolution).

Is it possible to use separatefields, encode the 720x288 at 50fps and use an output filter (like ffdshow), to splice the progressive half height frames back together into interlace fields for output on TV? (Yes I've read everything on 100fps :)

or alternately, which would be even better, keep the interlaced content (in a true interlaced form), and have an output filter that can take the fields, bob and 2xscale vert res on the fly? i.e. Instant 50 fps progressive playback on progressive displays, or, pristine smooth interlaced video for interlaced display, all from the original source?

Please please please, someone with the knowledge or someone who has done STTNG please let me know what you did. At this point the only way I can see of getting it 100% right is to decode the entire video, cut it into scenes, reverse the field ordering on the scenes that are wrong, cut all the scenes back together and then encode/mux audio and hope it all works.. Unfortunatley I don't have a week per episode of time to spend ;)

Any help, suggestions, etc will be very much appreciated.

Cheers,
Arkay.

[trbarry]

Have you tried DeComb? I thought it could handle that sort of thing.

- Tom

[-h]

I thought TNG was shot on film, and could just be IVTC'ed to give the 24fps source.

-h

[Ookami]

http://forum.doom9.org/showthread.php?s=&threadid=35106

http://www.stud.uni-karlsruhe.de/~usng/virtualdub-filter/TBDI_Doc/index.html

Good luck :) .

[Arkay]

Thanks for the replies guys. Unfortunately most of the other posts refered to deal with NTSC content, Telecined etc. I am ripping the PAL 25 fps of these discs.

The really wierd part and what I can't understand is that I've been using the SmoothDeinterlacer avisynth plugin which works a treat. It means you're outputting at 50 fps as each field is taken and delaced and then I scale vertically up to 480 (from 288). That's fine and motion is very smooth as it should be. The thing that I can't understand is that 90 percent of an episode appears to have top field first and I have no problems. Every now and then there is just one scene that appears to go to bottom field first. Using the smoothdeinterlace at 50 fps when this happens I end up seeing the frames in the wrong order, causing that scene to shimmer dramatically. That is, frames that should be ABABABAB come out as BABABABA, where frame B is a later point in time than frame A.

I concluded from this that there is no easy fix. What I don't understand is that playback of the original DVD to an interlaced display appears fine. Shouldn't you be able to notice a frame order reversal on a TV as well? If a scene switches from top field first to bottom field first how does the TV know which order to display them in?

Since I own the original DVD's, and it's legal to make backup copies of your originals, I'd be very interested to hear from someone who has done all this already, maybe I can just pay someone a reasonable fee to copy their copies for me and mail em over?

I'm nearing the give up point where I'm just going to use reduceby2 and live with the choppy playback...At least encoding won't be so slow.

Cheers,
Rick.

[mpucoder]

I'm not going to jump too deep into this, but did you consider that, although you have the PAL version, ST-TNG was originally made for NTSC? If, as one thread suggests, the series is hybrid, the problems could be traced back to 29.97 fps source.

[Guest]

Even with PAL, you can try applying Telecide(). Just don't add Decimate().

[wasco]

yeah i've had the same problems when trying to rip friends episodes from a pal disk. It seems that when they converted the original tapes to dvd ntsc and thereafter to pal they did a lousy job and now the interlace-lines are blended in the picture so it's almost impossible to get a clean image.
However, I spend almost 2 weeks trying out different filters and stuff and the avisynth script that gave me the best result is the following:

LoadPlugin("c:\rip\Software\aviSynth\MPEG2DEC.DLL")
LoadPlugin("c:\rip\Software\aviSynth\decomb.dll")
LoadPlugin("c:\rip\Software\aviSynth\SimpleResize.dll")


mpeg2source("c:\rip\files\FRIENDS\friends.d2v")
Telecide()
Crop(12,4,696,568)
SimpleResize(448,336)

so, to be short:=> like neuron2 says, stick with telecide!

[Guest]

>like neuron2 says, stick with telecide!

Some of the other IVTC solutions work very well, it must be pointed out. Decomb comes into its own in problematic situations, because it is flexible and willing to make some compromises that more rigid solutions do not. But for some clean clips, you may get better results from the other approaches.

So what I say is "try everything and use what works best for the problem at hand".

[Arkay]

Thanks for the advice guys.

I've actually had some luck, seems all but the first episode have their frame ordering right (in fact the first episode problems may now be resolved as well, still have to run some tests.)

Basically though the way I'm ripping these whilst time consuming is providing great quality.

My script looks something like this:

LoadPlugin("c:\rip\Software\aviSynth\MPEG2DEC.DLL")
LoadPlugin("c:\rip\Software\aviSynth\smoothdeinterlacer.dll")
mpeg2source("c:\rip\files\FRIENDS\friends.d2v")
trim(1,'whatever')
crop(12,4,696,568)
smoothdeinterlace(doublerate=true)
BicubicResize(640,480,0,0.75)

from memory.

This of course means you end up with a 50 fps progressive rip but that is where the smoothness comes from. As far as compressing double the frames the divx codec doesn't seem to care and the filesizes come out correct as Gknot suggests for 25 fps material.

Playback is fine on a PIII 733 so I have no complaints.

The only downside is encoding time, roughly 6 hours per episode on a P4 2.0Ghz.

Cheers,
Arkay.

[dragongodz]

just wondering if you have tried using DVDx,Xmpeg,Flask etc to see what results you get with them. would be worth the effort to test at least.

[Arkay]

I imagine I will get around to testing out some of the other codecs. To be honest I only got into this a few weeks ago and have gone through a fairly hefty learning curve. I'm very impressed with what these codecs can do, and the work being put in by the guys coding filters etc, all excellent stuff.

I'm hopefully getting a DVD writer (Pioneer DVDR-A05) next month, then I'll start looking at some other options. Like maybe working with mpeg2 and vobs, specially for home video (DV) stuff.

Cheers,
Arkay

[Xesdeeni]

Arkay:The really wierd part and what I can't understand is that I've been using the SmoothDeinterlacer avisynth plugin which works a treat. It means you're outputting at 50 fps as each field is taken and delaced and then I scale vertically up to 480 (from 288).There's something amiss here. If you have doublerate=true from a PAL source, then you should be getting 50 frames that are 576 lines tall.That's fine and motion is very smooth as it should be. The thing that I can't understand is that 90 percent of an episode appears to have top field first and I have no problems. Every now and then there is just one scene that appears to go to bottom field first.I concluded from this that there is no easy fix. What I don't understand is that playback of the original DVD to an interlaced display appears fine. Shouldn't you be able to notice a frame order reversal on a TV as well? If a scene switches from top field first to bottom field first how does the TV know which order to display them in?As you have realized (and I did while I was working on the SmoothDeinterlace port) that this is indeed impossible. In order to reverse the order of the fields, a field must be skipped, and that will almost never happen. If it does, it will DEFINITELY be visible on a regular TV (as you mentioned).Using the smoothdeinterlace at 50 fps when this happens I end up seeing the frames in the wrong order, causing that scene to shimmer dramatically. That is, frames that should be ABABABAB come out as BABABABA, where frame B is a later point in time than frame A.Have you tried complementing the parity (using tff= or ComplementParity())?

The other thing to ensure is that nothing between the VOB and SmoothDeinterlacer is affecting the stream. I don't know what some of the settings in DVD2AVI do, but I'm guessing "Force Film" would be bad ;) . Let me know what luck you have. But I'm certain that you should not have to trim off that first frame.

Edit: Oh, another thing to take a look at is what Bob() does. This will eliminate SmoothDeinterlace from the equation and see if it's really my fault :(

Xesdeeni

[Arkay]

Originally posted by Xesdeeni
Arkay:There's something amiss here. If you have doublerate=true from a PAL source, then you should be getting 50 frames that are 576 lines tall.

Ok. Now I'm really confused. I'm fairly new to this stuff so bare with me. I initially went though a phase of using separatefields and using the VirtualDub Smooth deinterlace filter. This is where I saw the symtoms mentioned above. After some searching I found the avisynth version (great work), and started fiddling with that. In the end I think all my problems came from selecting the wrong field ordering in dvd2avi. I have since been able to get smooth 50 fps rips of all the episodes I had previously had problems with. The only thing I've noticed though is that I must specify a trim of (1,xxxx) rather than (0,xxxxx) or smoothdeinterlace barfs up.

I had assumed that with doublerate=true the filter would take a field of 288, and remove deinterlace artifacts via comparison with it's partner field. Then I figured I would have to manually scale back up to 480 height. Is this not the case? I'm sure I tried to use smoothdeinterlacer without vertical resizing to see what I got..

My scripts look like this:

LoadPlugin("C:\APPLIC~1\Video\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\APPLIC~1\Video\GORDIA~1\smoothdeinterlacer.dll")
mpeg2source("E:\StarTrek TNG (2-046) Peak Performance.d2v")

trim(1,64209)
crop(14,2,696,570)
smoothdeinterlace(doublerate=true)
BicubicResize(640,480,0,0.5)

Please let me know your thoughts.


The other thing to ensure is that nothing between the VOB and SmoothDeinterlacer is affecting the stream. I don't know what some of the settings in DVD2AVI do, but I'm guessing "Force Film" would be bad ;) . Let me know what luck you have. But I'm certain that you should not have to trim off that first frame.

Edit: Oh, another thing to take a look at is what Bob() does. This will eliminate SmoothDeinterlace from the equation and see if it's really my fault :(

Xesdeeni

I think it was just me being stupid with field orderings. Funny thing is if I preview playback in dvd2avi it actually appears to be more "progressive" with no field order swap yet it will only encode properly with field orders reversed.

Not to worry anyway, I'm getting great results with the above script. Thank's for the reply!

Cheers,
Arkay

Hmmm. Now I've put all that down in text I'm beginning to wonder if the reason it's working the way it is is because I've reversed field order and used trim(1,xxxx).... I might go try normal field order with trim(0,xxxx)....

Edit: Nope, trim has to be 1, frame order normal causes field B to be shown before field A where A is the first in time. My mention of scaling from 288 up to 480 was a load of crap (I think I must've been thinking separatefields, I went through many many possibilites whilst working out this interlace puzzle and the memories seem to blur a bit)..

The scale to 640x480 is purely only for size considerations as suggested by compressability tests from Gnot.

[Xesdeeni]

I had a bug in SmoothDeinterlace regarding how it handled input from MPEG2DEC.dll, and I think I recall that a Trim(1,0) may have also fixed the problem before I updated the plugin. Make sure you have version 1.4. If it still fails, let me know (I've probably got another bug to track down :( ).

You can also override the polarity with either ComplementParity() or adding tff=false (with should be the opposite of what comes from DVD2AVI).

Xesdeeni

[Arkay]

Originally posted by Xesdeeni
I had a bug in SmoothDeinterlace regarding how it handled input from MPEG2DEC.dll, and I think I recall that a Trim(1,0) may have also fixed the problem before I updated the plugin. Make sure you have version 1.4. If it still fails, let me know (I've probably got another bug to track down :( ).

You can also override the polarity with either ComplementParity() or adding tff=false (with should be the opposite of what comes from DVD2AVI).

Xesdeeni

Ok. I'll download the latest and give it a try. Results to follow :)

Cheers,
Arkay.

Edit: ok... Tried it out and you're right, the need for trim(1,0) is now gone :) Thanks for the advice. I tried a section of STTNG with both field order normal and reversed (from dvd2avi) and strangely couldn't see a difference between the two regarding smooth motion. No jerking symtoms as I've mentioned above... Strange...

I've noticed with alot of this encoding it seems that your machine "learns how to do it" by itself :) Same with playback, I was playing my 50fps stuff back on my work laptop and it was skipping, next day it skipped less, now it doesn't skip at all and I haven't changed a thing... Wierd... Guess it has to do with the phase of the moon :)

Anyway, thanks again for your work on the plugin, much appreciated!

One other thing though that might be worth mentioning is this script:

LoadPlugin("C:\APPLIC~1\Video\GORDIA~1\mpeg2dec.dll")
LoadPlugin("C:\APPLIC~1\Video\GORDIA~1\smoothdeinterlacer.dll")
mpeg2source("E:\conversions\test.d2v")
Smoothdeinterlace(tff=true, doublerate=true)

fails with an Avisynth script failure: Invalid arguments to "smoothdeinterlace" in test.avs atline 4.

tff=false also fails.

I've not been able to get tff working in any version I've downloaded?

-Arkay

[Xesdeeni]

You are the second person to report this, but I cannot duplicate the problem on my end. What's worse, the way you specify the arguments is exactly the same for all of them (take a look at the very bottom of the code) and is handled by the AVISynth code, not the plugin, so I have no idea how to even guess at the issue. I'll keep trying, but in the mean time you can always use AssumeTFF() or AssumeBFF() as a workaround.

Xesdeeni

[Arkay]

Don't get stressed over it, there are plenty of ways to change field ordering, although I know how it feels when you've got a bug that you can't duplicate or find. I checked out the code just to see what the args should be and couldn't see anything wrong with anything. Maybe it's an avisynth problem, not sure. At any rate everything else appears to work fine. I did notice with an earlier version that some combinations of arguments wouldn't work too, at one point I attempted to pass every argument to smoothdeinterlace but didn't have luck with that either. In the end I settled on the smoothdeinterlace(doublerate=true), which for all intents and purposes gives me the best looking output anyhow, with the least amount of time consuming tweaking :)

Cheers,
Arkay.