PAL shifted fields

[wim]

hi guys. i am trying to encode a short film called "mr ikegami's flight", the source is pal dvd. when i advance frame by frame in the preview, there is a pattern of 13 progressive frames and then 12 frames with combing. some investigation: i tried creating the d2v with swapped field order and this causes positions which were combed to look progressive and vice versa. so i don't want to use a deinterlacer because the progressive information is all 'there', we just need to somehow swap field order every second. so is this the type of source mentioned in "THE GOOD: PROGRESSIVE DVDs" in the ivtc tut? what is the best way to recover the progressive information? i tried using telecide in the avs and this gets repeat frames every second, so i killed them with decimate. but i am not entirely happy with the results because it seems to make ghosting around objects..eg some skin colour seems to be smeared off people into the background.
another question i have is, why is this all happening? if i am understanding it right it's because the film is shot at 24 fps and another frame is added to get 25 fps pal. so shouldn't we be able to just swap field order every second and throw away that extra frame with no loss in quality? at the moment i am still seeing some ugly frames (usually at scene changes) ie ugly frames with combing or messed up colours - so i think i am doing something wrong

the avs:

SetWorkingDir("C:\PROGRA~1\GORDIA~1\")
LoadPlugin("mpeg2dec3.dll")
LoadPlugin("decomb.dll")
mpeg2source("C:\divX\short films\mr ikegami's flight\fswap\fswap.d2v")
crop(18,82,678,412)
Telecide()
Decimate(cycle=25)


ps: is
Telecide()
Decimate(cycle=25)
the same as
Telecide().Decimate(25)

[manono]

Hi-

I'm not sure which version of Decomb you're using, but it's not the latest (which I'd recommend you get). If I'm not mistaken, Chroma=True wasn't on by default in your version (check the doc to be sure). So adding that to Telecide may or may not help. You should probably also add Guide=2 to help out in the PAL field matching. In addition, the problem may be in the source, so you might try commenting out the Telecide-Decimate lines temporarily and adding SeparateFields() to the script to see if the color smearing/ghosting is in the source or not.

another question i have is, why is this all happening?

If they convert film to PAL correctly, they speed it to 25fps and that's all there is to it (the video anyway-audio's another story). But sometimes they'll add another frame every second, but they don't add it as a single frame. I think the idea is to avoid the slight pause that will result from having duplicate frames every second. So they add that extra frame as 2 fields spaced out, so the thing seems to change field order every half second. I think that's right, but neuron2 or hakko504 know more about such things than I (who has never seen a PAL DVD).

And yes, Decimate(Cycle=25) is the same as Decimate(25). But if you have other stuff in the Decimate line (such as Mode or Threshold), then you have to write out the full Cycle=25.

[scharfis_brain]

@wim: but i am not entirely happy with the results because it seems to make ghosting around objects..eg some skin colour seems to be smeared off people into the background.
please try telecide(post=false) instead of telecide().

The ghosting should be gone.
If not, the ghosting is already in your source.

@manono: your explantation is correct!

[wim]

speed up 24 -> 25 fps, do you mean they just decrease the period between each frame a little? that would make much more sense than stuffing around trying to invent another frame, i wonder why on earth it isn't done this way all the time?

okay guys i tried telecide(post=false,chroma=true). there is still ghosting. @scharfis_brain i am reluctant to believe the ghosting is in the source because when i run the same script with and without swapped field order, the ghosting occurs at opposite times. i mean there is clearly a place where there is ghosting present in a "fixed" part of the clip which is gone completely if i use the opposite field order in dvd2avi (but then 12 frames earlier now has ghosting where it was ok before). doesn't this suggest the artifacts are being produced by telecide? (see pic - since geocities are tight you might have to copy and paste the url rather than clicking)

i will post a vob fragment if you want to try it out yourself, but you'll need to tell me how to snip a section out :-)

http://www.geocities.com/wimiam/ghostpen.jpg

[manono]

Hi-

Yes, they just run the movies faster-25 fps instead of 24fps. PAL DVD movies are frequently about 4% shorter (by time) than their NTSC counterparts.

I wouldn't mind seeing a 10 MB or so piece of a vob from your movie. Just get Chopper XP (http://www.doom9.org/Soft21/Vobtools/chopperxp.zip). It's easy to figure out. I couldn't figure out how to see the picture you put up, though.

[scharfis_brain]

@wim: According to your picture: It seems to be, that the colorspace is handled wrong!

[wim]

hi guys, i did eventually work out the location of the problem and fix it, so i now have a beautiful avi with no ghosting, but i thought i'd post here to describe what the problem/solution was because i may have uncovered a small bug in gknot. or this source could just be weird and the problem not repeatable for most PAL dvds upped to 25 fps.

Originally posted by manono
In addition, the problem may be in the source, so you might try commenting out the Telecide-Decimate lines temporarily and adding SeparateFields() to the script to see if the color smearing/ghosting is in the source or not.

thanks manono this idea turned out to be very helpful. i got rid of telecide decimate and looked at the fields and there was still the pattern of ghosting. this of course means it was not caused at all by telecide (and actually telecide worked beautifully in the end). but i was still not convinced the ghosting was not in the source, so i tried removing some parts in the avs to try to locate where the problem was appearing. now when i commented out the resize, virtualdubmod says "Avisynth error: YV12 images for output must have a width divisible by 4 (use crop)". so i tried a crop width divisible by 4 and put the resize back in and all the ugly artifacts vanished. so here was my test script

SetWorkingDir("C:\PROGRA~1\GORDIA~1\")
LoadPlugin("mpeg2dec3.dll")
LoadPlugin("decomb.dll")
mpeg2source("C:\divX\short films\mr ikegami's flight\fnoswap\fnoswap.d2v")
#crop(6,88,704,400)
crop(18,82,678,412)
SeparateFields()
LanczosResize(576,320)

toggling those crops toggles the artifacts on/off. but there is no error messages when the resize to 576 (=0 mod4) is in there, even though the output frames get have the colourspace handled wrong (scharfis_brain guessed right).

i had just used auto crop -> "smart" crop all, which gets to mod2 but not mod4. since we are using yv12 now, maybe "smart" crop should make sure width is divisible by 4 (unless this is just a weird source).
i found some hosting space and posted vob fragment so you guys can try it yourself if you want, and i rehosted that pic

http://www.susanglenn-hume.com/wim/host/ghostpen.jpg
http://www.susanglenn-hume.com/wim/host/doom9.zip


ps: so decimate(25) actually makes the movie 4% shorter, then what happens to the audio track?! it seems gknot handled this automatically for me because audio is perfectly in synch and same length, but i am wondering how does the program actually process a vbr mp3 to make it 4% shorter. is decimation possible or does it have to be resampled completely?

[manono]

Hi-

Glad you got it sorted out. If I gave some helpful advice, it was completely by accident. I didn't even begin to suspect that the crop was at fault. scharfis_brain was much closer to uncovering the truth. :)

so decimate(25) actually makes the movie 4% shorter

No, Decimate(whatever) doesn't change the length or make the audio asynch. Back to the beginning. One common way to prepare film for PAL DVD is to speed it up from 24fps to 25fps. That method shortens the film by about 4%. In those cases the audio does have to be adjusted. The same number of frames being run at a faster framerate makes for a shorter film length. But that didn't happen for your DVD. They converted to 25fps by adding an extra frame every second. By doing that there were more frames, but they were being run at a faster framerate, so the film's length didn't change. By using Decimate(25) you were undoing the process. Fewer frames being run at a slower framerate (back to 24fps now) meant that the film's length stayed the same. Got that? Clear as mud? :)

[wim]

ah ok! easy.
so i guess that is a reason why they sometimes will invent another fps instead of just speeding up the film, it's so they don't have to muck around with the audio track.
thanks for the explanation

[wim]

i stopped using smart crop a while back because it would often muck me around with PAL+YV12 .. nice to see mod4 smart crops in gknot 0.28.7 ;-)