PAL interlaced confusion

[divxbeast]

Hi, I'm trying to convert some PAL DVD movies to DivX but I can't decide what type of deinterlacing or IVTC to perform. I read a guide about IVTC from this site and Hakko504's advices and I'm afraid that I don't understand how to make a difference beetwen DVDs converted from NTSC to PAL and Progressive DVDs. Also, progressive DVDs explanation is so confused for me. Can somebody explain it to me? Thanks

[jggimi]

You don't want to use IVTC on PAL discs, as that will drop your frame rate from 25 to 19 fps.

To determine if a PAL or NTSC disc will need deinterlacing, you will need to look at individual frames. Here's how I do it:[list=1]
In DVD2AVI, make sure Force FILM is not selected.
I select approximately 30 seconds of a scene with movement using the [ and ] buttons.
I save a test.d2v project.
I open the test project in GKnot.
I leave the preview window at full size, because I want to be able to more clearly see interlacing artifacts.
I press the > button to move forward frame by frame through the short scene.[/list=1]

If I see interlacing artifacts -- horizontal lines -- in every frame, I deinterlace.

Since my discs are NTSC, I look to see interlacing artifacts in 2 out of 5 frames. If so, then the content was Telecined. I use either Force FILM or an IVTC, depending on whether I can safely Force FILM or not.

If I can't use Force FILM, then I typically continue using that test.d2v, and play with different IVTC options (usually Donald Graft's DeComb tools -- Telecide and Decimate) in a test.avs script, running it frame-by-frame in Virtual Dub, using the right arrow key, until I get the type of results I want. I usually delete or comment out the resizing command, because I find it easier to see my results in Vdub when the content is original size (720x480 for NTSC).

For a complex PAL disc, such as an NTSC -> PAL conversion, I recommend using a test.d2v and test.avs, trying some of the DeComb suggestions for complex PAL content found in http://www.doom9.org/ivtc-tut.htm and see what works for you, before trying it on a multi-hour encoding.

[divxbeast]

Here is example: movie is Insider (Al Pacino, Rasell Crowe). DVD2AVI report that it is PAL (25fps) interlaced. But if I look throw GKnot or VDub without any filter such as telecide or ForceFILM I couldn't see any interlaced or duplicate frame (I tryed many parts and scenes of the movie). That's means it is progressive film only speedeed up to 25fps. Why then DVD2AVI report interlaced source?

[manono]

Hi-

Why then DVD2AVI report interlaced source?

Because DVD2AVI is just plain wrong many times when it says that. Most PAL DVDs are not interlaced. And most of the ones that do appear to be interlaced really just have the fields shifted and can be fixed by using Telecide() by itself.

This applies only to films. Many PAL TV series DVDs really are interlaced.

[divxbeast]

Originally posted by manono
Hi-

Why then DVD2AVI report interlaced source?

Because DVD2AVI is just plain wrong many times when it says that. Most PAL DVDs are not interlaced. And most of the ones that do appear to be interlaced really just have the fields shifted and can be fixed by using Telecide() by itself.

This applies only to films. Many PAL TV series DVDs really are interlaced.
Is it possible that PALDVD is trully progressive or I am wrong about the specified below (Insider). How to check does it have shifted fields?

[manono]

Hi-

It's real easy to tell if it's progressive (this is for PAL only). Just open your .d2v in GKnot, scroll to a place where there's movement, and then advance frame by frame. If you don't see interlacing, then it's progressive. If it appears interlaced in every frame, then you still can't be sure it's interlaced.

If it appears to be interlaced, then Load the Decomb Plugin and add Telecide(Post=false) to your .avs. Open it in VDub, find a place where there's movement, and start advancing frame by frame. If you no longer see interlacing, then it just had the common "shifted fields" problem, but if you still see lots of interlacing, then you have an interlaced DVD, and you will have to deinterlace.

If you had the shifted fields that were fixed by adding Telecide(Post=False) to the script, change it for the encoding to Telecide(). This adds the deinterlacer back into it to fix any few interlaced frames that might sneak through.

[divxbeast]

Thanks a lot. Your expanation is simple and precise. You solve all my misteries

[manono]

Hi divxbeast-

Good, I'm glad. Credit goes to hakko504 who taught me about these things. I've never seen a PAL DVD in my life. I only pretend to be an expert.:)

[Guest]

Originally posted by manono
I've never seen a PAL DVD in my life. I only pretend to be an expert.Me too. :rolleyes:

[divxbeast]

LOL!!! Great!!! Anyway thanks. I read hakko504's advices over and over but I can't understand some parts maybe becouse my english is poor.

[N_F]

Originally posted by divxbeast
LOL!!! Great!!! Anyway thanks. I read hakko504's advices over and over but I can't understand some parts maybe becouse my english is poor.

I've read it several times too and there are still some things I can't quite understand, specially about that field shifting. It's hard until you actually get the opportunity to work with the problem on some DVD (except for encoding gurus like neuron2 and manono :D )

[manono]

Hi-

Shifted Fields: I think I can explain that one. But it assumes a knowledge of what fields are. Think of the Progressive frame as being made up of two fields (each field being a bunch of thin horizontal video lines-288 of them or half of the 576 making up a frame of a PAL DVD). In this example, the number refers to the frame number, and t=top field , and b=bottom field. The top field refers to all the odd numbered horizontal video lines beginning with the first top one (1,3,5,7,etc.) The bottom field refers to all the even numbered video lines (2,4,6,8,etc.). They all fit together to form the frame the way a deck of cards fits together after you've shuffled them.

Progressive frames look like this:
1t 2t 3t 4t 5t
1b 2b 3b 4b 5b
But progressive frames with shifted fields might look like this:
1t 2t 3t 4t 5t
2b 3b 4b 5b 6b
The bottom fields have been shifted over by one frame. They will appear to be interlaced when you look at them in GKnot. But before going ahead and just deinterlacing them, and thereby lowering the quality of the final .avi by a lot, try and fix them with the methods outlined above. Telecide() can fix that problem very easily (and so can GreedyHMA).

And thanks divxbeast and N_F for mentioning that you are having problems understanding it. If you 2 both mentioned it, there may be many others out there also not understanding and just deinterlacing, when they could in many cases be encoding beautiful Progressive frames. Maybe we'll have to rewrite that section.

[N_F]

Thanks for a very imformative post! When you hear the same thing over and over again in different words you usually understand it after a certain point. You are that point ;).

One question: Does "Swap Field Order" in DVD2AVI have any place in all this? Hakko mentiones it in the document, but it's a bit vague and I'm not sure how to handle it.

It would probably be a good idea to add a little info to that particular part of the document. Especially that part about what top/bottom fields really are and that code where you show how the fields and frames would look like.

[manono]

Hi-

Does "Swap Field Order" in DVD2AVI have any place in all this?

Yes, very definitely. But *only* if they have all been shifted. If you can determine that they have all been shifted (by checking the .d2v very carefully in GKnot in many different places), you can go back into DVD2AVI and "Swap Field Order" and make a new .d2v. Then there will be no need to use Telecide() at all and the encoding will go faster. So here's the way hakko504 recommends that you proceed:

1. Make the .d2v with No Field Operation. Look at it in different places and if everything looks OK (the most common occurance), then go to 4. If everything does not look OK, then go to 2.

2. Make a .d2v with Field Operation-Swap Field Order. Look at it in different places and if everything looks OK, then go to 4. If everything does not look OK, then go to 3.

3. It may be progressive, but the fields may switch back and forth, either very often, or (in some cases) very rarely. Now you need Decomb. So Load the Decomb Plugin and add Telecide(Post=False) to your .avs script (before the crop line is the easiest way to do it, and it doesn't matter which .d2v you use-maybe use the first one to be safe). Open that .avs in VDub and look around. With luck, now everything is fixed, and you can go to 4. But before you do, change Telecide(Post=False) to Telecide() which is the same as Telecide(Post=True). This adds back the deinterlacer to fix any interlaced frames that might slip through. If it is still not fixed, then go to 5.

4. Most films at 24fps are converted to PAL 25fps by just speeding them up. But in some cases they will add an extra frame (actually 2 fields in different places every second) in every 24 to make it 25fps. If you don't feel like looking, then you might skip this step, but don't be surprised if you get a slight pause or jerk every second in the completed .avi. We here at Doom9 like to do things right, so it should be worth your time to do some further investigation. Open the .avs in VDub and find a place with motion and start counting. If you come across a doubled or duplicate frame, then start counting again, and you should see the duplicate frames every 25th frame. Then you want to remove those extra frames, so you add Decimate(25) to the script. If you came here from 1 or 2, then you don't need to add Telecide(). You can just load the Decomb Plugin and add Decimate(25). If you came here from 3, then add Decimate(25) after Telecide().

5. If you have the bad luck to need to read this section, then all the work you did above didn't fix it. Either you have a truly interlaced DVD (common for TV series DVDs, but uncommon for movies), and you must deinterlace, or perhaps you have noticed a lot of ghosting or doubled images. In that case, go back to the tutorial and check out hakko504's recommendations. But nothing can really completely fix the doubled images.

Now do you see N_F why I didn't mention Swap Field Order the first time? It can actually save you time in the long run (if it works), but it may be more trouble than it's worth. That's up to you to decide. But you can actually skip steps 2 and 3 by adding Telecide() to the script if it doesn't look good after step 1.

I think that step 4 is useful in cases when you have the doubled frames, but I don't know how common that is (I think not very common). And if you like to work with silent films, then using Decimate(xx) multiple times can be very useful in returning the film to its original frame rate.

Hey hakko504-did I leave anything out, say anything wrong, or make it sound more complicated than it really is? Is there anything you want to add.

[N_F]

*Saving thread*

Thanks for another very good post. I believe pretty much everything shoud be clear now.

One last (perhaps) question: If I found a duplicate frame every second and added Telecide(cycle=25) it would as I understand it bring the movie down to 24 fps and you'd get 4% more bits to each frame. Has anyone ever encountered this? In that case, what movie(s)? I guess the last question is mostly directed to Hakko since he's the one who's been working with PAL movies.

[Guest]

@N_F

I've never seen a full movie but I have several clips sent to me by people. Such movies definitely exist.

[hakko504]

Hello everybody. I've been working out of town for the last days and just found this thread.

@manono
I think you have described the situation very well. This is exactly how to test a PAL DVD for different cases of interlacing. I'll get back to via mail during the weekend.

@all
About how common the different cases are: First, if the film is letterboxed in any way, it is almost certainly NOT interlaced. So far I have never encountered a letterboxed DVD that isn't progressive. True interlaced stuff is always created as fullframe 4:3 or 16:9.
Then, true progressive DVD's is the most common case. I'd say that 99% of the new quality releases are true progressive (speeded up to 25fps). The stuff you need to watch out for are the cheap ones (and older ones). They can contain almost anything. I'm not sure how common each case is but I have encountered all cases described previously, fieldshifted, fieldshifted changing with regular intervals, fieldshifted changing with irregular intervals, and NTSC->PAL conversions.
I'd like to say something for those out there who do analogue captures: Some capture cards will drop fields to keep synch, creating fieldshifted videos changing with irregular intervals,making this a very common case for captures, even though the original broadcast wasn't fieldshifted. Also note that some broadcasters (like SVT) create stuff as 16:9 interlaced for digital broadcast, but letterbox it to 4:3 for analogue broadcast. This makes the captured clip both letterboxed and interlaced, a situation not found on DVD's.

[Swan]

@hakko504 Also note that some broadcasters (like SVT) create stuff as 16:9 interlaced for digital broadcast, but letterbox it to 4:3 for analogue broadcast. This makes the captured clip both letterboxed and interlaced, a situation not found on DVD's.
I have noted something else regarding SVT too. I have a digital satellite receiver with a Canal Digital subscription and when movies and TV-programs are shown in a format I would describe as "anamorphic" here, my friend, who has digital ComHem, gets a letterboxed version. Movies are shown letterboxed with black matte in Comhem, but "anamorphic" on SVT's digital broadcasts via Canal Digital. We have the same brand and make of TV. I need to set my 4:3 TV in 16:9 mode to view these "anamorphic" broadcasts.
Have you noted this too?

Great thread!

/Swan

[hakko504]

@Swan

Yes, that's the way it is supposed to be. SVT produces a lot of new stuff in genuine widescreen, 16:9, that is transmitted that way in the digital networks. ComHem might be re-encoding the video in letterboxed 4:3 in order to make the video compatible with as many TV's as possible. It is also possible that your friend's STB does the letterboxing automatically. Now, I use Boxer and a Nokia 9820T and if I set the box to 4:3 then the box adds the letterboxing, but if I set the box to 16:9 then it is shown the way it is transmitted. I havent studied ComHem's transmissions closely since I don't have access to their transmissions, but I know everything is OK with CanalDigital and Boxer.

Note for non-swedes:
Canal Digital and Viasat are satellited based digital services.
ComHem is the largest cable company
Boxer is the terrestrial distributor of digital TV

@Swan & N-F
Om det är något specifikt ni undrar över så kan ni skicka ett PM till mig så kan vi fortsätta diskutera på svenska.

[hakko504]

@divxbeast

Just remembered your last question, what the difference is between progressive and NTSC->PAL converted DVD's.
A progressive DVD is a clean source, with nice pictures without blended fields and so on. A (badly) NTSC->PAL converted will have lots of different fields blended into every frame. Worst case is a true NTSC interlaced clip that can produce as much as 5 or 6 different ghosts in every frame in the PAL DVD.

[sdris]

I know this thread is getting old, but it is excellent. It answered more of the questions I had on PAL DVDs than any article I've read so far.

Now for another question :)

I am trying to encode the Simpsons Season 2 PAL DVD, and I've noticed several places where the frames look interlaced. It seems to happen at regular intervals as I advance through the frames. I can't see a correlation between high-motion scenes and interlacing artifacts, so what could this mean?

Also, in the beginning sequence (where Bart is writing on the blackboard) there are a few successive frames (just before it zooms in on the letters) where ghosting is intense! It also looks interlaced, so I am not sure what filter would improve that part. Any ideas?

I tried swapping the field order in DVD2AVI, and the resulting .d2v comes out looking completely interlaced... What does that mean exactly? I presume I should use No Field Operation in the future, then try to fix the "interlaced" parts using telecide(), correct?

Thanks in advance.

[sdris]

An update... I made a test avs file using Gordian Knot, with a few seconds of video. If I've understood correctly, anything I use in the avs script will be applied on the video and will be viewable in VirtualDub, right? Now I tried using no filters at all (just opened the video using mpeg2source("D:\Simpsons\test.d2v")), and when I make it double size, EVERYTHING looks like it has horizontal lines! Why couldn't I see this in DVD2AVI?

What is confusing me is that I can't see any changes when I add lines to the script such as telecide() or FieldDeinterlace()...

Decimate(25) seems to remove frames, but I am not sure if I should use it. After all, it is a cartoon where double frames could simply be due to animation... Should I leave decimate(25) in there?

[manono]

Hi and welcome to the forum-

As I said earlier in the thread, I haven't seen any PAL DVDs. But The Simpsons is a very popular series, so there should be some people around that can give you some firsthand advice.

First. You might not see the interlacing in DVD2AVI because it doesn't advance frame by frame. So, you are much more likely to see it by advancing frame by frame in GKnot or VDub.

As for Telecide() not fixing it. In addition to putting in the "Telecide())" line (removing the "#" from in front) in the .avs, you also have to Load the Plugin up at the top. So remove the "#" from in front of "LoadPlugin("C:\DIVXST~1\GORDIA~1\decomb.dll")" or however it reads for your computer. Did you do that?

As for the ghosting. Sometimes the intros are created at 30fps, either interlaced or progressive. If that's true for The Simpsons, then in the process of converting it to PAL, fields get combined or blended or ghosted. There's one way to find out. For the purposes of testing, put "SeparateFields()" into the script before the resize (and without Telecide(). If you still see ghosting in there at those same places, then nothing you can do will get rid of it entirely.

Decimate(25). I don't think you need it in this case. But, if that interlacing is happening every 12 frames, then maybe you do need it, after getting Telecide() to work. And yes, you get a lot of doubled frames in animation. It's frequently drawn as 12fps or 8fps.

But like I said, maybe someone that has actually worked with the series will see this.

[sdris]

Thanks for the detailed reply. I think what I saw in VDub is not interlacing. I say this after testing a PAL DVD that is truly interlaced. It was much more obvious in almost every movement, and I could easily apply telecide() and see the results - it kind of blurs it, but it's still better than horizontal lines.

It may be digitization that was fooling me, since I had never seen an interlaced picture before.

Having said that, there seem to be frames at regular intervals that may be interlaced (not sure that's what it is). Telecide() and Decimate(25) don't get rid of these.

I probably won't add any of these when I do the final encoding. it has been a learning experience, and at least now I know what to look for.

Thanks guys!

[N_F]

Note that Telecide() should not be used for interlaced material, only for material where the frames have been shifted in some way. If you still see interlaced frames after trying Telecide(post=false) you should not use Telecide at all, but rather Fielddeinterlace() (or another deinterlacer if you will).

Read the steps from manono's post again and you'll see it says the same thing.

As have already been mentioned decimate(25) is most likely not correct to use in this case.

It's hard to tell without seeing the source, but perhaps Fielddeinterlace(full=false) would be appropiate?