Incorrect Field Order

[Monkeylord]

Hi,

I have come across a strange problem with a couple of DVD's that I'm trying to process.


They seem to be 29.970 progressive, but seemingly randomly a frame will appear with the fields the wrong way round. For example,

1T2T3T4T5T6T7T
1B2B3B5B4B6B7B

Because of this, I get a lovely encoded video with random single frames that look all interlaced, and when bobbed the frame jerks forwards and backwards.


Is there any functions in Avisynth or DGIndex that can detect and correct them?

Here's a very short clip that shows quite a few instances of the problem.

http://www.mediafire.com/?r5gld8uko4x98gg

I hope someone can help, because it's driving me nuts.

[Guest]

Is that from a commercial DVD? If so, which one?

tfm(pp=0)

...looks good to me.

[Monkeylord]

Is that from a commercial DVD? If so, which one?

tfm(pp=0)

...looks good to me.

Yup, it's the last episode on "Metalocalypse, Season 3". Season 1 and 2 are fine, but seasons 3 and 4 both seem to have this problem.

Would it be more likely to be an error due to the mastering, or could it possibly be a bad rip? (I used DVD Shrink, out of habit).

[Guest]

Looking more closely, I see it is just hard 3:2 telecined, but it's disguised a bit by the low animation rate.

So there's nothing wrong with it all. You just have to apply IVTC to take it back to 23.976 in the usual way. E.g.:

tfm()
tdecimate()

[Monkeylord]

I tried that, but I still get the problem. The best I got was using one of the "PP=" switches that caused the fields to blur rather than full-on interlace, but to be honest I'm interested in identifying and fixing the problem rather than just fudge a workaround.

It usually happens 3 or 4 frames apart over around 4 or 5 instances, and then doesn't happen again for a while.

You can see it on that clip when the bus first appears... every few frames it interlaces badly, and you can see clearly in the falling snow where the fields are screwed up.


It's not constant, but every time it happens I notice it. Once you see you can't unsee kind of thing.



If there's nothing more that can be done, then I'll deal with it, but I'd rather explore all possible avenues than just simply give up on it.

[paradoxical]

I downloaded your sample and did the same filters that neuron2 said above. All the frames are matched perfectly with no residual combing at all when single stepping. Also you mention "Metalocalypse" but your sample is of Ikki Tousen. Did you post the wrong file?

[poisondeathray]

You can see it on that clip when the bus first appears... every few frames it interlaces badly, and you can see clearly in the falling snow where the fields are screwed up.




What bus, what snow?:confused:

Did you maybe provide the wrong sample clip ?

[Monkeylord]

Also you mention "Metalocalypse" but your sample is of Ikki Tousen. Did you post the wrong file?

What!?

Yeah, I must have posted the wrong one. Lemme try again.

*EDIT*
Updated OP... here's the correct link - http://www.mediafire.com/?r5gld8uko4x98gg

[poisondeathray]

There is a problem with the new link .... it won't download...

[Monkeylord]

There is a problem with the new link .... it won't download...

Strange... it downloads fine for me. I've double checked and it's definitely set to public too.



I must add it to the list of things that are going totally wrong for me today (it's been one of those days).

[johnmeyer]

They seem to be 29.970 progressive, but seemingly randomly a frame will appear with the fields the wrong way round. The file in the latest download is 23.976, not 29.97.

[Guest]

No it is 29.97. You must have Forced Film set in the GUI.

The source appears hosed. Monkeylord, can you please post a larger stream but this time cut directly from the DVD VOB? You can cut it with DGSplit.

What did you rip this with? Made sure to rip only one angle?

[johnmeyer]

No it is 29.97. You must have Forced Film set in the GUI.Hmmm .... it's what Mediainfo reported, and also how it shows up in my editing program.

[edit]Here's the actual screen shot of Mediainfo:

http://i177.photobucket.com/albums/w208/johnmeyer/Framerate_zps192d0bfc.jpg

[Monkeylord]

No it is 29.97. You must have Forced Film set in the GUI.

The source appears hosed. Monkeylord, can you please post a larger stream but this time cut directly from the DVD VOB? You can cut it with DGSplit.

What did you rip this with? Made sure to rip only one angle?

Ok, I'll post a minute or two's worth in a few mins... just got in from work and I'm gasping for a coffee.


I ripped it with DVD Shrink, out of habit. Generally it's my 1st stop, and I only try other programs if it doesn't work in DVD Shrink (DVD FAB is next in line).


*EDIT*
Here we go, direct from one of the vobs on the DVD

[link removed]

[Guest]

Please rerip with a reliable ripper and make sure you have only one angle. Meanwhile I'll have a look at the VOB fragment.

[Monkeylord]

There's only one angle available.

I'll try using DVDFAB.


I appreciate you having a look. It's an odd problem.

[johnmeyer]

I used DGIndex with "honor pulldown" selected. I then used assumetff().separatefields() to look at the video field-by-field. I saw no instances of field reversal.

[Guest]

@Monkeylord

This is a website for all ages, and so hentai is not appropriate. I have removed your link. If you want further help on this issue, please post an appropriate stream sample. Hint: talking heads are not very useful.

[Monkeylord]

Sorry... I didn't notice that bit at the end (it's a show with violence and harsh language, but I totally forgot that S3 and S4 were released on DVD uncut).

Lemme try again.

[Guest]

Thank you!

[Monkeylord]

Just uploading a fresh clean bit now.

[Monkeylord]

Ah... here we go. It has a montage with a decent array of motion.

http://www.mediafire.com/?039o9yve05n0b5u


I've DEFINITELY watched through this one all the way.

*EDIT*
Added a better clip... keep an eye on the log on a conveyor belt going into a chipper about halfway through. The problem gets really obvious there. Most of the other times it happens are when people are talking, which you asked me to avoid.

[Guest]

I'm not seeing any residual combing with this script (except for some mouth combing that can be attributed to match failure and which postprocessing will mitigate):

dgsource("Metalocalypse S3 Sample 1.dgi")
assumebff()
telecide(post=0) # Use version 5.2.4 with Avisynth 2.6
decimate()

I don't see anything I would attribute to incorrect field order. The animation is not completely smooth but that is due to the low animation rate, not any pathologies, as far as I can see. Please describe the problem that you see by referring to specific frame numbers in your sample.

There is a field order transition at the end of the sample, but that could be due to your cutting.

[Monkeylord]

Thanks... I'll check when I get home from work and get back to you.

[paradoxical]

With his previous clips there were tons of bad matches with tfm no matter which mode was being used. Maybe Telecide is matching fields better.

[Monkeylord]

I managed to have a quick look on my laptop, and it seems that if I set it to TFF then there's tons of bad matches, but if I set it to BFF then there's a tiny amount of bad matches.

The bad matches in BFF mode are correct in TFF, so there seems to be field order transition error happening.


Would it be an idea to set up 2 versions of the video, and where BFF produces a combed frame have avisynth replace it with the good version from a TFF file?



(I have no idea if what I just typed made any sense)


For example, using:

AssumeTFF()
Telecide(post=0)
Decimate()

In my sample, this causes frames 794, 798, 801, 804, 807, 811, 818, 821, 824, 827, 830 and 834 to have interlacing left behind, but:

AssumeBFF()
Telecide(post=0)
Decimate()

In the same range causes only frame 835 to have interlacing artifacts.

[paradoxical]

You could always use something like YATTA as well to generate an override file for tfm so you can control the matching.

[Guest]

The clip is BFF so the second script is appropriate. I can look again at frame 835 this evening.

The usual simple way out is to enable postprocessing, so that one bad match gets mitigated.

[paradoxical]

Okay, so that explains all the bad matches I was seeing when I ran it with tfm. I didn't think to switch the field order assumption.

[Monkeylord]

You could always use something like YATTA as well to generate an override file for tfm so you can control the matching.

I shall have to look into it... never heard of YATTA before.

The clip is BFF so the second script is appropriate. I can look again at frame 835 this evening.

The usual simple way out is to enable postprocessing, so that one bad match gets mitigated.

Who wants a simple way out?!? lol

On a serious note though, you've helped me learn a lot over these past couple of days (previous clip content faux-pas, included!).

What I think I'll have to look into is how to generate a list of just the frames that need switching, and then how to switch the parity on just those ones. It's as much in the learning experience as it is the "getting it right" for me, at this stage.

[Guest]

Don't assume the parity is wrong for one frame! That's a leap you are making without cause. I'll look again this evening as I said.

A bad match just means that the matcher is not perfect, it doesn't mean that the frame has wrong parity. Have you tried pattern guidance?

[Monkeylord]

Have you tried pattern guidance?

I just tried Pattern Guidence, but if anything it made things worse.

[Guest]

Frame 835 looks clean to me with this script:

dgsource("Metalocalypse S3 Sample 1.dgi")
assumebff()
telecide(post=0)
decimate()

and with this one:

mpeg2source("Metalocalypse S3 Sample 1.d2v")
assumebff()
telecide(post=0)
decimate()

What source filter are you using?

[Guest]

Maybe Telecide is matching fields better. In some cases Telecide() does better and in some cases TFM() does better.

TFM/TDecimate have more features than Telecide/Decimate, but for what we do here they should be close in results.

[Monkeylord]

Frame 835 looks clean to me with this script:

dgsource("Metalocalypse S3 Sample 1.dgi")
assumebff()
telecide(post=0)
decimate()

and with this one:

mpeg2source("Metalocalypse S3 Sample 1.d2v")
assumebff()
telecide(post=0)
decimate()

What source filter are you using?

I made an error and it was frame 834, not 835. I feel a complete idiot.

Here's my exact script

mpeg2source("C:\Temp\Split\Metalocalypse S3 Sample 1.d2v")
AssumeBFF()
Telecide(post=0, show=true)
decimate()

Here's what I get when I run the avs in Virtualdub:

http://i.imgur.com/ZKRaQQz.png (http://imgur.com/ZKRaQQz)

The interlacing is on the log. It's the only thing moving in the frame.

I used separatefields() earlier, and navigated to the fields for that frame and it gives the telltale forward/backward jerking of the wrong field order, wheras all the frames leading up to it show the log progressing smoothly along.

[Guest]

That frame for me is 959 after decimation, so I don't know how you get 834. And that frame is clean for me. Are you using the exact same sample you uploaded?

Anyway, I can't duplicate your result.

The only differences I see is that you have Decomb 5.2.3 and you use VirtualDubMod. Can you try Telecide 5.2.4 and use plain vanilla VirtualDub?

[Monkeylord]

That frame for me is 959 after decimation, so I don't know how you get 834. And that frame is clean for me. Are you using the exact same sample you uploaded?

Anyway, I can't duplicate your result.

The only differences I see is that you have Decomb 5.2.3 and you use VirtualDubMod. Can you try Telecide 5.2.4 and use plain vanilla VirtualDub?

Done. Updated to 5.2.4 and used vanilla VDub. Initially I still had the same result, but I think I found the problem. In DGIndex, I found it was set to "Ignore Pull Down Flags". Once I set it to honour them I achieved the same result as you. Frame 959 and clean.



Thanks so much for all the help. I've learned a lot through all this turmoil.

[Guest]

You should spend more time reading user manuals, thereby avoiding such self-inflicted turmoils. :)

[Monkeylord]

Probably more a case of double checking my settings before embarking on a project.


Thanks again. You've been a massive help.

[johnmeyer]

I found the problem. In DGIndex, I found it was set to "Ignore Pull Down Flags". Once I set it to honour them I achieved the same result as you. Frame 959 and clean.Please note that this is what I recommended in my third post (http://forum.doom9.org/showthread.php?p=1617543#post1617543), using your original upload when I stated that there was no field reversal. Given the erratic nature of animation telecine, this setting is pretty much mandatory.