Field matching and IVTC of R1 NTSC Anime (VS pure film sources)

[kalam13]

Hi everyone,

this is my first post :) I have been lurking around for some time now;
let's hope this post is done properly ;) Please forgive my if i show
any (hopefully not so obvious) misconceptions below. I'll try to be as
precise as possible/necessary in explaining my problem.

To briefly state my issue first: I have problems with field matching
for performing proper IVTC of R1 NTSC anime sources.

So far I have only dealt with movies and tv series on DVD with clean
3:2 pulldown, such as my R1 Angel DVD boxes. In fact, they were purely
film ("100% Film" in DVD2AVI) and of progressive frame type with only
a few seconds in the intro that are not completely film material so
that I could use FORCE FILM and forget about messing around with
decomb (thanks for decomb neuron2! i do use it further below :)).

The anime sources I have - actually every anime DVD i checked so far
- are indicated as "NTSC" (above 95 % most of the time, and many as
pure NTSC) with frame type "Interlaced" in DVD2AVI. Therefore, I have
set field operation to "None" in the DVD2AVI settings to perform field
matching and IVTC operations in decomb. I have also checked the D2V
file, which for most sources consists entirely of 0's and 2's, while
some other sources' D2V file consists of 0's and 2's with 0123 mixed
in here and there for a couple of seconds.

Now to be more specific for further discussions, let me pick one
source:
Overman King Gainer from the R1 NTSC DVDs
(pretty clean i'd say) with the following specifications:
Video Type indicated in DVD2AVI:
97% NTSC (only 3.95% FILM as shown at the bottom of the D2V file)
Frame Type:
Interlaced (with Progressive mixed in here and there for a couple of seconds)
Frame Rate:
29.97 fps
Now, my problem is the following: For my telecined film sources I get
clean cccnn matches using the following AVS script to load them into
VirtualDub after determining field order:

LoadPlugin("C:\...\MPEG2Dec3.dll")
LoadPlugin("C:\...\Decomb521.dll")
MPEG2Source("sample.d2v")
Telecide(order=0,guide=1,show=true,post=0)
# adding Decimate(cycle=5) would then complete IVTC
I stated above that I use FORCE FILM for my purely film sources;
I have used this script just to check the field matches. Also, if
there were only strings of many c's (ccccccc...) i wouldn't touch
them at all.

For my anime source, however, after loading them with above script
into VirtualDub, changing guide=1 to guide=0, I get the following
100 field matches for frames 0 through 99 during the first 3.303
seconds (the second line of field matches is identical to the first
line, but with a space every 5 characters just in case you feel more
comfortable looking at it that way; also, it seems the first line has
an automatic linebreak):

ccncncnnccccnccccncccnncccnnccccnnccncnnncnncccccnnccnccccncccccncccnccccnnccncnccccncccccccccnccnnc

ccncn cnncc ccncc ccncc cnncc cnncc ccnnc cncnn ncnnc ccccn nccnc cccnc ccccn cccnc cccnn ccncn ccccn ccccc ccccn ccnnc
I must admit I am sort of perplexed at the sight of this string. Even
with guide=1 and allowing for 3:2 guidance I still get strings like
this. How am I supposed to do IVTC? In particular, find/confirm
correct field matches and decimate properly? I do believe I can't just
go ahead and decimate any duplicate frames, particular during static
scenes.

I have to add that, after using above script (no decimating),
I do NOT get any interlacing artifacts. So the clip seems to be fine
3:2 pulldown, and it's probably my misunderstanding of how to
interpret/handle a string like that. A possible solution I thought
about is decimating 1 in a cycle of 5, but then which one? sometimes
there are several identical frames, so which match (c/n) do i have to
decimate? For proper telecined material that would be the one which is
most similar to its predecessor frame. I guess this summarizes my
questions to this community (for now ;) ).

As a hint as to what solution I am looking for: I don't care if I have
to override all of decomb's telecide and decimate decisions and do
field matching frame by frame. Even if there were only a couple of
frames where I would need to do that, I would still prefer the manual
override using notepad. The reason being that I prefer to have a
knowledge of what is being done with the clip. (i really hope i don't
start to sound crazy/presumptuous at this point.)

I apologize for not providing a VOB clip. I do have a 15 sec, 8.28 MB,
VOB, which I used for the discussions above, but I don't have any
webspace and I am kinda at a loss about this. If anyone wants to help
me out with my issue above, I can send it directly per email or that
person can host it for a little while. I can send as many email as
necessary, so you if you're ok with it you could send me a pm with
your email address.

I also apologize if my post is too long; i am trying not to miss any
vital information before starting a thread. And I hope I do not cause
any infusion with this wordy post. Please let me know if i should
compress it a little more next time.

[Guest]

Nice post, except that it is too wide and thus a PITA to read.

Upload your VOB to my FTP server and tell me the filename here.

Often with anime, there are lots of duplicates and that makes it hard for a clean repeating 3:2 to be found. But let's see the VOB before jumping to conclusions.

[kalam13]

Thanks for the offer. I have named the file "sample_kalam13.vob" and
sent it to your FTP server.

I apologize for the formatting; i copied the text over from wordpad
and i was indeed wondering if the textwidth was due to my browser.
i'll enter linebreaks manually next time.

i have edited my first post to make it more pleasant to the eye.
(i have not changed the content)

[Guest]

Thanks, I'll look at it now.

Instead of all the line breaks, just make sure your code lines aren't too long. That is what was making it too wide.

[Guest]

The clip is some anime with a very low base picture rate. If you separate fields and then count successive duplicates, you see 7 duplicate pictures, then 8, then 7, then 8, etc.:

7 8 7 8 7 8 7 8 ... (fields)

So they show animation cell 1 for 7 fields, then cell 2 for 8, then 3 for 7, etc.

That means the base picture rate is only:

2/15 * 29.97 = 3.996 frames per second

I am unable to match this pattern to any 3:2 pulldown. Thus you can apply telecide(order=0,post=0) to recover the progressive frames. Then you can decimate to any desired rate, even down to 3.996 (except that most players don't play these low rates smoothly). Suppose you wanted to go to 23.976. Then you could just do decimate(5). Or if you wanted 25, you could use fdecimate(25.0).

The base frame rate is so low, that you can pretty much pick any target frame rate you want, and just pick it based on your target medium. If it's for display on a PC, even fdecimate(15) still looks fine. Of course, the more you decimate, the smaller will be the encoded result.

[kalam13]

Thanks a lot!

I wasn't even dreaming about looking for such (to me, abnormal) picture rates.

I will work on this and give you some feedback later. I'll also go through some other sources to see if i can apply a similar analysis there.

EDIT: Decimate(5) seems to work just fine. I noticed some irregular frame patterns here and there, even proper telecine for a couple of frames (those that were indicated by 0123 in D2V), but decimate(5) seems to consider these correctly.

[kalam13]

Originally posted by neuron2
The clip is some anime with a very low base picture rate. If you separate fields and then count successive duplicates, you see 7 duplicate pictures, then 8, then 7, then 8, etc.:

7 8 7 8 7 8 7 8 ... (fields)

So they show animation cell 1 for 7 fields, then cell 2 for 8, then 3 for 7, etc.

That means the base picture rate is only:

2/15 * 29.97 = 3.996 frames per second


On a second thought, shouldn't it be like this:

The field pattern

8 7 8 7 8 7 8 7 ...

results in a frame pattern

4 4 4 3 4 4 4 3 ...

using post=0, guide=0. This frame pattern indicates 4 different pictures in 15 (=4+4+4+3) frames, so that the frame rate of the clip is

4/15 * 29.97 fps = 7.992 fps, which is equal to 2 * 3.996 fps, rather than 1 * 3.996 fps.

Am I wrong here?

[Guest]

Yes, you are correct. I was hasty in my analysis. But hey, at least I got you thinking in the right direction. :)

[kalam13]

You did, thanks :).

And your FDecimate() help file (which I noticed after you mentioned the fdecimate() command) was also very informative. Especially the part "Why FDecimate() Cannot Be Perfect, and What We Can Do About It" made me further think about this matter and the issue with the observed irregularities in the frame pattern.