Deinterlacing and a pattern

[Akatsuker]

Hey you guys.
I'm working on an anime documentary and the video is not progressive. But there's a pattern: I see 3 ok frames, and 2 with those weird lines. It goes on and on just like that.
Here's the link for the m2v and the d2v of a sample, demuxed on the DGIndex.

http://www.sendspace.com/file/vk7vy6
Size: 67,44 MB

Could you give me any suggestions? Thank you.

[Guest]

It's 3:2 pulldown. Either use Force Film in DGIndex if it's soft pulldown, or use external IVTC, such as Telecide()/Decimate() if it's hard pulldown. (I can't tell you which it is because I am behind a firewall right now that blocks upload sites.)

[Akatsuker]

Okay, and how can I see about soft or hard pulldown here?

[Akatsuker]

Here's the d2v, if that helps:

DGIndexProjectFile16
2
E:\RIP\VTS_02_1.VOB
E:\RIP\VTS_02_2.VOB

Stream_Type=1
MPEG_Type=2
iDCT_Algorithm=6
YUVRGB_Scale=1
Luminance_Filter=0,0
Clipping=0,0,0,0
Aspect_Ratio=4:3
Picture_Size=720x480
Field_Operation=0
Frame_Rate=29970 (30000/1001)
Location=0,0,1,5df4c

d00 5 0 2048 0 1 1 92 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 32768 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 323584 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 550912 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 778240 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1005568 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1232896 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1458176 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2
900 5 0 1687552 0 1 1 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2

(...)

900 5 1 788105216 0 1 7 32 32 92 b2 b2 a2 b2 b2 a2 b2 b2 a2 b2 b2 a2 a2 ff

FINISHED  100.00% VIDEO

[manono]

If it's really in that 3:2 pattern as you described, then it's hard pulldown. You'd make the D2V using Honor Pulldown Flags (which you did) and then apply IVTC to bring it back to progressive 23.976fps.

I didn't download it, but am going by your description.

[Akatsuker]

Ok, then I have to correct the pattern that I said (maybe it's irrelevant, but who knows...):

I have 2 frames ok, and the next 2 have those weird lines. It's 2-2, not 3-2.

manono, there's a lot of IVTCs, right? Do you recommend me any of them?

[Guest]

Now, your new data changes everything!

I can have a look at your stream tonight, but now I am thinking it may be field blended, in which case IVTC is not going to be satisfactory.

You can have a look at the first question here and try to tell us DEFINITIVELY what type of video you have:

http://neuron2.net/faq.html

[Akatsuker]

Neuron2, here it is:

From FAQ page:

Now serve the script into VirtualDub and step through the fields in a high motion section. We will label a new picture with a new letter below. If you see a repeating field sequence like this:

... a a b b c c ...

then it is progressive video. If you see this:

... a b c d e f ...

then it is pure interlaced video. If you see this:

... a a a b b c c c d d ...

Then it is 3:2 pulled-down progressive video.

I did everything the faq said, and so the script for this pattern of frames is this one:

MPEG2Source("E:\RIP\[HPF]VTS_02_1.d2v") #HPF from the "Honor Pulldown Flags"
AssumeTFF()
SeparateFields()

I see this, when the documentary starts:

a a a a a b b b b b c c c c c d d d d d e e e e e f f f f f g g g g g ...

And this, on a highest motion section:

a a a b bc c c d d e e e f f g g g...


So, it is 3:2 pulled down. The variation for the first example... Is it usual?

[Guest]

It just means that before pulldown is applied, you have a progressive frame sequence:

A A B B C C ... # showing frames here, not fields

instead of:

A B C D ...

That is, the progressive frames were doubled prior to pulldown during the introduction.

[Guest]

So, you apparently have hard 3:2 pulldown.

You can start by trying Decomb. Save your project in DGIndex with Honor Pulldown Flags. Then make this script:

mpeg2source("file.d2v")
telecide()
decimate()

That should give you progressive video at 23.976 fps.

[Akatsuker]

Damn, it works now. Thanks.
If you don't mind, I have some fast questions (some related to this thread, and some other common questions, I think):

- I've seen a lot of Anime DVDs with so different technologies, you know... Progressive, or Telecined, Pure Interlaced, whatever. Why do we deal with so different stuff on the same world, with similar DVD Players around? I'd understand that if one of them is better than the other. Even if this was the case, on Japan, for example, same market, some are progressive at 23.976, as it should; and some are not, and so we've got fields and other things to figure out. Why the hell is that? Progressive is a better way? Or not?

- Although now I can handle a progressive video, I have a lot of lines that should be removed, or just do something and so it could reduce.







What is it called? I want to search about some solution on this.

- Is there any kind of "descriptive list" of these video problems that commonly appear? If there's not (and if this would be helpful for newbies like me), I could work on it.

[Guest]

Originally Posted by Akatsuker

- I've seen a lot of Anime DVDs with so different technologies, you know... Progressive, or Telecined, Pure Interlaced, whatever. Why do we deal with so different stuff on the same world, with similar DVD Players around? I'd understand that if one of them is better than the other. Even if this was the case, on Japan, for example, same market, some are progressive at 23.976, as it should; and some are not, and so we've got fields and other things to figure out. Why the hell is that? Progressive is a better way? Or not?

I'd have to write a book to answer that. You need to start studying video and its history. What we have today is a result of a historical legacy.

Although now I can handle a progressive video, I have a lot of lines that should be removed, or just do something and so it could reduce.

Your sample video changes from 3:2 pulldown to a pure interlaced section at the end. That is called "hybrid" video. It's a difficult thing to deal with. You can (1) force deinterlace the video part and allow it to be decimated to the film rate, or (2) force deinterlace the video part but leave the whole thing at video rate, or (3) consider a variable frame rate approach (VFR) using a container that supports it. You're just going to have to make a thorough study of this stuff as there are no easy answers.

Is there any kind of "descriptive list" of these video problems that commonly appear? If there's not (and if this would be helpful for newbies like me), I could work on it.

Not that I know of. It's kind of you to offer but I wouldn't attempt it until you become an expert in video analysis/processing. The sad part is that when people become experts they become less motivated to write basic guides.

[thetoof]

Originally Posted by neuron2

Not that I know of. It's kind of you to offer but I wouldn't attempt it until you become an expert in video analysis/processing. The sad part is that when people become experts they become less motivated to write basic guides.

I won't claim I'm an expert at all this, but in the process of writing and improving AnimeIVTC, I have encountered many problems of that type (hard telecine, "double" hard telecine, interlaced or progressive credits on top of a hard telecined background, field blending, "double chroma field blending", interlaced fade-ins and hybrid (mostly telecined with some 30p sections)).
The next v's docs (yeah, I know I've been blabbering about that for a few weeks now, but working 90h a week is a bit of a turn off for avs scripting) will include such a guide (which could be put somewhere else on the net if there's an interest for it) with detailed explanations backed with a lot of images.
neuron2, regarding your moderating comment a little while ago about me trying to turn every thread into an animeivtc advocacy thread, I'll keep that in mind by writing an objective guide listing as many alternatives as I know. Suggestions and demonstrations of (non-)interest are welcome... though that'd be OT... Never mind that, I'll most probably create a thread when the first version of the guide is ready.

Laters,
toof

[Guest]

@thetoof

I'd be happy to give you an area at my website to put your guide if you need somewhere to host it. I get some good traffic. Threads here are not ideal for that sort of thing. You could be right next to Ben Rudiak-Gould's hosted area. I think you may know who he is.

I'm also willing to review your guide materials as well.

[thetoof]

Thanks for everything, that'd be awesome!

It's true that threads are not ideal, but I thought it could be a way of getting many suggestions about the guide's structure and content (ensure all formulations are clear for everyone, get a lot of references to as many functions as possible with example scripts with their pros and cons and so forth)
In other words, using the amazing potential of this community to create a solid guide on how to identify the problems and use avisynth in all its glory against such issues.

[Guest]

Then do the best of both worlds. Have a Guide Development/Review thread here, and a permanent hosting of it.

[thetoof]

Yup, will do. Thanks again.

[Akatsuker]

Back:

@neuron2

or (3) consider a variable frame rate approach (VFR) using a container that supports it.

Yeah, I can work well with VFR, but the framerate and the fluidity of the video is not the problem here, is it? I was talking about those "scan lines" over there (the shots).

http://img444.imageshack.us/my.php?image=vid3rj1.png

The left side of the image is very clear on showing what I wanna clean up. The ear of the guy, the mouth, hair on front side...
I mean these lines. I've been trying some antialiasing (with antialiasing ()...), but it's strange. As I tweak the parameters' values up, the effects over the image are not being shown anymore.

[Guest]

It's interlacing. You use a deinterlacer.

[thetoof]

Just to expand on neuron2's advice:
List of deinterlacers : http://avisynth.org/mediawiki/Extern...#Deinterlacing
If you want to use a bobber (like tempgaussmc, mcbob, yadif or tdeint in mode=1... etc etc), make sure to use SelectEven() afterwards for proper VFR decimation.