Telecide(Guide=1,Agg, etc not enough

[Swan]

I've encountered a very bad film to DVD-transfer and I need some help.
The DVD is NTSC film (from a film source), but has not been telecined the normal way. DVD2AVI goes mad between FILM and NTSC.
I ripped the DVD without ForceFilm and and when examining the video frame by frame in VirtualDub, I see that the normal pattern, the tell-tale sign of telecine, is not there. Frames are randomly progressive and interlaced.
The regular:
Telecide(guide=1).Decimate(5)
leaves a lot of combed frames.
Even Manono's tip to use:
Telecide(Guide=1,Agg=True,threshold=5,dthreshold=5)
Decimate(5)
Does not remove all combing! But this is from a film source, the movie was not shot with a NTSC interlaced video camera, I am sure of it. What can I do? I want to bring the film down to 23.976 fps, and remove the combing.

I tried enabling the IVTC line in the script GordianKnot created, but when opening the avs in VirtualDub, I get an error message saying there is no such function.
Is there a dll I need to download for IVTC?

I am using Avisynth 2.52 build 1 Jun 1 2003 and GordianKnot 0.28

/Maria

[UHT]

Not really my field of expertise, but might this thread get more / more useful replies in the Avisynth Usage forum? Can move it across.

[Swan]

Thanks, UHT, yes move it.

[Guest]

Post a link to a VOB fragment that demonstrates the problem. There is no other way to help you.

One question: Does the D2V file have all 2s or all 0s, or does it have repeating 0 1 2 3 patterns in it? Does it have sections of 0 1 2 3 and sections of all 2s and/or all 0s?

[Awatef]

Wanna bet it is analog telecine?

Check my ivtc quick reference thread

[Swan]

If the tell-tale sign of an analog telecine is that there's no apparent or repetative pattern of interlaced and progressive frames and that many more frames appear interlaced than progressive, then yes, this movie certainly has been telecined the analog way.
After having encoded the video with
Telecide(guide=1).Decimate(5)
and
Telecide(Guide=1,Agg=true,threshold=5,dthreshold=5).Decimate(5)
I also see that the matching hasn't been perfect, as I get odd jerky motion on some scenes (but not all).

I read the Deinterlacing / IVTC quick reference and the suggestion there:
analog telecine (no 3:2 pattern)
Bob()
LanczosResize(320,240) [60fps output]
doesn't sound right for my purpose(I don't want to reduce the height and width *that* much). Or am I missing something?

Neuron2, I can't find a pattern, looks like the majority of the frames are interlaced with a few progressive thrown in sometimes.
Here is a sequence ( I start on a progressive frame):
P, I, I, I, I, I, I, I, I , I, I, I, I, I, P, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, I, P, etc.

I'll try to upload a clip, but I have to compress it (I have a 10 MB limit) so I'll use DivX 3 at the highest possible bitrate. Hope that works?

[Guest]

@Swan

I asked for the D2V file contents and you gave me something else. Why?

"Analog telecine" is a meaningless term that Awatef apparently made up. I suppose he is referring to "hard telecining". That is why I asked for the D2V file contents.

If you have no clear pattern, then you should disable pattern guidance by setting guide=0.

[Swan]

Neuron2, sorry, I thought you wanted to see part of the video.
Here's 50 lines of the d2v (the whole thing is very long, but if you want to see it, just let me know).:

7 0 0 2 2 2 2 2 2 2 2 2 2 2 2 2
7 0 3C 2 2 2 2 2 2 2 2 2 2 2 2 2
7 0 7F 2 2 2 2 2 2 2 2 2 2 2 2 3
7 0 131 0 1 2 3 0 1 2 3 0 1 2 3 0
7 0 181 1 2 3 0 1 2 3 0 0 0 0 0 0
7 0 1F4 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 266 2 3 0 1 2 2 2 2 2 2 3 0 1
7 0 2DD 2 3 0 1 2 3 0 1 2 3 0 0 0
7 0 355 0 0 0 1 2 3 0 1 2 3 0 1 2
7 0 3D1 3 0 1 2 3 0 1 2 3 0 0 0 0
7 0 44D 0 0 1 2 3 0 1 2 3 0 1 2 3
7 0 4C2 0 1 2 3 0 1 2 3 0 1 2 3 0
7 0 539 1 2 2 2 2 2 2 3 0 1 2 3 0
7 0 5B1 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 627 2 3 0 1 2 2 2 2 2 2 3 0 1
7 0 6D3 2 3 0 1 2 3 0 0 0 0 0 0 1
7 0 746 2 3 0 1 2 3 0 1 2 3 0 1 2
7 0 7B7 2 2 2 2 2 2 2 2 2 2 2 2 2
7 0 830 2 2 3 0 1 2 3 0 0 0 0 0 0
7 0 8AE 1 2 3 0 1 2 2 2 2 2 2 3 0
7 0 930 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 9B6 2 3 0 1 2 3 0 1 2 3 0 1 2
7 0 A4C 3 0 1 2 3 0 1 2 3 0 1 2 3
7 0 AC5 0 1 2 3 0 1 2 3 0 1 2 3 0
7 0 B64 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 C04 2 3 0 1 2 3 0 1 2 3 0 1 2
7 0 CB0 3 0 1 2 3 0 1 2 3 0 1 2 3
7 0 D7D 0 1 2 3 0 1 2 3 0 1 2 3 0
7 0 E16 1 2 3 0 0 0 0 0 0 1 2 3 0
7 0 EA8 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 F39 2 3 0 1 2 3 0 0 0 0 0 0 0
7 0 FCF 0 0 0 0 1 2 3 0 1 2 3 0 1
7 0 1054 2 3 0 1 2 3 0 0 0 0 0 0 1
7 0 10EC 2 3 0 1 2 3 0 1 2 3 0 1 2
7 0 1181 3 0 1 2 3 0 1 2 3 0 1 2 3
7 0 1204 0 1 2 3 0 1 2 3 0 0 0 0 0
7 0 129C 0 1 2 3 0 1 2 3 0 1 2 3 0
7 0 132E 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 13BF 2 3 0 1 2 3 0 1 2 3 0 1 2
7 0 1446 3 0 1 2 3 0 1 2 3 0 1 2 3
7 0 14D9 0 1 2 3 0 1 2 3 0 1 2 3 0
7 0 1569 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 15ED 2 3 0 1 2 3 0 1 2 3 0 1 2
7 0 167D 3 0 1 2 3 0 1 2 3 0 0 0 0
7 0 1714 0 0 1 2 3 0 1 2 3 0 1 2 3
7 0 17A0 0 1 2 3 0 1 2 3 0 1 2 3 0
7 0 1837 1 2 3 0 1 2 3 0 1 2 3 0 1
7 0 18BB 2 3 0 1 2 3 0 1 2 3 0 1 2
7 0 195F 3 0 1 2 3 0 1 2 3 0 1 2 3
7 0 1A06 0 1 2 3 0 0 0 0 0 0 1 2 3

[Guest]

No problem, Swan. Thank you for the follow up. If that is representative of the whole then it is claiming to be 3:2 telecined. You may have serious blended fields, though, that screw things up. Can you please make available the VOB you offered, so that we may reach a conclusive diagnosis. Thank you!

[Awatef]

@ swan
That's the tragedy with analog telecine sources (or hard telecine as neuron2 calls it).
But keep in mind that the height of a field is 240 anyway with NTSC sources.
If your PC is strong enough and you have enough bitrate, you may encode anamorph like this for example:

Bob()
BilinearResize(704,240)

And then resize manually while playing.
Don't forget to put your refresh rate on 120Hz, or the motion won't be perfect.

BTW, just got an idea, does it help if you enable "swap field order" in DVD2AVI.
Once I got a DVD with such a field order problem, after enabling it, I was able to IVTC perfectly.

[bilu]

@Awatef

Your method is similar to some of those here:

http://www.doom9.org/ivtc-tut.htm#InterlacingDeinterlacing

namely those that say:

"halves the resolution: should not be used for larger final resolutions as it may create shimmering"

Of course it will allways deinterlace perfectly :rolleyes:

I know, I used them too :) just look at the film titles or credits.

Bilu

[Swan]

Originally posted by neuron2
No problem, Swan. Thank you for the follow up. If that is representative of the whole then it is claiming to be 3:2 telecined. You may have serious blended fields, though, that screw things up.
Yes, there are blended fields, many of them!
Here is 12 seconds (9.6 MB) of the video:
http://w1.837.telia.com/~u83704103/test/

So, you guys think this film has been "hard telecined", i.e, telecined with analog equipment and then encoded to Mpeg-2 at 29.97 fps, instead of the usual 23.97 and pulldown flags method?
So, the DVD is made from an old master, perhaps one made for TV broadcasts or similar, could that explain it?

I don't want to take the resolution in half and then interpolate it up to 720 x 480, decimate to 23.97 and then resize and crop it for Xvid encoding. I tested it and motion was no longer smooth.

Isn't "Swap field order" for correction of field order, which field comes first and is sent first to the TV, etc? The material is 4:3, BTW.

[Guest]

When you think you've seen everything, something like this comes along. First, it has horrible blended fields, and lots of them. That alone effectively means the clip should be treated as video and not telecined progressive content. Second, there is something fishy about the field ordering because if you do assumetff.separatefields you'll see that there is jumping back and forth at some places, but if you use assumebff.separatefields there is jumping back and forth at different points!

So if it were me I would just do this:

TomsMoComp(1,5,0)

You can try to squeeze out a few good progressive frames with this, but it's hardly worth it:

telecide(blend=false,threshold=15)

It is clear that this clip is no candidate for any kind of pattern-guided inverse telecine operation.

[Swan]

Neuron2, thank you for taking the time to look at it and for the clear recommendation on what to do.
Imagine having paid 20 dollars for this DVD! :-(
I'll deinterlace it and leave it at 29.97 fps.

[Awatef]

I don't agree with neuron2
the bob method is much better, because it has much better motion and less ghosting.

[Guest]

Originally posted by Awatef
I don't agree with neuron2
the bob method is much better, because it has much better motion and less ghosting. Maybe Swan doesn't want a 60 fps clip and/or can't set the refresh rate to 120Hz and/or doesn't want to fiddle around with playback filter resizing. You act as if there is a right or wrong here. It depends upon Swan's needs.

[Swan]

I initially wanted to make a 23.97 fps Xvid or DivX 3 backup.
Since that is not possible, I prefer to make it a 29.97 fps backup. That means I have to deinterlace, which I really don't like. :)
But I guess the best thing would be an interlaced Xvid file?
However, one CD would not be enough then, do you think?
SVCD is also an option, of course.

BTW, setting the refresh rate etc for playback is useless on my system, so if anything but standard 50 Hz or 60 Hz (PAL or NTSC) is needed, I can't use it. I have a Matrox G450 and its Tv-out is independent of sceen resolution, refresh rates, etc. So no "refresh rate on 120Hz" for me. :)

Thank you all for your help.

[Awatef]

you can use 60hz too, I said 120 because it's much more comfortable for the eye.

[bilu]

Have you tried:

Telecide(post=false)
Decimate(mode=1) or (mode=3)
TomsMoComp(-1,30,1)

It looks nice to me in both modes 1 and 3.

Used Decomb 4.1b4.


Bilu

[Swan]

I've never in my life seen such a horrible film to DVD transfer.

Neron2s suggestion to use TomsMoComp of course worked, but it looked even worse than the DVD, loads and loads of blended fields and really, really bad motion (ghosting galore) when viewed on the TV. So I scrapped that one.

To use telecide was impossible. On some scenes it worked, but on many others, the video became choppy. Especially on pans, it was chucka-chucka-chucka.

I then encoded it as interlaced Mpeg-2 and put on an SVCD and then it at least looked as good (meaning bad)as the DVD.

Bach, I will test your script and see if it is better than what I did.
Thank you all for helping out.

[bilu]

@Bach

Well, I'll try as soon as I can, maybe tonight.

But decimating in separate fields and interleaving again could only provide better results if Decimate's match were wrong. And blending fields at the end just covers it up, as much as TomsMoComp :)

I've been making some tests with the metrics for MultiDecimate's first pass in this thread with this clip:

http://forum.doom9.org/showthread.php?s=&postid=327511#post327511

I can be wrong, no one answered at the time of this post.

But if I'm not, and this really is a cycle 5 Telecined movie in 81,7% of the stream, then Decimate(mode=3) is worth it, and TomsMoComp (never used BlendFields) can cover up the rest.

Of course mode=3 strategy can be better when the new decimating mode that neuron2 mentioned on the journal on 6-1-2003 is available ( I don't know if it already is).

Basically the differences between our methods seem to be:

- TomsMoComp vs BlendFields;
- Hybrid Decimate vs IVTC Decimate;

EDIT: TomsMoComp does a great job covering up the juttering from blended fields, that's why it's great over Decimate(mode=3).

@Swan

I'm glad we have the opportunity to learn from such a clip :)


Best regards,
Bilu

[bilu]

You maybe right, I'm somewhat newb on deinterlacing :rolleyes:

Have to test both ways to see which one I like more. To be honest, I had trouble seeing any difference between mode=3 and mode=1 with TomsMoComp because of the blended fields. Both looked better than the original.

But mode=3 can be appropriate here and look better when the new way is available. Then again this may be pure IVTC with duplicate metrics over 1 sometimes, and Decimate(5) could look better than Decimate (mode=3).


I'll look to it, thanks for the suggestion :)

PS: Do you think the new video decimation could help or disturb the process?

Bilu

[bilu]

@bach

Tested this mode

Telecide(post=false)
Decimate(mode=3)
FieldDeinterlace(dthreshold=0)

against yours

complementparity.separatefields()
odd=selectodd.decimate(5)
even=selecteven.decimate(5)
interleave(odd,even).weave()
telecide(post=false)
FieldDeinterlace(dthreshold=0) # equivalent to BlendFields()

Both modes look great, but mine is at film rate.

What really made the difference was FieldDeinterlace(dthreshold=0)
instead of TomsMocomp, it got much better. I tested with both mode=3 and Decimate(5), no difference in this clip.

Same for post=true and post=false, got the same size. But with post=false it looked a bit sharper.

Thanks for showing me that BlendFields (or equivalents) can have better looking than TomsMoComp.


Bilu

[trbarry]

TomsMoComp does its best on video sources, where all fields are equally separated in time, with no matches.

It will handle most anything else as sort of a fancy interpolation (bob) algorithm. This part will rarely be as detailed as a true field match when there is one.

And note TomsMoComp matches pixels, not fields, since it assumes it is being used when the fields don't match up (video source).

- Tom

[Guest]

Tom,

Are you going to say anything on the YUY2 problem in the 2.5 version as it may be causing some people's poor results with TMC?

[trbarry]

Are you going to say anything on the YUY2 problem in the 2.5 version as it may be causing some people's poor results with TMC?

Only if shamed into it. ;)

But yes, TMC for 2.5 seems to never use search effort > 0 if it is handling YUY2 instead of YV12. I'll put out a fix soon.

The more I think about it, there is maybe some additional code for TMC that should be added if it is now handling non-video material that could make it a bit more like GreedyHMA on film source. It would only slow it down a tiny bit but improve film performance when there is almost a field match. Hopefully Telecide would be handling those cases anyway but it's worth thinking about.

Donald - we should talk about that one offline.

- Tom

[bilu]

Originally posted by trbarry
The more I think about it, there is maybe some additional code for TMC that should be added if it is now handling non-video material that could make it a bit more like GreedyHMA on film source. It would only slow it down a tiny bit but improve film performance when there is almost a field match. Hopefully Telecide would be handling those cases anyway but it's worth thinking about.

Donald - we should talk about that one offline.

- Tom

Suspense... ;)

I hold great hope on this post :D


Bilu