Telecide predicament

[HyperSpad]

I capture analog NTSC television (cartoons), and one of the things I'm currently dealing with is telecined material with 3:2 pulldown.

I was initially very pleased with the way I found to get rid of it by using:

avisource("V:\record\deinterlace_testing.avi")

ConvertToYV12(interlaced=true)

Telecide(order=1,guide=1,post=0)
Decimate()

But, just as it says in the documentation for Telecide (Decomb Tutorial/Manual), Telecide will leave combed/interlaced frames behind. Now it recommends post processing to deal with this, but here's my problem with post processing.

Whenever I run a script using post processing...

avisource("V:\record\deinterlace_testing.avi")

ConvertToYV12(interlaced=true)

# also tried settings post=4, same problem
Telecide(order=1,guide=0,post=2,vthresh=24)
Decimate()

...all the frames that Telecide (without post processing) originally fixed now get gaps in their outlines (glasses and gun outlines):
http://www.ultronator.com/doom9/reducto_post.jpg
Telecide with post processing

Whereas Telecide without post processing does not have this problem...
http://www.ultronator.com/doom9/reducto_nopost.jpg
Telecide without post processing


So because Telecide lets some combed/interlaced frames through, I can't just use it without post-processing, but if I use Telecide with post processing I get those gaps shown in the 1st screen shot. Does anyone know of a way to get results with no degradation (second screenshot) and still take care of the combed/interlaced frames Telecide (without post processing) lets through? More advanced settings? Another filter?

[manono]

Hi-

Since you've read the DecombReferenceManual.html, you may have come across this:

vthresh (0.0-255.0, default 50.0) sets the combed frame detection threshold for the postprocessing. You may want to increase this value if too many good frames are being deinterlaced,

I suspect that what you see with the PostProcessing on is good frames being deinterlaced, so raise the Vthresh (not lower it).

Or leave Post=0, add FieldDeinterlace(Full=False) and tweak the Threshold if necessary. Or leave Post=0 and add TDeint(Full=False) tweaking if necessary.

Or read about the other Telecide tweaks, so that maybe no interlaced frames sneak through to begin with. Or use override files in Decomb, if you're having problems with only a few leftover interlaced frames..

Or try a different IVTC (IT, SmartDecimate, TIVTC) to see if any of them can do any better. Lots of ways to skin a cat. Others may have other suggestions.

[HyperSpad]

Since you've read the DecombReferenceManual.html, you may have come across this:

vthresh (0.0-255.0, default 50.0) sets the combed frame detection threshold for the postprocessing. You may want to increase this value if too many good frames are being deinterlaced,

The frames I was having problems with seemed to have very small areas of combing/interlacing. So I have to set vthresh to a low value if I want these frames to get detected and deinterlaced. Setting vthresh to a high value was causing these frames to be ignored. I seem to need a vthresh of about 22 for the post-processing to even pick up on these small combed/interlaced areas.


Or leave Post=0, add FieldDeinterlace(Full=False) and tweak the Threshold if necessary. Or leave Post=0 and add TDeint(Full=False) tweaking if necessary.
So I tried skipping post-processing and using a deinterlacer like FieldDeinterlace or TDeint:
Telecide(order=1,guide=1,post=0)
Decimate()
FieldDeinterlace(full=false,threshold=0,dthreshold=255,blend=false,show=true,map=true)
# also tried TDeint
# TDeint(order=1,type=3,full=false,cthresh=0)
But no matter how low I set the thresholds for these stand-alone deinterlacers, they didn't seem to pick up on any interlaced/combed content if I ran Telecide first. I couldn't even get them to show debug output if Telecide was run first! Is there something wrong with my code above that would be keeping these from working?

Or read about the other Telecide tweaks, so that maybe no interlaced frames sneak through to begin with. Or use override files in Decomb, if you're having problems with only a few leftover interlaced frames..
I don't think I'm quite insane enough to try overriding Telecide by specifying individual frames to process in an input file. As I'm sure this would produce the best results, I really don't think it would be worth the time. There has got to be a quicker way.

What are these Telecide tweaks you speak of? Can they be found in the manual? The only thing resembling a tweak seemed to be the overriding thing.


Or try a different IVTC (IT, SmartDecimate, TIVTC) to see if any of them can do any better.
I also tried using TFM followed by TDecimate in place of Telecide and Decimate and the results seemed to be nearly identical. I also tried SmartDecimate a bit, but it didn't seem to be going well either.


I did discover that blending (blend=true) causes much less harm to the frames telecide fixed than interpolating does (aka the outlines in the screens above). I guess this would be adequate. I still think that there's something I'm missing that's keeping the above script from working properly though.

[neuron2]

You're unrealistically expecting perfection from an automatic heuristic process! When the combed area is small it gets swamped by noise and by normal signal having the spatial frequency of combing.

The other tweaks refer to pattern guidance, the noise threshold, etc. If you'd like to upload an uprocessed VOB sample showing a problematic section to my FTP server I'd be willing to give you what I think is the best treatment using Decomb. The larger the file the better. You can put up to 200MBytes on the server.

Since perfection is improbable without manual tweaking, you should play the video at full speed and normal size and decide if it really is acceptable or not. Freezing on frames and zooming in and then agonizing over minor residual combing is not representative of the perception of a viewer experiencing normal playback.

You may also have some field blending or other pathology that could account for the postprocessor kicking in more than it should. That's another good reason to make a source VOB available for inspection. If you had good consistent 3:2 pulldown, you should be able to get good results with pattern guidance. Let's find out why you are getting residual combed frames.

If you can make a better comb detector please let us all know.

[HyperSpad]

Let me first start off by saying that it was not my intention to bash Decomb or any of the filters associated with it in my last post. I believe that it is definetly possible to tweak Decomb to handle my situation quite well. My last post was mainly meant to express my frustration in failing to find those optimal settings for my situation.

I would be more than happy to upload an unprocessed VOB sample to your server so you can better understand exactly what I'm dealing with here (as soon as I read up on how to create one of those). I will try to upload something by the end of today (after carefully considering the best area to include). I'm not sure how long of a clip can be packed into a 200MB VOB, but I know it's not much for an uncompressed AVI.

I understand that the goal of any script or filter is not to get perfect results, but rather to get the best output possible, such that it appears nearly perfect to the common viewer, and nothing significant sticks out to hinder the presentation. I know that I'm at the point where it is very easy to obsess over little imperfections, but lately I've been carefully trying to draw the line between perfect and acceptable.

If you look back at what I originally posted, I think you can see that the results seem to stem more from an improper combination of settings rather than any shortcomings that may exist in the filter.

Finally I would like to say that I appreciate and fully respect all the work you and others on the forum do to provide everyone with powerful filters to suit our needs. I have neither the skills nor the knowledge to write a better combing detector than any of the ones already out there.


Thanks,
-HYPERSPAD

[neuron2]

I would be more than happy to upload an unprocessed VOB sample to your server so you can better understand exactly what I'm dealing with here (as soon as I read up on how to create one of those). I will try to upload something by the end of today (after carefully considering the best area to include). I'm not sure how long of a clip can be packed into a 200MB VOB, but I know it's not much for an uncompressed AVI. Don't upload uncompressed AVI! Sorry I thought you had a VOB, so forget that. Compress the unprocessed source AVI losslessly using HUFYUV or Lagarith and upload that.

Thanks for the words of appreciation. I hope I can help you when I see your sample.

[HyperSpad]

I'm uploading a sample now, and by the time you read this, it looks like it'll still be uploading it. It's telling me 6 - 10 hours to complete. Is that ok?

[neuron2]

Is that OK? In terms of performance, no -- I would upgrade to broadband if I were you. From my perspective, sure, no problem. :)

[HyperSpad]

I switched to a better ftp agent (that will actually stay connected), looks like about 5 hours now. And I have a cable modem, so I'm not sure why it's going soo slow.

I tried tweaking a few more settings in Telecide that I feel stupid for not having noticed before. Here's my latest script:

avisource("V:\record\birdman_huffyuv_full.avi")

ConvertToYV12(interlaced=true)

# Edit:
# DeDot(20,20,15,5)

Telecide(order=1,guide=1,post=2,blend=true,vthresh=22,dthresh=40,chroma=false,show=false)
Decimate(mode=2)

Things seem to have improved a bit more, but still not perfect. They are very close to the acceptable range though. I'm curious if you can do even better assuming the clip uploads successfully.

By the way, the choice for the segment to upload was not easy. I pretty much chose the part I've been working with and looking at the most. However, there were some other isolate parts I wish I could've included, but didn't.

[neuron2]

How big is the file you are uploading?

I don't see any file or upload in progress.

EDIT: I see an upload now, but it's pitifully slow.

[neuron2]

I have two main points. First, when downsampling from RGB for Decomb, I prefer to stay in YUY2 for processing. Second, you have a noisy analog capture, so a higher noise threshold helps. Here is the script I tried:

avisource("birdman.avi")
converttoyuy2()
telecide(order=1,guide=1,nt=70,post=0)
decimate()

Note that I have no post processing, but the result appears perfect. If you remove the Decimate, you get a very small mismatch on Birdman's mouth, but it is imperceptible at normal speeds.

With the script above, you could add postprocessing with a high vthresh if needed to catch bad edits, etc.

It's always better to try to eliminate stray combs, rather than to postprocess them away.

[HyperSpad]

I'm sorry to say, that I don't know the differences between YUY2 and YV12 and all those other ones with Y's in 'em. I know that certain filters only work with certain colorspaces. Is there actually a clean-cut difference between YV12 and YUY2? (other than the two just being different formats). Also, with huffyuv, isn't that natively YUY2 and if so, why did you need to convert my source to YUY2?


Second, you have a noisy analog capture, so a higher noise threshold helps.

Wow, I completely missed that variable, I breezed over it along with y0 and y1. Does it really help that much? I loaded your script into virtualdub and quickly going through the frames, I don't see much of a difference between it and the script I posted last. I'm running your script on the entire episode now and I'm going to compare the result with the one my latest script (last post) outputted. I think I like yours better though because it doesn't use post-processing which always runs the risk of causing that problem I originally posted about. I was able to minimize it by setting dthresh higher and using blending (which probably isn't the best way, but it seemed to be working for me). But yeah, I'll post my opinion in a bit once your script finishes running and I get to watch the two different versions of the episode.

Speaking of noise and the like (dot crawl), now that you've experienced my source, first-hand (and it's a pretty good example of the kind of problems most of my cartoon captures are gonna have), would you be able to make any specific recommendations on filters (and/or specific settings on filters) I can use to clean it up?


From the DecombReferenceManual:
mode=2...This mode is the most reliable with anime and other material where the motion may occur only in every second, third, or fourth (etc.) frames. If you use mode=0 on such clips, there is a danger that incorrect decimation may occur, causing jerkiness.So is it better to run Decimate using mode 2 in my case?


Oh yeah, and...If you remove the Decimate, you get a very small mismatch on Birdman's mouthUh, Birdman's not in the clip I sent you. You must mean either Reducto (short green guy with the gun) or Black Vulcan.

[neuron2]

I'm sorry to say, that I don't know the differences between YUY2 and YV12 and all those other ones with Y's in 'em. I know that certain filters only work with certain colorspaces. Is there actually a clean-cut difference between YV12 and YUY2? (other than the two just being different formats). Do some research about it. YV12 is subsampled vertically for chroma, while YUY2 is not. When performing interlace detection, it's better to have no vertical subsampling.

Also, with huffyuv, isn't that natively YUY2 and if so, why did you need to convert my source to YUY2? The HUFYUV documents are quite clear. If your input is RGB and you do not select Convert to YUY2 in RGB Compression Method, then your output will be RGB. I needed the conversion because you did not select the option to convert to YUY2.

Does it really help that much? It eliminated at least one obvious stray combed frame in your short clip. Your question is hard to answer without quantifying "that much". Generally, a higher noise threshold helps with noisy analog captures. That's why it was implemented.

Speaking of noise and the like (dot crawl), now that you've experienced my source, first-hand (and it's a pretty good example of the kind of problems most of my cartoon captures are gonna have), would you be able to make any specific recommendations on filters (and/or specific settings on filters) I can use to clean it up? No. My only recommendation is to capture using S-Video or component (not composite) to avoid dot crawl. Please open a new thread for general noise filtering questions.

So is it better to run Decimate using mode 2 in my case? I doubt it. Mode 2 was designed for films like Princess Mononoke, where there are exactly two copies of every frame. You clip is not like that and has such irregular motion that it probably wouldn't be helped. But try it and see what you like better.

[HyperSpad]

Very difficult decision, (as far as choosing between yours and my scripts). Neither is flawless, of course, but I don't think I can decide based on this cartoon alone. The advantage of post-processing is that it can blend the interlaced stuff to make it less noticeable (character's mouths is a good example, although you only got to see a little bit of that in the clip I gave you) as opposed to just ignoring it. However, post-processing can also deinterlace the noise ridden outlines and make them have gaps.

In conclusion, what I'm talking about is really picky stuff and the truth is that I could probably use either of the two scripts and the common person wouldn't see anything wrong with either of them. I'm gonna continue testing and tweaking the scripts using a few different cartoons. Whatever works best on the majority of the stuff will be what I end up using.


Do some research about it. YV12 is subsampled vertically for chroma, while YUY2 is not. When performing interlace detection, it's better to have no vertical subsampling.
I found that ConvertToYUY2() tends to make edges pixelated while ConvertToYV12(interlaced=true), which is what I've been using lately, seemed to make edges smoother (yes I tried YUY2 with interlaced=true, no difference). Maybe converting to YUY2 before telecide and converting to YV12 after telecide would work. I guess I'll have to research these colorspaces a bit more.


Thanks for all your help
-HYPERSPAD

[Chainmax]

Try setting blend=false to use interpolation instead of blending (blending causes ghosting). Also, maybe using the back setting could help.