Dot-Crawl and Rainbowing in a telecined source

[bkman]

Is there any filter that can deal with these artifacts baked into a DVD and telecined? Not just that, but they are on constantly moving parts of the image.

I've tried running various of the popular filters (like guavacomb, etc) after IVTC but they make no difference. The closest I have come to a filter that worked was the VirtualDub DotCrawl Comb filter by Scott Eliott (in Spacial mode), but it lost a lot of detail. Is there anything better?

The source in question are the Beast Wars DVDs, if anyone is wondering.

[Richard Berg]

Try TComb if you haven't. Also, depending on how it was mastered, it might be better to run before IVTC. (You always do it in this order with TV captures, for instance.)

[charleski]

I've found DeRainbow (http://www.avisynth.org/DeRainbow) to work extremely well in removing luma->chroma artifacts from composite sources (which is what I assume you mean by rainbows)

[bkman]

Tcomb does seem to help, although only in the low motion parts of the video. I suppose that it can't get much better than that...

[Chainmax]

For dotcrawl remocal, I recommend to use DeDot() (http://nullinfo.s21.xrea.com/#DeDot_YV12) before deinterlacing/IVTCing.
For rainbow removal, try BiFrost (http://yatta.mellbin.org/misc/), but you have to find out what kind of rainbows you're dealing with (the included readme tellls you how to know) in order to know wether to put it before or after IVTCing/Deinterlacing, as putting it in the wrong place renders it useless. Another great rainbow remover is foxyshadis's SmartSSIQ mod (http://forum.doom9.org/showthread.php?t=103191&page=2). It needs SSIQ (http://yatta.mellbin.org/misc/).
To sum it up, your script would look like this:

BiFrost() <-- if rainbows are Type 1
ConvertToYV12(interlaced=true)
DeDot()
IVTCing/Deinterlacing
Bifrost(interlaced=false) <-- if rainbows are Type 2
SmartSSIQ() <-- Another rainbow remover
rest_of_the_filters

[bkman]

DeDot seems to work about as well as tcomb (although without the artifacts that tcomb makes when the image is shaking), but the problem is that they are both temporal filters, and most of the dotcrawl in my source is present in moving details.

The rainbowing I am not as worried about, but it would be great to get rid of the dotcrawl entirely. Unfortunately, no-one seems to know a filter that will do it.

[hartford]

A long shot, but you might try to DeRing. One or both together might work.

<a href="http://forum.doom9.org/showthread.php?t=96592&highlight=hqdering">HQDering Script</a>. Requires Deen v1.0 beta 1 and MaskTools v 1.4.16

BlindPP internal filters of DGDecode. This function is explained
in the documentation.

Both filters are spatial.

Good luck to you.

[well, my "html code" is off and can't be changed, apparently; sorry for sloppy link.]

[bkman]

It was a long shot, and I'm sorry to say it didn't work.

I guess this kind of problem doesn't come up all that often, or else someone would have written a good spacial filter for it by now. Scott Eliott's vdub filter just blurs it too much.

[Mug Funky]

@ bkman:

sadly, rather than being a problem that doesn't come up often and is thus ignored, rainbowing belongs to that other class of problem that comes up all the time but is intrinsically difficult to deal with.

when a picture is static, it can be dealt with pretty well, but when stuff moves the rainbows become unbeatable.

[mg262]

when a picture is static, it can be dealt with pretty well, but when stuff moves the rainbows become unbeatable.Mmm... so this is a shot in the dark, but have you tried a mocomped attack? Motion vectors are (AFAIK) always computed on pure luma, so they won't be misguided by chroma artefacts. If its stuff on which motion compensation doesn't work very well, it's probably old animation... in which case, can I have a sample ;)?

[Mug Funky]

hmm. i've never tried motion compensation, but i suspect the results will be only slightly better - the crosstalk can only be cancelled because it inverts every field. if the fields were moved about it'd go in and out of phase, so something like DeDot would be of limited effectiveness. the motion engine would have to take the crawls into account and do the compensation such that they cancel out. and of course that would give spoiled motion, which would mean it doesn't filter enough, which leaves you only slightly better than with no compensation at all.

well, that's my gut feeling anyhoo :)

[edit]

The source in question are the Beast Wars DVDs, if anyone is wondering.

didn't read that the first time... sorry about that.

Beast Wars is a bit of a problem child... it was originally made in PAL (some french studio) and several strange edits were made (some scenes look to have been rendered interlaced, then deinterlaced in editing using field-discarding). if you're in NTSC land, you'll have standards-converted versions of this footage, and it's a different edit again. also, the masters are analog. Betacam SP. though it's a component format (i think... hard to tell, but on the monitors i saw no crawls so that'll do me), it's quite possibly been output as composite, then standards-converted to NTSC, then encoded for DVD. if this is the case, you're SOL because the standards-converter probably muddled the rainbowing up enough that no filter can remove it without nuking the chroma.

the more agressive De-Rainbowers should help though. and don't worry too much about leaving them in, because for all it's faults, beast wars seems to be pretty compressible.

or... you could wait for the R4 release because it'll be slightly better - still a bit noisy, but no standards-conversion and no rainbows. also a fair bit sharper (due to the apparent lack of antialiasing in the original renders).

@ clouded:

motion compensation works pretty well on beast wars - it's all CG with no motion-blur, but mvflowblur works a treat on it :)

[Didée]

hmm. i've never tried motion compensation, but i suspect the results will be only slightly better - the crosstalk can only be cancelled because it inverts every field. if the fields were moved about it'd go in and out of phase, [...]
If it works for general noise, why shouldn't it for crosstalk? You have to pay attention that your ME doesn't "snap on" to the artefacts, i.e. the clip ME is performed on should be prefiltered.

[bkman]

Beast Wars is a bit of a problem child... it was originally made in PAL (some french studio) and several strange edits were made (some scenes look to have been rendered interlaced, then deinterlaced in editing using field-discarding). if you're in NTSC land, you'll have standards-converted versions of this footage, and it's a different edit again. also, the masters are analog. Betacam SP. though it's a component format (i think... hard to tell, but on the monitors i saw no crawls so that'll do me), it's quite possibly been output as composite, then standards-converted to NTSC, then encoded for DVD. if this is the case, you're SOL because the standards-converter probably muddled the rainbowing up enough that no filter can remove it without nuking the chroma.

Yeah, its a bit oddly mastered. The strange thing is that Seasons 2 and 3 (the ones I have right now) are mastered differently. Season 2 seems to have been frame-blended up to 60 fields a second (so I'm using dgbob, it looks decent), but Season 3 is properly telecined from ~25fps -> ~30fps, and I can get back the source through IVTC. Its with the latter that the dotcrawl (as I mentioned, I am not as concerned about the rainbows) is most noticeable, as it is quite clear.

I don't know too much about image processing, but couldn't a filter be made to kill crawl based on looking at the high-contrast detail where it is likely to occur, identifying the ladder pattern, and blurring it away?

[hartford]

<i>looking at the high-contrast detail where it is likely to occur, identifying the ladder pattern, and blurring it away?</i>

Well, if you have a few years for processing, I'm sure that could be done ;)

(note: why is my html off? and, how do I turn it on? Can't find the switch.)