Weird DeDot-caused artifact, why does it happen?

[Chainmax]

I'm backing up my Simpsons S7 DVDs and found a peculiar situation with the very first episode. It was pretty easy to clean, but upon inspection of random spots using VDubMod, I came across the following artifacting:

http://img245.imageshack.us/img245/8355/sourcelu3.png (http://imageshack.us)
http://img245.imageshack.us/img245/815/resultbz7.png (http://imageshack.us)

Testing the filters one by one showed that the culprit was DeDot's location. I usually put it right after MPEG2Source or ColorMatrix, but the only way I got rid of the artifacting was putting it right before the TDecimate line:

MPEG2Source("C:\SimpS7\e1\Epi1.d2v",info=3)

ColorMatrix(hints=true,interlaced=true)

BiFrost()

DeCross()

AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,type=1,edeint=Interp)
TFM(d2v="C:\SimpS7\e1\Epi1.d2v",order=1,mode=6,PP=7,slow=2,mChroma=false,micmatching=3,Clip2=Deinted)

DeDot()

TDecimate(mode=1)

FFT3DFilter(sigma=3.5,plane=3,bw=32,bh=32,bt=3,ow=16,oh=16)

DeGrainMedian()

DeHalo_Alpha()

Crop(8,0,702,480,align=true)

Spline36Resize(640,480)

RemoveGrain(mode=1)

FFT3DFilter(sigma=1,bw=32,bh=32,bt=3,ow=16,oh=16)

aWarpSharp(depth=16,cm=1)

dull=last
sharp=dull.LimitedSharpenFaster(SMode=4,Strength=150)
Soothe(sharp,dull,25)

Putting DeDot right after TDecimate or anywhere before IVTCing would cause this. Any ideas as to why it happened? If you want , you can download a sample clip with the scene in the pictures from [sorry :o].

[Mug Funky]

just a consequence of how dedot determines what is a crawl and what isn't. when it hits the wrong part of a 3:2 pattern (especially on a simple, flat source like animation) it'll effectively remove parts of the picture.

note that TComb doesn't do this - i switched when i found out and haven't suffered any bad effects (though i left that job shortly after the discovery and didn't test it on many discs).

you'll see this effect on "shaking up and down in 3:2" footage. if there happens to be back-and-forth movement in the particular 2 frames in the 3:2 cycle that are interlaced, you'll see chunks disappearing.

[Chainmax]

Ok, I'll try TComb and compare both. In the meantime, how would you say they compare in dotcrawl removal effectiveness and how much actual detail they remove?

[DarkT]

As much as I love Tritical(he also just solved a problem I had a day ago) - Tcomb, sometimes utterly trashes the pic, I had to do lots of replaceframes with fumoffu... But Tcomb also is the best one to remove DotCrawl that I've found up untill now...

Toget rid of most dotcrawl, I use the following pre-ivtc

Tcomb(mode=0,fthreshL=7,othreshL=10)
DeGrainMedian(interlaced=true, mode=2)
hqdn3d(5)
VagueDenoiser(threshold=1.5, method=1, nsteps=6, chromaT=2.0, interlaced=true)

Some might question what the heck is DeGrainMedian/hqdn3d are doing there, all I can say is that they help in removing the horrible dotcrawl/grain that fumoffu has...

Simpsons are a rather "2d" animation, it doesn't have too many details and all - so I don't think you'll see the problems Tcomb can cause, but be wary anyway.

Edit:

BTW, I was getting a very similiar, though much worse "artifact" when I did "DFMDeRainbow()" pre IVTC... Then I put it after and it was all good(after I started disabling them all to check what causes the artefact).

[Chainmax]

Well, I tried it and must say I'm very impressed. Not only did it remove all the dot crawl, I could even ditch the chroma smoothing via FFT3DFilter :eek:. I did notice some very slight image differences between the old filterchain and the new one throughout the comparison, but nothing too evident. Thanks for the advice though, DarkT, I'll keep my eyes open http://smilies.vidahost.com/otn/wink/thumb.gif.
I hope someday tritical gets the itch to develop a halo remover to complement the already wonderful DeHalo_Alpha :).

[Alain2]

On films the halo remover built in ftt3dfilter is quite good imho (and dehalo_alpha remains better for me on animation, but not on real-life content)

[DarkT]

Chainmax - it's very rare, let's say 1% of the video - it happens... Only when there are lots of details and it's all a bit fuzzy to begin with - my best bet is that Tcomb catches it as "noise" or something and just tries to "clean" it all... The result is that you see it all blurry, as if through one of those photoshop filters... That's the best I can describe it, I'll post pics later, since I'll be QC'ing my encode later on.

Again, my guess is that you won't see it in simpsons, but you know - just a heads up for the future, since I remember breaking my head over it - I just didn't suspect Tcomb for some reason... go figure.

[JuanC]

Here, two very important recommendations about the correct placing of comb filters in a script: from the GuavaComb filter by "High Speed Dubb" Readme file: ... Where should I put it in my script? ... Comb filtering should definitely
be run before any inverse telecine steps. Color crosstalk can badly mislead
inverse telecine, so it is worth cancelling it out before trying to figure
out the pulldown pattern.

What the Heck is a Comb Filter?

In technical terms, it averages out chrominance/luminance crosstalk caused by
imperfect separation from a composite signal. It’s called a “comb” for a really
obscure reason — because the frequency response looks a (very little) bit like
a comb. Apparently, signal processing engineers are obsessed with grooming. ... From the DeDot filter by "thejam" readme file: ... Only used for interlaced materials. ...And from the TComb filter by "Tritical" readme file: ... It must be used before ivtc or deinterlace in order to work. In terms of
what it does it is similar to guavacomb/dedot. ... Personally I am very happy with the results I get with my Analog TV captures using TComb. Real-life or 3D/2D Animation. Sometime ago I started using Guava, then DeDot and now TComb. The artifacts you get are caused as "Mug Funky" explains by the metrics these filters use for determining what are and what are not dot-crawl or rainbows caused by chrominance/luminance crosstalk. You can play with the parameters in these filters. Here a translation of the DeDot readme file, I don't recall where I got it from (¿niiyan?), so you can play with it too. Translation of DeDot.txt

DeDot (DeDot 0.3) : thejam

It reduces DotCrawl and Cross-Color. Maybe.

Changes
0.3
debug Check Cross-Color more strictly.
0.2
debug Remove green points flickering on the upper right corner.
debug Reduce garbage.

Requirements
MMX2 CPU.
Only used for interlaced materials.

Usage
Write in your avs file as follows:
LoadPlugin("path\De.dll")
# your sources and other filters
DeDot()

Parameters
DeDot(luma2d,lumaT,chromaT1,chromaT2)
default 20,20,15,5

luma2d
Spatial luma threshold.
This value is lower, DeDot detects more pixels as Dot Crawl.

lumaT
Tolerance of Temporal luma signals.
This value is higher, DeDot detects more pixels as Dot Crawl.

chromaT1
Tolerance of temporal chroma signals.
This value is higher, DeDot detects more pixels as cross-color.

chromaT2
Temporal chroma threshold.
This value is lower, DeDot detects more pixels as cross-color.

Warning
Filter before deinterlace and resize.
If pixels are mixed, difficult to detect.

I don't know well, but different TV tuners and Capture cards have
different appearing patterns of Dot Crawl and Cross-Color ?
If so, you may not get good results depending on environment.

Disclaimer
At your own risk.

[Chainmax]

Thanks for the advice, but I already knew that. As is said in the OP, the only way to avoid the artifacts was to put DeDot right after fieldmatching (i.e: right in the middle of the IVTCing); not before, not after.

[BigDid]

DeDot (DeDot 0.3) : thejam

It reduces DotCrawl and Cross-Color. Maybe.

Changes
0.3
debug Check Cross-Color more strictly...

Hi,

Thanks for the infos. Would you, by chance have a link for the Dedot 0.3 rev cause after checking, I have only the 0.2
:thanks:

Did

[Terranigma]

Chainmax, DarkT. What would you guys recommend for high-quality real-life sources to remove dotcrawl? Using Tcomb destroys the image, and dedot, as you've witnessed, can mess up a frame. I've witnessed that too a good while back and stopped using DeDot ever since.

[JuanC]

@BigDid: About DeDot, I'm afraid I have the same version yo do. But I found out where I got the translation from. It was "niiyan" who provided the translation of some guides into English for some Japanese avisynth filters at his home page: niiyan's web page (http://niiyan.s8.xrea.com/avisynth/index_en.html). It points to the place (http://nullinfo.s21.xrea.com/#DeDot_YV12) where you can get the filter, but they have the same version (0.2) we have.

If you look at the source code of the filter you'll see at the first two lines: // De.dll v0.0.02 Copyright(C) 2002 thejam79, 2003 minamina
// Avisynth Plugin - DeDot (YUY2 and YV12 Only, DeDot 0.3 baseThis suggests that another developer by the name of "minamina" took DeDot v0.3 by "thejam79" as a baseline to produce the ¿newer? version we have.

You could ask niiyan to find out if he has any newer version. He's a user of Doom9's forum. niiyan also has an english home page at: http://niiyan.net/?FrontPage

@Chainmax: I have tried some things with these comb filters. Placing them before, after, in between, the IVTC process. I never got perfect results, but placing them before IVTC has yielded the results I like the most. Sometimes I play around with the parameters, and to get more satisfying results, I adjust them for every particular video source. Finally, another quote from Dedot's readme: Warning
Filter before deinterlace and resize.
If pixels are mixed, difficult to detect.

[BigDid]

Yep,

Too bad, thanks for coming back anyway.

If I get it I will post a link.

Did

[Chainmax]

Terranigma: if you find TComb is destroying the image, I'd recommend you to use wat DarkT suggested minus TComb, i.e:


...
Toget rid of most dotcrawl, I use the following pre-ivtc

DeGrainMedian(interlaced=true, mode=2)
hqdn3d(5)
VagueDenoiser(threshold=1.5, method=1, nsteps=6, chromaT=2.0, interlaced=true)
...

as I know for a fact that DeGrainMedian is indeed a very good dotcrawl remover as Mg Funky said a long time ago. The other two are probably good as well then. Try different combinations of those (before/after IVTC, more/less aggressive settings, etc).

[niiyan]

@BigDid and JuanC
"DeDot 0.03" is the version number of an original version of DeDot, not DeDot_YV12.
So, the latest version of DeDot is DeDot_YV12 v0.0.02 (http://nullinfo.s21.xrea.com/#DeDot_YV12).

DeDot was originally developed by thejam79.
But he stopped development of his AviSynth plugins.
Then, minamina added AviSynt 2.5 and YV12 support to DeDot 0.0.3 and released it as DeDot_YV12 v0.0.01(the plugin name is still De.dll).

[BigDid]

@BigDid and JuanC
"DeDot 0.03" is the version number of an original version of DeDot, not DeDot_YV12.
So, the latest version of DeDot is DeDot_YV12 v0.0.02 (http://nullinfo.s21.xrea.com/#DeDot_YV12).

Hi niiyan,

That's the version I have except mine was named dedot.dll; yours is de.dll but rev number is the same and last modified date is also the same.

:thanks:

Did

[JuanC]

Thanks for the info niiyan!

And thanks for your efforts to help us non-japanese speakers to understand and know about all those filters.

[Chainmax]

...
@Chainmax: I have tried some things with these comb filters. Placing them before, after, in between, the IVTC process. I never got perfect results, but placing them before IVTC has yielded the results I like the most. Sometimes I play around with the parameters, and to get more satisfying results, I adjust them for every particular video source. Finally, another quote from Dedot's readme: Warning
Filter before deinterlace and resize.
If pixels are mixed, difficult to detect.

Like I said, I already know that. I also said that for this particular source any other location than after fieldmatching would cause the artifacting.

[niiyan]

I wrote all I know about DeDot here: http://niiyan.net/?DeDot
You can find English translation of DeDot docs (DeDot_YV12.en.txt and DeDot.en.txt).