Dotcrawl-like effect won't go away

[Elegant]

Hi everyone, I'm new to these forums :)

I'm currently trying to create a script for a DVD I have where the source seems to have many issues. I have fixed most of them but I've come across this one situation where it looks like a case of dot crawl but could be a color bleed(?). This screenshot is from the source (https://www.dropbox.com/s/0hk8fox8whenwm8/Sample.m2v_20140721_153932.031.jpg).

The problem area is when the the orange hair clips the ground, it seems to create some kind of a dot crawl that I just can't remove without some intense spatial denoising. TBilateral did a fine job eliminating it but basically destroyed the detail. I can reduce it somewhat with TComb(0, 28, 29) and Checkmate(0, 1, 5) but I'm wondering if there's a better solution so I don't have to have TComb's values be so high. I'm also using a maa2() to remove some low frequency dot crawl but since this doesn't really have a black edge of any kind it doesn't apply but again this screenshot is from the source.

A sample from the source can be found here (https://www.dropbox.com/s/5lkojg1uzj1zgy8/Sample.m2v).

[Bloax]

There's also some rainbowing happening if you pay attention to the colors of the (supposedly) black lines. It's pretty noticeable along the top ear of the lying digimon.

[Elegant]

Yes, again that particular screenshot and sample are from the source. The current script covers rainbows and is as follows:

TComb(mode=0)
Checkmate(thr=0, max=1, tthr2=5)
AnimeIVTC(mode=1)
crop(10, 0, -6, 0)
Undot()
maa2()
MergeChroma(RemoveDirtMC,0.80)
ChubbyRain2()
DDComb()
TTempSmooth()
LSFmod(strength=100)

Screenshot with script (https://www.dropbox.com/s/5b6hgyf14zrhcul/Ep1026648.jpg). The "dot crawl" is still present in this screenshot as well though not nearly as noticeable.

[fvisagie]

I'm unsure how important it is to you, but ghosting seems to be more pronounced in your last screenshot.

[Elegant]

I did notice that cranking up TComb introduced some ghosting in previous frames, I toned it down just now. I didn't notice it in the current screenshot frame though, could you point it out by chance? I've been very fixated on the crawls and I don't think I can see much else XD

[foxyshadis]

Kill it with fire or nuke it from orbit is the only solution for crazy heavy dot crawl. I've settled on the following for all the no-budget composite-captured training DVDs I get:
TComb(fthreshL=20,fthreshC=16,othreshL=20,othreshC=16)
TDeint()
fft3dfilter(sigma=10,plane=3,bt=5)
fft3dfilter(sigma=2,plane=3)
dfttest(dither=1)
LSFmod(defaults="fast")
It's ugly but effective. The original DVDs were captured from a VHS deck, so it's not like there's anything much to destroy. YMMV with this.

[Elegant]

Yeah, I think what I have works a bit better for the moment; I may tinker with yours a bit more tomorrow though. Thanks!

I'm thinking it may even be best to just use a spatial denoiser to fix the last bit of dotcrawl. I'm not sure if the quality lost was really there to begin with; tough call though.

[fvisagie]

could you point it out by chance?

Along the edges of the standing character, comparing your source screenshot with the one in post #3. To be fair, there is a degree of haloing (better term this time, sorry) in the source already, and it being more pronounced in post #3 may purely be the result of denoising and sharpening.

[Bloax]

I wonder if applying the "nuke from orbit" along an aggressive edge mask would do anything to salvage a tiny bit of information, hm.

[Elegant]

@fvisagie I didn't notice that! I see it now right along his wing and his head. As you said, it's also in the source just masked by the noise and rainbows around it. I'm working on it now though the fact that his wing is also quite bright near the halo is not helping...

@Bloax If combined with maa2() it seems to clear up most of the issues. Still having to resort to Checkmate though as a high TComb is introducing artifacts. I'll probably make another stab at finding better TComb values later, not sure if I'll need the chroma part though as all rainbows looked to be fixed with the nuke.

EDIT: I think I'll take the hit in detail; it doesn't seem all that bad compared to other spatial filters I was planning to use. I think I'll stick with the following:

TComb(mode=0, fthreshL=6, othreshL=7)
Checkmate(thr=0, max=1, tthr2=5)
AnimeIVTC(mode=1)
crop(10, 0, -6, 0)
Undot() # Minimal Noise
maa2()
FFT3DFilter(sigma=10,plane=3,bt=5)
FFT3DFilter(sigma=2,plane=3)
DDComb()
dfttest(dither=1)
LSFmod(defaults="fast")
DeHalo_alpha(rx=1.4, ry=1.4, darkstr=0.4)

[LemMotlow]

TComb()
AnimeIVTC(mode=1)
ColorYUV(cont_y=30,off_y=-20)
a=last
e=a.SelectEven().SPline36Resize(360,480).Santiag().Spline64Resize(720,480)
o=a.SelectOdd().SPline36Resize(360,480).Santiag().Spline64Resize(720,480)
Interleave(e,o)
LimitedSharpenFaster(edgemode=1,smode=4)
FastLineDarken(strength=48,thinning=86)
DeHalo_Alpha()
TemporalSoften(2,2,4,15,2)
Santiag()
Overlay(last,a,mode="chroma")
GradFun2DBMod(thr=1.5,mask=false)
return last

sample_new.mp4 http://www.mediafire.com/file/2z62r5hfhc96e0d/sample_new.mp4

http://forum.doom9.org/attachment.php?attachmentid=14286&stc=1&d=1406152688

[Elegant]

I contemplated using this tactic to remove the crawl but a complete resize removes far too much from the picture quality. Anything small is completely lost, I think at one point a guy's shoes looked like carved rocks. Changing the color scheme up though seemed interesting I might look into that. For now though I think I'll stick with my last iteration. Thanks though!

[LemMotlow]

I contemplated using this tactic to remove the crawl but a complete resize removes far too much from the picture quality. Anything small is completely lost, I think at one point a guy's shoes looked like carved rocks. Changing the color scheme up though seemed interesting I might look into that. For now though I think I'll stick with my last iteration. Thanks though!What "picture quality" do you mean? It looks like a careless VHS to DV capture in appearance if not in fact, with an elevated IRE and bad clipping. Depends on what kind of disturbance you prefer to watch. I find bad interlace or processing with hard-coded telecine, and permanent loss through clipping during capture to be more annoying. I guess everyone has their idea of what each piece of video ought to look like. Everyone has different priorities.
:)