I need some advice for removing chroma artifacting on a capture

[Chainmax]

I am creating a filterchain for a somewhat recent VHS capture. Its almost finished, but I need some help removing chroma artifacts. The rainbows are easily removed, but what really is causing me trouble are some yellow and red color splotches. You can see an example here:

http://img203.imageshack.us/img203/7842/source6iy.png

Now, my last try looks like this:

http://img203.imageshack.us/img203/681/filtered9nv.png

And it uses SSIQ(diameter=11,strength=50,interlaced=false) for rainbow removal and Cnr2("xxx",4,5,255) for extra rainbow removal and color splotch removal. As you can see, the color splotches, while somewhat diminished, are still there. Also, Both SSIQ and aWarpSharp cause those green smudges at the sides.

I'd like to know if you could suggest me something to replace these two (I am trying to find an effective alternative to aWarpSharp as well). I thought about LB's SmartSSIQ function, but I don't know if it can handle the color splotches. Any help will be greatly appreciated.

[Revgen]

Msmooth combined with FFT3dfilter worked well for me for removing rainbows, regular noise, and mosquto noise at high settings. I used it to clean up my crappy Garfield cartoon episodes.

Unfortunately, I can't remember the exact settings I used.

[Chainmax]

I'm using LRemoveDust_YV12 for denoising and found it better suited than FFT3DFilter(sigma=1) for this source. Are you sure FFT3DFilter and MSmooth can remove rainbowing?

[Revgen]

It worked for me. It might work for you too.

I was orginally using them to remove all the noise. I figured that I'd encode to Huffyuv then figure out how to remove the rainbows later. But it turned out that it also removed the rainbows too.

It's very slow if you don't have a highend processor. I probably wouldn't have used them if the noise wasn't so bad.

Just use Trim() and do a short encode and find out.

[Soulhunter]

...better suited than FFT3DFilter(sigma=1) for this source.
You know that FFT3DFilter(sigma=1) alone will only processes the luma, no?

And no, imo FFT3D doesnt work well for "TV-style" interferences... :\


Bye

[Chainmax]

I didn't, but a quick glance at the readme confirms that you're right, but then again:

plane - processed color plane: 0 - luma(Y), 1 - chroma U, 2 - chroma V (default = 0)

so, you can either filter luma or one chroma plane at a time?

[Mug Funky]

you could use a frankenstein's monster script by separating the channels and applying fft3d on them all, then merging them again.

bt=3 will help a LOT for things like chroma noise chunks. and finish it off with a fairly aggressive temporalsoften on only the chroma.

a nice GPU will help if you like fft3dfilter - use the GPU version (with an appropriate overlapping setting and a reasonably high sigma). the only reason i got a PCI-e viddy card was to use this filter, and i wasn't disappointed :)

[mg262]

For what it's worth, this is what I sent to ChainMax when we were discussing it by PM before this thread came up:
I made it a bit better by using this:

imagesource("source6iy.bmp")
converttoyuy2

source = last
reduceby2
spatialsoften(12, 20, 100)
bilinearresize(width()*2, height()*2)
source.mergechroma(last)

although it is pretty slow. You could downsample more to fix that, although that would increase the chroma bleeding. In any case, it's not so much this particular solution I wanted to suggest so much as the idea that using some pretty heavy filtering followed by mergechroma might be a good idea.It was reasonably effective but very slow... the problem being that four times as many luma pixels as chroma were being processed, to no earthly purpose.

a nice GPU will help if you like fft3dfilter - use the GPU version (with an appropriate overlapping setting and a reasonably high sigma). the only reason i got a PCI-e viddy card was to use this filter, and i wasn't disappointed I was thinking about getting a GPU just to try speeding up some of my filters... some are both very parallelisable and very slow. But unfortunately (I believe) I only have a AGP slot...

[tsp]

mg262: Buy an Asus V9999GT. It's AGP, cheap and rather fast. The GPU should matter more than the interface. I use one right now and also used it when I created fft3dGPU.

[mg262]

@tsp,
Thanks! I shall look into it.

[Chainmax]

So, is there no way to tackle Y, U and V all at once in one single FFT3DFilter call? If there was, this would make it an ideal filter for this (denoising, chroma cleanup and first stage sharpening would be awesome).

[Didée]

If it has to be by all means, you could do
stackvertical(stackhorizontal(o.UtoY(),o.VtoY()),o)

FFT3DFilter(...)

YtoUV(crop(0,0,width/2,height/3),crop(width/2,0,-0,height/3),crop(0,height/3,-0,-0) )
Filter settings are not separateable between the planes, and (eventually, dunno) some inprecisements at the borders might happen.

[Backwoods]

YToUV: Y clip does not have the same height of the UV clips! (YUY2 mode)

I recieve that error with this script:

LoadPlugin("C:\Program Files\AviSynth 2.5\yea\FFT3dFilter.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\yea\Decomb521.dll")
o=AVISource("FOTNS_SAMPLE.avi").Telecide(0).Decimate(5)
StackVertical(stackhorizontal(o.UtoY(),o.VtoY()),o)
FFT3DFilter(sigma=1)
YtoUV(crop(0,0,width/2,height/3),crop(width/2,0,-0,height/3),crop(0,height/3,-0,-0))

[Didée]

Ah, YUY2 does exist, too ... ;)

Okay, excercise:

change the cropping values laid for YV12

UV
YY
YY

to fit to YUY2

UV
YY

Hint: look out for odd numbers.

:)

[mg262]

Mmmm... what was the problem with having three separate filter calls?

[Chainmax]

Unnecessary slowdown. Anyway FFT3DFilter(plane=1,bt=3).FFT3DFilter(plane=2,bt=3) (like Mug Funky suggested) would only remove chroma artifacts, so no denoising would take place, right? Would using the GPU version imply any difference besides speed? Finally, what overlapping settings and sigma would you recommend me for artifacts like the ones in the picture?

[tsp]

with the GPU version fft3dgpu(plane=1,bt=3) will process both chroma-planes

[Backwoods]

Ah, YUY2 does exist, too ... ;)

Okay, excercise:

change the cropping values laid for YV12

UV
YY
YY

to fit to YUY2

UV
YY

Hint: look out for odd numbers.

:)

Perfect, thank you.

[Revgen]

Would using the GPU version imply any difference besides speed?

Here IMO are the most important differences:

* It doesn't produce "griding" artifacts at high settings.

* No Kalmar filtering.

I believe there are other minor technical differences, but IMO these differences shouldn't change the result as much as the ones above.

[Mug Funky]

the griding appears in the GPU version because the default overlap is for speed rather than quality. you just need to use the 50% overlap method (i forget which one that is - it's not set up on my work machine).

[tsp]

mode=1 or mode=2. The current version of fft3dGPU is missing most of the new features of fft3dfilter(>version 1.1). Might add some of those when I'm done with the MultiThreaded version of avisynth

[Backwoods]

the griding appears in the GPU version because the default overlap is for speed rather than quality. you just need to use the 50% overlap method (i forget which one that is - it's not set up on my work machine).

bw=48,bh=48,ow=24,oh=24

Like that?

[tsp]

no just mode=1 and for even less grid use mode=2 instead. Also note that the bw,bh has to be a power of two (32,64,128 etc).
so something like this:
fft3dgpu(bw=32,bh=32,mode=1)
or
fft3dgpu(bw=32,bh=32,mode=2)

[Backwoods]

Nice, I'll have to test it out. I've been getting alot of gridding with the settings I try.

[Chainmax]

Ok, here's a comparison of SSIQ+CNR2 and 2 FFT3DFilter calls:


SSIQ+CNR2:
http://img348.imageshack.us/img348/2981/ssiqcnr21kj.png

2xFFT3DFilter:
http://img348.imageshack.us/img348/491/fft3dfilterx25iy.png

Do you see any difference between the two? If so, which one you feel is better?

[Revgen]

I can't see any difference when looking at these stills.

Maybe there is a subtle difference when the clip is played.

<OFFTOPIC PONDERING>

Hmm... is there an MVTools based rainbow remover?

Perhaps motion compensation can be used to track the difference between the rainbows in the chroma planes and the true content in the luma planes.

Or maybe a combination of the masktools methods in SSIQ and MVtools methods mixed together.

Or maybe I'm just getting crazy ideas. :rolleyes:

[mg262]

CM, Can we have a chunk of source?

[Chainmax]

Sure thing. It took a while to upload due to my crappy connection, but here (http://www.31012.com/~azulftp/X-Men%201x00%20-%20Opening%20(HUFF).01.avi) it is.

[mg262]

Wow -- that'll take me a while to download! Thank you very much.

[Chainmax]

Confucious say: be careful what you wish for, you might just get it ;) :).

Sorry if it's too long, it's an almost unprocessed (only IVTC and color correction were done) capture. I didn't think about posting a shorter clip, I apologize :o.

[mg262]

No, it's not too long, it's perfect -- thank you!

Edit: Grrr. Those lightning effects are annoying.

Edit: So -- when I played this, I found that the shaking was noticeable and distracting, especially on the title screens; on the other hand, I know that a)I am particularly sensitive to this and b) I've been working on it recently. Does anyone else notice it?

[Backwoods]

Does anyone else notice it?

Yea, it shifts around a bit.

[Chainmax]

So you think I should use DePanStabilise (or maybe CelStabilise once it gets some speedup)?

[mg262]

I haven't personally managed to stabilise animation effectively with DePanStabilise or VirtualDub's DeShaker, but I put this down to my own difficulty in setting all the parameters appropriately. I would definitely encourage you to try them -- and would be very interested in the results!

I'm trying Stabilise on your clip myself; that wasn't my intention when I downloaded it, but as I said, I find shaking particularly distracting. The filter has been sped up a bit and is running at 5-6 FPS on my machine -- but I'm by no means finished. What kind of speed would you consider acceptable? Incidentally, can I ask which of SSE/SSE2 you have?

[Chainmax]

I have a Sempron 2200, so I only have SSE. As for tests with and without stabilizing, you can download a clip without stabilizing I made a short while ago:

http://rapidshare.de/files/5580334/Pryde.mp4.html

Since you downloaded the source and know much more about this than me, I think you would be a much more suitable person to try your stabilizer on it than me. By the way, I tried DePanStabilize as follows:

AVISource("X:\wherever\X-Men 1x00 - Opening (HUFF).01.avi")
i = ConvertToYV12()
mdata = DePanEstimate(i)
DePanStabilize(i, data=mdata)
AddBorders(16,16,16,16)
SSIQ(diameter=11,strength=50,interlaced=false)
Cnr2("xxx",4,5,255)
LRemoveDust_YV12(17,2)
aWarpSharp(depth=16,cm=1)
Crop(18,18,716,476,align=true)
Lanczos4Resize(640,480)
RemoveGrain(mode=1)
HQDering()
LimitedSharpen(ss_x=2,ss_y=2,Strength=128)

And found out that a thin black border appeared on the left side of the screen from frames 12 to 25 (maybe elsewhere too, didn't confirm).

[Chainmax]

So, have you tried your stabilizer on the clip?

[mg262]

I tried earlier, but wasn't terribly happy with the results; there were particular problems with scenes where a character "ran out of the screen" at you (hope that make sense), because at the end of the scene there was no background left to track.Plus as I said earlier the lightning in some scenes makes a total mess of things! I'm going to fiddle with things to try and work around this, but real life has been pretty busy...

[Revgen]

@Chainmax

Have you tried Tritical's TComb filter instead of SSIQ?

[Chainmax]

No, I haven't. What are your experiences with it?

[Revgen]

I haven't tried it yet. I just went to Triticals homepage (http://bengal.missouri.edu/~kes25c/) yesterday and saw it. I figured you'd be interested.

[Chainmax]

Well, it doesn't hurt to try. Thaks for the advice :).