80's Anime LD capture... need some help cleaning it up!

[BBQmyNUTZ]

Ok, maybe this will be fun for some of you guys who are experienced and enjoy this sort of thing.

I'm working on an 80's anime called TO-Y, captured & recorded to DVD-RW from it's incredibly rare 1986 LD pressing to a settop DVD recorder. It's only a 45min LD, so it was recorded at the highest possible quality settings. Unfortunately, I'm not able to do the capturing to lossless AVI, because the friend that owns the LD is not interested in shipping his precious relic to me, it's the best source I will be able to work with.

Now, here comes the fun. Luckily (for me at least), the source IVTC's very nicely, so at least that's one problem out of the way when I go to re-encode it. The picture is pretty sharp, but is (at least to me) VERY noisy, and it exhibits those wonderful rainbow artifacts everyone loves so much pretty heavily. I am looking to re-encode this to MPEG2 for DVD authoring.

Here be the vob fragment ^_^ :

http://home.comcast.net/~briankai/VTS_01_1.VOB (20mb) (http://home.comcast.net/~briankai/VTS_01_1.VOB)

So far, the best result I've been able to get was by serving to VDUB and hitting it with SmartSmoothIQ (aka: the really really really slow filter) to get rid of most of the rainbows. GuavaComb, which I read was highly recommended, didn't even make a dent.

For cleanup, served to Vdub again, I hit it with Vdub's deinterlacer (blend), then SmartSmoother (radius 3, strength 25), then WarpSharp (strength 40). The resulting picture is cleaner, but alot of detail was lost.

So anyways, if you wouldn't mind taking a look at it and maybe get some good results I would love to see them and learn how to apply your techniques. Thanks!

Kai

[Mug Funky]

care to post a pic, for me who has crap bandwidth :(

and excuse my terrible ignorance... but what's an LD?

[BBQmyNUTZ]

Originally posted by Mug Funky
care to post a pic, for me who has crap bandwidth :(

and excuse my terrible ignorance... but what's an LD?

LD = Laserdisc.:)

And, I guess I can post a pic or two, if it would be helpful. What kind of terrible connection do you have that a 20meg file is too much to handle? Should we start a collection up for you?(=P Just kiddin'.)

Kai

[slk001]

LD = LaserDisc

I looked at your video and I don't see any rainbows to speak of. Tell me what frame you are refering to. Maybe you have a CODEC or display issue. To me, the video looked quite clean.

[Piper]

@Mug Funky
LD = Laserdisc

I would tackle this one with just AVS filters, it's been quite a while since I've even looked at VDub filters (I should probably rethink that).

I just started downloading the file, though from what you describe, I'd tackle it with something like:

Undot()
GuavaComb(Mode="NTSC",Recall=75,MaxVariation=5,Activation=90)

# Cropping and/or letterbox here

Telecide()<--You've already determined values
Decimate()

# In order of preference. Parameters omitted
Convolution3D()
or
VagueDenoiser()
or
PeachSmoother()

FluxSmooth()

Limiter()

It's pretty vague, but this is the general formula I typically use.

One interesting thing regarding FluxSmooth. I've been working with a very noisy VHS source lately, and In order to get the effect I wanted I had to increase FluxSmooth to (35,-1), but I noticed that anything above FluxSmooth(10,-1) caused a horrible trailing/ghosting effect. On it's own FluxSmooth(10,-1) didn't provide as much filtering as I wanted. It then occurred to me to try something like the following:
FluxSmooth(10,-1)
FluxSmooth(10,-1)
FluxSmooth(10,-1)

This may be overkill for your application, but it worked wonders for my noisy VHS. Even on clean analogue captures I'm now using FluxSmooth(1 or 2,-1). It's subtle but it does make a difference.

Hope this helps.

[BBQmyNUTZ]

Originally posted by slk001
LD = LaserDisc

I looked at your video and I don't see any rainbows to speak of. Tell me what frame you are refering to. Maybe you have a CODEC or display issue. To me, the video looked quite clean.

Well, there aren't rainbow-like artifacts all over the picture, they're very noticible in the dark lines of the characters, like when the girl is spinning the cat around. I'm viewing the clip unfiltered, frameserved from avs to VDub.

And I'm suprise you didn't notice that incredible amount of grain, which is VERY noticible.

Kai

[BBQmyNUTZ]

Originally posted by Piper
@Mug Funky
LD = Laserdisc

I would tackle this one with just AVS filters, it's been quite a while since I've even looked at VDub filters (I should probably rethink that).

I just started downloading the file, though from what you describe, I'd tackle it with something like:

Undot()
GuavaComb(Mode="NTSC",Recall=75,MaxVariation=5,Activation=90)

# Cropping and/or letterbox here

Telecide()<--You've already determined values
Decimate()

# In order of preference. Parameters omitted
Convolution3D()
or
VagueDenoiser()
or
PeachSmoother()

FluxSmooth()

Limiter()

It's pretty vague, but this is the general formula I typically use.

One interesting thing regarding FluxSmooth. I've been working with a very noisy VHS source lately, and In order to get the effect I wanted I had to increase FluxSmooth to (35,-1), but I noticed that anything above FluxSmooth(10,-1) caused a horrible trailing/ghosting effect. On it's own FluxSmooth(10,-1) didn't provide as much filtering as I wanted. It then occurred to me to try something like the following:
FluxSmooth(10,-1)
FluxSmooth(10,-1)
FluxSmooth(10,-1)

This may be overkill for your application, but it worked wonders for my noisy VHS. Even on clean analogue captures I'm now using FluxSmooth(1 or 2,-1). It's subtle but it does make a difference.

Hope this helps.

Thanks for the feedback, though definitely take a look at it and let me know what kind of results you get. Like I said earlier, I tried to use Guava and it wasn't able to even touch the rainbow artifacts (perhaps because of the amount of grain/noise in the picture?)

Kai

[Piper]

btw, what parameters did you use with Telecide?

[Mug Funky]

laserdisc. heh. should have known that one. bit before my time, really. story of laserdisc was no-one could afford it, then suddenly it was obsolete :)

*slapping myself in the face*

[BBQmyNUTZ]

Originally posted by Piper
btw, what parameters did you use with Telecide?

Telecide(order=1,guide=1,post=0)
Decimate(cycle=5,quality=3)

Using Telecide's post processing has always given me nothing but grief, so I use another deinterlacer... just incase you were wondering. ^_^

BTW, for those who wanted it, here's a cropped screen capture, exhibiting the original capture vs. filtered with smartsmootherIQ.

Original:
http://home.comcast.net/~briankai/orig.bmp

Filtered:
http://home.comcast.net/~briankai/filt.bmp

The most noticible instances of the "rainbow artifacting" being near the bottom of the girl's shirt, and the line drawing the arm in front of her face. Big difference in the final output (at least to me).

Kai

[qwerpoi]

One method of removing rainbows which works well for me is Mipsmooth, although not many people seem to mention this for de-rainbowing. Check out the following script I used on your vob (using a d2v file made from DVD2AVI):

input = mpeg2source("D:\Temp\test.d2v")
output = input.MipSmooth(downsizer="lanczos",upsizer="lanczos",temporal_chroma=225,spatial=0,temporal=0,spatial_chroma=50,scenechange=1)
Subtract(input,output)


Sorry for the obnoxiously long line of code, MipSmooth has a lot of parameters which you could leave at default, but in this case I'm only trying to demonstrate removing rainbows. The subtract command is there just to see how much rainbow is removed, the final script should simply have "output" in its place. Please try it out before you pass judgement- if you look at the result, practically all rainbows are removed without any loss of detail or ghosting for this clip. If you want to remove more grain, try tweaking the spatial parameter. Also, if you want to deinterlace, you may want to consider manually deinterlacing with TMPG (http://www.doom9.org/mpg/tmpg-ivtc.htm) and tprivtc.dll (http://kurosu.inforezo.org/avs/TPRIVTC/index.html) , since the clip is only 45 minutes long. That's only if you want to be extremely picky about the quality, though.

A little off topic: I was wondering if it is unwise to use VDub to preview an avisynth script for yv12 material due to a potential colorspace conversion- does VDub convert to RGB for the preview? I know that you can set the preview to true color, but I never was able to find out anything specific about colorspaces. Does anyone know anything specific about the matter?

[BBQmyNUTZ]

[QUOTE]Originally posted by qwerpoi
[B]One method of removing rainbows which works well for me is Mipsmooth, although not many people seem to mention this for de-rainbowing. Check out the following script I used on your vob (using a d2v file made from DVD2AVI):

input = mpeg2source("D:\Temp\test.d2v")
output = input.MipSmooth(downsizer="lanczos",upsizer="lanczos",temporal_chroma=225,spatial=0,temporal=0,spatial_chroma=50,scenechange=1)
Subtract(input,output)


Sorry for the obnoxiously long line of code, MipSmooth has a lot of parameters which you could leave at default, but in this case I'm only trying to demonstrate removing rainbows. The subtract command is there just to see how much rainbow is removed, the final script should simply have "output" in its place. Please try it out before you pass judgement- if you look at the result, practically all rainbows are removed without any loss of detail or ghosting for this clip. If you want to remove more grain, try tweaking the spatial parameter. Also, if you want to deinterlace, you may want to consider manually deinterlacing with TMPG (http://www.doom9.org/mpg/tmpg-ivtc.htm) and tprivtc.dll (http://kurosu.inforezo.org/avs/TPRIVTC/index.html) , since the clip is only 45 minutes long. That's only if you want to be extremely picky about the quality, though.QUOTE]

Nice! Thanks for the suggestion! It did manage to at least get rid of some of the rainbow effects, although there was alot more left behind than with the VDub filter I was using. Should I be using this filter prior to IVTCing the clip, or does it matter? Also, which parameters need can be increased or decreased to try and catch more of the rainbow's (if any)? I'll also be sure to check out the manual deinterlacing you suggested, thanks for the tip. ^_^

Kai

[qwerpoi]

The settings for MipSmooth I suggested were specifically geared for getting rid of temporal chroma noise (ie, changes in chroma over time) without ghosting, so I'm not surprised that different filters remove more rainbows than settings I suggested- I chose a conservative value for the spatial_chroma parameter to make sure I did not adversely affect the rest of the image. Try bumping up the spatial_chroma to remove more rainbows, it can go up to 255 (I wouldn't go that high, try it out to see what it will do to your image). MipSmooth is a great filter precisely because it has so much configurability. You should note that it's pretty slow, although not the slowest filter around :) . If you want to increase it's speed, try using different downsizer and upsizer methods. Lanczos is the slowest, but I found it to be the best quality. If you're feeling particularly crazy, try adding the parameter scalefactor=1.5, this will potentially be higher quality (never tried it myself) but the time to encode will be enormous.

As far as IVTC goes, I would imagine it wouldn't really matter where the MipSmooth filter goes, but it depends on the deinterlacing method, and in general I suggest to put it after the IVTC (maybe I'm wrong). It becomes a moot point if you use manual IVTC, since you are doing the pattern detection yourself. I suggest to anyone to try manually deinterlacing at least a couple files, it teaches you a lot about the process and lets you know what can and cannot be done to a clip by an automatic filter such as neuron2's Decomb.

[BBQmyNUTZ]

Originally posted by qwerpoi
The settings for MipSmooth I suggested were specifically geared for getting rid of temporal chroma noise (ie, changes in chroma over time) without ghosting, so I'm not surprised that different filters remove more rainbows than settings I suggested- I chose a conservative value for the spatial_chroma parameter to make sure I did not adversely affect the rest of the image. Try bumping up the spatial_chroma to remove more rainbows, it can go up to 255 (I wouldn't go that high, try it out to see what it will do to your image). MipSmooth is a great filter precisely because it has so much configurability. You should note that it's pretty slow, although not the slowest filter around :) . If you want to increase it's speed, try using different downsizer and upsizer methods. Lanczos is the slowest, but I found it to be the best quality. If you're feeling particularly crazy, try adding the parameter scalefactor=1.5, this will potentially be higher quality (never tried it myself) but the time to encode will be enormous.

As far as IVTC goes, I would imagine it wouldn't really matter where the MipSmooth filter goes, but it depends on the deinterlacing method, and in general I suggest to put it after the IVTC (maybe I'm wrong). It becomes a moot point if you use manual IVTC, since you are doing the pattern detection yourself. I suggest to anyone to try manually deinterlacing at least a couple files, it teaches you a lot about the process and lets you know what can and cannot be done to a clip by an automatic filter such as neuron2's Decomb.

It seems like a really nice filter, for sure. Just to see how it went, I went ahead and used the smoothIQ filter on top of mipsmooth, and the result combining the two was better than each one individually. ^_^ Very nice result, as far as getting rid of the rainbows goes.

BTW, I sent you a PM with a couple of off-topic questions I hope you don't mind answering. You've been incredibly helpful, thanks!

Kai

[Cyberia]

These rainbowing artifacts look like chroma problems to me. Yellow and purple banding.

Try looking at it from that angle. There are chroma tools out there.

[BBQmyNUTZ]

Originally posted by Cyberia
These rainbowing artifacts look like chroma problems to me. Yellow and purple banding.

Try looking at it from that angle. There are chroma tools out there.

Any recommendations? This is a little advanced for me beyond what I'm used to doing. Searching for "chroma problems" and other derivatives tend to bring up a ton of stuff when I'm not sure exactly what I'm looking for.

Kai

[madoka]

Have you tried using MSmooth, then MSharpen? In my experience this gives me good results on noisy anime. The downside is that a little detail is lost, and that it's *very* slow, since the costly boundary calculation is done twice.

[Piper]

@qwerpoi

That's why I visit this forum, to learn about filters like that and how to use them. Thanks!

@BBQmyNUTZ
I've gone over it, and I think this is how I would approach this clip. The Telecide settings you provided don't seem to pattern match all that well, but I admit I'm having a hard time getting it to match any better.

It's personal preference, but I would also resize to Half-D1. To me it looks cleaner without sacrificing any detail. Any detail that may have been lost would've already been done at this point (compression artifacts). Had you the original Laserdisc source, then I'd perhaps I'd keep it full res.

mpeg2source("c:\test.d2v")

# Remove Rainbows
MipSmooth(downsizer="lanczos",upsizer="lanczos",temporal_chroma=225,spatial=0,temporal=0,spatial_chroma=100,scenechange=1)

Undot()
ConvertToYUY2()

GuavaComb(Mode="NTSC",Recall=75,MaxVariation=5,Activation=90)

# Cropping and/or letterbox here
Crop(8,0,-8,0)

Telecide(order=1,guide=1,post=0)
Decimate(cycle=5,quality=3)

VagueDenoiser(threshold=4,method=1,nsteps=6,chroma=true,debug=false)
FluxSmooth(10,-1)

LanczosResize(352,480)

Limiter()

[qwerpoi]

@Piper
No problem!

Actually, you bring up some interesting points in your script. Not many people seem to know that a very good way to get rid of rainbows and temporal noise is to downsize the resolution (actually, this is part of the principle behind MipSmooth). Furthermore, if you downsize to half the original height, you do not need to deinterlace (strange but true). However, in this case, I will argue that downsizing to half the original height and with are a bit too much, at least for my tastes. Check out a script like this (it was the first thing I tried after getting the vob):

input = mpeg2source("C:/test.d2v")
output = input.LanczosResize(input.width/2,input.height/2).LanczosResize(input.width,input.height)
Subtract(input,output)

You'll see that you have removed some rainbows (not all) and some temporal noise, but a lot of linework also is lost. Again, that's personal tastes, and you should take a look at the output after encoding a section before passing judgement. The point is that doing a final resize (not as drastic as above) is a good way of removing some final rainbow and temporal noise which your other previous filters left behind (as Piper points out), meaning you don't have to go overkill with the settings on the filters. One good point in using a simple script like the one above is the speed- it's way faster than using a dedicated de-rainbowing filter, it does not require deinterlacing, and for some really ugly transfers, it's just about the only way to remove awful temporal noise which sometimes shows up around red parts of a video.

[BBQmyNUTZ]

Originally posted by Piper
@qwerpoi

That's why I visit this forum, to learn about filters like that and how to use them. Thanks!

@BBQmyNUTZ
I've gone over it, and I think this is how I would approach this clip. The Telecide settings you provided don't seem to pattern match all that well, but I admit I'm having a hard time getting it to match any better.

It's personal preference, but I would also resize to Half-D1. To me it looks cleaner without sacrificing any detail. Any detail that may have been lost would've already been done at this point (compression artifacts). Had you the original Laserdisc source, then I'd perhaps I'd keep it full res.

mpeg2source("c:\test.d2v")

# Remove Rainbows
MipSmooth(downsizer="lanczos",upsizer="lanczos",temporal_chroma=225,spatial=0,temporal=0,spatial_chroma=100,scenechange=1)

Undot()
ConvertToYUY2()

GuavaComb(Mode="NTSC",Recall=75,MaxVariation=5,Activation=90)

# Cropping and/or letterbox here
Crop(8,0,-8,0)

Telecide(order=1,guide=1,post=0)
Decimate(cycle=5,quality=3)

VagueDenoiser(threshold=4,method=1,nsteps=6,chroma=true,debug=false)
FluxSmooth(10,-1)

LanczosResize(352,480)

Limiter()

Thanks for the input, I'll try it out and see how it pans out. I tend to judge a good IVTC by the smoothness of the motion and if it has gotten rid of the most obvious combing... but I'm learning that this may not be the best approach for this particular job. Thanks to qwerpoi I'm going to learn how to manually IVTC it, so that should help quite a bit.

Kai

[Boulder]

Didée once suggested something like this for removing rainbows:


xx = 576 # desired
yy = 272 # output resolution

AviSource( Whatever )

chro = BicubicResize(xx/2, yy/2, 1.0, .0).BicubicResize(xx, yy, -.6, .8)
lum = BicubicResize(xx, yy, .0, .5)
MergeChroma(lum, chro)

[SeeMoreDigital]

Sorry to go back to the beginning of the thread!

When capturing from LaserDisc 'LD' player it is possilbe to remove quite a bit of the 'rainbow' effect noise by simply killing the voltage to (or removing) the players RF tuner/modulator.

Most players had one of these fitted as standard because (LaserDisc was launched back in the days when only a few TV's had scart sockets and/or composite inputs) it was the only way to get a connection to a TV or VCR.

I know it's a bit late in your case but might help people in the future!

Cheers

[mf]

Ooo, this is a very good source! The lines are so thin and sharp, you'd almost not need sharpening! I say almost cause I couldn't resist the temptation to try mfToon, which made it look very nice except for the lines which are already so thin that warpsharp makes them even thinner! :scared: So now I'll have to release a version of mfToon where you can turn off warpsharpening (currently, you can only turn off sharpening altogether). I'll come up with my script after that :D. Btw, did you know your Telecide() settings leave interlacing? I tried to get rid of it but as interlacing is such a bitch I decided to focus myself on cleaning the other stuff.

[BBQmyNUTZ]

Originally posted by mf
Ooo, this is a very good source! The lines are so thin and sharp, you'd almost not need sharpening! I say almost cause I couldn't resist the temptation to try mfToon, which made it look very nice except for the lines which are already so thin that warpsharp makes them even thinner! :scared: So now I'll have to release a version of mfToon where you can turn off warpsharpening (currently, you can only turn off sharpening altogether). I'll come up with my script after that :D. Btw, did you know your Telecide() settings leave interlacing? I tried to get rid of it but as interlacing is such a bitch I decided to focus myself on cleaning the other stuff.

You know, that's really odd... There have been a few times I've loaded up the same script and gotten artifacts, and others when it was totally clean. Not really sure why, but that's probably as good a reason as any to do the IVTC manually as was suggested.

I tried mfToon as well, and liked the result except for the oversharpening. I had no problems with the sharpness of the original source at all... as a matter of fact I was very impressed with it upon first viewing. It was just the noise and chroma issues that really bug me, but thankfully I'm getting some great tips on how to get around those from you guys. ^_^

Kai

[Piper]

I tried mfToon as well, and liked the result except for the oversharpening. I had no problems with the sharpness of the original source at all... as a matter of fact I was very impressed with it upon first viewing. It was just the noise and chroma issues that really bug me, but thankfully I'm getting some great tips on how to get around those from you guys. ^_^

This is why I suggest cropping to 704x480 and resizing to 352x480. If you filter then resize using BiCubic or Lanczos, your result will be nice and sharp without relying on any additional sharpening filters. I also find that unless I have an exceptional source, resizing to Half D1 doesn't sacrifice any detail, it may even help to reduce noise just a touch more as well (IMHO).

[mf]

Originally posted by BBQmyNUTZ
I tried mfToon as well, and liked the result except for the oversharpening. I had no problems with the sharpness of the original source at all... as a matter of fact I was very impressed with it upon first viewing.
That's what I said as well. This source is really good. Anyway, if you want you can always try mfToon(sharpen=false), which just bypasses the whole sharpening step, and as a side-effect is also a lot faster.

[BBQmyNUTZ]

Originally posted by Piper
This is why I suggest cropping to 704x480 and resizing to 352x480. If you filter then resize using BiCubic or Lanczos, your result will be nice and sharp without relying on any additional sharpening filters. I also find that unless I have an exceptional source, resizing to Half D1 doesn't sacrifice any detail, it may even help to reduce noise just a touch more as well (IMHO).

I did try this out last night before I went to bed, and I really wasn't happy with the result. It just looked really blurry to me, and I can't really understand how halving the resolution of a video could be a good thing? Couldn't a similar result be acheived by either doing a full-frame deinterlace or blur, then running a sharpen filter to get some of the detail back?

Kai

[BBQmyNUTZ]

That's what I said as well. This source is really good. Anyway, if you want you can always try mfToon(sharpen=false), which just bypasses the whole sharpening step, and as a side-effect is also a lot faster.

That was a question I had about your filter I was going to ask later, but I may as well do it now... Wasn't your script at one point or another able to darken or thicken black lines, at the cost of slightly darkening the entire image? I've tried messing around with it a bit, but I wasn't able to do anything like that with it. Maybe I'm thinking of another filter?

Kai

[sh0dan]

Ok - after an idea I had, I threw together this "Rainbow Killer" (tm). It completely removes rainbows on the test clip, without any visible quality loss.

I requires msharpen by Donald Graft and MaskTools by Kuruso and YV12 material.


org = last
msharpen(threshold = 10, mask=true).greyscale()
reduceby2()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)
b = blankclip(org)
m_mask = ytouv(uv,uv,b)

filtered = org.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,0,255,3,2)
MaskedMerge(org, filtered, m_mask)


It can be made faster by being a filter, and a few tricks, but it's already quite acceptable.

[Kurosu]

Ok, I guess I need to change nicks and use now "Cross/ClothWithJapanesePronounciation". :D

Edit: FitYtoUV() (I should modify MaskTools 1.x to list that possibility) does what you achieve by ReduceBy2. And I'm sure the former uses MMX code ;)

[BBQmyNUTZ]

Originally posted by sh0dan
Ok - after an idea I had, I threw together this "Rainbow Killer" (tm). It completely removes rainbows on the test clip, without any visible quality loss.

I requires msharpen by Donald Graft and MaskTools by Kuruso and YV12 material.


org = last
msharpen(threshold = 10, mask=true).greyscale()
reduceby2()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)
b = blankclip(org)
m_mask = ytouv(uv,uv,b)

filtered = org.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,0,255,3,2)
MaskedMerge(org, filtered, m_mask)


It can be made faster by being a filter, and a few tricks, but it's already quite acceptable.

I just tested out this script as you posted it, and this is what I got:

http://home.comcast.net/~briankai/00.bmp

I'm doing something wrong, aren't I? ^_^

[Edit: Haw... I was using the wrong version of MaskTools. The newest version of masktools that your script requires doesn't seem to work with MFToon, though. It works VERY well, nearly equal to SmoothIQ, and it's much faster! Thanks a ton!]

Kai

[Piper]

It looks effective, though perhaps sh0dan would be kind enough to provide us simple users with the means to invert "Rainbow Killer"s (tm) results.

I don't know what it is about this clip, it's just so "happy". I'm starting to have dreams of that spinning cat with its tounge hanging out. I thought of doing that with my own over-weight Calico, but I don't think she'd enjoy it. :D

[sh0dan]

@Kurosu: Heh - sorry - I was probably a bit fast on the keyboard there.

@Piper:
You should probably update MaskTools - I have changed the script slightly - I don't know MaskTools that well, and there was some notes about the 'Y/U/V' parameters. And since they weren't needed they can be removed.

The theory: I noticed that rainbow artifacts are always present where there is high luma frequencies. So I do an edgedetect, and apply heavy smoothing on chroma, where luma is very detailed.


#Create an edgemap. Remove chroma information.
msharpen(threshold = 10, mask=true).greyscale()

#reduce it to half size. Now luma is the same size as chroma.
reduceby2()

#Make edges thicker, so it includes both sides of lines.
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

#Luma must not be filteres.
b = blankclip(org)

#Place the UV mask in U and V, and make luma black.
m_mask = ytouv(uv,uv,b)

#Create a heavily blurred chroma clip.
filtered = org.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,0,255,3,2)

#Replace chroma in the original image with the blurred chroma, where luma is detailed.
MaskedMerge(org, filtered, m_mask)

Hope that clarifies how it works.

Edit: Whoops - I accidently edited BBQMyNutz' post - I only changed the script - sorry!

[Piper]

@sh0dan

Thanks for taking the time to explain how your script works. I've updated MaskTools and tried your modified script but I'm a little puzzled as I still see a great number of rainbows. Perhaps I'm somehow implimenting it wrong.

Can you also describe why you came up with this method as opposed to using

MipSmooth(downsizer="lanczos",upsizer="lanczos",temporal_chroma=225,spatial=0,temporal=0,spatial_chroma=100,scenechange=1)

as qwerpoi initially suggested? By increasing spatial_chroma to 100 from 50, MipSmooth appears to be very effective, at least to my eyes.

[sh0dan]

@Piper: Try reducing threshold. You could add ".coloryuv(off_y=-256)" after both temporalsoften to visualize the smoothed areas in the script below:

30% faster version:


org = last
org_u = utoy(org)
org_v = vtoy(org)

msharpen(threshold = 10, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

ytouv(u_final, v_final, org)

[qwerpoi]

@Piper
When I compared them side by side, I saw very little visual difference between Shodan's latest script and my MipSmooth settings (I was comparing Subtract() clips, not the final clips). In terms of encoding time, Shodan's script is a touch faster.

However, I would probably recommend using Shodan's script, since it is specifically geared towards attacking rainbows on edges, using an edge mask. The problem with my MipSmooth settings is that it attacks temporal and spatial chroma in general, without any concept of edges. In order to very effectively eliminate rainbows, I shot the temporal_chroma parameter to its max value, which would normally destroy your video (ghosting), but at the same time put the scenechange parameter at its lowest setting, meaning the temporal cleaning will only be active when there is little overall motion in the clip. The spatial_chroma parameter is set to get rid of any remaining non-temporal rainbows, but again, it has no knowledge of edges, so it will start adversely affecting the image if set too high. Despite all these downsides, I found that my MipSmooth settings are very effective for getting rid of rainbows without badly damaging the original image. At the time I came up with it, I preferred my method over other de-rainbowing methods, but Shodan's new script seems more appropriate now for the job, since it will be easier to tweak, and has similar properties in terms of image and encoding time (and since Shodan also wrote MipSmooth, I'll trust what he has to say about removing rainbows :) ).

[Piper]

@qwerpoi

Thanks for the detailed explanation. I now understand where this is all coming from. It seems this discussion has proved somewhat useful as we now have a new "Rainbow Killer" (tm) to play with <-- now 30% faster thanks to sh0dan.

[BBQmyNUTZ]

Well, I'm glad this has turned out to be a useful thread. ^_^ I know I learned quite a bit so far.

After several tests, I've concluded that for speed/quality, Sh0dan's Rainbow Killer(tm) script handles the source the best. For best quality, Rainbow Killer(tm) & SmartSmootherIQ just comes out awesome.

Now that the source has been "de-rainbowed", so to speak, I'll be working on which filtering methods give out the best image based off all the great info I've gotten here. So far, the results have been VERY good.

On a side note, turns out we will be attempting to recapture the show with the use of the ADVC-100 I'm about to buy. Turns out, his LD player (a Pioneer elite model) wasn't used to do the original transfer... and apparently the cables used **** **** too. Hopefully, should things go well, the resulting capture will be a bit cleaner chroma wise.

If any of you are interested in getting a copy of the finished result let me know, I'll be more than happy to share. The show is about an underground punk band trying to get their first record deal, despite the recording industries pressure for them to do more "mainstream" music. Great soundtrack, and a good story all around. Kind of an anime cult classic, if you will. ^_^

Kai

[Chainmax]

I'd be very interested in having it hosted on an FTP. Mail me at gsztajn@hotmail.com in order to discuss details :).

[Piper]

I'm also very interested. Please PM me with details.

[Piper]

@sh0dan

Is there a possibility "Rainbow Killer" could be tweaked for a YUY2 source? For my analogue sources (YUY2), being able to reduce or eliminate rainbows using "Rainbow Killer" without having to do any colour conversions would be ideal.

My understanding is SwapUV/UToY/VToY/YToUV requires YV12. Is there a YUY2 equivalent?

[Wilbert]

Get the latest binaries and read:

http://www.avisynth.org/index.php?page=Swap

[sh0dan]

The functions you mention have recently been implemented as YUY2, as Wilbert states - I do however not believe that MaskTools does!??!

[Piper]

All research on Masktools deadends at http://kurosu.inforezo.org/ as it appears Kuroso has shutdown his Avisynth plugin DLL area. He must have closed it only recently as I downloaded Masktools from his site only a few days ago. I did find mention of it only supporting YV12 at http://www.avisynth.org/index.php?page=mfRainbow. Shame.

[LB]

Hey ya'll,

I've been working with some really dirty DVDs and have found TWO really good lines of code:

First, the ultimate rainbow killer imho. I've tried EVERYTHING else, even tim's IQ, but this hands down beats them all and it's faster. ^.^
YUY2 only and you need a copy of AntiBlink2.dll (yes that's right, I can't get the newer antiblink to work, so you gotta scrounge up a copy of the older one). Use this EXACT line and your rainbows will vanish:

MergeChroma(Reduceby2().AntiBlink(15,6,false,2).SpatialSoften(7,0,50).blur(1.5).BilinearResize(720,480))

Second, I've really found guavacomb handy, but it's weird... I'm using the numbers opposite how the guide says to use them, and am getting MUCH better results. Here is my A+ dot-crawl remover. And remember, ALWAYS put this BEFORE any IVTC/RESIZE:

GuavaComb(Mode="NTSC", Recall=21, MaxVariation=30, Activation=30)

Welp, tell me what you think. ;)

LB

[MasterYoshidino]

main problem with low recall is that you are far more prone to temporal artifacts from guavacomb, higher thresh = encode and forget
(too easy to overlook those fine miscolor bands from guava messing up and trying to correct what is not composite noise ...)

[High Speed Dubb]

Yep, using a low Recall value is asking for trouble. A low activation value can work just fine, though — It’ll mess up with some specific kinds of video (like stock tickers or MTV-style fast cuts), but otherwise it tends to work quite well. My standard Activation is about 40.

Trivia fact — Activation is automatically scaled such that any value will give a decent picture. “100” means that the filter triggers only if the area of the screen has been shimmering continuously; “0” means that the shimmering has been occuring just often enough to distinguish it from a) Lots of shimmering followed by one field without shimmering evidence, and b) One field of motion interrupting a static background.

[LB]

Let there be life! :p

Just wanted to post an update on a few things.

Rainbows: Been using SmoothIQ. Slow as hell, but it removes 99% of all the rainbows I throw at it. Give it a shot if you haven't.

Dotcrawl: Yeah, the settings I was using produced a lot of crap on other thows, but the show I was working on it worked decent. But after a LOT of searching I located deDot and holy hell... nothing beats that in the area of dotcrawl removal. Best darn dot cral remover ever. Pick it up if you get time.

[Chainmax]

BBQ, has this gone anywhere? I haven't received mail from you in ages...

[Valky]

Originally posted by sh0dan
@Piper: Try reducing threshold. You could add ".coloryuv(off_y=-256)" after both temporalsoften to visualize the smoothed areas in the script below:

30% faster version:


org = last
org_u = utoy(org)
org_v = vtoy(org)

msharpen(threshold = 10, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

ytouv(u_final, v_final, org)



Usually screenshots can tell you more than just words and I am trying to somehow restore these horrible Fawlty Tower episodes on PAL dvd. There are many problems in these episodes, but rainbow effect is one of them. So I tried your scripts and it really helps a lot. Not perfect, but I am still trying other methods too.

My script is simple:

TomsMoComp(-1,5,0)
AutoCrop(0,wMultOf=4, hMultOf=4)



org = last
org_u = utoy(org)
org_v = vtoy(org)

msharpen(threshold = 10, mask=true)
reduceby2()
greyscale()
uv = blur(1.5).levels(0,2.0,255,0,255).blur(1.5)

filtered_u = org_u.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)
filtered_v = org_v.blur(1.5).blur(1.5).blur(1.5).temporalsoften(2,255,0,3,2)

u_final = MaskedMerge(org_u, filtered_u, uv)
v_final = MaskedMerge(org_v, filtered_v, uv)

ytouv(u_final, v_final, org)

bilinearresize(512,384)





Screenshots can be seen here:

http://students.turkuai.fi/users/toviital/temp/rainbows.png


The other scripts is without your script.

Next problem with these episodes is to somehow reduce the oversharpness or whatever causes those linebreaks in picture edges. I was earlier talkin about these episodes in here:

http://forum.doom9.org/showthread.php?s=&threadid=75652&highlight=fawlty