Removing mosquito noise and halos in YUY2

[springbox]

I'm having some consistent problems with two specific types of noise in most of my captures. Currently I am capturing an already compressed video stream over a composite connection (don't ask.)

The source is mostly good except there is some pretty noticable mosquito noise in areas with a lot of motion. The second problem (halos) is probably the result of whatever filtering the analog capture card is applying to the composite signal.

I've already tried searching for a way to remove such types of noise, but I've mostly found filters that only work in YV12! I would really like to know an approach for removing both types of artifacts natively in YUY2 (either that or RGB is OK I guess.)

For mosquito noise, I've already tried MSmooth, but it has a problem of smearing fine details at higher strengths. I'm going to play around with TBilateral later and see if that does anything.

Any ideas?

Thanks

[foxyshadis]

fft3dfilter works in yuy2, as does removegrain (by placing it in planar yuy2) and degrainmedian, PixieDust, PeachSmoother, GrapeSmoother, TTempSmooth, VagueDenoiser, undot, NoMoSmooth, and FluxSmooth. (Some of those are YUY2 only.) So you have a lot of choice still, even if some are a bit dated and you can't use masks.

Oh right, for haloing, if unfilter can't help, you'll probably have to convert to yv12 anyway and try dehalo_alpha.

[Serbianboss]

you forget Convolution3D

[Didée]

Most of these artefacts are predominant on the luma channel. So you've always the possibility do do

- convert clip to YV12
- process with what is needed
- convert back to YUY2
- merge back chroma of original YUY2 clip

Note: pretty often there's too much "I want/must stay in YUY2" hype, just because someone has heard of YUY2 having better chroma resolution. It may be of concern for certain, top-exact intermediate filtering in long processing chains. (Well, it certainly IS in certain contexts.) But in the majority of cases, it is of no concern in fact, because the intended end format is YV12 anyways (true for probably 99.99% of all encoding that gets done in all homes all around the world).

If the very slight gain of ultimately staying in YUY2 comes at the cost of not being able to use dedicated filtering, but only broad general filters instead, then this strategy has something of "fighting for peace" or "fu**ing for virginity" ...

[springbox]

Note: pretty often there's too much "I want/must stay in YUY2" hype, just because someone has heard of YUY2 having better chroma resolution. It may be of concern for certain, top-exact intermediate filtering in long processing chains. (Well, it certainly IS in certain contexts.) But in the majority of cases, it is of no concern in fact, because the intended end format is YV12 anyways (true for probably 99.99% of all encoding that gets done in all homes all around the world).

Well the reason is that I only wanted to downsample to YV12 when I was ready to encode with XviD to get better quality from the picture. (I'm still hoping that some codecs will start to support 4:2:2 profiles sometime soon though.)

I was thinking about it a bit anyway and it might not be a huge deal to downsample to YV12 and then run a single dehaloing filter. I've tried DeHalo_alpha already but it seems to have the unfourtnate side effect of dimming certain areas. (I'm using LimitedSharpenFaster as the only YV12 filter at the moment though.)

[springbox]

Here are some results from the tests of various filters. The "source" image is the image processed up to the point where nothing has been done to remove the mosquito noise or halos.

These filters are applied to the "source" image:
TComb, TFM (IVTC), and
VagueDenoiser(threshold = 1.5, method = 3, nsteps = 6, chromaT = 2.0, percent = 80, interlaced = false)

The TBilateral image is the source image after filtering with TBilateral:
TBilateral(iDevL = 5.0, iDevC = 5.0)

The DeHalo_alpha image is the source image after filtering with DeHalo_alpha:
DeHalo_alpha(rx = 2.0, ry = 2.0, darkstr = 0.5)

Although, I couldn't find any settings for DeHalo_alpha that effectively removed the halos.

The halos in this image were not very strong
http://i57.photobucket.com/albums/g226/springbox/th_halos0_default.jpg (http://i57.photobucket.com/albums/g226/springbox/halos0_default.jpg)http://i57.photobucket.com/albums/g226/springbox/th_halos0_tbilateral.jpg (http://i57.photobucket.com/albums/g226/springbox/halos0_tbilateral.jpg)http://i57.photobucket.com/albums/g226/springbox/th_halos0_dehalo_alpha.jpg (http://i57.photobucket.com/albums/g226/springbox/halos0_dehalo_alpha.jpg)
Source -> TBilateral -> DeHalo_Alpha

The Halos in this image were particularly bad in some places
http://i57.photobucket.com/albums/g226/springbox/th_halos1_source.jpg (http://i57.photobucket.com/albums/g226/springbox/halos1_source.jpg)http://i57.photobucket.com/albums/g226/springbox/th_halos1_tbilateral.jpg (http://i57.photobucket.com/albums/g226/springbox/halos1_tbilateral.jpg)http://i57.photobucket.com/albums/g226/springbox/th_halos1_dehalo_alpha.jpg (http://i57.photobucket.com/albums/g226/springbox/halos1_dehalo_alpha.jpg)
Source -> TBilateral -> DeHalo_Alpha

I also didn't want to smudge small details too much. I could have increased the effectivness of TBilateral, but it would have smeared the cloth texture in this image. DeHalo_alpha also makes the white/black area to the left a bit fuzzy.
http://i57.photobucket.com/albums/g226/springbox/th_detail0_source.jpg (http://i57.photobucket.com/albums/g226/springbox/detail0_source.jpg)http://i57.photobucket.com/albums/g226/springbox/th_detail0_tbilateral.jpg (http://i57.photobucket.com/albums/g226/springbox/detail0_tbilateral.jpg)http://i57.photobucket.com/albums/g226/springbox/th_detail0_dehalo_alpha.jpg (http://i57.photobucket.com/albums/g226/springbox/detail0_dehalo_alpha.jpg)
Source -> TBilateral -> DeHalo_Alpha

Here's an image where mosquito noise is particularly bad. VagueDenoiser takes care of some of it but increasing its strength usually kills the image. TBilateral is usually pretty good at cleaning up this type of noise when it's not as strong. I wasn't expecting DeHalo_alpha to do much with this, but it cleans it up a bit.
http://i57.photobucket.com/albums/g226/springbox/th_noise0_source.jpg (http://i57.photobucket.com/albums/g226/springbox/noise0_source.jpg)http://i57.photobucket.com/albums/g226/springbox/th_noise0_tbilateral.jpg (http://i57.photobucket.com/albums/g226/springbox/noise0_tbilateral.jpg)http://i57.photobucket.com/albums/g226/springbox/th_noise0_dehalo_alpha.jpg (http://i57.photobucket.com/albums/g226/springbox/noise0_dehalo_alpha.jpg)

TBilateral seemed to work the best. DeHalo_alpha removes some halos but you can still see some faint halos and diming in certain areas.

[springbox]

Putting FFT3DFilter before TBilateral seemed to help a bit with the noise without blurring too much:
FFT3DFilter(bt = 2, sigma = 2, plane = 0, bw = 128, bh = 128, interlaced = false)
FFT3DFilter(bt = 2, sigma = 2, plane = 1, bw = 128, bh = 128, interlaced = false)

http://i57.photobucket.com/albums/g226/springbox/th_noise_with_fft3dfilter.jpg (http://i57.photobucket.com/albums/g226/springbox/noise_with_fft3dfilter.jpg)

One of these days I'm actually going to encode this video

[springbox]

Turns out DeHalo_alpha is doing what it's supposed to. Those dark areas were halos being wiped out correctly. I guess that's what happens when you're always zoomed in to the images. The final script that I'm going to use for now looks something like this:


Import("/scripts/DeHalo_alpha.avs")
Import("/scripts/LimitedSharpenFaster.avs")

# TComb, TFM, etc
VagueDenoiser(threshold = 1.5, method = 3, nsteps = 6, chromaT = 2.0, percent = 80, interlaced = false)

FFT3DFilter(bt = 2, sigma = 2, plane = 0, bw = 128, bh = 128, interlaced = false)
FFT3DFilter(bt = 2, sigma = 2, plane = 1, bw = 128, bh = 128, interlaced = false)
TBilateral(iDevL = 5.0, iDevC = 5.0)

ConvertToYV12()
DeHalo_alpha(rx = 1.25, ry = 1.25, darkstr = 0.4)
LimitedSharpenFaster()


Everything looks pretty nice. I might post some more pictures later showing the results.

[Mug Funky]

you might want to avoid strong temporal filters, as inuyasha has already had one applied (that aside, it's a very good source. whatever method they're using to reduce grain is actually really good - almost perfect for filmed anime).

hehe... is this show any good? i've seen bits of it but was never the least bit interested in watching it. just tooo many episodes.

btw, the US DVDs are very good quality. of course you wouldn't learn anything if you gave up on your video and grabbed the DVDs...