Light snowing noise in source

[Seraphic-]

Did some searching on the forums to try and find some answers/suggestions. Came across a few relevant postsm along this one here HERE (http://forum.doom9.org/showthread.php?t=73865&highlight=denoise+snow), but it's from 2004. Wanted to know if anyone can maybe suggest a few more filters that I could try that would work well to remove light snowing noise in video. Sample image is below and any help would be appreciated. Thanks

http://www.seraphicgate.com/1.png

[Guest]

A edge-masked smoother like my MSmooth works great on clips like that.

http://neuron2.net/misc/girl.png

[Seraphic-]

Wow, that looks pretty good. But I guess I should ask, did I diagnose that correctly? Would that be considered "snow" or something else? Also, what quality downsides come when using filters that work in this area? Not trying to put your script down or anything (I plan to try it), but the image looks to have have some color bleeding and blurring (or maybe that is just my eyes and/or monitor).

Thanks again for your help. I'll see if I can find a link to your "MSmooth" somewhere on the forums along with some sample code.

[Didée]

Well, tastes are different ... for me, msmooth looks pretty poor. Overly smoothing on big areas, loss of shadings on smaller areas, not-filtered edges, and hard transitions between all of these.

Puzzled together during lunch:

o = last
vinverse(2.6)
minblur(1,1)
removegrain(2,0)
minblur(1,1)
gradfun2db(1.51)
soft = last
softD = mt_makediff(soft,o,U=1,V=1)
sharp0 = mt_lutxy(soft,soft.removegrain(11,-1).removegrain(20,-1),
\ "x x y - abs 20 / 1 2 / ^ 20 * x y - 2 ^ x y - 2 ^ 9 + / * x y - x y - abs 0.0001 + / * +",U=1,V=1)
sharp1 = sharp0.repair(o,1,0)
sharp0D = mt_makediff(soft,sharp0,U=1,V=1)
sharp0DD= mt_lutxy(sharp0D,sharp0D.removegrain(11,-1),
\ "x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=1,V=1)
sharpDD = mt_lutxy(sharp0DD,softD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=1,V=1)

soft.mt_makediff(sharpDD,U=2,V=2)
http://img257.imageshack.us/img257/9111/snownoiseupdowniz5.png (http://imageshack.us)


Not that it necessarily had to be done this way ... but it's always fun to get a competitive result by doing the major work with nothing but kernel/median filters. :D

There's a bunch of easy-to-use filters that might deal well with this ... try e.g. FFT3DFilter or FrFun7.

[Guest]

Didée, you know perfectly well that MSmooth has configurable parameters. I emphasized the smoothing to show its effect. Your result is more subtle and I could easily have achieved a result close to that. And yours is probably one of those 1 fps scripts, eh? :)

Seraphic-, of course there's blurring, that's what smoothers do! But you are imagining the color bleeding.

[check]

ImageReader("1.png").crop(0,1,0,0).converttoyv12().crop(28,74,-28,-74)
fft3dfilter(sigma=3,plane=4,bt=1,beta=1.2)

This works pretty well :) Compared to didee's, it's far smoother, but some of the subtle gradients are gone. Mostly it's a good thing, I think they were actually noise. The luma is also a little oversmoothed, but if you want to fix that, just filter the luma & chroma seperately.

http://img444.imageshack.us/img444/4805/checkersfft3dlc3.png

[Didée]

Of course I know about the parameters, neuron2. But I still doubt the general approach of msmooth and msharpen (do you remember that many years ago, a newbie called Didée spoke his doubts about that?;)).
The problem is the basic "do-or-dont" method. Whatever one sets 'strength' and 'treshold', the result will always be that blurred pixels are standing right beneath not-blurred pixels, because of the binary masking. There'll always be hard boundaries somewhere, and tweaking the parameters will only change their location and severeness.

BTW, regarding "1fps script": I modded the script alittle (don't process chroma where it's not needed), and got the following results on my agy 2.6GHz Celeron:

msmooth(strength=2,threshold=10) : ~31 fps
my modified script : ~33.5 fps

So, speed is not all that bad, I'd say. :)

[Terranigma]

Nice Didée, I find your result most pleasing, but I have a question about your custom script. What is "o"? It looks as if you missed out on specifying an argument for it. =P

[Didée]

oops, there should be an "o = last" at the very top. Just managed to miss it during copy & paste.

The fft3dfilter result looks pretty good to me. It's more smooth, yes, but detail presentation is much better. On fine detail like e.g. the hair, there's even an (apparent) improvement over the original shot.

Could be worth trying to fiddle in some constrained sharpening after fft3d ... only thing is the slight ringing of fft3d, which would respond again, too.

Fiddling ...

[check]

hmm, you're one step ahead of me in thinking didee, but I'm one step ahead in posting results!
ImageReader("1.png").crop(0,1,0,0).converttoyv12().crop(28,74,-28,-74)
fft3dfilter(sigma=3,plane=4,bt=1,beta=1.2,dehalo=0.5)
soothe(limitedsharpenfaster(ss_x=1,ss_y=1,strength=40),last,70)
http://img164.imageshack.us/img164/3473/checkersfft3dlsf01ku2.png

I'm not sure if the dehalo parameter is really doing anything, but I am using the 2d filter (bt=1) which is a lot stronger than the 3d one at the same sigma, so I put it in as a Just In Case

[Didée]

Soothe might be good in practice, but as long as we're dealing with a single screenshot, it has zero effect. ;)

In the following, I took your fft3d line and added some "source restricted" sharpening (i.e. no pixel might be sharpened more than what previously was removed in the ultimate neighborhood of the pixel) :
o = last
fft = o.fft3dfilter(sigma=3,plane=4,bt=1,beta=1.2)
fftX = fft.minblur(1)#.mt_lutxy(fft.minblur(2),"x y > x 1 - x y < x 1 + x ? ?")
fftshrp = mt_lutxy(fftX,fftX.removegrain(11,-1),"x x y - abs 16 / 1 2 / ^ 32 * x y - 2 ^ x y - 2 ^ 6 + / * x y - x y - abs 0.0001 + / * +",U=1,V=1)
fftshrpD = mt_makediff(fftX,fftshrp)
fftD = mt_makediff(fft,o)
fftshrpDD = fftshrpD.repair(fftD,12)

fft.mt_makediff(fftshrpDD,U=2,V=2)

# for comparison:
interleave(o,fft,last)

The difference is this: using limitedsharpen, you're limiting the sharpening result by the denoised source. This means, consequently, that you cannot restore signal amplitude in spots where same was reduced by the noise filter.
However when limiting by the original source, you can. :)

Everyone compare the results for him/herself - I'm getting tired of linking the (almost) same shot over & over again. :p

[Seraphic-]

Whoa, didn't think I would get so many replies... but everyones input and suggestions are really appreciated. Unfortunately, I've gotta head out to work now, but I look forward to experimenting with these ideas later when I get home.

If anyone is interested, I put together five sample clips that I think best reflect the different view-points of the game and I'll be using them to be run these denoise tests. You're welcome to download them and run some additional tests if want to try some more ideas with video.

Sample 1 - Full Motion Video (scene from original post) (http://www.seraphicgate.com/TestVideo1.rar)
Sample 2 - Menu Interface (http://www.seraphicgate.com/TestVideo2.rar)
Sample 3 - Dungeon Demonstration (http://www.seraphicgate.com/TestVideo3.rar)
Sample 4 - Game-play Event Scene (http://www.seraphicgate.com/TestVideo4.rar)
Sample 5 - Battle Demonstration (http://www.seraphicgate.com/TestVideo5.rar)

[check]

for the love of..! What's with you and uncompressed samples? If you HuffYUV compress them they'll drop in size by around 30% compared to the inefficient zip routine.

Anyway, sample 1 shows that the denoising currently being used is all shimmery, I had a quick play and can't really reduce this. I think doing things related to block size and windowtype should help, ditto with using the inbuilt sharpener rather than an external one.

[Guest]

There'll always be hard boundaries somewhere, and tweaking the parameters will only change their location and severeness. The original poster did not notice or object to any threshold effects and his reaction was "wow". One could even argue that the effect provides a kind of desirable edge enhancement for anime-like video. (MSmooth could be enhanced to roll-off the effect, I suppose, for purists like you.)

I don't remember you ever being a newbie, even on your first post. :)

[Seraphic-]

for the love of..! What's with you and uncompressed samples? If you HuffYUV compress them they'll drop in size by around 30% compared to the inefficient zip routine.

Apologies if that was an inconvenience to you. I guess the best way to explain it is that it's just my preferred way and method to work (everything uncompressed).

Also, the files actually come out smaller when WinRAR'ing a raw YUY2 video then when WinRAR'ing a HuffYUV.

Anyway, sample 1 shows that the denoising currently being used is all shimmery, I had a quick play and can't really reduce this. I think doing things related to block size and windowtype should help, ditto with using the inbuilt sharpener rather than an external one.

As I'm sure you know, Sample 1 (well, all of them) are just raw videos, so they haven't been tampered with. Are you suggesting that the noise is more or a blocking issue then a snowing effect? Also, if you don't mind me asking, what inbuilt sharpener are you referring too?

The original poster did not notice or object to any threshold effects and his reaction was "wow". One could even argue that the effect provides a kind of desirable edge enhancement for anime-like video. (MSmooth could be enhanced to roll-off the effect, I suppose, for purists like you.)

Again, I'm in no way trying to put down your work here... the result you posted did look good in terms of removing the "snow" noise. But I would have to agree with a the posters, that it did (at least to me) look somewhat "over-smoothed". But like you said, that can be improved with different configurations.

I'm going to do some searching to try and find some different configurations or a guide for "MSmooth" ( as I'm not sure how to write a valid config script for it or what settings can be used with it).

# Import File
avisource("C:\Documents and Settings\Desktop\TestVideo1.avi")
# ----------------------------------------------------

# Color Depth
converttoyv12()
# ----------------------------------------------------

# Import Scripts
import("E:\Other\Encoding Backup\Filter\MCBob_v03c.avs")
# ----------------------------------------------------

# Import Dynamic-link Library Files
LoadPlugin("E:\Other\Encoding Backup\Filter\RemoveGrainS.dll")
LoadPlugin("E:\Other\Encoding Backup\Filter\vinverse.dll")
LoadPlugin("E:\Other\Encoding Backup\Filter\mt_masktools.dll")
LoadPlugin("E:\Other\Encoding Backup\Filter\gradfun2db.dll")
LoadPlugin("E:\Other\Encoding Backup\Filter\Repair.dll")
# ----------------------------------------------------

# Avisynth Code
o = last
vinverse(2.6)
minblur(1,1)
removegrain(2,0)
minblur(1,1)
gradfun2db(1.51)
soft = last
softD = mt_makediff(soft,o,U=1,V=1)
sharp0 = mt_lutxy(soft,soft.removegrain(11,-1).removegrain(20,-1),
\ "x x y - abs 20 / 1 2 / ^ 20 * x y - 2 ^ x y - 2 ^ 9 + / * x y - x y - abs 0.0001 + / * +",U=1,V=1)
sharp1 = sharp0.repair(o,1,0)
sharp0D = mt_makediff(soft,sharp0,U=1,V=1)
sharp0DD= mt_lutxy(sharp0D,sharp0D.removegrain(11,-1),
\ "x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=1,V=1)
sharpDD = mt_lutxy(sharp0DD,softD,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=1,V=1)

soft.mt_makediff(sharpDD,U=2,V=2)
# ----------------------------------------------------

Decided to give Didée's suggestion a go first, but after several different videos, I'm finding that light "snow" noise still somewhat remains... making me wonder if I left something out or called the script wrong? Also Didée's, when you said "I modded the script alittle (don't process chroma where it's not needed)", could you perhaps post that change if you get a chance? I'm curious to see how that works. Anyway, I'll try some different things to see what I can come up with. Thanks again for everyones help.

http://www.seraphicgate.com/2.png

[check]

Well, smaller blocksize and larger overlap helps significantly.
The smaller blocksize means all the mosquito noise around the mouth and eyes from before is more or less completely gone. I also split the denoising into luma & chroma.
My eyes are a little bored of looking at this same image for so long, but I think this is close to as good as it gets.
I think it now needs a smidge more sharpening, but I don't know masktools, and the only thing from poland I like is sausage. Help please didee :)

Also note the ingenious use of the ncpu parameter, which I only remembered after the thing stopped running in realtime. If you have two CPUs, this script is probably 25% slower or so.
avisource("TestVideo1").converttoyv12().crop(24,74,-20,-74)
o = last
fft = o.fft3dfilter(sigma=2,plane=0,bw=24,bh=24,ow=12,oh=12,ncpu=2).mergechroma(o.fft3dfilter(bt=5,sigma=3,plane=3,bw=24,bh=24,ow=12,oh=12,ncpu=2))
fftX = fft.minblur(1)#.mt_lutxy(fft.minblur(2),"x y > x 1 - x y < x 1 + x ? ?")
fftshrp = mt_lutxy(fftX,fftX.removegrain(11,-1),"x x y - abs 16 / 1 2 / ^ 32 * x y - 2 ^ x y - 2 ^ 6 + / * x y - x y - abs 0.0001 + / * +",U=1,V=1)#.limitedsharpenfaster(60)
fftshrpD = mt_makediff(fftX,fftshrp)
fftD = mt_makediff(fft,o)
fftshrpDD = fftshrpD.repair(fftD,12)

fft.mt_makediff(fftshrpDD,U=2,V=2)


Also, the files actually come out smaller when WinRAR'ing a raw YUY2 video then when WinRAR'ing a HuffYUV.
Since you'll be compressing to YV12 anyway I did this test in YV12:
o huffyuv in latest ffdshow-tryouts. median prediction w/ adaptive tables: 56 910kbyte
o Raw YV12, then RAR'ed with winrar 3.51 'best' compression: 58 961kbyte
'nuff said :)

[*.mp4 guy]

The source image has aliasing in it (likely a bad scaling routine in the game console) this oversharpened image (http://img507.imageshack.us/img507/6544/52325709te5.png) shows the problem well. Because of the aliasing, you will probably want to stick to mild sharpening to avoid enhancing it.

[Terranigma]

Hi guys, I have a question. If I were to choose to resize an image, would it be wise to use gradfun2db before or after I resize an image, or does it really matter?

[Seraphic-]

The source image has aliasing in it (likely a bad scaling routine in the game console) this oversharpened image (http://img507.imageshack.us/img507/6544/52325709te5.png) shows the problem well. Because of the aliasing, you will probably want to stick to mild sharpening to avoid enhancing it.

*.mp4 guy, could you run that aliasing test and over sharpen this image as well (I would but I'm not sure what you used)? I tried something and will go into more detail if it worked.

http://www.seraphicgate.com/3.png

Well, smaller blocksize and larger overlap helps significantly.
The smaller blocksize means all the mosquito noise around the mouth and eyes from before is more or less completely gone. I also split the denoising into luma & chroma.
My eyes are a little bored of looking at this same image for so long, but I think this is close to as good as it gets.
I think it now needs a smidge more sharpening, but I don't know masktools, and the only thing from poland I like is sausage. Help please didee :)

Also note the ingenious use of the ncpu parameter, which I only remembered after the thing stopped running in realtime. If you have two CPUs, this script is probably 25% slower or so.
avisource("TestVideo1").converttoyv12().crop(24,74,-20,-74)
o = last
fft = o.fft3dfilter(sigma=2,plane=0,bw=24,bh=24,ow=12,oh=12,ncpu=2).mergechroma(o.fft3dfilter(bt=5,sigma=3,plane=3,bw=24,bh=24,ow=12,oh=12,ncpu=2))
fftX = fft.minblur(1)#.mt_lutxy(fft.minblur(2),"x y > x 1 - x y < x 1 + x ? ?")
fftshrp = mt_lutxy(fftX,fftX.removegrain(11,-1),"x x y - abs 16 / 1 2 / ^ 32 * x y - 2 ^ x y - 2 ^ 6 + / * x y - x y - abs 0.0001 + / * +",U=1,V=1)#.limitedsharpenfaster(60)
fftshrpD = mt_makediff(fftX,fftshrp)
fftD = mt_makediff(fft,o)
fftshrpDD = fftshrpD.repair(fftD,12)

fft.mt_makediff(fftshrpDD,U=2,V=2)



Since you'll be compressing to YV12 anyway I did this test in YV12:
o huffyuv in latest ffdshow-tryouts. median prediction w/ adaptive tables: 56 910kbyte
o Raw YV12, then RAR'ed with winrar 3.51 'best' compression: 58 961kbyte
'nuff said :)

Not sure if this would help at all, but I have a filter with my card (uncertain if software or hardware based) but it will not kill chroma when luma saturates. However, like I said, not sure if that would help here. Anyway, I would agree it might need some more sharpening, but I haven't had much of a chance to use many of the different sharpening filters. My CPU is one core, but I'll still give your script a try. Thanks

Also, I used a huffyuv encode done with VBDub and it game out about 5MB larger then the YUY2 (after RAR'ing both videos). Did you use "create solid archive"? Although, it doesn't really matter. Let's just say it's about the same. :)

[*.mp4 guy]

*.mp4 guy, could you run that aliasing test and over sharpen this image as well (I would but I'm not sure what you used)? I tried something and will go into more detail if it worked.

It looks, perhaps a little better, but the aliasing is still there (http://img201.imageshack.us/img201/6866/13460800tf5.png), I wouldn't worry about it, its a not easy to see without sharpening.

[Didée]

@ twolfe18
That's pretty much OT for this thread. Denoising of HD material is quite another story. Especially if it's film grain ... due to the bigger resolution, spatial filters (or those with spatio-temporal coupling) get less effective on HD, and more destructive if one cranks up the parameters to compensate for that.

[edit: it seems twolfe18's post was deleted.)

[Didée]

Dunno, but perhaps like this?

http://img408.imageshack.us/img408/5196/snownoise182moreprocessyu7.png (http://imageshack.us) (frame 182)

http://img338.imageshack.us/img338/4976/snownoise239moreprocessiy9.png (http://imageshack.us) (frame 239)


vinverse(2.6)
fft3dfilter(sigma=32,sigma2=2,sigma3=2,sigma4=2,bt=5,bw=16,bh=16,ow=8,oh=8,degrid=1)
v2=minblur(2,2).removegrain(11,0)
v2D=mt_makediff(v2)
v2.seesaw(NRlimit=0,NRlimit2=4,Slimit=8,szp=12,sootheT=60,sootheS=24)
\.mt_adddiff(v2D,U=2,V=2)
limitedsharpenfaster(ss_x=1.2,strength=100,overshoot=0)
\.awarpsharp(depth=4.0,blurlevel=1).awarpsharp(depth=3.0,blurlevel=1).awarpsharp(depth=2.0,blurlevel=1)
\.mergechroma(last)
merge(dctfun5(7),0.61)

[survivant]

@Didée

I just try your script.. it give a really good image quality for an anime movie. I'm just starting to play with filter to encode x264, and for now you have the best combinaison.

the one I had before was this :

fft3dfilter(sigma=1.5, bt=5, bw=32, bh=32, ow=16, oh=16, sharpen=0.4)
degrainmedian(mode=1)
LimitedSharpenFaster(ss_x=1.5, ss_y=1.5, smode=4, strength=60, overshoot=1, undershoot=1)

[shadowhaze]

@Didée

:eek: Awesome script for anime - much better than anything I've been able write myself or find so far. :thanks:

To speed things up, I use FFT3dGPU instead of FFT3dFilter and modified the script accordingly.

# Didée's Anime Script ("DAS") modified to use FFT3dGPU instead of FFT3dFilter as follows:
# Changed syntax order to match FFT3dGPU, bt=5 to bt=3 as bt=5 not supported yet by FFT3dGPU (v0.82),
# increased sigma by 0.4 to adjust for slight luma change caused by using bt=3 instead of bt=5, mode=1 added, and precision=2 added)

AVISource("C:\...avi")
vinverse(2.6)
fft3dGPU(sigma=32.4,sigma2=2,sigma3=2,sigma4=2,bw=16,bh=16,bt=3,mode=1,precision=2,degrid=1,ow=8,oh=8)
v2=minblur(2,2).removegrain(11,0)
v2D=mt_makediff(v2)
v2.seesaw(NRlimit=0,NRlimit2=4,Slimit=8,szp=12,sootheT=60,sootheS=24)
\.mt_adddiff(v2D,U=2,V=2)
limitedsharpenfaster(ss_x=1.2,strength=100,overshoot=0)
\.awarpsharp(depth=4.0,blurlevel=1).awarpsharp(depth=3.0,blurlevel=1).awarpsharp(depth=2.0,blurlevel=1)
\.mergechroma(last)
merge(dctfun5(7),0.61)

Using FFT3dGPU, my encoding speed is almost 3x faster than with FFT3dfilter.

[survivant]

I try fft3dgpu 0.8.2 but I can't get it to work form me. THe image is corrupted. I have a screenshot here

http://24.202.22.217:8282/fft3dGPU.png

I have Windows XP, ATI X800 Pro 256 megs. DirectX 9.0c.

the script is simple.. it work if a remove the line for fft3dpgu and replace it by fft3d

movie.avs

DGDecode_mpeg2source("C:\DVD\encode\megui\intro.d2v",info=3)

the line for fft3dgpu

fft3dgpu()

or fft3dgpu(movie)

or I try the line from the manual

fft3dGPU(mode=1,precision=2)

it give me always a corrupted image.

I have recent DirectX and Drivers.

there is a way to launch fft3dGPU with debug information or verbose mode.. I could run it and post the result.

thanks

[Terranigma]

fft3dgpu relies on the prowess of a gfx card; what gfx card do you have?

[Leak]

fft3dgpu relies on the prowess of a gfx card; what gfx card do you have?
I have Windows XP, ATI X800 Pro 256 megs. DirectX 9.0c.
*cough* :D

[Terranigma]

*cough* :D

Now I feel embarassed. :o
I should've been wearing my glasses. =P

[shadowhaze]

FWIW, I made some more changes to my "DAS" script in order to keep the output color close to the original for more general use with anime. As before, if you want to use fft3dfilter instead of fft3dgpu, change the sigma to 32 and bt to 5.

vinverse(2.6)
fft3dGPU(sigma=32.5,sigma2=2,sigma3=2,sigma4=2,bw=16,bh=16,bt=3,mode=1,precision=0,degrid=1,ow=8,oh=8)
v2=minblur(2,2).removegrain(11,0)
v2D=mt_makediff(v2)
v2.seesaw(NRlimit=1,NRlimit2=4,Slimit=8,szp=14,sootheT=49)
\.mt_adddiff(v2D,U=2,V=2)
limitedsharpenfaster(ss_x=1.2,strength=100,overshoot=0)
\.awarpsharp(depth=3.0,blurlevel=1).awarpsharp(depth=2.0,blurlevel=1).awarpsharp(depth=1.0,blurlevel=1)
\.mergechroma(last)
merge(dctfun5(7),0.61)

Didee (or anyone else), I've also got some motion blocking issues and I'd like to know what's the proper place to put Deblock_QED. Right now I'm placing it between vinverse and fft3dgpu. Is that correct?

Thanks.