avisynths spatial compared to sshq

[Red-Scorp]

I've been doing encodes of animes for awhile. Most of them are
Dragonball Z. My question is regarding the noise in the dvds. I want to get rid of the noise without loosing any detail. Im using avisynth. Ive done many tests but cant come to a resultive conclusion. I used a combination of temporalsmoother with spatialsoften to remove the noise. I also did tests using temporalsmoother and smart smoother high quality to remove noise. I need someone really experienced to answer this question. From tests and experience that u may have, what should i use? spatial or sshq to remove noise without loosing detail. Thanks.

[OUTPinged_]

try one of these:

TemporalSoften2(1,10,1)

SpatialSoftenMMX(1,1,0,false,false,10,10,1,1)

they wont filter all of noise but also wont kill backgrounds and will increase compressability as well as sshq will

in the end, it doesnt matter, does you encode have noise or not, you kill noise because it takes away bits.

[dividee]

Just by looking at the algo of both filters, I'd say SmartSmootherHQ is better for almost any source.

Unfortunately, there aren't a lot of spatial denoising filters for avisynth. That's probably because spatial filtering is harder to do fast in YUY2 compared to RGB32. And if you can't do it faster than converting to RGB and filtering there, there is not much point.

[sh0dan]

When I'm finished optimizing average-pixels mode, I'll move on to YUV2-processing for Avisynth.
I'm currently looking for some documentation on YUV2, and pixel placement. YUV2 seems to be using full 32bits (for two pixels?), which would require some modifications, since I'm currently relying on 24 bits/pixel. I found some nice documentation some time ago, but now it's gone :(

[dividee]

Very cool sh0dan :) I'm eagerly awaiting this port.

YUY2 is not that hard to understand:
Scanlines are stored top to bottom (contrary to RGB).
each 32 bit DWORD contains information for 2 pixels; the lower byte (byte 0) is luma (Y) for the first pixel, byte 2 is luma for the second pixel, byte 1 and 3 contains chroma (U,V) information for both pixels (so in hexa this is 0xVVYYUUYY (C notation)).

If you filter a pixel using it's 3 components, you usually need to dedouble chroma (U,V) and average them after filtering.

If you want some source code, I guess the easier and shorter one is the Tweak filter by Donald Graft.

[sh0dan]

Thanks a bunch for the info. Seems like YUV2 in itself provides a nice speedup, since it only requires half the memory-bandwidth of RGB.

Can YUV-values be linear interpolated (as RGB-values), the that (U1+U2)/2 and (V1+V2)/2 gives the correct color inbetween?

Seems like I'll get some use for pshufw. ;)

I'll have to give it a good thought on how to make the filter fast. The most obvious way seems to always process two pixels at the time, thus setting the horizontal window size to 5,9 or 13 pixels, but preserving full luma resolution would be preferable. Perhaps an option could be added later.

The filter doesn't care about top/bottom - it's actually coded as top-bottom ordering.

I have already looked at Tweak/decomb and your filters, and they help a lot - this just added the final understanding! :)

[dividee]

Can YUV-values be linear interpolated (as RGB-values), the that (U1+U2)/2 and (V1+V2)/2 gives the correct color inbetween?
Theorically I don't know, but at least in practice it seems to work :)

I'll have to give it a good thought on how to make the filter fast. The most obvious way seems to always process two pixels at the time, thus setting the horizontal window size to 5,9 or 13 pixels, but preserving full luma resolution would be preferable. Perhaps an option could be added later.
Processing two pixels at a time is the usual approach. It can become quite tricky if thresholding isn't done independently for each channel. It's easy to lose the edge given by the memory bandwidth savings.

YUY2 fun:
Let's have a two pixel block you want to filter: Y1 U1 Y2 V1
And another one you're filtering with: Y3 U2 Y4 V2
Let's say pixels 2, 3 and 4 pass the threshold for pixel1, but only pixel1 and pixel4 pass the threshold for pixel2. (understand: pixel2 is (Y2,U1,V1) )
What would be the correct (unweighted) averaged pixel pair ?
Unpack each pixel and average:
pixel1: Y U V = (Y1+Y2+Y3+Y4)/4 (U1+U2)/2 (V1+V2)/2
pixel2: Y U V = (Y1+Y2+Y4)/3 (2*U1+U2)/3 (2*V1+V2)/3
Average chroma:
((U1+U2)/2+(2*U1+U2)/3)/2 = (7*U1+5*U2)/12
(same for V)

final pair: (Y1+Y2+Y3+Y4)/4 (7*U1+5*U2)/12 (Y1+Y2+Y4)/3 (7*V1+5*V2)/12

I don't know if this makes sense. It's late and I'm utterly tired. It may be rubbish :(

[sh0dan]

Actually it does make sense. I use 16 bit weights, so pixels further away are blended less than closer pixels, so the final pixel (in RGB) is:

w = accumulated weight for pixel (max. 65536)
iw = 65536-w
P = Accumulated other pixels multiplyed by weights
cP = Current pixel (0->255)

Final pixel = (P+(CP*iw))>>16

This also avoids using any divides in 'weighed average' mode.

I finished the Average MMX-optimization, and looked a bit at YUV-processing, and it seems like 2 pixel processing is out of the question. It will generate a lot of special conditions that would generate a lot of random braching.

The solution seems to be to do two seperate functions - one for even pixels, and one for odd pixels. That way no special braching is necessary within the function. I plan on using seperate weights for each Y-component, and storing UV-data so that:

Y1=(Y1+(cY1*IW1))>>16
Y2=(Y2+(cY2*IW2))>>16

and
iw = (IW1+IW2)>>1
UV = (UV+(cUV*iw))>>16

Both fuctions will only be called for each even pixel (and so store both odd and even pixel with one memory access).

I hope very much that the speed gained from not doing YUV->RGB->YUV is enough to make up for the complexity of handling YUV-data.

[Acaila]

I hope very much that the speed gained from not doing YUV->RGB->YUV is enough to make up for the complexity of handling YUV-data.
Well, since I currently suffer from a three-fold increase in encoding time using your filter, I would be happy with ANY speedups :D

[jopereira]

Originally posted by dividee
Just by looking at the algo of both filters, I'd say SmartSmootherHQ is better for almost any source.



@ dividee

This is really a beginners question but, where do one get SSHQ?
(until now in my tests I've only used TS(4,3) or TS(3,2), but it's seems there is a better one...)

TIA,

[dividee]

Get it from Donald Graft's site (http://sauron.mordor.net/dgraft/). To use it in avisynth, look in the new VirtualDub Filters Import File, v1.2 (http://forum.doom9.org/showthread.php?s=&threadid=23804) thread.

I was comparing SSHQ and SpatialSoftenMMX, not TS.
TS is a temporal denoiser while SSHQ is a spatial denoiser; in some case it might even be useful to combine them.

[sh0dan]

@jopereira:
Get it at Donald Graft's filter page (http://sauron.mordor.net/dgraft/) it's in the hosted section.

@Acaila: If you're using "Average pixels" mode then you can mail me (kp@interact.dk) for the v2.1 beta, which gives a >100% speedup - and even surpasses the original Smart Smoother and Dividee's optimized Spatial Smoother. If you are the more patient type, the new version should be out within a week.

[Acaila]

If you are the more patient type, the new version should be out within a week.
Most good things in life are worth the wait :)
I don't mind the slowdown all that much anyway, because I'll be encoding on my new system (hopefully) after this weekend. On that dual Athlon 2000+ even the un-optimized version will probably be quite fast :D.
So you can understand I'll be busy for a while getting it up and running (and stable!), but after that I'd be happy to help you with beta-testing.

[jopereira]

Originally posted by dividee
I was comparing SSHQ and SpatialSoftenMMX, not TS.
TS is a temporal denoiser while SSHQ is a spatial denoiser; in some case it might even be useful to combine them.

Thanks.

I realised that you were talking about spatial filters, while my experience is with temporal ones. I would like to make a combination of both, if the results are better...

I'll make some tests but before that, and because many of you have already made many tests, I will ask: isn't most of DV and VHS noise temporal?
I mean: in VHS we have noise from tape itself, while in DV most of the noise is due to "capture" device (CCD), mostly indoor scenes.
Since I will have both sources (DV been most of them), I would like to have same kind of standard for denoising my sources.

TIA,

[sh0dan]

It seems like Donald is very busy at the time - and since my new version contains a rather big bug-fix, you can grab Smart Smoother v2.1 here (http://cultact-server.novi.dk/kpo/vdub/index.html). (All users should upgrade!)

[Acaila]

Yummy :D

[h00z]

Any word on the YUY2 native (AVISynth) port? I'm foaming at the mouth waiting for it ;)