chroma denoising

[kassandro]

Below you find as an attachment (my server is down unfortunately) a tiny plugin for chroma denoising. It is yet for YUY2 only, because analog capturing is the most likely application.

It consists of three filters CNRT, CNRS and CNRF.
CNRT is very similar to MarcFD's CNR2, but uses a two sided approach in contrast to one sided approach in CNR2. The two sided approach has the advantage, that one doesn't have to care about scene switches and it should improve overall quality as well. The variables and the defaults are the same, but the value of the scene switch variable scdthr is ignored. Since I use different functions for calculating the weights for averaging, these default values may not be optimal. CNRT is purely temporal, whence it can be used for interlaced content.

CNRS is a spatial analog of CNRT and has the same variables and defaults as CNRT plus one mandatory additional clip variable. The default use is the follwoing:
CNRS(input, blurred_input)
Here blurred_input is a blurred version of the clip input, RemoveGrain(input,mode=12) for instance. The same variables as for CNRT or CNR2 can be used to tweak the filter. Being purely spatial CNRS can only be used for progressive input, though if blurred_input is only horizontally blurred, it can be used for interlaced input as well.
Because CNRT and CNRS are similar to CNR2 and uses lookup tables, it cannot be SSE optimised. The filter CNRF is very different and can be SSE optimised (would have about 1000 fps on my Prescott monster). It embodies more clearly than CNRS the following philosophy: the spatial variation of the chroma is limited by the spatial variation of the luma. The usage of CNRF is gain
CNRF(input, blurred_input)
but now there are no further variables. What does CNRF do? To answer this question y, u, v be the luma and chroma values of a pixel of the clip input and yb, ub, vb be the same pixel in the clip blurred_input. Then CNRF first calculates the luma distance d=|y - yb| and then u is clipped at ub - d, ub + d, i.e. the new value = min(max(u, ub - d), ub + d) and similarily v is clipped at vb - d, vb + d. Of course, it would also be possible to replace d by a (possible fractional) multiple of d.

The current version can be downloaded from here (http://home.pages.at/kassandro/Cnr2k.zip)

[Kagura]

Would it be possible to host this on another site? Currently, the attachment system doesn't work very well. I don't see it showing up anywhere on your message. =(

[kassandro]

I can't see the attachment either. Download it from here (http://home.pages.at/kassandro/Cnr2k.zip). When I edit the above message, the forum software insists that the attachment is there. It's the first time I tried such an attachment, though.

[Boulder]

A moderator needs to approve the attachment before it's available.

Thanks for the plugin, I've got another capture coming in today so it's time to play again:D

[Mug Funky]

nice stuff. only just started playing with it (can't find many really dirty sources to be honest...) and it seems pretty fast.

btw, it doesn't throw an error if you feed it yv12 - it just gives me green :)

[kassandro]

Originally posted by Mug Funky
nice stuff. only just started playing with it (can't find many really dirty sources to be honest...) and it seems pretty fast.

btw, it doesn't throw an error if you feed it yv12 - it just gives me green :)
This concerns only CNRF and the reason is simple. For CNRF, but not the other two (which are not really my kids), I want to write both an SSEx-YUY2 and an SSEx-YV12 version, if the current slow YUY2 version (the SSE2 version should be 20 times faster) turns out to be useful. The description of CNRF given above is somewhat simplified, because for each "chroma pixel" there are two (and in YV12 even four) "luma pixels". CNRF can also be tweaked to be more aggressive or more conservative, but I like the philosophy implemented in CNRF (with a suitable blurred_input) that the chroma should not oscillate more than the luma.

Originally posted by Boulder
A moderator needs to approve the attachment before it's available.

Thanks for the information, I was desperately searching the attachment without any success and I though that I simply too stupid for making an attachment, but for a whimpy 8 KB attachment Moderator approvement should really not be neccessary.

Anyway my server is back again!

[Boulder]

Originally posted by kassandro
This concerns only CNRF and the reason is simple. For CNRF, but not the other two (which are not really my kids), I want to write both an SSEx-YUY2 and an SSEx-YV12 version, if the current slow YUY2 version (the SSE2 version should be 20 times faster) turns out to be useful. The description of CNRF given above is somewhat simplified, because for each "chroma pixel" there are two (and in YV12 even four) "luma pixels". CNRF can also be tweaked to be more aggressive or more conservative, but I like the philosophy implemented in CNRF (with a suitable blurred_input) that the chroma should not oscillate more than the luma.

Yep, the optimizations would be appreciated as it's always a good slice of time off the encoding process, which can be painfully slow when interlaced content needs to be treated. Smart bobbing means twice the amount of frames so the processing length approximately doubles..anything less is always a good thing:)

for a whimpy 8 KB attachment Moderator approvement should really not be neccessary.

I guess it's mostly for virus checking? I can't think of any other really good reason as you can link to files that are off-site.

[Boulder]

I did a short test and noticed that CNRF changes the colours slightly.

See the three screenshots, one from the original MJPEG avi, one with CNR2("xoo") and one with CNRF() , blurred clip made with RemoveGrain(mode=12).

Original (http://www.saunalahti.fi/sam08/org2.jpg)
CNR2 (http://www.saunalahti.fi/sam08/marcfd.jpg)
CNRF (http://www.saunalahti.fi/sam08/kassandro.jpg)

Pay attention to the core of the ship, the one with bluish color. It's a more light blue with CNRF, more like the original with CNR2.

[kassandro]

Sorry Boulder, I overlooked your last posting until yesterday. Because I couldn't find the bug by inspecting the source code, I finished the SSE YUY2 version and compared it with the C routine, which was used by yourself. It turned out that there was a big difference between both and I finally could find the mistake in the C version. In hindsight I really wonder that the output of CNRF looked so good. It is even difficult to spot in your pictures. I will provide a fixed version later today.

Many thanks for realising the problem.

[Boulder]

Oh, those CNR2 and CNRF screenshots were taken from filtered clips (LRemoveDust+CNR2/CNRF) and the original one is completely unprocessed. That's why they look a lot smoother;) Nice to hear that you found the cause, got another Get Smart! episode on the air tomorrow so I've got raw material to use for testing.

[kassandro]

I have removed the attachment of the first posting in this thread and replaced by a link to the corrected. I have also updated the link from last sunday. Both links now point to the same location. I have corrected a difficult to find bug (no crash and hard to see). Actually only one character was incorrect, which led to wrong V values. The new plugin contains also an SSE/SSE2 version of CNRF. However, with standard YUY2 the C filter of CNRF is used. The SSE is used only with planar YUY2.

What is planar YUY2? For Avisynth there is no difference between planar and the normal interleaved YUY2. Only the organisation of the data is quite different. Thus if you display a planar YUY2 clip, you will see only garbage. The advantage of planar YUY2 is, that one can process planar YUY2 almost the same way as YV12, i.e. SSE/SSE2/SSE3 can be used nicely. Virtually any YV12 can be extended to planar YUY2. Over the course of the next months I will extend the filters RemoveDirt and RemoveGrain to planar YUY2 and remove the support for interleaved YUY2 in RemoveDirt. Especially the YUY2 performance of RemoveDust will be improved quite substantially.

If there are no "natural" planar YUY2 clips, how can one utilise it? Answer: through the conversion filters Interleaved2Planar and Planar2Interleaved of my yet inofficial plugin SSETools (SSE2Tools is for SSE2 cpus, i.e. P4/Athlon64/Sempron 3100). Interleaved2Planar is called at the beginning of the filter chain and Planar2Interleaved is called at the end of the filter chain, but all the filters of the chain must support planar YUY2 and currently CNRF is the only one, though many temporal filter, but not CNRT, work also correctly with planar YUY2. Interleaved2Planar and Planar2Interleaved are highly optimised (should be almost bitblt speed). Since Avisynth doesn't know of planar YUY2 (actually there is also planar RGB24 and planar RGB32), one has to tell a filter, that the input is planar and not interleaved. This is usually done with boolean variable "planar". Thus

function CNRF_SSE(clip input)
{
smooth = RemoveGrain(input, mode=12).Interleaved2Planar()
planar = Interleaved2Planar(input)
return CNRF(planar, smooth, planar=true).planar2interleaved()
}

does the same as

function CNRF_C(clip input)
{
smooth = RemoveGrain(input, mode=12)
return CNRF(input, smooth)
}

but should be somewhat faster. Unfortunately the interleaved RemoveGrain(input, mode=12) is currently a big brake in CNRF_SSE. Once we have support for planar YUY2 in RemoveGrain

function CNRF_SSE(clip input)
{
planar = Interleaved2Planar(input)
smooth = RemoveGrain(planar, mode=12, planar=true)
return CNRF(planar, smooth, planar=true).planar2interleaved()
}

should be a lot faster.

I hope to post an YV12 version of CNRF next weekend.

[Chainmax]

I can't wait to try CNRF_YV12 on a VHS capture that's proving to be a real bitch :). You say that CNRT should be better than CNR2, what about CNRF? I'm currently using Cnr2("xxx",4,5,255) in order to reduce some small red and yellow bands that appear on said capture and would love to test an alternative.

[Boulder]

cnr2kSSE2.dll crashes on my system - when I use the function cnrf_sse(), VDubMod simply closes itself. Similar behaviour to what happened in SSE2-optimized RemoveGrain with some modes in the first version. The plain cnr2k.dll works.

[kassandro]

Originally posted by Boulder
cnr2kSSE2.dll crashes on my system - when I use the function cnrf_sse(), VDubMod simply closes itself. Similar behaviour to what happened in SSE2-optimized RemoveGrain with some modes in the first version. The plain cnr2k.dll works.
Boulder, you are quite right. It was the same kind of bug as with earlier SSE2 versions of RemoveGrain. A data alignment bug. Actually the bug was more a consequence of tiredness. Testing was simply insufficient. I have tested the SSE2 version only for a 720x576 video. If the YUY2 frame address and the frame width are each a multiple of 16, then the fast routine for aligned access is chosen. This routine was correct and was the only one, whihc was tested. But unfortunately I simple copied the routine for unaligned access was simply a copy of the aligned one and this it simply work. I have uploaded a corrected SSE2 version.

[Boulder]

I just did a very quick-n-dirty test.

CNRF, speed ~0.31RT, filesize 9019164 bytes
CNR2 (defaults), speed ~0.56RT, filesize 7209380 bytes

CNR2's output looks better, it removes nasty rainbowing quite well, leaving only slight blotches which would probably be unnoticable on the TV. It's also quite a bit faster which I always appreciate.

I used this script:

AVISource("d:\temp\captures\testi.avi",pixel_type="YUY2")
LeakKernelDeint(order=1,sharp=true,threshold=7) # for testing only
Crop(0,4,696,568,true)
BilinearResize(672,544)
#org=CNR2()
org=CNRF_SSE()
ConverttoYV12()
LRemoveDust_YUY2(4,4)
ConverttoYUY2()
MergeChroma(org)
ColorMatrix(mode="rec.601->rec.709")
AddBorders(16,16,16,16)

The script is practically the same (apart from cropping) I use for nearly all my captures. I decided to simply deinterlace (which I almost never do) for faster processing.

[kassandro]

Originally posted by Boulder
I just did a very quick-n-dirty test.

CNRF, speed ~0.31RT, filesize 9019164 bytes
CNR2 (defaults), speed ~0.56RT, filesize 7209380 bytes

Thanks for the test. Quite an impressive difference. It shows again that temporal smoothness is more significant for compression than spatial smoothness.
As far as speed is concerned I can do much better. When all is converted to planar ConverttoYV12(), ConverttoYUY2(), MergeChroma(org) will be superfluous. Anyway, your numbers - obviously you use CCE - do not imply that CNR2 is so much faster than CNRF. Obviously the output of CNR2 is much smoother and because of this CCE is much faster. Even if all the filters would be planar, the overall encoding would still be slower, although the time spent with Avisynth will almost be neglible then. Similarily, an unfiltered video takes about 30% more time to encode with XviD than a video with RemoveDust.
CNRF is only for progressive input. Thus deinterlacing was necessary.


CNR2's output looks better, it removes nasty rainbowing quite well, leaving only slight blotches which would probably be unnoticable on the TV. It's also quite a bit faster which I always appreciate.

Does CNRT remove the rainbows? It should act very similarily to CNRS but doesn't have and doesn't need the poor scene change detection, which may easily confuse motion with a scene change. Overall CNRT should be somewhat more conservative than CNR2. Because it is twosided it is probably a bit slower than CNR2, but I tried to close the gap by most efficient programming as much as possible.

You can monitor the agressiveness of a filter nicely with the difference filter contained in SSETools:

difference(filtered, unfiltered)

Then the SAD difference and the number of different pixels is reported to the debugview utility (actually a pixel may be counted three times, for the Y, U and the V value). The output of difference is identical to "filtered". The difference filter has many options. One is the tolerance option

difference(filtered, unfiltered, tolerance=5)

ignores all difference <= 5. The higher the total differences, the stronger the filtering.
I can make CNRF more aggressive. However, rainbows seem to be a temporal rather than a spatial phenomenon, otherwise CNR2 couldn't remove them. As I virtually never have rainbows on my mostly progressive digital recordings, could you please provide me a 3 frame Huffyuv video, with a rainbow frame in the middle. Then I could make tests on my own. CNRF can also be made more aggressive by replacing RemoveGrain(mode=12) by RemoveGrain(mode=12).RemoveGrain(mode=12) (unfortunately quite slow currently). Unlike CNR2 or CNRT, CNRF can never creat ghosts.


I used this script:

AVISource("d:\temp\captures\testi.avi",pixel_type="YUY2")
LeakKernelDeint(order=1,sharp=true,threshold=7) # for testing only
Crop(0,4,696,568,true)
BilinearResize(672,544)
#org=CNR2()
org=CNRF_SSE()
ConverttoYV12()
LRemoveDust_YUY2(4,4)
ConverttoYUY2()
MergeChroma(org)
ColorMatrix(mode="rec.601->rec.709")
AddBorders(16,16,16,16)


What I do not like in this script, is that you crop and resize so early. This certainly makes the script slightly faster, but you also loose quality. Only for RemoveDirt or any block based filter early cropping and resizing makes sense. Also CNRF_SSE would have more impact, if the luma is cleaned before the application of CNRF_SSE. Of course, the same applies to CNR2, but CNRF would probably profit a lot more. Keep always in mind, that the amount of cleaning by any of these CNR filters depends heavily on luma smoothness. I would use the following script:

AVISource("d:\temp\captures\testi.avi",pixel_type="YUY2")
org=LeakKernelDeint(order=1,sharp=true,threshold=7) # for testing only
ConverttoYV12(org)
LRemoveDust_YUY2(4,4)
ConverttoYUY2()
MergeChroma(org)
#CNR2()
CNRF_SSE()
Crop(0,4,696,568,true)
BilinearResize(672,544)
ColorMatrix(mode="rec.601->rec.709")
AddBorders(16,16,16,16)

[Boulder]

OK, I uploaded a 5-frame HuffYUV clip, download it at http://www.saunalahti.fi/sam08/rainbow_test.avi . I couldn't get a clip with lots of visible rainbows but you can see them easily by using UtoY() or VtoY(), there's a nice amount at the edge of the tie and the shirt.

CNRT works almost like CNR2, probably would just need some settings adjusting. I didn't make a speed test yet as I've got an encode going on but I'll try to find some time later today. I'll also try filtering before cropping and resizing and move chroma denoising after LRemoveDust. Funny thing that I never thought that denoising luma first would help with cleaning the chroma channels, I guess that tells how much I really understand what I read:D

[Boulder]

OK, here are results of another quick test:

CNR2 - 0.47RT - 20 315 364 bytes
CNRT - 0.35RT - 21 381 396 bytes
CNR2 after LRemoveDust_YUY2 but before crop+resize - 0.45RT - 23 185 852 bytes
CNR2 before LRemoveDust_YUY2 but before crop+resize - 0.45RT - 23 211 384 bytes

[kassandro]

Originally posted by Boulder
OK, here are results of another quick test:

CNR2 - 0.47RT - 20 315 364 bytes
CNRT - 0.35RT - 21 381 396 bytes

I am not too surprised, because I discovered today, that the initialisation of the lookup tables was not correct. If u[n,x,y] denotes the u value of the pixel at position x,y of the n-th frame. Then CNR2 replaces this value by (1-t)u(n,x,y) + t u(n-1,x,y), where the averaging coefficient t depends on y and u value differences and CNRT is similar. However t should never be > 1/2 and in my case it may attain the value 1. The same happened unfortunately with CNRS. I hope to upload a corrected version, where not only the lookup table is correctly initialised but also the algorithm is improved (CNRT and CNRS, CNRF remains unchanged).


CNR2 after LRemoveDust_YUY2 but before crop+resize - 0.45RT - 23 185 852 bytes
CNR2 before LRemoveDust_YUY2 but before crop+resize - 0.45RT - 23 211 384 bytes
This result is very surprising. the difference is absolutely neglible 0.1%.

If you have time, you may test the following temporal application of CNRF (deinterlacing is not necessary). It uses the scene change detection of TemporalSoften (seems to work well with planar YUY2):

function TCNRF(clip input)
{
planar = Interleaved2Planar(input)
smooth = TemporalSoften(planar,1,255,255,15,2)
return CNRF(planar, smooth, planar=true).planar2interleaved()
}

You may also tweak the radius parameter of TemporalSoften.
However, there is no change to beat CNR2. It is far too tame.
Here are the unfiltered/filterd differences of some sample frames with TCNRF:

[2868] [870] total difference = 12412, different pixels = 10788
[2868] [871] total difference = 14866, different pixels = 13011
[2868] [872] total difference = 15015, different pixels = 13567
[2868] [873] total difference = 15638, different pixels = 13927
[2868] [874] total difference = 20339, different pixels = 18251
[2868] [875] total difference = 19653, different pixels = 17808
[2868] [876] total difference = 15567, different pixels = 13524
[2868] [877] total difference = 14163, different pixels = 12723
[2868] [878] total difference = 14560, different pixels = 12829
[2868] [879] total difference = 20423, different pixels = 17792
[2868] [880] total difference = 19168, different pixels = 17142
[2868] [881] total difference = 20069, different pixels = 18194
[2868] [882] total difference = 15699, different pixels = 14080
[2868] [883] total difference = 20574, different pixels = 18222

and here are the same frames now with CNR2:

[2868] [870] total difference = 152017, different pixels = 132789
[2868] [871] total difference = 204315, different pixels = 179442
[2868] [872] total difference = 184501, different pixels = 163621
[2868] [873] total difference = 212530, different pixels = 191829
[2868] [874] total difference = 259744, different pixels = 228510
[2868] [875] total difference = 249652, different pixels = 222470
[2868] [876] total difference = 294953, different pixels = 258043
[2868] [877] total difference = 269431, different pixels = 248090
[2868] [878] total difference = 258365, different pixels = 242860
[2868] [879] total difference = 275824, different pixels = 249133
[2868] [880] total difference = 369295, different pixels = 295538
[2868] [881] total difference = 463517, different pixels = 322928
[2868] [882] total difference = 487876, different pixels = 320502
[2868] [883] total difference = 485125, different pixels = 303761

Note that there are only 720*576 "chroma" pixels. Thus CNR2 changes in the average about 60% of the chroma with its default values, while CNRT changes only less than 5% of all "chroma" pixels. I have to make CNRF more aggressive.

[Boulder]

Sure, I'll give TCNRF a go tomorrow when I have the time.

The problem with tweaking default values for chroma denoisers is IMO that the differences are very hard to notice and you practically have to either use Subtract+Levels or U/VtoY to see any changes. They might look OK in most cases but could easily smack you in the face in some strange point.

Apart from occasional chroma ghosting, CNR2 looks good to me so it could possibly be regarded as some kind of a calibration point. The ghosting almost always appears in clips that already have some kind of ghost problems, so it's not an everyday thing. I've managed to get rid of most of that by switching back from PVR-250 to the BT8x8 card, I guess that improved chroma resolution (YUY2 vs. YV12) and all-keyframe encodes can make a huge difference:)

[Chainmax]

kassandro, did you miss my post?

[kassandro]

Originally posted by Chainmax
kassandro, did you miss my post?
No, but your post didn't seem to require an answer. I think my filter is yet in test phase. Currently, it is simply too conservative, especially CNRF, to compete with CNR2 as far as compression is concerned. CNRT, CNRS should not be used at the moment. I will post a new version on saturday. Tests and comments are very wellcome!

[Chainmax]

I'll be happy to test CNRF_YV12 once it's ready, and I will post pictures comparing it to CNR2 :).

[kassandro]

Originally posted by Chainmax
I'll be happy to test CNRF_YV12 once it's ready, and I will post pictures comparing it to CNR2 :).
I think, I have to postpone the YV12 routine until next weekend. But why are you using YV12 for analog capture? If you capture progressive material, then YV12 is not really disadvantageous, but if is is interlaced material, then YUY2 is vastly superior, because interlaced YV12 is really a bad thing. The relation quality loss versus compression gain is far worse for interlaced YV12 than for progressive YV12 material. Probably you have a capture card with an mpeg2 hardware encoder, which are easy to use, but with respect to quality they are obviously inferior to the much cheaper YUY2-avi capture cards.

[kassandro]

I have now uploaded a new version (available at the old links). There have been a lot of changes. CNRT and CNRS should now work as expected, i.e. better compression and less ghosts for CNRT. I have also added the the boolean variable aggressive to CNRT. If aggressive=true, then CNRT should operate more similarily to CNR2, but it will also a similar amount of ghosting. The aggressive=true mode is one sided, thus some scene change code should be added, which is not yet there. However, for testing the current version is sufficient. As far as CNR2 and CNRT(aggressive=true) is concerned, there is an issue with multipass encoding: Avisynth may serve slightly different frames in each pass. All three filters have been made slightly more aggressive.
No YV12 version yet.

[Chainmax]

Originally posted by kassandro
I think, I have to postpone the YV12 routine until next weekend. But why are you using YV12 for analog capture?
The capture was either made with professional DV equipment (in RGB24 IIRC) or a hauppauge 401 (in YUY2 AFAIK), but what I'm working on was manually IVTCed via TMPG (no auto IVTC method produced satisfactory results) so it's progressive. The only reason I'm converting to YV12 is because I'm compressing it to Xvid for testing purposes and because some of the filters I use need YV12 input.

[kassandro]

Usually any filter for YV12 can be quickly turned into a planar YUY2 filter, because all planes are processed separately. CNRF is an exception of this rule. In YUY2 each chroma "pixel" belongs to two luma "pixels" while in YV12 there are already 4 luma "pixels" for each chroma pixel and if you want to go for the best quality these differences require different assembler routines, because the CNR filters unlike most other filters relate the chroma to the luma, more precisely chroma differences to luma differences. Because of this I am waiting for things to settle before I turn to the YV12 routine. At the moment CNRF simply seems to be to conservative.