RemoveGrain

[kassandro]

RemoveGrain is a spatial denoiser for progressive video. Because of SSE optimisation, it is fairly fast (above 100 fps on my whimpy 1.3 GHZ Celeron). There are now significantly faster SSE2, SSE3 versions. For more details go to www.RemoveGrain.de.tf (http://www.RemoveGrain.de.tf).
16|07|04 version 0.4 (first public version) released.
17|08|04 version 0.5 released (for a change log see the posting below)
12|09|04 version 0.6 released (for a change log see the posting below)
13|02|05 version 0.7 released (for a change log see the posting below)
09|04|05 version 0.8 released (for a change log see the posting below)
01|05|05 version 0.9 released (for a change log see the posting below)

[Boulder]

Nearly missed this one..I'll test it tomorrow on my P4 system:)

[Boulder]

I can't get the SSE2 version working. I extracted first both RemoveGrain.dll and RemoveGrainSSE2.dll in my plugins folder and then tried removing RemoveGrain.dll. In both cases I got an error message "there is no function named DRemoveGrain".

[ARDA]

Same here; and would like to test this filter

Thanks ARDA

[kassandro]

Sorry, I was a little bit too fast and too tired (after midnight), when I rushed out the plugin. Actually RemoveGrainSSE2.dll was the non-test version, which I wanted to release only after satisfactory testing. I hope that it's correct now. Thanks in advance for testing.
I also made some slight corrections in the documentation.

[Boulder]

OK, I tested the SSE2 version, and there were no differences in the DebugView log. However, the only modes I got working were 5,6,7 and 8, all the others gave an access violation when the script was loaded in VirtualDub. I used a 720x576 MPEG-2 capture as the test material.

Other than that, the filter seems to work very well, mode 8 did an excellent job with my TV caps when followed by RemoveDirt(). Which reminds me, are you planning on adding SSE2 optimizations to RemoveDirt as well?

[kassandro]

First, many thanks for testing.

Originally posted by Boulder
However, the only modes I got working were 5,6,7 and 8, all the others gave an access violation when the script was loaded in VirtualDub. I used a 720x576 MPEG-2 capture as the test material.


Hmmm, was this a read access or a write access violation? Did the access violation also appear with the non-SSE2 version (the only one I could test)? Does the access violation persist if you crop before RemoveGrain with crop(4,4,-4,-4, align=false)? Finally, there may be a bug in the difference filter, though this is extremely unlikely, because it works with the other modes. But nevertheless one should make a try without it. It is enough to test this for the simplest mode=1, where surprisingly the access violation already happens. I expect that you did the tests with an yv12 clip, which is the natural one for mpeg2. The other color spaces are not well tested. In fact, I am stunned about the very poor performance for the non-yv12 color spaces, but I postponed the investigation of this problem for a while.


Other than that, the filter seems to work very well, mode 8 did an excellent job with my TV caps when followed by RemoveDirt(). Which reminds me, are you planning on adding SSE2 optimizations to RemoveDirt as well?
I did some work in this direction, but the performance gain will be quite small as long as the block width is 8. Changing the block width to 16 SSE2 will become very valuable, but then the frame width has to be a multiple of 16. Also unlike RemoveGrain, RemoveDirt heavily uses the psadbw instruction and this instruction behaves differently on 128 bit SSE registers (used with SSE2) and 64 bit mmx registers used (which must be used by the non-SSE2 version). Thus, while I get the RemoveGrain SSE2 version for free (it only requires testing) using smart macros, this is no more the case for RemoveDirt, where I have to make modification at some places, where I use the psadbw instruction. But since Intel has adopted the Athlon64 platform, we all will move to this platform within the next 2-3 years, because with its 16 general purpose registers (14-15 regsiters will then be usable, currently only 6-7 registers can be used) C++ compilers can generate significantly more efficient code even for 32 bit apps on the Athlon64 platform. Also the video resolutions will increase (HDTV). These two reasons more than justify a block width of 16 (the block height will stay at 8), which also has some slight disadvantages.

[Chainmax]

This is the filter that you mentioned works similarly to Undot, right?

[Boulder]

@kassandro: I'll do the tests tomorrow and let you know what happens.

[kassandro]

Originally posted by Chainmax
This is the filter that you mentioned works similarly to Undot, right?
Yes, RemoveGrain(mode=1) = UnDot(). Even the speed of the SSE version (with mode=1, of course) is identical to that of Undot. Like Undot and many other spatial denoisers, also the other eight modes are based on the eight neighbours of a pixel. The details of the algorithms can be found in the documetnation

@Boulder: many thanks already. So far, I can't reproduce these ugly access errors. A thorough inspection of the source code also didn't help.

[Boulder]

Hmmm, was this a read access or a write access violation? Did the access violation also appear with the non-SSE2 version (the only one I could test)?


It gives a read access violation, the exact message is "Avisynth: caught an access violation at 0x012f151a, attempting to read from 0xffffffff". The violation doesn't appear on the non-SSE2 version, all modes are OK.


Does the access violation persist if you crop before RemoveGrain with crop(4,4,-4,-4, align=false)?


Yes, modes 4-8 are the only ones that work.


Finally, there may be a bug in the difference filter, though this is extremely unlikely, because it works with the other modes. But nevertheless one should make a try without it. It is enough to test this for the simplest mode=1, where surprisingly the access violation already happens.


No go, so the problem must be somewhere within RemoveGrain.


I expect that you did the tests with an yv12 clip, which is the natural one for mpeg2. The other color spaces are not well tested. In fact, I am stunned about the very poor performance for the non-yv12 color spaces, but I postponed the investigation of this problem for a while.


Yep, YV12 it is, I don't deal with YUY2 anymore since I bought the PVR-250.

Hope this all helps:)

[kassandro]

Thank you Boulder for all your efforts, but I'll have to withdraw the SSE2 version. I spend quite some time to find the bug(s) using your precise observation, but all I found was one wrong SSE2 macro in for mode 1 and nothing in the other modes. Surprisingly my Celeron with Tualatin P3 design swallows the SSE2 version without any access error (also with 720x576 mpeg2 input). Of course, the output is not quite correct. While with the true SSE2 one can handle 16 pixels at a time, with the P3 SSE2 only the first 8 bytes of the 16 byte SSE register are processed. Thus if chrom processing id disabled (i.e. modeU=0), then after 8 correctly processed pixels there are 8 incorrectly processed pixels with the SSE2 version on Tualatin P3 (I don't know what happens on a Katmai or Coppermine P3 or on the various Athlons). I simply have to get my hands on a P4 to finish the job. Programming for the P4 without having one simply does not work.

[Boulder]

Well, fortunately the speedup isn't that big, I'd estimate somewhere along 8-12% on my system. I suppose that mode 8 (which I also prefer) works OK, at least according to the difference debug log, so the SSE2 version can be used and I probably will.

I don't know if there are any developers who have a P4 system and could help..maybe it would be worth asking around a bit.

Thanks for your efforts anyway:)

[dbzgundam]

Why can't I access the page?

It's loading but I see a blank page. :confused:

[kassandro]

Originally posted by Boulder
Well, fortunately the speedup isn't that big, I'd estimate somewhere along 8-12% on my system. I suppose that mode 8 (which I also prefer) works OK, at least according to the difference debug log, so the SSE2 version can be used and I probably will.

8-12% is quite disappointing. The following formula should hold

SSE processing time - SSE mmemory access time = 2 * (SSE2 processing time - SSE2 mmemory access time)

Now in SSE2 mode, RemoveGrain tries to read and write 16 Bytes in one stroke, while in SSE mode only 8 bytes are read and wrote in one stroke. In 90-100% of all cases RemoveGrain reads from the L1 cache. However, reading and writing is usually unaligned. In SSE3 (Prescott P4) there is even a special instruction for unaligned reading from memory, which may be helpful.


I don't know if there are any developers who have a P4 system and could help..maybe it would be worth asking around a bit.

I think that by the end of the year I will buy a Prescott Celeron. Then I will certainly return to the SSE2 version.

Originally posted by dbzgundam

Why can't I access the page?

It's loading but I see a blank page.

Why are you young guys always so impatient? If you click on the above link, the server from www.AlpenNIC.com (http://www.AlpenNIC.com) maps the domain www.RemoveGrain.de.tf (http://www.RemoveGrain.de.tf) to a path on an Austrian server, where RemoveGrain and all my other plugins are hosted. As this server is advertisment free (however AlpenNIC generates some popups) and probably publicly funded, it is not the fasted (thousands of austrian home pages are hosted there). So you have to be patient. If it doesn't even work patience, I will give you a direct link.

[Boulder]

Originally posted by kassandro
8-12% is quite disappointing. The following formula should hold

SSE processing time - SSE mmemory access time = 2 * (SSE2 processing time - SSE2 mmemory access time)

Now in SSE2 mode, RemoveGrain tries to read and write 16 Bytes in one stroke, while in SSE mode only 8 bytes are read and wrote in one stroke. In 90-100% of all cases RemoveGrain reads from the L1 cache. However, reading and writing is usually unaligned. In SSE3 (Prescott P4) there is even a special instruction for unaligned reading from memory, which may be helpful.


That's probably theory and real life colliding:D The speedup might have (and probably would have) been bigger if RemoveGrain was the only thing eating CPU cycles, there's always other processes and Avisynth and CCE probably cause some loss to the actual gain.

[kassandro]

I just put up version 0.5 to the web site.
There have been substantial internal changes. The code for handling color spaces other than yv12 has changed, however the speed still remains surprisingly poor compared with the very satisfactory speed for yv12. The assembly code for mode=2,3,4 has been optimised. Here is a comparison of the old and new version on my machine (1.3 GHZ Celeron):

mode old version new version
2 149.5 fps 150.5 fps
3 141 145.5
4 142 149.5

I also made a comparison of trbarry's Undot and RemoveGrain(mode=1). It turned out that Undot and RemoveGrain(mode=1) differ for the pixels on the right border. By the very nature of the algorithm border pixels cannot be processed and should therefore be left unchanged. While Undot does just this for the left, the top and the bottom border. it makes a mistake on the right border, where instead of copying the last pixel on the line, it copies the penultimate pixel to the last pixel, which simply doesn't make sense and should be considered as a bug.

[Boulder]

Thanks, RemoveGrain's one of my standard filters for processing analog TV caps:)

EDIT: Has the SSE2 version changed in any way or am I better off using the SSE optimised one? I use mainly mode=8.

[Audionut]

In Vitualdubmod, I get an error,

Unable to load "C:\RemoveGrain.dll"
Same goes for the SSE2 version.

LoadPlugin("C:\PROGRA~1\ARCALC~1\AVS_Plugins\Mpeg2dec3.dll")
LoadPlugin("C:\RemoveGrain.dll")
Mpeg2Source("z:\*.D2V")
RemoveGrain(mode=4,mode=U2)

[Boulder]

Works fine for me.. did you try extracting the dll to the Avisynth plugins directory?

[Audionut]

I was missing a couple of dll's.

Got the same problem with the SSE2 version as Boulder.

[kassandro]

Originally posted by Boulder
Has the SSE2 version changed in any way or am I better off using the SSE optimised one? I use mainly mode=8.
The SSE2 version has changed in the same way as the SSE version (thus only for mode=2,3,4). Actually I use macros for the registers (for SSE I take the 64 bit mmx registers and for SSE2 I take the 128 bit SSE registers) and for the instructions, where SSE and SSE2 differ. Thus all changes effect both versions equally and simultaneously. Unfortunately there must be something wrong with this very economical way of programming.

Originally posted by Audionut
In Vitualdubmod, I get an error,

Unable to load "C:\RemoveGrain.dll"
Same goes for the SSE2 version.

You probably have not installed "msvcr70.dll" properly (see the installation section of www.AvsTimer.de.tf (http://www.AvsTimer.de.tf) for more details). You may also use the staticly linked version RemoveGrainS.dll instead. In fact, this is the only reason, why I provide it.

[kassandro]

Originally posted by Audionut
I was missing a couple of dll's.

Got the same problem with the SSE2 version as Boulder.

My response to your problem in the previous posting is obsolete (your posting was almost simultaneous). Thanks you for your test. I will buy an SSE2 capable cpu (2.66GHZ Prescott Celeron + motherboard with Intel chip set + 512 MB of DDR-RAM is now well below a very affordable 200 Euros) by the end of the year. Then I should be able to track down the problem.

[Audionut]

Originally posted by kassandro
My response to your problem in the previous posting is obsolete (your posting was almost simultaneous). Thanks you for your test. I will buy an SSE2 capable cpu (2.66GHZ Prescott Celeron + motherboard with Intel chip set + 512 MB of DDR-RAM is now well below a very affordable 200 Euros) by the end of the year. Then I should be able to track down the problem.

Thankyou, for this fine filter. Hope you can get the SSE2 version working, without many hassles.

[Audionut]

I love this filter.

6000 frame clip (Clean PAL DVD) single pass, const. quant 2, with no filter's returned 2979kbps bitrate.
With Undot(), returned 2828kbps
With removegrain(), returned 2662kbps bitrate.:D
With removegrain() and unfilter(-5,-5) returned 2328kbps.

Thankyou again.

[ARDA]

@kassandro

First of all thanks for the great work you're doing.
Second I want to make a question about a reference of Undot you've posted.

kassandro wrote:
By the very nature of the algorithm border pixels cannot be processed and should therefore be left unchanged. While Undot does just this for the left, the top and the bottom border. it makes a mistake on the right border,where instead of copying the last pixel on the line, it copies the penultimate pixel to the last pixel,which simply doesn't make sense and should be considered as a bug.
Please would you please be more specific about that bug; some time ago I've made some test with Undot sources and don't remember such a thing. Is it in YV12 or YUY2 colorspace?

Thanks ARDA

[kassandro]

Originally posted by ARDA
Please would you please be more specific about that bug; some time ago I've made some test with Undot sources and don't remember such a thing. Is it in YV12 or YUY2 colorspace?

First of all, the Undot bug is not severe. It doesn't cause a crash and concerns only the last pixel on a line. It is therefore hardly noticable. Nevertheless it is instructive to see how I uncovered the Undot bug. I started with the following script:

input=MPEG2Source("test.d2v")
Undot_clip=Undot(input)
RemoveGrain_clip=RemoveGrain(input, mode=1)
difference(Undot_clip, removegrain_clip)

The difference filter is taken from my AlignFields (http://www.AlignFields.de.tf) plugin. It reports for each frame the SAD difference as well as the number of different pixels to the debugview utility. Each pixel may be counted up to three times (Y, U, V values). Thus for a standard 720x576 clip there may be roughly up to 600000 different pixels for each frame.
Running the script with Vdubmod I was surprised to get the following output from debugview:

[1016] [129] total difference = 1, different Pixels 1
[1016] [130] total difference = 3, different Pixels 3
[1016] [131] total difference = 2, different Pixels 2
[1016] [132] total difference = 1, different Pixels 1
[1016] [133] total difference = 2, different Pixels 2
[1016] [134] total difference = 4, different Pixels 4
[1016] [135] total difference = 5, different Pixels 5
[1016] [136] total difference = 4, different Pixels 4
[1016] [137] total difference = 5, different Pixels 5
[1016] [138] total difference = 3, different Pixels 3

Thus there were differences between Undot and RemoveGrain(mode=1) but only very few. The border pixels were the immediate suspects (Undot and RemoveGrain require 8 adjacent pixels, whence border pixels cannot be processed and should therefore be left unchanged). To prove this I ran the following modified script:

input=MPEG2Source("test.d2v")
Undot_clip=Undot(input)
Undot_clip=crop(Undot_clip, 2,2,-2,-2)
RemoveGrain_clip=RemoveGrain(input, mode=1)
RemoveGrain_clip=crop(RemoveGrain_clip, 2,2,-2,-2)
difference(Undot_clip, removegrain_clip)

and fortunately I now got

[1016] [204] total difference = 0, different Pixels 0
[1016] [205] total difference = 0, different Pixels 0
[1016] [206] total difference = 0, different Pixels 0
[1016] [207] total difference = 0, different Pixels 0
[1016] [208] total difference = 0, different Pixels 0
[1016] [209] total difference = 0, different Pixels 0
[1016] [210] total difference = 0, different Pixels 0
[1016] [211] total difference = 0, different Pixels 0
[1016] [212] total difference = 0, different Pixels 0

Thus the differences between Undot and RemoveGrain(mode=1) are indeed on the border only. Next I asked the question: Are all border pixels handled differently or only a part? Running the following modified script:

input=MPEG2Source("test.d2v")
Undot_clip=Undot(input)
Undot_clip=crop(Undot_clip, 0,0,-4,0)
RemoveGrain_clip=RemoveGrain(input, mode=1)
RemoveGrain_clip=crop(RemoveGrain_clip, 0,0,-4,0)
difference(Undot_clip, removegrain_clip)

I still got the zero difference ouput

[1016] [251] total difference = 0, different Pixels 0
[1016] [252] total difference = 0, different Pixels 0
[1016] [253] total difference = 0, different Pixels 0
[1016] [254] total difference = 0, different Pixels 0
[1016] [255] total difference = 0, different Pixels 0
[1016] [256] total difference = 0, different Pixels 0
[1016] [257] total difference = 0, different Pixels 0
[1016] [258] total difference = 0, different Pixels 0
[1016] [259] total difference = 0, different Pixels 0
[1016] [260] total difference = 0, different Pixels 0
[1016] [261] total difference = 0, different Pixels 0

Thus only the border pixels on the right side are processed differently. Having located the differences I now turned to the Undot source code and found the following cause:

// do last qword
lea esi, [esi+eax-8-BPP] // point at last qword
movq mm0, qword ptr[esi+ebx] // move 1st 4 pixel
movq qword ptr[edi+eax-8], mm0

The comment "point at last qword" is incorrect. Because of -BPP=-1 he doesn't "point at the last qword". Removing "-BPP" from the source code, Undot and RemoveGrain(mode=1) should become identical. Right now, because of -BPP, Undot copies the penultimate pixel of a line of the input frame to the last pixel of the same line of the output frame, which may improve compression a tiny little bit but is simply not correct and was not intended either.

[Boulder]

Heh, that's one nice post about troubleshooting:D Maybe you should notify trbarry so that he might fix the bug once he has the time. I haven't seen him in a while though.

[Fizick]

Tom Barry can be notified by e-mail (recently I inform him about other filter).

Kassandro, may I to use some part of your RemoveGrain code in my DeGrainMedian code ? :)

[ARDA]

@kassandro
Thanks for the long explanation, I think it is very instructive; at least for me.
I've borrowed this function modified a little and moving the slide in virtualdub could see the difference between your plugin and undot on the right side of amplified difference; and as you say it's hardly noticeble.


clip=MPEG2Source("mysource")
v1 =clip.UnDot
v2 =clip.RemoveGrain(mode=1)
sub = v2.subtract(v1)
substrong = sub.levels(122,1,132,0,255)
v3 = StackVertical(StackHorizontal(substrong.subtitle("Difference amplified"),
\v1.subtitle("UnDot")),StackHorizontal(sub.subtitle("Difference"),v2.subtitle("original")))
return v3

Thanks again for this great work.

ARDA

[kassandro]

@Boulder:
Yes, I haven't seen trbarry here for quite a while. He seems to be busy with other things. I will draw his attention to this slight problem, when he returns.

@ARDA:
Thanks for the nice sript. Actually, my input source was not very suitable for the test this morning. It had a black stripe on the right hand side. Thus the penultimate and the last pixel on a line were almost always identical. Only noise did give small differences. Cropping away the black stripe, your script displayed the differences nicely. Actually it shows that the chroma difference effects the last two pixels on the right hand side.
Since trbarry's assembly routine is nearly the same for YUY2, but with BPP=2 instead of BPP=1 for YV12, the same copying mistake is made for YUY2. On the other hand for YUY2, Undot and RemoveGrain(mode=1) cannot be compared, because Undot doesn't process the chroma of YUY2 clips. He could have done it with nearly the same assembly routine and BPP=4, but more attention at the border would have been necessary. Thus Undot and RemoveGrain(mode=1, modeU=0) are nearly identical for YUY2, but for YUY2 Undot is much faster.

[kassandro]

Originally posted by Fizick
Kassandro, may I to use some part of your RemoveGrain code in my DeGrainMedian code ? :)
Sorry, Fizick, for having overlooked your posting yesterday, which I cannot understand, because it was just between Boulder's and ARDA's last posting.
Of course you can use my code. However, as I mentioned earlier, the programming style is very different from trbarry's. Because I have always SSE2 in mind, I use macros where SSE and SSE2 differ, which makes the code more difficult to understand. Understanding is also hampered by my passion to overoptimise. Though the SSE2 code doesn't work currently, I should be able to fix it easily once I have a SSE2 capable cpu. In the end, we all have to switch to SSE2, because the Athlon64 platform, which will abolish the current 32 bit platform within 2 years, because Intel has adopted it, has no mmx registers anymore in 64 bit mode. Rather it has 16 SSE 128 bit registers. It would have been much better to have 8 256 bit registers instead, with which one could easily double dct/idct performance and process 32 pixels instead of 16 simultaneously with various Avisynth filters.

[Fizick]

Kassandro,
Thanks for your permission!
I am quite easy understand your code. It is very well optimized and structured.
I already put it to my filter beta (will be released soon, of course under GPL). But i remove SSE2.

About SSE2. I think its time is not come for the present
(not compatible with Athlon XP, so SSEMMX is standard now). I do not think, that SSE2 speed can be used in every filter. And the speed is not most important thing, but algo and quality (for me). But the next generation of filter-writers will use SSE4 of course! :)

BTW, why you do not like Adobe so much? ;)

[kassandro]

If there are no special restrictions, like in RemoveDirt, where the block size is fixed, then one should be able to produce a SSE and an SSE2 version simultaneously by using some simple macros for registers and some load and store instructions. I just learned that there is one important difference, which I am was completely unaware, when I made RemoveGrain. For 128 bit memory operands of instructions like paddusb, pminub etc. the memory operand must always be aligned (i.e. the address must be a multiple of 16), while 64 bit operands as used for SSE are allowed to be unaligned. I have to blame Intel for my ignorance, because Intel always speaks only of 128 bit memory operands and not of 128 bit aligned memory operands. Only when Intel discusses exceptions, which I never read, because I simply do not expect such exceptions for my programs, then one can read, that a memory alignment error triggers a read access exception, as reported by Boulder. Fortunately I read at least one time the exception stuff and then everything was obvious for me. Now it is my programming style to load into a register, if the data is used more than once (that is a key difference between Undot and RemoveGrain(mode=1)), because only one instruction with a memory operand can be executed at a time (that is the reason why RemoveGrain(mode=1) is about 4 fps faster than Undot). Thus for some modes I don't use at all instructions with a memory operands and these were exactly the modes reported to work by Boulder.
I just uploaded a new SSE test version (it must be called with DRemoveGrain) to the web site. The other parts of the binary archive as well as the source archive have not changed. The SSE2 problem should now be fixed, though I may have overlooked 1 or 2 instructions with a memory operand.

[Boulder]

Great, I'll have a go ASAP!

[Boulder]

OK, I tested the SSE2 version. It looks like all the other modes except number 4 work and produce no differences compared to the SSE version, viewed with DebugViewer. If I use mode=4, VirtualDubMod just closes without giving any error messages whatsoever.

Here's another bug as well, I noticed it earlier but forgot to post:scared: If I use modeU=-1 to disable chroma processing, I first get a screen like this:


http://www.saunalahti.fi/sam08/rg_1.jpg


After scrolling for a few dozen frames in VDubMod, the output turns like this (notice the odd ghosting) :

http://www.saunalahti.fi/sam08/rg_2.jpg

If I use modeU=0, everything's OK. With mode=-1, I also get weird results. This occurs on both SSE and SSE2 version.

[kassandro]

Originally posted by Boulder
OK, I tested the SSE2 version. It looks like all the other modes except number 4 work and produce no differences compared to the SSE version, viewed with DebugViewer. If I use mode=4, VirtualDubMod just closes without giving any error messages whatsoever.

I looked again at the source code and the SSE2 bug seems to be corrected. On the other hand, there have been a lot of changes meanwhile and the current SSE2 binary is only a snapshot. In fact, the next version 0.6 will contain a new plugin, which is derived from the same source file, such that both plugins can be maintained simultaneously. Unfortunately, the technique to creat different plugins from the same source code has one drawback: the source code gets a little bit messy.


Here's another bug as well, I noticed it earlier but forgot to post:scared: If I use modeU=-1 to disable chroma processing, I first get a screen like this:

This is not a bug. If you want to leave the chroma unchanged, you have to use modeU=0. If you want the chroma to be not processed at all (not even copied), which is faster, you should choose modeU=-1. If modeU=-1, then the chroma becomes random (in your case green and then it changes again). Why modeU=-1?. If you have black&video, you can save time if you use modeU=-1 and then at the end you simply use the greyscale command. The chroma is automaticly erased if RemoveDirt is used with grey=true.
If you you use mode=-1, then the luma becomes random as well. Of course, this doesn't make sense even for b&w material.

[Boulder]

Originally posted by kassandro

This is not a bug. If you want to leave the chroma unchanged, you have to use modeU=0. If you want the chroma to be not processed at all (not even copied), which is faster, you should choose modeU=-1. If modeU=-1, then the chroma becomes random (in your case green and then it changes again).

Oh, then it was a mistake on my part. I actually thought that using modeU=-1 ended up in a b&w clip, but that has to be done either with RemoveDirt(grey=true) or Greyscale().

When can we expect the v0.6? :)

[kassandro]

Originally posted by Boulder
Oh, then it was a mistake on my part. I actually thought that using modeU=-1 ended up in a b&w clip, but that has to be done either with RemoveDirt(grey=true) or Greyscale().

From this design you can see that RemoveGrain was designed as a precleaner for RemoveDirt. Unfortunately the very sophisticated RemoveDirt becomes more and more obsolete. The new plugin derived from the RemoveGrain source, currently called Repair, leads to a much more compression efficient way of cleaning: firstly I apply a very simple and very fast temporal clenser (essentially RemoveDirt without any artifact protection), then the Repair plugin compares the clensed with the original frames and instead of removing grain it removes the massive clenser artifacts (unfortunately it restores also medium and big temporal dirt). Finally RemoveGrain is applied to erase some left over. While this filter, called RemoveDust, can never remove medium or big dirt like RemoveGrain, it is much more efficient on dust than RemoveDirt and much more efficent on grain than RemoveGrain. My first tests show very remarkable compression gains with only slight motion blurring. The static parts have the same softness as the corresponding RemoveGrain modes.
Now already, Fizick in his denoiser plugin, made a temporal extension of RemoveGrain mode 5-9, but for mode=9 (trbarry's ST-median) an additional change limitation is necessary to avoid very unpleasant artifacts. While this temporal extension makes a lot of sense, the gain over the purely spatial variants is limited. Like RemoveGrain with mode >=5 it can never remove grain consisting of three equal pixels on a line segment, even if this piece of grain is only on one frame. Thus Fizick can't really exploit the temporal nature of grain either. That will change with RemoveDust.
I will supply various compression comparisons of RemoveDust with RemoveGrain and DeGrainMedian. A comparison with convolution based smoothers doesn't make sense to me.


When can we expect the v0.6? :)
The most difficult work is the documentation and of course a lot of tests are necessary. I hope to release it within one to three weeks.

[Boulder]

Originally posted by kassandro
From this design you can see that RemoveGrain was designed as a precleaner for RemoveDirt. Unfortunately the very sophisticated RemoveDirt becomes more and more obsolete. The new plugin derived from the RemoveGrain source, currently called Repair, leads to a much more compression efficient way of cleaning: firstly I apply a very simple and very fast temporal clenser (essentially RemoveDirt without any artifact protection), then the Repair plugin compares the clensed with the original frames and instead of removing grain it removes the massive clenser artifacts. Finally RemoveGrain is applied to erase some left over. While this filter, called RemoveDust, can never remove medium or big dirt like RemoveGrain, it is much more efficient on dust than RemoveDirt and much more efficent on grain than RemoveGrain. My first tests show very remarkable compression gains with only slight motion blurring. The static parts have the same softness as the corresponding RemoveGrain modes.

Sounds interesting, it will once again be one to try for me. I just hope you'll be able to keep one of the strong points of RemoveDirt - the incredible speed;) Do you have any estimates on when it might be released? I'll have 2-4 analog TV capture clips to encode weekly so I might find the filter very useful for my purposes.

[kassandro]

Originally posted by Boulder
Sounds interesting, it will once again be one to try for me. I just hope you'll be able to keep one of the strong points of RemoveDirt - the incredible speed;) Do you have any estimates on when it might be released? I'll have 2-4 analog TV capture clips to encode weekly so I might find the filter very useful for my purposes.
You may download the current Repair.dll from (it contains also the clenser "Clense") from deleted and use it within the following function

function RemoveDust(clip input, int _mode, bool _grey)
{
repmode=2
clensed = Clense(input, grey=_grey)
rep=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode)
return RemoveGrain(rep, mode=_mode, modeU = _grey ? -1 : _mode)
}

I currently run my second big test with RemoveDust(4, false) (false= color input). My first big test the movie "Heimatland" broadcasted yesterday on the ARD channel did crunch this movie with xvid 1.0.1 1 pass quality mode with quantiser=4 below 700 Mb, while without RemoveDust it was almost 1200 Mb. It was a dusty digital DVB capture with a lot of grain from 1955. With repmode=1 you get higher compression but motion is blurred more.

Edited: the quantiser was 5 not for 4.

[Fizick]

Kassandro,
It is very interesting evolution of your "Remove..." plugins.

it can never remove grain consisting of three equal pixels on a line segment, even if this piece of grain is only on one frame.
These three equal pixels can be not grain, but some object't edge...:confused:

Thus Fizick can't really exploit the temporal nature of grain either.
Poor Fizick :)

I will supply various compression comparisons of RemoveDust with RemoveGrain and DeGrainMedian.
It is interesting.
But i think, I know results right now: for different sources we must use different plugins.

The more we have, the better.

Good luck!

[kassandro]

Originally posted by Fizick
These three equal pixels can be not grain, but some object't edge...:confused:

Well, that is the question! If you are that conservative, you never can remove scratches. If these three pixels do not appear on one of the two neighbour frames, they are removed by the clenser. Now Repair works as RemoveGrain. However, the center pixel is taken from the clensed frame, while the eight neighbours are taken from the original frame and the center pixel is clipped by one pixel pair of these eight neighbours as in RemoveGrain (well, that's only the almost-truth). It's a very simple but remarkably good artifact removal. It results only in some motion blurring (the amount depends on the repair mode), but the big artifacts removed. Unfortunately, the big dirt is restored too.
Repair can also be used to remove sharpening artifacts of a sharpener or at least makes a sharpened clip more compression friendly. Of course it gets a little bit softer too.



But i think, I know results right now: for different sources we must use different plugins.

That is certainly true. But as a collector of old films my sources are always of the same type. Their origin is old chemical non-anime film, mostly black&white and I receive them over digital sat-tv by recording the transport stream. The most significant difference is, whether these progressive clips are messed up (shifted fields) or not. My denoiser/cleaner results are always for these kinds of sources.

[Audionut]

Mode=4 in sse2 version gives me an avisynth read error: avisynth: caught an access violation at 0x017a14c2, attempting to read from 0xffffffff

Everything else works like a charm.
Thanks again for your work.

[Boulder]

Originally posted by kassandro
It was a dusty digital DVB capture with a lot of grain from 1955. With repmode=1 you get higher compression but motion is blurred more.


I tried it on a small TV capture clip, not particularly noisy but more than enough to annoy. It appears that RemoveDust is quite a bit more aggressive than RemoveDirt, so I hope there will be a mode which would be something between RemoveDirt with its current defaults and repmode=2. I didn't try to watch the result on my TV since I don't have any DVD+RW's to spare at the moment, but on a frame-by-frame comparison, the motion blur looked a little too aggressive. Compressibility-wise RemoveDust wins hands down;)

As I said, it looks very interesting, and I'll be following the development closely.

[kassandro]

Originally posted by Boulder
I tried it on a small TV capture clip, not particularly noisy but more than enough to annoy. It appears that RemoveDust is quite a bit more aggressive than RemoveDirt, so I hope there will be a mode which would be something between RemoveDirt with its current defaults and repmode=2. I didn't try to watch the result on my TV since I don't have any DVD+RW's to spare at the moment, but on a frame-by-frame comparison, the motion blur looked a little too aggressive. Compressibility-wise RemoveDust wins hands down;)

While my first two big tests were quite satisfactory (both were quite soft color movies from fifties), the third big test, which I ran over night was less satisfactory. The source was a very sharp Olsen brother film from the seventies. Knowing that RemoveGrain(mode=4) doesn't like sharp movies, I cautiously took mode=8 instead of mode=4. The output was fairly crisp (RemoveGrain(mode=8) sharpens edges), of course, with some motion blurring, the compression gain was much less impressive (I expected this to a certain extend), but what really hit me hard was the enormous thin line flicker (that certainly worsened compression). Already RemoveGrain(mode=4) not only destroys thin lines like most spatial denoisers but also creats thin line flicker (on one frame a thin line is preserved, on the next it is removed) unlike convolution based denoisers, which however blur more. However, the flicker of RemoveDust(mode=8) was much stronger than that of RemoveGrain(mode=4). RemoveDust seems to strongly amplify the good but also the bad properties of RemoveGrain. Your analog captures are certainly less sharp than my digital ones. Hence the flicker problem should not be such a big issue for you. If you want less motion blurring, then you should try repmode=3. In general, all modes of RemoveGrain are available for Repair. The more aggressive mode is for grain removal the more aggressive it is for artifact removal, but repmode=4 is far too aggressive, it almost completely abolishes the clenser and the output looks like applying RemoveGrain(mode=4) twice, which isn't good either. Also repmode=9 doesn't seem to be a great idea. I have added a new mode 10, which is also available in the recent RemoveGrain SSE2 test version, which you gratefully have tested. It seems to be between mode=1 and mode=2.

The flicker problem forced me to think almost the entire day about a fast, conservative flicker filter and I am coding the first lines right now.

[trbarry]

kassandro -

Nice spotting on the UnDot bug. As others have mentioned I've been away recently and right now don't even have access to my web page, which is going to relocate soon. But trbarry.com is redirected and it should be back up at a new location (with an UnDot and some STMedianFilter fixes) in a couple days.

- Tom

[Fizick]

trbarry,
we will wait your return :)

[kassandro]

Tom,
thank you for your friendly comment. I learned SSE/MMX programming from the source code of your plugins, especially Tomsmocomp. You really can't learn it from the Intel manuals, which can only be used for look up. Before I saw from your plugins how virtuosely it can be used, the integer SSE/MMX didn't make much sense to me.
Having said that, I did found another problem with your STMedian filter, which has also propagated to Fizick's DeGrainMedian. For each output frame you need three input frames, the previous, the current and the subsequent frames and you tacitly make the assumption that all three frames have the same pitch for each plane. On the other hand, your own filter Uncomb may creat a clip, where frames do not all always have the same pitch, if Crop was applied before. In order to improve performance, Uncomb simply passes through frames without combs. On the other hand combed frames are replaced, by frames created with env->NewVideoFrame(vi), which have a smaller pitch, if the child clip was cropped substantially. Now, the reason for assuming that all frames have the same pitch was not negligence, rather it was the register shortage of the Intel architecture. Intel only gives you 6 registers. On the other hand, you have to keep the pointer for the four frames in registers and the pitches for all three source frames. Even, if you decide to reload the pointer of the destination frame each time, you then need one register for reloading. Thus you still cannot keep all the three pitches for the three source frames in registers. Once you start reloading, you have to do even more reloading, which will then have a significant impact on performance. To cope with this ugly problem, I would wrap around a class, say, Homogenate around the child clip, which does the following: if the frame has the standard pitch for each plane, i.e. the same pitch as a frame created by env->NewVideoFrame(vi), then the frame from the child clip is passsed through to STMedian unchanged (this is the usual case). If the frame has not the standard pitch, a new frame is created with env->NewVideoFrame(vi) and the child frame is copied to this new frame, which is then processed by STMedian. Proceeding in this way not only the three source frames have the same pitch, but also the destination frame.

[trbarry]

Yuk! You are right but that is more elaborate a change than I have time to make right now, even though I do have 2 other STMedianFilter bug (and suggested fixes) I have to apply.

I guess a quick workaround would be to precede STMedianFilter by UnDot or UnFilter in the above case. Either runs almost instantly and will copy all the frames causing a constant pitch. UnDot is probably preferable here. Something like UnFilter(-7,-7) instead works best immediately following STMedianFilter() to get better compressibility.

Or I suppose you could just crop after UnComb instead of before. But I'll try to look at some better fix that doesn't involve waiting for Athlons-64's with more registers. ;)

- Tom

[Fizick]

Kassandro,
Of course I know about this pitches limitation in Stmedian and degrainmedian. But I forget to note it in doc, sorry all.
Now I use only simplest workaround to prevent possible memory access crash - degrainnmedian uses minimal pitch of 3 frames.
I think it is very rare situation. Nobody found problem in real coding with STMedian for about 1 year.
If somebody will post bug report, i will think about it if have a time.

[kassandro]

Originally posted by Fizick
Kassandro,
Of course I know about this pitches limitation in Stmedian and degrainmedian. But I forget to note it in doc, sorry all.
Now I use only simplest workaround to prevent possible memory access crash - degrainnmedian uses minimal pitch of 3 frames.
I think it is very rare situation. Nobody found problem in real coding with STMedian for about 1 year.
If somebody will post bug report, i will think about it if have a time.
I agree, it is very unlikely that the constant pitch assumptions causes problems, but when it does it will be difficult to spot. Nevertheless one should not reload the pitches all the time. Though I never had the necessity, I will write a class which guarantees frames with standard pitch, i.e. the pitch of frames returned by env->NewVideoFrame(vi). That class should also have the option, to require the frames to be properly aligned. This is particularily imüportant for SSE2, where an even more significant gain if access is aligned.

[kassandro]

I just put up version 0.6 to the web site. As anounced, there is now additional plugin Repair containing the filters Repair and TemporalRepair. I have also added the filter Clense RemoveGrain and there are the new modes 10, 11 for RemoveGrain. Motion blur and flicker of RemoveDust has been reduced substantially using TemporalRepair.

[Audionut]

Thanks kassandro.

Can you give me an example of how to use repair/temporalrepair.
I really do not understand the examples given at the download page.

I use this script.

Mpeg2Source("C:\JEEPER~1\VIDEO_TS\JEEPER~1.D2V")
removegrain(mode=4, modeU=2)
removedirt()
Crop(8,74,-8,-74)
a=trim(0,1769).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
b=trim(1770,141093).Lanczos4Resize(704,288)
c=trim(141094,149818).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
a+b+c
removegrain()
Unfilter(-7,-7)

[kassandro]

Originally posted by Audionut
Thanks kassandro.

Can you give me an example of how to use repair/temporalrepair.
I really do not understand the examples given at the download page.

I use this script.

Mpeg2Source("C:\JEEPER~1\VIDEO_TS\JEEPER~1.D2V")
removegrain(mode=4, modeU=2)
removedirt()
Crop(8,74,-8,-74)
a=trim(0,1769).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
b=trim(1770,141093).Lanczos4Resize(704,288)
c=trim(141094,149818).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
a+b+c
removegrain()
Unfilter(-7,-7)

Wow, you are doing a lot of blurring. Let me explain TemporalRepair.
Say you have clip input, which is processed by a filter and let's call the resulting clip filtered. Now let's call y(input,n,x,y) the luma of the pixel with coordinates x,y of frame n of clip input and y(filtered, n,x,y) the same pixel but in the filtered clip. Then TemporalRepair first calculates m(input,n,x,y)=min(y(input,n-1,x,y), y(input,n,x,y), y(input,n+1,x,y)) and M(input,n,x,y)=max(y(input,n-1,x,y), y(input,n,x,y), y(input,n+1,x,y)). Then y(filtered,n,x,y) is clipped at m(input,n,x,y) and M(input,n,x,y), .i.e. min(M(input,n,x,y), max(m(input,n,x,y), y(filtered, n,x,y))). In other words, if the filter has changed the luma of the pixel so much such that it got out of the range spanned by m(input,n,x,y) and M(input,n,x,y) it is clipped back into this range. Repair(mode=1) works similar but spatial instead of temporal. Instead of 3 temporally adjacent pixels, one has now 9 spatially adjacent pixels. Though it is not quite true, one should think about Repair the following way: it works just like RemoveGrain, but with two instead of one clip and the center pixel is taken from the first clip, while the neighbour pixels are taken from the second clip. Actually this thought has evolved into a general concept of deriving artifact removing filters from cleaning filters.
I hope this helps. Everybody is wellcome for ideas to illuminate this concept.

[Audionut]

Originally posted by kassandro
I hope this helps.

Not really, unfortunally.
I simply do not understand all that a,b,x,y stuff.:(

If possible, can you or someone else, edit my above script, to include repair.

Thanks.

Originally posted by kassandro
Wow, you are doing a lot of blurring.

The results are quite good at quant 5.
http://www.queenslander.net/users/lioneldavey/test10.jpg

Of course, ffdshow's sharpening and add noise, improve playback quality.

[kassandro]

Originally posted by Audionut
Not really, unfortunally.
I simply do not understand all that a,b,x,y stuff.:(

Hmm, to understand what is going on here, you should first try to understand how trbarry's Undot = RemoveGrain(mode=1) works: the luma of a pixel, called the center pixel in the sequel, is compared with that of its eight neighbours. If it is larger than the minimum of the eight neighbours and smaller than the maximum of the eight neighbour, then the pixel is left unchanged. If it is smaller than the minimum of the eight neighbours than the luma of the center pixel is replaced by the minimum of the eight neighbours. If it is larger than the maximum of the eight neighbours than the luma of the center pixel is replaced by the maximum of the eight neighbours. Quite natural, isn't it? Now, once you have understood this simple idea, we can continue to explain RemoveGrain, Repair, ST-Median, DeGrainMedian, which are all more or less subtle refinements of this basic idea.

[krieger2005]

Hi kassandro,

i thought you want write a mode=10, which is something between 1 and 2? Is this history?

--- EDIT:
I see, your RemoveGrain 0.6 does this... sorry

[kassandro]

Originally posted by krieger2005
Hi kassandro,

i thought you want write a mode=10, which is something between 1 and 2? Is this history?


mode 10 didn't live up to my expectations. When I combined Repair(mode=10, limit=5) with Sharpen(1.0) it did perform much worse than mode=1,2. Fortunately, together with my forthcoming plugin AmplifyDifference it gives a nice, fast, edge friendly sharpener. Thus mode=10 is not completely useless.
To have more choice I will make Repair specific improvements for mode=3 in the next version as I did already for mode 1,2,10. Currently Repair(input, input, mode=3) is identical to RemoveGrain(input, mode=3) because if mode is not 1,2,10 the center pixel of the second clip is not used for artifact removal and this is not optimal.

[kassandro]

"RemoveTemporalGrain" would be a much more appropriate name for the script function "SharpRemoveGrain" in the RemoveGrain. Of course RemoveGrain(mode=4) looks much softer than SharpRemoveGrain(mode=4), because the latter cannot make changes in static areas of the frame. On the hand RemoveDirt/Clense can only remove grain if that grain is absent on both adjacent frames, while SharpRemoveGrain can already remove grain, if it is absent on one adjacent frame. Of course, unlike RemoveDirt/Clense SharpRemoveGrain can only remove very very thin dirt. RemoveDust can remove substantialy thicker grain and dust at the expense of some motion blurring. All in all SharpRemoveGrain is a very conservative denoiser and should only be used with mode=4. It will be renamed "RemoveTemporalGrain" next time.

I also tested the deinterlacer RGDeinterlace. In the compression benchmark that deinterlacer was almost as good as AlignFields(mode=2). However, the ticker tape test was a desaster. Now this test where an interlaced fast moving stock market ticker tape has to be deinterlaced is a tough cookie for any smart deinterlacer. So far any motion adaptive sharp deinterlacer /including the most recent TDeint)shows substantial artifacts under 400% Vdub magnification, but as a blur deinterlacer RGDeinterlace should have had it easier, but it produced the worst results I have ever seen in this test. TemporalRepair simply restores too many pixels which it shouldn't. Without this problem RGDeinterlace would probably beat AlignFields(mode=2) as far as compression is concerned, but at the moment it cannot be recommended for real use.

[kassandro]

I just put up a new version of my RemoveGrain package to the web site (http://www.removegrain.de.tf). The SSE2 version should work fine now (no test version anymore). There are now even SSE3 versions of the plugins. Though Intel added only one new ISSE instruction to SSE3 this single instruction makes a big difference. Though I didn't have time for extensive benchmarks, it is justified to say that SSE3 versions delivers what the SSE2 promised but didn't deliver, namely almost doubling the performance of the SSE version. The performance difference between the SSE3 and the SSE2 version is much larger than between the SSE2 and the standard SSE version.
The remarkably poor performance (already realised by Boulder) of the SSE2 version is a consequence of the movdqu instruction used for unaligned reading of 16 bytes into SSE registers. The instuctions really stalls the cpu. Once it is replaced by lddqu in SSE3 unaligned "almost" sequential reading of 16 byte chunks is almost as fast as alinged reading. In this way Intel really removed a bottlekneck. For Athlon64 and also the Sempron 3100, which also have SSE2, movdqu and whence the SSE2 version may perform much better. The SSE3 version can only be used on Prescott P4s/Celerons.
With my new system I have also upgraded to Visual Studio 2003, which may result in missing DLLs.
The SSE3 version can only be compiled with the Intel Compiler (the Microsoft is simply too old). Because the Intel Compiler generates larger executables than the Microsoft compiler) the SSE3 files are larger than the SSE files.
Various new modes have been added as well. The limit variables available for some modes have been disabled, because they turned out not to be really useful.

[Boulder]

Humm, looks like RemoveGrainSSE2.dll is the same as in v0.6. That is, mode 4 doesn't work and there is no mode 12. Unfortunately I can't test the SSE3 version as I have a Northwood P4.

[kassandro]

Originally posted by Boulder
Humm, looks like RemoveGrainSSE2.dll is the same as in v0.6. That is, mode 4 doesn't work and there is no mode 12.
Sorry, Boulder, but you see it from the date of the SSE2 version, that this was the old version. I did test the SSE3 version extensively since the beginning of February and millions of frames have been processed with various modes of RemoveGrain/Repair. There should be no problem with the SSE2 version because it is almost identical with the SSE2 version. I did finish updating the dicumentation yesterday evening and then I did compile the various version (except the SSE3 version, where I did take just the ones which I have used for quite a while). Unfortunately it was a little bit too late, when I pulled the files into the archive and the old SSE2 simply remained there and got uploaded again. I have now replaced the old version and uploaded it. Modes 1-14 should work now.

Unfortunately I can't test the SSE3 version as I have a Northwood P4.
In general I would say that the Prescott CPU design is the worst, which Intel ever made, but with the SSE3 versions of RemoveGrain/Repair it really shines. I would have loved to buy a Sempron 3100 instead of a Prescott CPU, but it was simply too expensive. A boxed Sempron 3100 did cost 110€ and a suitable main board a further 70€, while I payed only 155€ for a boxed Celeron 330D + P4V88 Mainboard + 512 MB DDR400 RAM. Nevertheless I would be interested how the SSE2 version does relative to the standard version on the various Athlon64 and Sempron 3100 systems. The questions is whether the movdqu is equally poor on AMD systems as it is on Intel systems.

[Boulder]

From what I've heard, SSE2 code runs pretty much slower on AMD's processors in some applications.

I don't know how Prime95 works, but I'd say that program is the best optimized single executable for Intel's SSE2..I do recall saying that the biggest difference is the registers, the amount or the size, don't remember which one.

[kassandro]

I just made a short look the source code of Prime95. It is an assembler monster. However, it seems to use SSE only for FFT type problems. In particular, only floating point SSE seems to be used in Prime95, while we RemoveGrain and some other avs filters use integer SSE. Only DGDecode/Mpeg2Dec3 and Fizick's FFT filters seem to use floating point SSE. SSE3 brings substantial improvements for FFT/DCT/IDCT (most of the new instructions are only useful for this purpose). Thus the above plugins should profit from it, once they specificly use SSE3. I will also release a filter called SSETools (formerly I called it AmplifyDifference), which uses floating point SSE. I changed the name after adding a general color transform filter for YV12 and YUY2, which ccontains all the ColorMatrix transformations as very special cases and should do it more precisely. It will be interesting to compare performance with Tritical's mmx version of the ColorMatrix filter (YV12 only), which uses integer arithmetic only.
However, a new version of RemoveDirt will come first.

[len0x]

Does SSE3 version require any dlls or it is staticly linked? (I have no idea how intel compiler works in this case).

[kassandro]

Originally posted by len0x
Does SSE3 version require any dlls or it is staticly linked? (I have no idea how intel compiler works in this case).
No, it is dynamically link (I hope).

The SSE3 version had to be compiled with the Intel compiler and I don't know this compiler well enough. Unfortunately, the Intel compiler does not integrate into the german Visual Studio GUI.

The library mscvcr71.dll should be enough. Version 0.6 was compiled with the english Visual Studio 2002. It required mscvcr70.dll. Unfortunately each version of Visual Studio requires a different msvcrxx.dll.

[len0x]

Originally posted by kassandro
No, it is dynamically link (I hope).


But if its intel compiler then MSVC libraries should not matter, right? What are the other libraries that might be required by intel compiler?

[kassandro]

Originally posted by len0x
But if its intel compiler then MSVC libraries should not matter, right? What are the other libraries that might be required by intel compiler?
As far as I know, under Windows the Intel compiler uses the Microsoft libraries, under Linux it uses its own libraries. On the other hand, Intel binaries are 50% larger than the Microsoft ones. Perhaps I didn't use the optimal compiler options to get the file size down. At the moment I have not the time to explore the Intel compiler (version 8.1) any further.

[Fizick]

Only DGDecode/Mpeg2Dec3 and Fizick's FFT filters seem to use floating point SSE.
I only use external FFT library with float SSE.
However VagueDenoiser has 3DNow! and float SSE code by Kurosu.

[kassandro]

Originally posted by Fizick
I only use external FFT library with float SSE.

Yes, I thought the it is neccessary to use float SSE in FFT dll, because you can no more use integer approximations (the usual way in Avisynth to avoid floating point arithmetic) in a big FFT, otherwise accumulation of approximation inaccuracies would just give garbage.





However VagueDenoiser has 3DNow! and float SSE code by Kurosu.
I don't know the VagueDenoiser details, but I thought that VagueDenoiser uses a "small" transformation on 8x8 blocks and for these integer approximation should be faster and reasonably accurate. With SSE2 you could process even 8 pixels simultaneously by 16 bit integer approximation.

[tsp]

it seeems as both NASM and MASM support SSE3.
http://www.intel.com/cd/ids/developer/asmo-na/eng/167741.htm?prn=Y

[kassandro]

Originally posted by tsp
it seeems as both NASM and MASM support SSE3.

Thanks, for the information, but I prefer inline assembler over using an external assembler. Firstly, external assemblers require a lot of administrative overhead like segment directives etc. Secondly, with an external assembler using function arguments is difficult and may even become messy, if changes are made afterwards. Similarily, accessing class elements is easy with inline assembler and almost impossible with an external assembler. On the other hand, an external assembler has the big advantage that one can nearly always use the ebp register. Unfortunately, the Microsoft compiler insists on accessing stack variables with the ebp register instead of the esp register for function with inline assembler (the /Oy option is ignored for such functions). Surprisingly, for pure C++ function it can use the ebp register for optimisation and uses the esp register to accesses stack variables, if the /Oy option is used.

[Didée]

Sorry to interrupt the technical talk ...

Thanks for the new version, kassandro! After reeading the doc for a longer while, I think I found all new features ;)

Could you elaborate a little on the "smooth" switch? It sounds interesting, but "considers also fluctuations of neighbor pixels" is a rather vague description. Just a small hint or two, about what is actually compared, figured and done? :)

I ask because some time ago, I happened to play with a similar idea:

(Forgetting about foreground/background motion transitions or such things,) The difference of a pixel to its temporal neighbors should be rather similar for all pixels in a small neighborhood. Thus, the [spatial] difference between the temporal pixel differences should be sort of an indicator for the actual local noise. Hence I tried to even out those temporal differences for all pixels in a 3x3 neighborhood. But the milage was pretty poor: the frame got more blurring than the noise reduction was worth...

[kassandro]

Originally posted by Didée
The difference of a pixel to its temporal neighbors should be rather similar for all pixels in a small neighborhood. Thus, the [spatial] difference between the temporal pixel differences should be sort of an indicator for the actual local noise.

I had the same idea in mind, when I designed TemporalRepair with smooth=1 (actually in the source code, there is also smooth=2, 3 but I disabled it, because it doesn't improve much).
To describe the details, let me first recall the old TemporalRepair(fclip, oclip, smooth=0). Here fclip is the filtered and oclip is the original clip. Let further f(n,x,y) (resp. o(n,x,y)) the luma of the pixel with coordinates x,y of the n-th frame of fclip (resp. oclip). Finally let Min=Min(n,x,y)=min(o(n-1,x,y), o(n,x,y), o(n+1,x,y)) and Max=Max(n,x,y)=max(o(n-1,x,y), o(n,x,y), o(n+1,x,y)). Then in the output of TemporalRepair(fclip, oclip, smooth=0) the pixel f(n,x,y) is clipped at Min and Max, i.e. it is replaced by min(max(f(n,x,y), Min), Max). Now with smooth=1 the interval for clipping is enlarged. To this end let DMin(n,x,y)=o(n,x,y)-Min(n,x,y) and DiffBelow(n,x,y)=max(DMin(n,x,y), DMin(n,x-1,y), DMin(n,x+1,y), DMin(n,x,y-1), DMin(n,x,y+1), DMin(n,x-1,y-1), DMin(n,x-1,y+1), DMin(n,x+1,y-1), DMin(n,x+1,y+1)) and similarily DMax(n,x,y)=Max(n,x,y)-o(n,x,y), DiffAbove(n,x,y)=max(DMax(n,x,y),...). Now f(n,x,y) is clipped at o(n,x,y)-DiffBelow(n,x,y) and o(n,x,y)+DiffAbove(n,x,y).
If smooth=0, then f(n,x,y) is completely replaced by o(n,x,y) if o(n-1,x,y)=o(n,x,y)=o(n+1,x,y). If smooth=1 then the property o(n-1,x,y)=o(n,x,y)=o(n+1,x,y) must hold also for each of its 8 neighbours to ensure that f(n,x,y) is completely replaced by o(n,x,y). Thus if there are no temporal fluctuations in a neighbourhood of a pixel, the spatial cleaning or blurring is removed. If combined with the rather aggressive RemoveGrain(mode=4), then the static parts of a video are not touched at all, while compression is still much better than with the other, more conservative, modes of RemoveGrain.

[kassandro]

Didée,
after one night I now think that TemporalRepair(smooth=1) has nothing do with your idea. However, from time to time I think about a very similar idea, namely to apply spatial filters to the differences of subsequent frames rather than the frames itsself. In this way, spatial filters become interesting filters, which usually preserve static detail.
Why didn't I implement this idea yet?
Firstly, there technical obstacles. The difference of two pixels doesn't fit into a byte anymore, because the sign of the difference is very important. Thus one has to process two difference frames. One for postive and one for negative differences. Furthermore, it is a bad idea to use differences of unprocessed frames. If this is done, then mistakes made at one frame would propagate to all the subsequent frames. Thus one has to take differences between the current unprocessed frame and the previous processed frame. This unfortunately implies that the filter works well only sequentially. However, there is a work around even for this nasty recursion problem. The main argument for rejecting this kind of temporal cleaning is that only two frames are used for temporal cleaning, which makes it much more risky than the three frame approach, which I use in RemoveDirt or RemoveDust.

[Didée]

Huhu, seems I'll need pen, paper and some time to work that up. But sure I'll get through.

Thank you very much for the exact explanation. :)

-----
edit: yes, I also concluded that the idea is not so unproblematic as it might appear on first glance. But still, it has a grain of reasonable ground, somehow.
If its only about speed, let me squeeze it in a script. People are used to my scripts being ridiculously slow ;)
-----

BTW, regarding the "issue" I brought up: yesterday I quickly did this little comparison (http://x4.putfile.com/videos/5905203518.avi) (6 MB) (source/PixieDust(2)/RemoveDust(4)/LRemoveDust(4,2) (http://forum.gleitz.info/showpost.php?p=169704&postcount=32)). Perhaps worth having a look at.

[kassandro]

Originally posted by Didée
Huhu, seems I'll need pen, paper and some time to work that up. But sure I'll get through.

Yes, the matter unfortunately quite technical.

Thank you very much for the exact explanation. :)


BTW, regarding the "issue" I brought up: yesterday I quickly did this little comparison (http://x4.putfile.com/videos/5905203518.avi) (6 MB) (source/PixieDust(2)/RemoveDust(4)/LRemoveDust(4,2) (http://forum.gleitz.info/showpost.php?p=169704&postcount=32)). Perhaps worth having a look at.
Thanks for your tests. However, I must say that your avi is a bit unfair, because it contains virtually no static parts. You even did cut out the sky. This video exhibts the weak points of RemoveDust: if the moving objects have thin details, then they get blurred as if they were shot with a slower shutter speed and also compression deterioates considerably. Under average circumstances, the compression advantage over PixieDust is much higher than just 10% as in your avi. The static parts of the video are even improved. You also have chosen a motion speed, which seems to be slow enough for PixieDust's motion compensation and fast enough to exhibit the weakness of RemoveDust. It is also interesting that your limited version of RemoveDust comes fairly close to the unlimited version as far as compression is concerned, while keeping the video sharp. I have to check whether this also the case under average conditions.

[Valky]

Originally posted by Didée
Huhu, seems I'll need pen, paper and some time to work that up. But sure I'll get through.

Thank you very much for the exact explanation. :)

-----
edit: yes, I also concluded that the idea is not so unproblematic as it might appear on first glance. But still, it has a grain of reasonable ground, somehow.
If its only about speed, let me squeeze it in a script. People are used to my scripts being ridiculously slow ;)
-----

BTW, regarding the "issue" I brought up: yesterday I quickly did this little comparison (http://x4.putfile.com/videos/5905203518.avi) (6 MB) (source/PixieDust(2)/RemoveDust(4)/LRemoveDust(4,2) (http://forum.gleitz.info/showpost.php?p=169704&postcount=32)). Perhaps worth having a look at.

Good looking comparison. What was the speed cpmparing to pixiedust? Default settings in removedust seems to blur lots of details in sleeve. Also would be nice to see similiar comparison with more noisier source like tv-capture etc.

[kassandro]

Didée
what are you doing with the chroma in your script function LRemoveDust? More specifically, what is U=2,V=2 in yv12lutxy(last,input,yexpr=STR,U=2,V=2) good for? I expected values for uexpr and vexpr instead (the same string as for yexpr).

[Didée]

kassandro -
as it is currently written, that LUTxy command performs limiting on the Y plane only. U & V planes are passed through from RemoveDust just as they are. To do the same limiting for chroma, one would specify uexpr & vexpr with the same string as yexpr, and call U=3 & V=3 in the LUTxy comand. However when I tried that, I didn't see any remarkable visual difference. So I took the faster option with just passing chroma untouched.

Granted, that clip snipplet doesn't show all possible aspects. But then, that wasn't my aim. It is so that I really like RemoveDust in general. But there are motion characteristics where I don't like the plain RemoveDust at all, which held me back from using it. The most important one is just this: loss of detail in *slowly* moving areas. If motion is slow, then detail loss is easily noiticeable. In high motion, it's much less of a problem.
And so I just took a random example that showed the problem I'm mostly concerned about. It wasn't meant at all to be a "general" comparison.

Then, one could see such "hard limiting" as a poor technique. In fact it is, because it's just a dumb "cutting-off the effect". But looking at it from another angle, one could say as well like this: Generally, RemoveDust does the right thing. But in certain cases, it just does too much. Therefore, if limiting the effect to rather low values still is able to remove the noise/grain we want to remove, then there is no need to do any more. Or, in other words: The filter is working in the right direction, but is working too diligent. We better tell it where to stop. ;)

Keeping in mind that all of this is working without true motion compensation, I think that this is a really great filter combination for noise and grain removal. And I could imagine that it gets even more impressing when combined with motion compensation, even if the incredible speed is lost then. Alas I've currently not enough free time to look into that. perhaps someone might want to try that.


Valky:

Just try it and see for yourself. Not that difficult, it's just a few lines.
Very roughly from memory: Together with XviD encoding, I got ~5fps for PixieDust and ~20fps for LRemoveDust. LRemoveDust encoding was only marginally slower than without any filtering - XviD was the most limiting factor, there.
For plain rendering without encoding, (L)RemoveDust is in the range of 8~10 times faster than PixieDust. Regarding *that* difference in performance, I think the results are so close to PixieDust, it's almost impertinent :)

[Boulder]

Didée,

would it be possible to make a YUY2 version of LRemoveDust? It seems very interesting but would require converting to YV12 and then back to YUY2 to feed to CCE.

[Didée]

Yes and no. Kassandro's RemoveDust function supports YUY2 natively (in fact, it supports all colorspaces, afaik). But the limiting for LRemoveDust is done through MaskTool's "yv12lutxy" ... the name says it all. Because of that, at least for doing luma limiting there's no way to avoid the yuy2->yv12->yuy2 journey (unless someone codes a plugin for that). However, passing through the untouched "original" yuy2 color planes (as produced by RemoveDust, without limiting) is perfectly possible, with a small speed penalty.

If I made no error (you know ... writing a script without possibility to test it), the following should do it:

function LRemoveDust(clip input, int _mode, int "limit")
{
limit = default(limit,4)
LL = string(limit)
STR = "x "+LL+" + y < y "+LL+" - x "+LL+" - y > y "+LL+" + x ? ?"
repmode = 2
clensed = Clense(input)
rep = Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
trep = TemporalRepair(rg, rep)
trep2 = isyuy2(input)
\ ? trep.converttoyv12()
\ : trep
input2 = isyuy2(input)
\ ? input.converttoyv12()
\ : input
limit!=0 ? yv12lutxy(trep2,input2,yexpr=STR,U=2,V=2) : last
out = isyuy2(input)
\ ? last.converttoyuy2().mergechroma(trep)
\ : last
return out
}

But even if it's correct, still no support for RGB input. Sorry tmpeg users :|

[Boulder]

Thanks a bunch, I'll have to try it on my next capture tomorrow:)

EDIT: A quick test shows it works, at least no errors pop up;)

[Boulder]

I did some tests and found LRemoveDust wonderful! The YUY2 version is quite a bit slower (something like 0.29RT vs. 0.44RT), and I noticed that conversion to YV12 and back to YUY2 (and then back to YV12 in the encoder) is not noticable with TV captures so I ended up using the original function. LRemoveDust keeps details in motion better than RemoveGrain().RemoveDirt() so it looks like it'll become the next allround filter for my analogue captures -- at least till kassandro updates RemoveDirt;)

[kassandro]

Originally posted by Boulder
I did some tests and found LRemoveDust wonderful! The YUY2 version is quite a bit slower (something like 0.29RT vs. 0.44RT)
Yes, the YUY2 and RGB versions are of RemoveGrain etc. are a lot slower. I use a very simple class to separate the components of interleaved color space into planes such that I can apply the same functions as for YV12. Though I have written the interleaved2planar and planar2interleaved routines in C, I am quite surprised about the enormous slow down. I knew this for quite while, but didn't yet investigate the cause. Both conversions shouldn't be much slower than a bitblt operation. I should be able to privide faster assembly routines at least for YUY2. On the other hand, in RemoveDust and LRemoveDust filters from RemoveGrain/Repair are used several times and thus one has several times the interleaved2planar and planar2interleaved routines, which is simply stupid. If there would be planar versions of YUY2, RGB, RGB32 in Avisynth we wouldn't have these problems. We would simply convert to planar at the beginning of the script and back to interleaved at the end. Converting to YV12 and then back to YUY2 is not a good idea, because it blurs the chroma quite a bit. But Boulder, I do not understand, why you need YUY2 output. It could only be for CCE which wants to have YUY2 input although it produces YV12 output. This makes only sense if the higher resolution YUY2 chroma is used for searching motion vectors or a combined luma/chroma motion vector search is employed, but I doubt that CCE is doing that.

When I did my first analog capture recently, I saw that when using RemoveGrain with AVIsource, VdubMod crashes upon exit. I had the same problem with RemoveDirt once. The probable reason is that I didn't deallocate some objects properly and AVISource doesn't like that. This is a bug of AVIsource as well and I recommended to fix it long ago, because under certain circumstances objects cannot be deallocated. This is certainly not a serious problem and it will be fixed with the next version of RemoveGrain in the summer.

YV12LUTxy is a very useful, reasonably fast and also very smart (because of using look up tables) filter, which should be made internal, but in LRemoveDust it can be replaced by the much faster LimitChange. However, when I tested an optimised version two days ago it didn't work anymore, but I hope to post an inofficial version next weekend, which should speed up LRemoveDust a little. It will also come with a modified inofficial version of Clense, which then can be used together with mvtools to provide a slow, motion compensated version of RemoveDust. It should reduce motion blurring considerably but also worsen compression.


-- at least till kassandro updates RemoveDirt;)
Probably by the end of the month. I am testing right now.

[Boulder]

Originally posted by kassandro
Converting to YV12 and then back to YUY2 is not a good idea, because it blurs the chroma quite a bit.

With analogue captures it's OK because there is a substantial amount of chroma noise.

But Boulder, I do not understand, why you need YUY2 output. It could only be for CCE which wants to have YUY2 input although it produces YV12 output. This makes only sense if the higher resolution YUY2 chroma is used for searching motion vectors or a combined luma/chroma motion vector search is employed, but I doubt that CCE is doing that.

I think the main reason is that Cinema Craft doesn't want people screwing up things badly and then say that the program is crap. There's a thread about the whole thing somewhere in the CCE forum. It is a real shame they don't support direct YV12 input.

Probably by the end of the month. I am testing right now.
That's good news:)

[Boulder]

I just had a walk with the dog and this thought came to my mind:

1)source is YUY2
2)convert to YV12
3)use LRemoveDust_YV12, enable greyscale mode for even faster processing
4)convert to YUY2
5)put the original chroma information back
6)denoise the chroma (CNR2)

This way there would be no loss, right? Since luma noise is much more easily seen by the human eye, it wouldn't matter if chroma wasn't processed by LRemoveDust and you can use other plugins like CNR2, which is also fast, for that.

[kassandro]

Originally posted by Boulder
I just had a walk with the dog and this thought came to my mind:

1)source is YUY2
2)convert to YV12
3)use LRemoveDust_YV12, enable greyscale mode for even faster processing
4)convert to YUY2
5)put the original chroma information back
6)denoise the chroma (CNR2)

This way there would be no loss, right? Since luma noise is much more easily seen by the human eye, it wouldn't matter if chroma wasn't processed by LRemoveDust and you can use other plugins like CNR2, which is also fast, for that.

Yes, you can do that.

Actually, I can outwit Avisynth's color space limitations. For Avisynth a planar YUY2 frame will look just like an ordinary interleaved YUY2. However, the information is organized differently on the planar YUY2 frames. Say, you have a 720x576 interleaved YUY2 frame. Then the first byte is the luma of the first pixel, the second byte ist the U value of pixels 1 and 2, the third byte is luma of pixel 2 and the 4th byte is the V value for pixels 1 and 2 and then it goes on with pixels 3 and 4, etc. On a planar YUY2 frame, you have first the 720 luma values of the first line, then the 360 U values of the line and finally the 360 V values of the line. Thus I have only rearranged the values of the pixel lines, but any yv12 filter could handle such frames with only tiny changes just like yv12 frames. However, ordinary spatial YUY2 filters like a horzontal crop would be very destructive (a vertical crop wouldn't hurt) for such frames. Temporal filters, which treat luma and chroma equally could be applied to planar and interleaved YUY2 in the same way.

[kassandro]

Here is my motion compensated RemoveDust:

function MCRemoveDust(clip input, int _smooth, bool _grey)
{
repmode = 2
clmode = 4
quad = Quadruple(input) # each pixel is quadrupled, SSETools are needed
qpel = RemoveGrain(quad, mode=12, modeU=-1) # blur the luma for searching motion vectors
bvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=true)
fvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=false)
backward = MVCompensate(quad, bvectors)
forward = MVCompensate(quad, fvectors)
clensed = mcclense(quad, backward, forward, grey=_grey).shrinkby2(grey=_grey) #SSETools are needed for shrinking
rep=Repair(clensed, input, mode=repmode, modeU=_grey ? -1 : repmode)
rg = RemoveGrain(rep, mode=clmode, modeU=_grey ? -1 : clmode)
return TemporalRepair(rg, rep, grey=_grey, smooth=_smooth)
}

You need MVTools 0.9.9.1 and the following inofficial plugins (http://home.pages.at/kassandro/inofficial.zip). It contains a new inofficial version of RemoveGrain (a special version of clense for motion compensation has been added) and of SSETools (no documentation). SSETools contains also a filter LimitChange, which can be used for a faster version of Didee's LRemoveDust:

function LRemoveDust(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input)
rep=Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
rd=TemporalRepair(rg, rep)
return LimitChange(rd, input, limit, limitU=255)
}


Here is a version which also "limits" the chroma:

function LRemoveDust(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input)
rep=Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
rd=TemporalRepair(rg, rep)
return LimitChange(rd, input, limit)
}

[Boulder]

That's a nice one, got another capture coming in later today so I guess it's time to play:)

[kassandro]

Sorry, Boulder. Something must have gone wrong here. I just ran a test and it turned out, that MCRemoveDust gives exactly the same as RemoveDust. There must be misunderstanding of the MVTools on my side.

[kassandro]

Unfortunately, the Filter mcclense, part of a slightly changed RemoveGrain plugin, was working exactly as clense, whence MCRemoveDust did work exactly as the RemoveDust. I have now uploaded a corrected and tested version. The script function remains unchanged. I did a test with a short 5500 frames part of a fairly clean black&white movie from 1952. Motion was definitely sharper as with the original RemoveDust. On the other hand in such old movies motion is always blurred a little. Thus this video was not very suitable for MCRemoveDust. Here are the compression results:

MCRemoveDust 18.896.476 Bytes
RemoveDust 18.791.986 Bytes

Thus with motion compensation one looses only 0.6%, which is very cheap for a significant improvement in quality. Unfortunately, encoding time roughly tripled with MCRemoveDust, where XviD was used with all advanced features except GMC. More detailed results will be posted within the next weeks.

[kassandro]

I just made a comparison with a high quality source from 2002 (63958 frames, 720x576, with ogg vorbis audio):

unfiltered 347.059.657 Bytes
RemoveDust 284.028.286 Bytes
MCRemoveDust 280.028.971 Bytes
LRemoveDust 297.983.847 Bytes

This time MCRemoveDust is ahead of RemoveDust by about 1.5% and contains visibly more detail in moving parts of a frame.
Because the source contained relatively little grain and was virtually dirt free, the compression gain was only 18.2% with RemoveDust and 19.3% with MCRemoveDust, less than half what I experience in the average with my usually much older films.
Encoding time with MCRemoveDust was a little bit more than two times the encoding time with RemoveDust.
Using LRemoveDust (should be identical with Didee's version) the output is 6.4% larger than with MCRemoveDust. The comparison with LRemoveDust is a little bit unfair, because it uses TemporalRepair with smooth=0, the default value, but RemoveDust, MCRemoveDust use smooth=1, but the impact should not be very significant (1-2%), because TemporalRepair only corrects the spatial denoiser RemoveGrain, such that it cannot touch (temporally) static pixels.

[Boulder]

How does LRemoveDust (luma only) compare? Using MCRemoveDust is unfortunately out of my league since the encoding time is already quite long and I can't run the computer overnight. However, it could be used on shorter clips and the result first saved in lossless format to speed up the encoding to the final format.

[kassandro]

Originally posted by Boulder
How does LRemoveDust (luma only) compare? Using MCRemoveDust is unfortunately out of my league since the encoding time is already quite long and I can't run the computer overnight. However, it could be used on shorter clips and the result first saved in lossless format to speed up the encoding to the final format.
I just finished the test with LRemoveDust and added the results to the above posting. I will run one more test with a much grainy source from the early nineties. Here the differences should be bigger.

[vigi_lante]

Sorry, but I can't get LRemoveDust to work. This is my script...

LoadPlugin("c:\filters\decomb521.DLL")
LoadPlugin("c:\filters\RemoveGrainS.dll")
LoadPlugin("c:\filters\RepairS.dll")
Import("c:\filters\LRemoveDust.avs")

AviSource("c:\capture (2005-03-08 at 00-05-53).avi")

Trim(0,7322)

Telecide(order=1,post=0)
Decimate(cycle=5,mode=2)

ConvertToYV12()

LRemoveDust()

The LRemoveDust code is inside LRemoveDust.avs

Is there anything wrong ?

[vigi_lante]

OK. I got LRemoveDust to work with this...

LoadPlugin("c:\filters\decomb521.DLL")
LoadPlugin("c:\filters\RemoveGrainS.dll")
LoadPlugin("c:\filters\RepairS.dll")
LoadPlugin("c:\filters\MaskTools156.dll")
Import("c:\filters\LRemoveDust.avs")

AviSource("c:\capture (2005-03-08 at 00-05-53).avi")

Trim(0,7322)

Telecide(order=1,post=0)
Decimate(cycle=5,mode=2)

ConvertToYV12()

LRemoveDust(4,2)



Just try it and see for yourself. Not that difficult, it's just a few lines.
Very roughly from memory: Together with XviD encoding, I got ~5fps for PixieDust and ~20fps for LRemoveDust. LRemoveDust encoding was only marginally slower than without any filtering - XviD was the most limiting factor, there.
For plain rendering without encoding, (L)RemoveDust is in the range of 8~10 times faster than PixieDust. Regarding *that* difference in performance, I think the results are so close to PixieDust, it's almost impertinent

Well, I using an AthlonXP 2400+ and LRemoveDust speed was not even 2 times faster than PixieDust(5).

I wonder what is wrong...

[Didée]

The question was:Originally posted by vigi_lante
Well, I using an AthlonXP 2400+ and LRemoveDust speed was not even 2 times faster than PixieDust(5).

I wonder what is wrong...

The answer is:Telecide(order=1,post=0)
Decimate(cycle=5,mode=2)

The promised acceleration values for a car don't hold up when the car is pulling a caravan.

[Boulder]

It should be faster, I do it like this:

org=last
ConverttoYV12()
LRemoveDust(4,2)
GreyScale()
ConverttoYUY2()
MergeChroma(org)

and it's still much faster than PixieDust. I have tweaked the function so that only luma is processed by adding the necessary grey=true and modeU=-1 parameters where needed, but the difference shouldn't be too big.

Then again, I have a SSE2 CPU which gives a nice boost thanks to kassandro's optimizations:)

EDIT: Didée's right..if you've tried using PixieDust in your IVTC script, it will show you what slow is;)

[vigi_lante]

Not really. I was using, of course, PixieDust with that exactly script, including IVTC.

I tried to use it without IVTC, and speed is just a little faster. That IVTC script also have "post" disabled too, so it's not pushing too much.

Anyway, this filter is great! I just wish I could get that speed too.

[kassandro]

I just ran a further small test. Source was fairly poor. The film was mastered about 30 years ago on a tape and the quality of the tape detioroated over the years (4420 frames).

unfiltered 27.183.680 Bytes
RemoveDust 15.103.098 Bytes
MCRemoveDust 15.969.790 Bytes
LRemoveDust 17.426.043 Bytes

LRemoveDust clearly profited from not limiting the cleaning of the chroma, because the chroma was even worse than the luma.

[vigi_lante]

What about using filters like Undot or Cnr2 with LRemoveDust ? Could it give better results ? (with analog TV source)

[kassandro]

Originally posted by vigi_lante
What about using filters like Undot or Cnr2 with LRemoveDust ? Could it give better results ? (with analog TV source)
Read the beginning of the documentation of RemoveGrain for its relation with Undot. Also this thread contains lots of information about the RemoveGrain/Undot relation.
CNR2 is a filter, which only effects the chroma. It is essentially a temporal filter. The basic idea is to average the chroma of the current frame with that of previuos frame, where the weight of the averages depend on the luma and the chroma differences. This is certainly a good idea and it can also be implemented in a spatial context, though such a filter would be much slower. Having said that I cannot understand the logic behind the computed weights. Actually these weights do not make much sense to me. If somebody is interested I can easily provide a variant of this filter with simpler and more reasonable weights, which would be closer to the basic idea of this filter.


EDIT: A spatial version would have the advantage of avoiding ghosts!

[Boulder]

If you can make a tweaked CNR2, I'm all for it:) The problem with the filter is that it causes chroma ghosting very easily, even at the default values. Then again, it removes noise very effectively when dealing with analogue material.

Out of interest, what causes the spatial version to be much slower?

[kassandro]

Originally posted by Boulder
If you can make a tweaked CNR2, I'm all for it:) The problem with the filter is that it causes chroma ghosting very easily, even at the default values. Then again, it removes noise very effectively when dealing with analogue material.

I probably can't remove the ghosting significantly. However, MarcFD uses only the previous frame for temporal cleaning and ignores the subsequent frame, which would be equally good. Because temporal cleaning ord enoising is quite risky, it is a golden rule to use both the previous and the subsequent frame. I would simply choose for each pixel the best one. As a byproduct I wouldn't have to worry about scene switches as MarcFD. On the other hand, the memory access costs are 50% higher. But the very simple algorithm is very cache friendly, so it shouldn't be a big penalty.


Out of interest, what causes the spatial version to be much slower?
MarcFD relates a pixel of a frame only to the same pixel on the previous frame. My modification would also consider the same pixel on the subsequent frame. If I would do it spatially, then I would have to consider all the eight spatial neighbours of the pixel, which is quite a drag, if one can't use SSE/MMX.

It is fairly easy to do the temporal version outlined above. Probably I will do it next weekend.

[vigi_lante]

CNR2 is very fast too.

But it's recommended to use CNR2 after or before LRemoveDust ?

[kassandro]

Originally posted by vigi_lante
CNR2 is very fast too.

I would call it moderately fast. It cannot compete with SSE/MMX optimised filters like Undot or RemoveGrain (YV12 version only).


But it's recommended to use CNR2 after or before LRemoveDust ?

I have only done one analog capture and that was a b&w VHS movie. Usually I just record DVB streams. Thus I am not a user of CNR2 and really can't answer this question. However, applying CNR2 after LRemoveDust should denoise more than in reverse order.

[vigi_lante]

Read the beginning of the documentation of RemoveGrain for its relation with Undot. Also this thread contains lots of information about the RemoveGrain/Undot relation.


Sorry my ignorance, but it's very hard for me to understand all those tech talk...so, just to keep everything easy: Undot with LRemoveDust is good or bad ? :) Because the compressibility gain with Undot is very nice.

One more thing...is there any other option for LRemoveDust to work with very noise sources ?

[vigi_lante]

I just record DVB streams

Here, the same cable that delivers analog TV also delivers DVB stream. Right now I'm wondering if would be better to capture analog or digital, because DVB uses MPEG-2 compression for video...so you can't avoid artifacts. Analog is more noiser, but without artifacts.

[kassandro]

Originally posted by vigi_lante

Undot with LRemoveDust is good or bad ?

It doesn't really harm to combine it, but it really doesn't make sense either.



Because the compressibility gain with Undot is very nice.

The compression gain with Undot (or equivalently RemoveGrain(mode=1)) is neglible (at least with my digital sources) compared with that RemoveDust. If your extremely conservative, you may use RemoveGrain(mode=2) instead of Undot. The compression gain is about twice as much and the artifact risk is nearly as low as with Undot.


Here, the same cable that delivers analog TV also delivers DVB stream. Right now I'm wondering if would be better to capture analog or digital, because DVB uses MPEG-2 compression for video...so you can't avoid artifacts. Analog is more noiser, but without artifacts.
In static areas DVB is vastly superior over analog and in motion areas DVB may sometimes be inferior to analog, but altogether it would be nightmare for me to go back to analog.
Also recording DVB streams is so easy. You do not have to worry about frame drops and a/v sync and disk space. While recording streams you can easily encode a video with Vdub. This is unthinkable during analog capturing. With massive analog capturing you also reduce the life time of your hard disks substantially. Also you can convert the DVB stream into DVD without reencoding.

[vigi_lante]

In static areas DVB is vastly superior over analog and in motion areas DVB may sometimes be inferior to analog, but altogether it would be nightmare for me to go back to analog.
Also recording DVB streams is so easy. You do not have to worry about frame drops and a/v sync and disk space. While recording streams you can easily encode a video with Vdub. This is unthinkable during analog capturing. With massive analog capturing you also reduce the life time of your hard disks substantially. Also you can convert the DVB stream into DVD without reencoding.

WHAT !? I didn't know this was possible.

But what I need to directly record those streams ? Any special capture card ?

I was thinking it was just like capturing analog signal.

Here I have an settop-box to receive DVB broadcasts. This settop-box convert the digital signal to analogue, in order to work with a normal TV. So, I didn't know there were other ways to capture, besides capture the analogue output from the settop box.

[Boulder]

You need a DVB capture card to do that. Then you can get the same stream that comes to the set-top box.

[vigi_lante]

But what about if the signal is encrypted and needs the settop box to decode it ? It's possible to happen ?

[Boulder]

The streams that you have to pay for are encrypted.

This is getting OT, so you'd better see the appropriate forum section for more and detailed info.

[vigi_lante]

Originally posted by Boulder
The streams that you have to pay for are encrypted.

This is getting OT, so you'd better see the appropriate forum section for more and detailed info.

The problem is that I pay for everything... :scared:

Sorry for the OT.

[vigi_lante]

you may use RemoveGrain(mode=2) instead of Undot. The compression gain is about twice as much and the artifact risk is nearly as low as with Undot.


In this case, I need to use RemoveGrain(mode=2) + LRemoveDust ?

[Boulder]

I wouldn't use RemoveGrain with LRemoveDust, MCRemoveDust or RemoveDust. All of those functions actually include RemoveGrain(mode=4) by default.

[vigi_lante]

Originally posted by Boulder
I wouldn't use RemoveGrain with LRemoveDust, MCRemoveDust or RemoveDust. All of those functions actually include RemoveGrain(mode=4) by default.

What about LRemoveDust with Undot ?

[Didée]

Originally posted by vigi_lante
What about LRemoveDust with Undot ?
/*getting nervous*/

f-o-r-g-e-t i-t! Please!

There is *no need* to assist LRemoveDust with *anything* like undot or removegrain.

[vigi_lante]

Originally posted by Didée
/*getting nervous*/

f-o-r-g-e-t i-t! Please!

There is *no need* to assist LRemoveDust with *anything* like undot or removegrain.

What I'm just asking if there is any side effect using Undot. Because if not, I see no reason to not use it, since it is very fast and gives a very good compression gain.

[Boulder]

LRemoveDust already does the same work - and more - it will remove the stray pixels UnDot would. UnDot is almost exactly the same as RemoveGrain(mode=1) as kassandro already told you. Thus, no need to use it.

[Didée]

Well, after having a second thought about it, there could be cases where it IS useful to to let LRemoveDust follow a RemoveGrain(1).

E.g. LRemoveDust(4,2) will change no pixel by more than +/- 2. Now, if the source contains stray pixels that are standing out very much in value, those would be filtered too weakly, and a final Removegrain(1) would make sense.

Haven't encountered something like that so far, at lest not visually. But it would be possible. My apologizes to vigi_lante.

[scharfis_brain]

mcremovegrain gave me weird blocking artifacts on non predictable movements like flames.
also, it blurs the entire frame like hell.

anything I can do against it?

[Didée]

Originally posted by scharfis_brain
mcremovegrain gave me weird blocking artifacts on non predictable movements like flames.
also, it blurs the entire frame like hell.

anything I can do against it?
Yes. Yesterday I tried something like

function LRemoveDustMC(clip input, int _smooth, bool _grey, int "limit")
{
limit = default( limit, 3 )
repmode = 2
clmode = 4
quad = Quadruple(input) # each pixel is quadrupled, SSETools are needed
qpel = RemoveGrain(quad, mode=12, modeU=-1) # blur the luma for searching motion vectors
bvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=true)
fvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=false)
backward = MVCompensate(quad, bvectors)
forward = MVCompensate(quad, fvectors)
# clensed = mcclense(quad, backward, forward, grey=_grey).shrinkby2(grey=_grey) #SSETools are needed for shrinking
# rep=Repair(clensed, input, mode=repmode, modeU=_grey ? -1 : repmode)
# rg = RemoveGrain(rep, mode=clmode, modeU=_grey ? -1 : clmode)
interleave(backward,input.Quadruple(),forward)
LRemoveDust(4,limit)
SelectEvery(3,1)
shrinkby2(grey=_grey)
# return TemporalRepair(rg, rep, grey=_grey, smooth=_smooth)
return last
}
Actually, I forgot to take the function made yesterday with me. The above is just re-typed, from memory. Hope it works. (If not, you get at least the idea).

[scharfis_brain]

Thanks
it works, BUT
all those functions I saw here de only filter over a window of 3 frames. (radius = 1)
For a temporal filter this is definitly not enough!
only when I call them twice:
lremovedustmc().lremovedustmc()
i get a nice denoised result.
but this probably has not been in the intention of the inventor.

for temporal filtering a radius of two or three would be very nice

[scharfis_brain]

It would be great, if one of those denoisers here could fight with noise amounts like manaos filter:
http://forum.doom9.org/showthread.php?s=&threadid=89555

unfortunately it never processes moving edges :(

[Piper]

@Kassandro: I've just tried numerous attempts at www.RemoveGrain.de.tf to find a download location for RemoveGrain and each time I get a Gateway Timeout error. I've googled/searched the forum for an alternative download site but so far have come up with nothing. Is there an alternative URL where I can find a current version of RemoveGrain.dll? Thanks

[kassandro]

Originally posted by Piper
@Kassandro: I've just tried numerous attempts at www.RemoveGrain.de.tf to find a download location for RemoveGrain and each time I get a Gateway Timeout error. I've googled/searched the forum for an alternative download site but so far have come up with nothing. Is there an alternative URL where I can find a current version of RemoveGrain.dll? Thanks
The server in Austria, where my files are located, seems to be down. If that continues until tomorrow night, I will upload the files to my Geocities account and map the domain to this server.

[kassandro]

Originally posted by scharfis_brain
mcremovegrain gave me weird blocking artifacts on non predictable movements like flames.
also, it blurs the entire frame like hell.

anything I can do against it?
It is true, that if the motion compensation fails from both sides, i.e. low quality motion vectors, then motion gets blurred by MCRemoveDust in the same way as with the original RemoveDust. If it fails only from one side, then denoising may be poor, but there shouldn't be any significant damage. If the moving objects are homogeneous, i.e. have no fine details, then the edges should be still sharp, when filtered with RemoveDust or when MCRemoveDust cannot get good motion vectors. With RemoveDust moving thin lines or structures may flicker, when they are in motion. With MCRemoveDust that flicker is quite rare, because in these cases the motion compensation usually works quite well. On the other hand big artifacts should be equally impossible as cleaning of big spots. This is simply assured by the Repair plugin.
Scharfi, could you please post an example, i.e. the unfiltered and the filtered frame with the weird artifacts.

[Piper]

Originally posted by kassandro
The server in Austria, where my files are located, seems to be down. If that continues until tomorrow night, I will upload the files to my Geocities account and map the domain to this server.

Thanks, sorry to be the one to point out server difficulties. Hope you're able to get it sorted without too much hassle.

[kassandro]

Originally posted by scharfis_brain
It would be great, if one of those denoisers here could fight with noise amounts like manaos filter:
http://forum.doom9.org/showthread.php?s=&threadid=89555

unfortunately it never processes moving edges :(
I couldn't download badnoise.zip. But from the picture shown in this thread, it is obvious that this is interlaced material. Virtually all the filters in RemoveGrain are for progressive material only and RemoveDust is very destructive for interlaced material. In fact, if moving information mastered afterwards on a film (it is nearly always interlaced), as it is often done by tv broadcasters, this interlaced information is treated quite poorly by RemoveDust. I simply apply RemoveDust(mode=14) before RemoveDust on those parts and only on those parts usually at the end or the beginning of a movie. mode=13,14 are true bob modes (a true bob is allowed to use only one field, must preserve this field and interpolate the other) very similar to Tomsmocomp(1,0,0), but substantially faster. Actually the RemoveGrain in inofficial.zip (see the link in an earlier posting) has also the bob modes 15,16 which are slightly slower than modes 13,14 but are of higher quality (better compression and in my objective quality test for true bob filters they should be very difficult to beat).

If the entire video is interlaced, then one may denoise the fields seperately, but this is definetly not recommendable. Instead one should denoise both RemoveGrain(input, mode=13) (bottom field preserved, top field interpolated) and RemoveGrain(input, mode=14) (top field preserved, bottom field interpolated) or better use modes 15,16 instead and then merge the two denoised clips together (the bottom field from the first and the top field from the second clip). This is very similar to a standard technique of scharfis_brain, which he uses also for resizing and other spatial filtering of interlaced content.

[scharfis_brain]

I am not referring to interlacing!
look at the scripts and you will notice, that the filter only works with progressive frames (deinterlacer before it).

(I always bob-deinterlace interlaced video, to be able to treat it like progressive video!)

[kassandro]

Originally posted by scharfis_brain
I am not referring to interlacing!
look at the scripts and you will notice, that the filter only works with progressive frames (deinterlacer before it).

(I always bob-deinterlace interlaced video, to be able to treat it like progressive video!)
Of course, I have seen that, but for a real test the video should always be genuinely progressive. No matter how you do it, the interpolated field will always contain less information than the preserved field. Thus the two fields asymmetric and this is simply not natural. This is also the key reason, why deinterlacers, which always preserve one field, perform so poorly as far as compression is concerned.

[scharfis_brain]

man.
I never told, that I want to denoise interlaced contents!

It is just a very grainy and rainbowy PAL progressive video.

[kassandro]

I meant the video in the thread http://forum.doom9.org/showthread.php?s=&threadid=89555. Though it can no more be downloaded, it is obviously interlaced as shown by the picture there.

[scharfis_brain]

I know. But again,
I asked this:

all those functions I saw here de only filter over a window of 3 frames. (radius = 1)
For a temporal filter this is definitly not enough!
only when I call them twice:
lremovedustmc().lremovedustmc()
i get a nice denoised result.
but this probably has not been in the intention of the inventor.

for temporal filtering a radius of two or three would be very nice

and

It would be great, if one of those denoisers here could fight with noise amounts like manaos filter:
http://forum.doom9.org/showthread.php?s=&threadid=89555

unfortunately it never processes moving edges

So is there any chance to get an answer related to this?

[Didée]

The answer most probably is "no". All work done by combinations of RemoveGrain/Repair and their derivatives is based and restricted to working on the values of a 3^3 cube.

For bigger radii, the simplest way is to interleave more compensated frames, and do temporal averaging over those. Just like MVDenoise does internally.
However, that way one can only do a "straight" linear denoise ... kinda like an arrow shot through the block of compensated frames. I could think of something that does smart temporal averaging over such compensated frames, what effectively would be similar to a second stage of motion compensation.

Imagine a panning frame. In the frame there is a clock with fastly moving hands. "Normal" motin compensation will be able (at best) to match the clock positions from frame to frame. But the position of the clock's hands still has moved from frame to frame, and cannot be compensated by ME (information is too small, spatially). That's the point where the smart temporal denoising would work its way through the stack of compensated frames, including the most probable pixel positions from on frame to the next.

[kassandro]

Originally posted by Didée
The answer most probably is "no". All work done by combinations of RemoveGrain/Repair and their derivatives is based and restricted to working on the values of a 3^3 cube.

you couldn't describe it better. It's all about the 3^3 cube and in most cases it is only the 3x3 square. I am not so pessimistic about using more than 3 frames, though. However, it should be put into the MVCompensate code. I just did an inspection of this code. It is very simple and only one frame + motion vectors is used, unless MVCompensate is applied with mode=2, which doesn't make much sense, however, because are artifacts propagate from one frame to the next and probably accumulate to some extent over time. Actually, inspecting MVCompensate showed me that, MCRemoveDust doesn't do, what I wnated it do, because mode=0, which means that the first clip is ignored, is the default and not mode=1. Thus my old RemoveDust script is indeed equivalent to

function MCRemoveDust(clip input, int _smooth, bool _grey)
{
repmode = 2
clmode = 4
quad = Quadruple(input) # each pixel is quadrupled, SSETools are needed
qpel = RemoveGrain(quad, mode=12, modeU=-1) # blur the luma for searching motion vectors
bvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=true)
fvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=false)
backward = MVCompensate(qpel, bvectors)
forward = MVCompensate(qpel, fvectors)
clensed = mcclense(quad, backward, forward, grey=_grey).shrinkby2(grey=_grey) #SSETools are needed for shrinking
rep=Repair(clensed, input, mode=repmode, modeU=_grey ? -1 : repmode)
rg = RemoveGrain(rep, mode=clmode, modeU=_grey ? -1 : clmode)
return TemporalRepair(rg, rep, grey=_grey, smooth=_smooth)
}

which should be somewhat softer than intended. Here is the corrected script:

function MCRemoveDust(clip input, int _smooth, bool _grey)
{
repmode = 2
clmode = 4
quad = Quadruple(input, grey=_grey) # each pixel is quadrupled, SSETools are needed
qpel = RemoveGrain(quad, mode=12, modeU=-1) # blur the luma for searching motion vectors
bvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=true)
fvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=false)
backward = MVCompensate(quad, bvectors, mode=1)
forward = MVCompensate(quad, fvectors, mode=1)
clensed = mcclense(quad, backward, forward, grey=_grey).shrinkby2(grey=_grey) #SSETools are needed for shrinking
rep=Repair(clensed, input, mode=repmode, modeU=_grey ? -1 : repmode)
rg = RemoveGrain(rep, mode=clmode, modeU=_grey ? -1 : clmode)
return TemporalRepair(rg, rep, grey=_grey, smooth=_smooth)
}

I made also another change to speed up b&w movies slightly. Thus I have to run benchmarks again. The more general mode=1 should really be default mode. Overall, the script should become slightly slower. Probably the additional softness due to my mistake is partially responsible for the better than expected compression performance. I now have to run some more tests again. Unfortunately the big test below was still been done with the old MCRemoveDust.

[kassandro]

This should have been my last benchmark, if there wouldn't have been the mistake in MCRemoveDust. Nevertheless, it's worthwhile to publish the results here. The source is about ten years old (it's from the series "Ein Bayer auf Rügen"), but it has been remastered about two years ago, because it was sold to another broadcaster. Obviously the film deterioated a lot during these 8 years. There are tons of grain but otherwise there is no bigger dirt. The video has 67367 frames. Here are the results:

unfiltered 464.303.331 Bytes
RemoveDust 275.641.562 Bytes
MCRemoveDust 267.164.004 Bytes
LRemoveDust(4,4) 318.499.154 Bytes

Why does LRemoveDust(4,4) perform so purely? The grain has simply too much contrast. With a limit of 4 the grain could only be removed partially. Here are the pictures.
The unfiltered frame:
http://home.pages.at/kassandro/testun.png

the RemoveDust frame:
http://home.pages.at/kassandro/testrd.png

and finally the LRemoveDust(4,4) frame
http://home.pages.at/kassandro/testlrd.png

[Didée]

Jeeze, that is grain! On such a source, LRemoveDust is clearly out of the race - it's mostly good as long as all noise or grain can be pressed down with a reasonable limit. This is clearly not the case here ...
(However looking at the filesizes, LRemoveDust wasn't all that bad, either. Try to filter that source with e.g. Convolution3D ... ;) )

Another thought - I'm not sure how sensitive MVTools' motion engine actually reacts on noise. But perhaps it would be good to replace the clip's cleaning that MCRemoveDust is doing by RemoveGrain(mode=12) (before creating the motion vectors) by an instance of RemoveDust itself? I could imagine that MVTools like this better, if the source contains *so* much grain.

[kassandro]

Originally posted by Didée

Another thought - I'm not sure how sensitive MVTools' motion engine actually reacts on noise.

mvtools uses SAD, i.e. it sums up the absolute differences of the pixels of the blocks. I used the same routine in RemoveDirt and it is quite noise sensitve. However, mvtools is already designed such that SAD can easily ber replaced by other block difference functions and in the upcoming new RemoveDirt I have a modified SAD which ignores a certain noise level, i.e. instead of summing up |x_i - y_i| I sum up max(0, |x_i - y_i| - noise), where noise is usually a small number below 10. This is in a way opposite to what you have done with RemoveDust. The modified SAD just ignores that noise, which is cleaned by LRemoveDust. Clearly the modified SAD is slower than the original one, but with SSE2 the modified SAD should be about as fast as the simple SAD with ordinary SSEMMX. Because RemoveDirt spends most of its time measuring block differences I will therefore provide an SSE2 version. Similarily MVAnalyse spends well over 90% of its time measuring block differences.


But perhaps it would be good to replace the clip's cleaning that MCRemoveDust is doing by RemoveGrain(mode=12) (before creating the motion vectors) by an instance of RemoveDust itself? I could imagine that MVTools like this better, if the source contains *so* much grain.
That is a good idea. When I designed this script, I wanted to make it as fast as possible and I have now even a faster version of RemoveGrain(mode=12), but it really doesn't matter, because the time spent within these filters is neglible compared with the two MVAnalyse calls. It may be even be faster, because the motion vector search terminates as soon as it has found a resonably good vector. That is the reason, why XviD with RemoveDust is much faster than without. In fact, in the last benchmark the unfiltered encoding took almost 50% longer than with RemoveDust (the unfiltered encoding took only 30% less than with MCRemoveDust). In my encodings I always use XviD qpel motion vector search with Turbo mode. Without Turbo mode the speed would probably be even larger, but without qpel probably a lot smaller.

[morsa]

You see..that is my problem.
Super8 black an white film has way more grain than that and far more contrasty...
So none of the filters here can remove its noise, at least with more or less normal settings...
(just a comment):(

[redfordxx]

Hi,
I have tried your scripts and wanna share experience (I have low noise source LOTR TTT).
RemoveDustMC: Probably same experience as Scharfis brain --- Blurred blocks, and for me label unusable:( I didn't try Didée's LRDMC, since I read it just now. Probably helps... (after trimming first frame before SelectEvery, if I understand the idea correctly?)
LRemoveDust: much better experience --- removed the noise I wanted to... large surfaces (sky, snow..). But little problems with motion: When an object is moving relatively fast, it gets smaller (wow, hope Einstein doesnot hear me). The reason is, I think, that Repair not able to fully compensate the brutality with which Clense attacks moving objects. So, as I am beginner in the area, what do you think of following line as an aid (between Clense and Repair)clensed=YV12LUTxy(clensed,input,"x y - abs 6 > y x ?","x y - abs 6 > y x ?","x y - abs 6 > y x ?",Y=3,U=3,V=3) where 6 is threshold to be tuned. (I suppose limitchange does something like this, but inserting it here shoud provide better results, i think)
Other idea can be
input.MotionMask(.Inpand.Expand???)
MaskedMerge(...where is motion, use input, otherwise clensed)

[redfordxx]

Is somewhere described ShrinkBy2, LimitChange...?

[redfordxx]

As you are presenting some comparisons, I add one as well if u don't mind (low noise source, H.263, 1passQ2):
unfiltered: 0.400 bit/(pixel*frame)
LRemoveDust(3 (or 4?),1): 0.357 bit/(pixel*frame)(IMHO bpp gives more info than filesize)

[kassandro]

Originally posted by redfordxx
As you are presenting some comparisons, I add one as well if u don't mind (low noise source, H.263, 1passQ2):
unfiltered: 0.400 bit/(pixel*frame)
LRemoveDust(3 (or 4?),1): 0.357 bit/(pixel*frame)(IMHO bpp gives more info than filesize)
The terrrible compression results are easily explained. With LRemoveDust(3 (or 4?),1) you allow LRemoveDust to change the Luma value of each pixel only by 1. With LRemoveDust(3 (or 4?),0) the Luma wouldn't be changed at all. Please use more reasonable values like 4 for the second parameter.

[redfordxx]

Originally posted by kassandro
The terrrible compression results are easily explained. With LRemoveDust(3 (or 4?),1) you allow LRemoveDust to change the Luma value of each pixel only by 1. With LRemoveDust(3 (or 4?),0) the Luma wouldn't be changed at all. Please use more reasonable values like 4 for the second parameter. In fact, this was the last of my three post I expected being answered...
By "terrrible compression results" you understand small compression gain, right? Well as I said, I have low noise source and don't need heavy denoising. I want to preserve some detail. Moreover, I dont know exactly what LimitChange does... (my second post)
I also suppose, that the sources you used would have higher bpp than 0.400, so the potential is higher. (that's why I wrote it gives more info, than filesize)
Well, maybe I'll experiment with the limit parameter.

BTW: I limit chroma as well I hope (no limitU=255 parameter)
BTW2: Some experiment results: MotionMask does not preserve the moving object as well as YV12LUTxy...(maybe it is about threshold setting)

[Bexley]

I'm trying to play around with MCRemoveDust, but I can't get SSETools to work. VDubMod throws an error like "AVISynth open failure: AVISynth: script open failed!" If I delete it from my plugins directory, the script loads, but of course the MCRemoveDust function doesn't work.

Am I doing something dumb again, or does SSETools not work on an Athlon T-Bird?

[kassandro]

Originally posted by redfordxx
In fact, this was the last of my three post I expected being answered...
By "terrrible compression results" you understand small compression gain, right? Well as I said, I have low noise source and don't need heavy denoising. I want to preserve some detail. Moreover, I dont know exactly what LimitChange does... (my second post)

LimitChange does the same as YV12LUTxy with special yexpr, uexpr, vexpr (see Didee's original LRemoveDust). It simply limits the amount of change. For instance if the filtered pixel has the y value 200 and the unfiltered pixel has the y value 100, then the y value is replaced by 104 if LimitChange(filtered, unfiltered, 4) is used. If you use LimitChange(filtered, unfiltered, 1) then the y value is replaced by 101.


I also suppose, that the sources you used would have higher bpp than 0.400, so the potential is higher. (that's why I wrote it gives more info, than filesize)
Well, maybe I'll experiment with the limit parameter.

You can safely take 4 instead of 1. The difference is hardly visable. Moreover, codecs certainly make substantially more significant changes through quantising.


BTW: I limit chroma as well I hope (no limitU=255 parameter)
BTW2: Some experiment results: MotionMask does not preserve the moving object as well as YV12LUTxy...(maybe it is about threshold setting)
That's correct. If you drop limitU=255, then LimitU inherits the value 4. You may safely take limitU=10, because the human eye is much less sensitve to chroma changes than to luma changes. Filters like CNR2 change the chroma much more aggressively.

Originally posted by Bexley
I'm trying to play around with MCRemoveDust, but I can't get SSETools to work. VDubMod throws an error like "AVISynth open failure: AVISynth: script open failed!" If I delete it from my plugins directory, the script loads, but of course the MCRemoveDust function doesn't work.

Am I doing something dumb again, or does SSETools not work on an Athlon T-Bird?
SSETools has some filters, which use floating point SSE (however, this filters are not used with MCRemoveDust). Probably, the Athlon Thunderbird only supports integer SSE. I may provide a version without checking for full SSE next weekend. It will of course crash if some of the floating point SSE filters is used, but they aren't yet documented either.

[Heini011]

hi kassandro,

i have an athlon xp and have the same problem. ssetools and the current RemoveGrain.dll doesn't work. RemoveGrainS.dll is ok.

system: win 2k, avisynth 2.55

VirtualDub Error

Avisynth open failure:
LoadPlugin: unable to load "C:\Programme\Avisynth 2.5\plugins\SSETools.dll"

greetings.

[kassandro]

Originally posted by Heini011
hi kassandro,

i have an athlon xp and have the same problem. ssetools and the current RemoveGrain.dll doesn't work. RemoveGrainS.dll is ok.

system: win 2k, avisynth 2.55

VirtualDub Error

Avisynth open failure:
LoadPlugin: unable to load "C:\Programme\Avisynth 2.5\plugins\SSETools.dll"

greetings.
The Athlon XP certainly has floating point SSE. Thus the problem is a missing DLL. I think you need msvcp71.dll (but I am not sure about it). Until the end of 1004 I had Visual Studio 2002 and it required msvcp70.dll. As I have now moved to Visual Studio 2003 also the DLL has changed. Each version of the C++ compiler has its own DLL.

Probably the them is true for Bexley's Athlon Thunderbird.

[Leak]

Originally posted by kassandro
The Athlon XP certainly has floating point SSE. Thus the problem is a missing DLL. I think you need msvcp71.dll (but I am not sure about it). Until the end of 1004 I had Visual Studio 2002 and it required msvcp70.dll. As I have now moved to Visual Studio 2003 also the DLL has changed. Each version of the C++ compiler has its own DLL.

Probably the them is true for Bexley's Athlon Thunderbird.

For those missing a DLL:

http://www.dependencywalker.com/ is a good tool to check for missing DLLs - it's free and it's not hard to use... :)

np: Funkstörung - Captured In Tones (ft. Sarah Jay) (Disconnected)

[Bexley]

I may provide a version without checking for full SSE next weekend. It will of course crash if some of the floating point SSE filters is used, but they aren't yet documented either.
Thanks, kassandro. I checked and the DLL isn't my problem, so that would be awesome. :)

[kassandro]

I have uploaded a new inofficial.zip. It contains a staticly linked SSEToolsS.dll.

@Bexley:
Did you copy several versions of SSETools in the plugin directory. Only one version of SSETools (in the future the SSE2 version will be called SSE2Tools and the SSE3 version will be called SSE3 Tools) and RemoveGrain should be in the plugin directory. Otherwise it is not predictable, which plugin is chosen be Avisynth. If the SSE2 or SSE3 is chosen, then Athlon32 owner will have troubles. Actually, the old SSETools did only check for integer SSE and not full SSE.

[Bexley]

Ugh... I have a big problem, then. :(

I only have one version of SSETools.dll and RemoveGrain.dll in my plugins directory. I wonder if some other plugin might be causing a conflict... I'll try removing all unnecessary plugins when I get home and see if that helps.

[kassandro]

Originally posted by Bexley

I only have one version of SSETools.dll and RemoveGrain.dll in my plugins directory. I wonder if some other plugin might be causing a conflict... I'll try removing all unnecessary plugins when I get home and see if that helps.
You also need Repair.dll from the official RemoveGrain.zip on www.RemoveGrain.de.tf (http://www.RemoveGrain.de.tf). I am sorry for that mess, but it will take a while until I can return to RemoveGrain.

[Bexley]

No joy. :(

I removed all unnecessary plugins and made sure I have the most current versions of everything, and VDub still throws an error. The only way to load a script is to remove SSETools from the plugin directory. Oh well.... I'm still having fun with LRemoveDust, so I'll keep playing with that for a while. ;)

[kassandro]

Originally posted by Bexley
No joy. :(

I removed all unnecessary plugins and made sure I have the most current versions of everything, and VDub still throws an error. The only way to load a script is to remove SSETools from the plugin directory. Oh well.... I'm still having fun with LRemoveDust, so I'll keep playing with that for a while. ;)
You may also use Didde's original version. It doesn't require LimitChange. YV12LUTxy is used instead. On the other hand it would be nice to know the reason for failure. Avisynth's error message are unfortunately not very informative.

[kassandro]

I just have uploaded a new version of my RemoveGrain plugin. It became necessary because in the thread http://forum.doom9.org/showthread.php?s=&threadid=91823
a bug in the YUY2 version of TemporalRepair was discovered. Rather than fixing the YUY2 code, I dropped the interleaved YUY2 support altogether and replaced it by planar YUY2 (this is the reason why I have included inofficial versions of SSETools, which contain the filters Interleaved2Planar and Planar2Interleaved). The above thread contains also a claim by scharfis_brain, that the deinterlacer RGDeinterlace contructed with RemoveGrain is a "superblurrer", which blurs static areas. This is false as the test scripts in the above thread show, but scharfis_brain is not willing to withdraw his claim.
I have added also new bob modes 15,16 which are slightly better than modes 13,14 but also slightly slower.
However, the most import change is the new mode 17. It is a denosing mode for truely progressive film only, just like modes 1-10. I really love it. When the very aggressive mode 4 doesn't destroy thin lines, mode 17 uses this mode for denoising. If mode 4 is destructive, then mode 17 denoises more conservatively such that thin lines are preserved. The first test results with a b&w video clip (3962 frames) are impressive:

no cleaning 32.463.010 Bytes
RemoveGrain(mode=8) 29.964.964 Bytes
RemoveGrain(mode=9) 29.994.246 Bytes
RemoveGrain(mode=17) 28.221.649 Bytes
RemoveGrain(mode=4) 28.043.482 Bytes

Thus mode 17 clearly beats my old favorite mode 8 and comes quite close to the very aggressive mode 4. Of course, these results have to be confirmed by more extensive tests. Mode 17 seems to be ideal for any kind of postprocessing as, for instance, in RemoveDust. Previously mode 4 was used for this purpose and to avoid excessive softness it had to be corrected by TemporalRepair. This correction is no more necessary with mode 17. Compression should improve slightly while softness is reduced.

There are also substantial changes for the Repair filter. Modes 5-9 have now become useful by including the center pixel. Mode 9 repairs all thin lines, mode 5 now behaves similaroutput files to mode 2 and modes 6-8 are between mode 5 and 9. Of course, if mode 9 is used in RemoveDust, it has negative impact on compression (are 5-20% larger). However, mode 9 is ideal for the new RemoveDirt. It is used for cleaning the non-static areas, which are identified by the filter RestoreMotionBlocks of the new RemoveDirt plugin, and keeps these areas quite sharp, while for the static areas Repair isn't applied at all. My first tests with this new RemoveDirt script function show slightly better compression than with RemoveDust (with Repair(mode=2)) with significantly sharper non-static areas.

[scharfis_brain]

The above thread contains also a claim by scharfis_brain, that the deinterlacer RGDeinterlace contructed with RemoveGrain is a "superblurrer", which blurs static areas. This is false as the test scripts in the above thread show, but scharfis_brain is not willing to withdraw his claim.

I dislike, when poeple do big assumptions like I
would lie about test results (1) or my will to answer again (2).

1) I now DLed the new version an now with the new version rgdeinterlace dosen't do the overall superblurring.
But with the versions before, it definitely did!
But now it also leaves some combing behind.
If you want to, I can send you the my all purpose test clip with this cute asian girl.

2) The only thing I saw was: superblurring.
no matter what I did. so I decided to gave up to discuss this and went to non-computerized holidays
(oh, and they were really nice. Driving my Folding bike and so :) )

And now, the day I am back at home,
you seem to have solved the blurring problem yourself.

[Didée]

Thanks for the new version, kassandro! Mode=17 was a good idea.

... I do have a question. One might call it a request.

These days I've come across an application where the following is desireable to do:

> IF { Repair(clip1, clips, mode=1) does change a pixel }
> THEN { apply Repair(clip1, clip2, mode=3) to that pixel }

where the actual values might change. Would it be possible to implement a version of RemoveGrain with a *pair* of modes as arguments, like e.g.

> TestRepair(clip1, clip2, TestMode, RealMode )

or perhaps (since RealMode > TestMode must be true anyways, and probably easier to integrate into the current framework of the plugin)

> TestRepair(clip1, clip2, mode, mode_offset)

Because, basically the operation is extremely simple. However, if it has to be simulated through scripting, things become annoyingly slow:

> test = Repair(clip1, clip2, mode=1)
> real = Repair(clip1, clip2, mode=3)
> test_diff = difference(source,test)
> test_diff = test_diff.binarize()
> MaskedMerge(source, test, test_diff)

This is pretty much work for a close-to-trivial operation. And to make things worse: In the given case of application, this operation has to be done several times (2, 4, 6 or even 8 times), and therefore is not really practical.
But I assume when done by a plugin itself, it should be breezingly fast, since all work is still done on 3*3 pixels only.

BTW, it's not intended for some obscure special operation. A nice & handy all-purpose application, probably of public interest, is what I have in mind ;)


EDIT:

Seems I hadn't my brain with me at 3 o'clock in the night.
Functionally, the description was correct, but I'm aiming at Repair(), not at RemoveGrain() ... :o

Corrected.

[Boulder]

I can't get the new RemoveGrain working.

SegmentedAVISource("e:\temp\captures\lapparit_2.avi",pixel_type="YUY2")
LeakKernelBob(order=1,sharp=true,threshold=10)
Interleaved2Planar()
RemoveGrain(mode=17)
Planar2Interleaved()


---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
RemoveGrain: only planar color spaces are supported
(E:\Temp\Captures\testi.avs, line 4)
---------------------------
OK
---------------------------

I also tried the SSE versions of the plugins but no luck.

[kassandro]

Since Boulder's problem can be answered quickly, I do it first. The posts of scharfis_brain and Didée will be answered in about an hour.

Boulder, if you want to use RemoveGrain (and all the other filters which do not accept interleaved color spaces) you have to use this filters with "planar=true". The reason for this extra work is explained in the section "Color Spaces" at the end of the RemoveGrain documentation. Here is the corrected script:

SegmentedAVISource("e:\temp\captures\lapparit_2.avi",pixel_type="YUY2")
LeakKernelBob(order=1,sharp=true,threshold=10)
Interleaved2Planar()
RemoveGrain(mode=17, planar=true)
Planar2Interleaved()

[Boulder]

Oh damn, missed that one:o Thanks, I've got some 4hrs of VHS sources to torture with a revised LRemoveDust;)

EDIT: To be 100% sure, could you confirm that this function is OK? (I left TemporalRepair out as you said it can be removed with mode=17)

function LRemoveDust_planar(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input,grey=true)
rep=Repair(clensed, input, mode=repmode, planar=true, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, planar=true, modeU=-1)
return LimitChange(rg, input, limit)
}

YV12 input and the original chroma merged back after the function call, hence the greyscale mode and no Interleaved2Planar/Planar2Interleaved calls.

[kassandro]

@scharfis_brain:
I wanted to be a little bit aggressive but not offensive. Firstly, I never the said that you lied. Instead I claimed that you were wrong, which is a big difference. In the above thread I did concede that you were right, if the color space was not YV12. Actually, my own tests fail, if the ImageReader clip is not converted to YV12. Since your opinion has considerable weight (which is well deserved) in this forum, I wanted you to make some reasonable YV12 tests, in order to get rid of this "superblurrer" branding, which was justified then only for non-YV12. The YV12 code didn't change at all - at least as far as RGDeinterlace is concerned. As you didn't answer my latest contribution in this thread for quite a while, I did launch this little attack. The sole purpose was to make it clear, that your reputation in this forum will be reduced, if you are not able to withdraw from statements, which have turned out to be false. Unfortunately, I had to withdraw many times from some of my statements, because they were wrong. But it is better to withdraw from a statement rather than to hope that it will be forgotten.

As far as residual combs are concerned you are probably right. I would rather prefer to say that there are deinterlacing artifacts. Most smart deinterlacers exhibit some kind of artifacts under 400% magnification in Vdub. On the other hand, The compression results speak for themselves. If the residual combs or artifacts would be significant, these results would be impossible. I claim that except my earlier Deinterlacer AlignFields no other Deinterlacer, which doesn't blur static areas, comes even close to RGDeinterlace (I mean the iterated version of jstelly in the above thread) as far as compression is concerned.


@Didée:
There are three ways to implement your request and before doing it. Clearly the fasted way is to implement for any two modes a routine, which does exactly what you want. If you n modes, then this results in a whopping (n-1)*n routines. That is simply too much. Of course, you may do it for a few selected combinations. But for each mode I use a substantial amount of mmx/sse registers. In order, to avoid unpleasant reloading from the cache, I would have to do both modes simultaneously, which is currently not possible for most mode combinations, because there are not enough mmx/sse registers. This problem will be gone, once we have moved to the Athlon64 platform. On this platform, there are a whopping 16 sse registers, twice as much as we have now. Avisynth may well be the only occassion, where one could really use that many sse registers. Nevertheless, I would have preferred 8 256-bit sse registers instead of 16 128-bit registers.
Altogether I would say that this first and fasted solution to your request, is not feasible right now.
The second approach is to use only one mode but two input clips. If the pixel of the first clip is changed by the specified mode, then the pixel is replaced by the same pixel in the second clip, which may be processed with RemoveGrain(mode=secondmode) or something completely different. This approach is quite feasible right now and the memory access costs are twice as much as with the first approach. With this approach the execution time should be 40-80% longer.
And then there is a third very general approach. It is bases on a filter with three input clips. If the same pixel is different in the first and the second clip, then this pixel in the first clip is replaced by the same pixel in the third clip. This is similar to MaskedMerge, but instead of a mask it uses the difference of two clips. The memory access costs are now three times as high as with the first optimal solution. On the other hand, I can exploit SSE2 much better. Thus the execution time penalty should be only 50-90% with SSE2, if the frames are properly aligned.

Before implementing one of these solutions, some additional consideration are in order. The above "alternative" processing produces similar effects as thresholds. Didée, you and I, we are the only ones to stress repeatedly the principle "use as few thresholds as possible", because of threshold artifacts and there negative impact on compression. I therefore would suggest the following "smooth" version of your request: let y1, y2, y3 be the value of a pixel in three different clips and 0 < M < 256 a fixed constant (chosen as an Argument in the Avisynth script. Letting a = min(|y1 - y2|, M), then y1 is replaced by (y1*(M-a) + a*y3)/M. In other words, if the difference between y1 and y2 is very larger ( >= M ), then y1 is replaced by y3. If it is 0, y1 remains unchanged. Otherwise some "smooth" intermediate values are chosen. In this way, you clearly avoid the nasty threshold like effects. If M is a power of 2, i.e. 1,2,4,8,16,32,64,128,256, then such a general purpose filter can be implemented fairly efficiently with SSE2 being quite helpful.

Actually, in my first version of mode=17, I did implement the following: If for a single pixel mode=4 destroys thin lines, then use mode 1 for this pixel. Although about 2/3 of the mode 4 changes were done exactly the same way also by this unofficial mode 17, the compression results were shocking and this "threshold like" effect must be the sole reason for it. Earlier I was already quite careful, to avoid such effects in Repair and TemporalRepair.

[kassandro]

Originally posted by Boulder
Oh damn, missed that one:o Thanks, I've got some 4hrs of VHS sources to torture with a revised LRemoveDust;)

EDIT: To be 100% sure, could you confirm that this function is OK? (I left TemporalRepair out as you said it can be removed with mode=17)

function LRemoveDust_planar(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input,grey=true)
rep=Repair(clensed, input, mode=repmode, planar=true, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, planar=true, modeU=-1)
return LimitChange(rg, input, limit)
}

YV12 input and the original chroma merged back after the function call, hence the greyscale mode and no Interleaved2Planar/Planar2Interleaved calls.
Yes, it is ok. Actually, Interleaved2Planar/Planar2Interleaved also support grey=true and then only the luma is processed (I didn't test it, though). Being purely temporal, LimitChange works also for
planar YUY2 as long as you do not specify values for LimitU, LimitV, which are different from Limit. LimitChange with LimitU=256 processes only the luma (at least I think so), but this works currently only with YV12. Once SSETools is moved to planar color spaces, it will do so also for planar YUY2, RGB24, RGB32. LimitChange will continue to support interleaved color spaces as well, because it costs only very little additional work. Because Interleaved2Planar/Planar2Interleaved is much faster than ConvertToYV12/ConvertToYUY2 (especially with grey=true), it would be nice if you would try it. Of course, if you use YV12, you don't need "planar=true".

[Boulder]

I think the culprit is RemoveGrain and it's YUY2 performance.

At first I tried feeding YUY2 in that function (with the appropriate changes, of course) and noticed that the process is considerably slower than converting to YV12, processing and then going back to YUY2 and merging the original chroma back. I didn't make any accurate measures, but the difference was something in the region of 25-30%.

I'll make another test as soon as my current encode finishes to verify this, I'll also do a compressibility and a quick artifact comparison between mode=4+TemporalRepair and mode=17 without TR.

[Boulder]

I did the tests and I'm puzzled:D

1)YV12->RD(17,4) operating in b/w->YUY2->MergeChroma : 0.37RT , 22 922 588 bytes
2)YV12->RD(4,4) operating in b/w->TemporalRepair->YUY2->MergeChroma : 0.35RT , 22 273 212 bytes
3)YUY2->RD(17,4) limiting chroma as well : 0.25RT , 26 192 444 bytes

What the heck happens with YUY2 input? I did the same chroma denoising in all cases so that can't cause the difference. The third case looks a bit noisier though.

Based on quick examination, there's not much difference between 1 and 2.

[Didée]

@ kassandro

Sorry, I had one stupid error in my last post: I meant to speak about Repair(), but somehow wrote RemoveGrain() instead ... probably it was too late in the night ;)

I've to think a little about your three suggestions, to not miss arising aspects.

But to make the task easier: Since in fact a clip shall be repaired by another one, the only modes in question are the modes 1 to 4, i.e. full evaluation of the 3*3 square. The higher "line" modes are of no use in this very case.
Therefore (and for the fact I've only little clue about that lowlevel coding you do ;)), I thought it should be not too hard to implement:

For each pixel being processed:

a = source
b = repair( mode = mode1 )
c = repair( mode = mode2 )

(b != a) ? c : a

In words: do two repairs, and if the first did something, use the second one instead.

Your lastly proposed, alternative method here is not suited, I think. What I'm actually doing is post-processing frames that have been motion compensated. Generally, Repair(MC_frame,orig_frame) does an awesome job in removing blocks in those places where MC has failed, and makes the MC'ed frame perfectly suited to be included in the aftercoming temporal processing. In fact, the "repaired" blocks become like the original, but with some sort of "noise-on-their-own", which is an side-effect that appears like a gift :)
The downside is: The MC'ed clip has to be Repair()'ed by at least mode=3, to sufficiently remove the block artefacts. But when working with a fixed mode=3, then all those areas where MC was successful are processed way too strong, and the end result of MC'ed denoising therefore turns out too soft, to not say blurry.

Currently, things like turning this (http://img156.exs.cx/img156/2999/01thenoise3hl.png) into this (http://img156.exs.cx/img156/3091/092xltsmc4au.png) are possibe: Insane amount of noise can be removed, and the result is *calm*. But when working on rather clean sources with only little noise, things get too soft if a fixed repair mode of 3 is used.

That's the problem I'm currently facing.

[kassandro]

Originally posted by Boulder
I think the culprit is RemoveGrain and it's YUY2 performance.

At first I tried feeding YUY2 in that function (with the appropriate changes, of course) and noticed that the process is considerably slower than converting to YV12, processing and then going back to YUY2 and merging the original chroma back. I didn't make any accurate measures, but the difference was something in the region of 25-30%.

25-30% of the frame serving process would be acceptable, but 25-30% of the encoding process would be a lot. If you use grey=true, then in both cases the same amount of pixels is processed in the same way. From this point of view, there should be no difference. However, in the planar YUY2 case 720 Y values are followed by 360 U Values, then 360 V values and then again 720 Y values, while in the YV12 case 720 Y values are followed by 720 Y values etc. (the clip width is 720). Thus in the planar YUY2 you have always an L1 cache miss at the end of each line, which you don't have in the YV12 case, but this slight L1 cache inferiority cannot cause such a huge performance difference. The YUY2 version uses also more memory, because the YUY2 frames are 33% larger than the YV12 frames. But with 512MB this should be irrelevant. There must be other causes.


1)YV12->RD(17,4) operating in b/w->YUY2->MergeChroma : 0.37RT , 22 922 588 bytes
2)YV12->RD(4,4) operating in b/w->TemporalRepair->YUY2->MergeChroma : 0.35RT , 22 273 212 bytes
3)YUY2->RD(17,4) limiting chroma as well : 0.25RT , 26 192 444 bytes

The difference between 1) and 2) is too high compared with the very short benchmark done by myself. Remember that in my benchmark mode 17 and mode 4 were compared on an equal footing. In your case, mode 4 was tamed by temporalrepair while mode 17 was not restricted and nevertheless the difference is higher on a percentage level. On the hand, denoising truely static areas doesn't impact compression very much.
Now the difference between between 1) and 3) is dramatic. There must be something wrong either on my side or on your side. You can check this with difference filter, my main debugging tool, which is included in SSETools. I suggest the following test:

difference(YUY2Filter(input).greyscale(), YV12Filter(input).greyscale())

If non-zero differences are reported to debugview something must be wrong. Note, that you can only compare clips of the same kind with difference (same color space, width and height). It would also be nice to post both scripts here.

[kassandro]

Originally posted by Didée
@ kassandro

Sorry, I had one stupid error in my last post: I meant to speak about Repair(), but somehow wrote RemoveGrain() instead ... probably it was too late in the night ;)

That was a good mistake. Because the main point was the same, only the setup was somewhat simpler, which probably made it easier for me to understand, what is going on.



But to make the task easier: Since in fact a clip shall be repaired by another one, the only modes in question are the modes 1 to 4, i.e. full evaluation of the 3*3 square. The higher "line" modes are of no use in this very case.

The modes 5-9 also inspect to the full 3*3 square, but the neighbours are only used only as line pairs. Modes 3 and 4 are poor, because they ignore the most important pixel, the pixel in the center. However, I can change this. Probably you want mode 3, because mode 4 simply repairs too much and even generates softness like RemoveGrain(mode=4). If you want strong repair, you should try mode=9. It is the sole Repair mode, in my upcoming RemoveDirt script function and so far I like what I see, though the real visual test will come, when I burn all these RemoveDirt videos on a DVD and watch it with my DVD player. My very brilliant but also very slow plasma screen is not appropriate for such tests.



For each pixel being processed:

a = source
b = repair( mode = mode1 )
c = repair( mode = mode2 )

(b != a) ? c : a

In words: do two repairs, and if the first did something, use the second one instead.

The general filter, which I have proposed as the "third approach" can be used also here. But you have here the same "either or". In your very specific case, however, I probably can do also the very fast version and it won't be much slower than mode=3 (of course the center pixel must be included as well). On the other hand, the disadvantages of an "either or approach" remain and although your denoising result in your two pictures is very impressive (you obviously didn't use boring change limiting), but you won't be fully rewarded by the compressor. To make my point more clear, let's say that you have two pixel, which are close too each other. For the first pixel Repair(mode=1) makes no change, while for the second Pixel it makes only a change of 1. Then the first pixel is not touched by Repair(mode=3), while the second may be massively changed by Repair(mode=3). In other words, through your filtering you introduce new discontinuities (you certainly remove also a lot of discontinuities) and this is simply not DCT friendly. Thus you always should avoid such "either or techniques" and replace it by "smooth transition".
[/B][/QUOTE]

[Boulder]

I did the difference test, and there is a substantial amount of difference between the two versions. This is the script I used:

AVISource("i:\lapparit_1.avi",pixel_type="YUY2")
AssumeTFF()
Trim(82655,121925).FadeIn(5).FadeOut(25)
KillAudio()
LeakKernelBob(order=1,sharp=true,threshold=10).AssumeTFF()
yv12=ConverttoYV12().LRemoveDust_planar(17,4).ConverttoYUY2()
yuy2=Interleaved2Planar().LRemoveDust_planar(17,4).Planar2Interleaved()
difference(yv12.GreyScale(), yuy2.GreyScale())

function LRemoveDust_planar(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input, grey=true)
rep=Repair(clensed, input, mode=repmode, planar=true, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, planar=true, modeU=-1)
return LimitChange(rg, input, limit)
}

This is a snippet of the debug log:

[3420] [46] total difference = 1356, different pixels = 1101
[3420] [47] total difference = 10954, different pixels = 7634
[3420] [48] total difference = 35615, different pixels = 21908
[3420] [49] total difference = 56094, different pixels = 35110
[3420] [50] total difference = 88070, different pixels = 53263
[3420] [51] total difference = 118387, different pixels = 74082
[3420] [52] total difference = 152732, different pixels = 90588

The output of the YUY2 one looks rather odd. See these screenshots:
YUY2
http://www.saunalahti.fi/sam08/rd_yuy2.jpg
YV12
http://www.saunalahti.fi/sam08/rd_yv12.jpg
Similar behaviour occurs even if I set mode to something else than 17. Looks like chroma is screwed up somehow but there's difference in the luma channel as well. This can be seen when the original chroma information is merged back to both YV12 and YUY2 branches. The YUY2 version is less aggressive and leaves a slightly noisier video.

[kassandro]

Thanks for your test, Boulder. I see no problem with your script. The problem must be on my side. My first suspect is the YUY2 version of LimitChange. The low level routine is the same for both YUY2 and YV12, but I may have false arguments to the low level routine in the YUY2 case. Because Didee's LRemoveDust was based on YV12LUTxy, which unfortunately accepts only YV12 input, I could make comparisons only in the YV12 case.
I will make my own tests to reproduce the problem. I am too tired to make these tests now. If I am locked out by neuron2 tomorrow, I probably can say more.

[kassandro]

I just did reproduce the problem. My guess seems to be correct. I used the following script

function LRemoveDust_planar(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input, grey=true)
rep=Repair(clensed, input, mode=repmode, planar=true, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, planar=true, modeU=-1)
return LimitChange(rg, input, limit)
}

input=MPEG2Source("input.d2v", cpu=4, ipp=false)
yv12=LRemoveDust_planar(input,17,4).ConverttoYUY2
yuy2=ConverttoYUY2(input).Interleaved2Planar().LRemoveDust_planar(17,4).Planar2Interleaved()
difference(greyscale(yv12), greyscale(yuy2), tolerance=4)

If tolerance=4, then the difference filter ignores differences <= 4. Note that limit=4!!!. If tolerance=4 then I get zero differences. If I lower the tolerance, then I get non-zero differences. Thus it is very likely that LimitChange messes up the clip. Since I did test the YV12 version, the problem must be in the YUY2 version. I will hunt this bug tomorrow more concretely. Now, if LimitChange messes up the clip, the encoding process slows down, because the input is less smooth. This would also explain the speed differences.

[kassandro]

I just have uploaded a new binary archive. It corrects a silly bug in LimitChange (contained in SSETools). I have also corrected a problem with Avisynth's PLANAR_ALIGNED flag in SSETools. The same problem is in Repair, TemporalRepair, MCClense, but it cannot cause damage. However, these filter may reject valid input clips because of this problem. Since there are no complaints, I will fix it later in version 0.8.1, which will improve Repair(mode=3) to include the center pixel, because Didee has a nice application of this mode.

[Boulder]

I still have the same problem with the fixed SSE2Tools.dll, I also tried SSETools.dll. The same script as above.

[kassandro]

Originally posted by Boulder
I still have the same problem with the fixed SSE2Tools.dll, I also tried SSETools.dll. The same script as above.

Sorry, I am really a fool. I simply ran the test with tolerance=4 and with this tolerance I had already zero differences before. I have to resume the bug hunt.

[kassandro]

Boulder, I hope that it's correct now.

[Boulder]

Yes, looks OK to me. The speed difference remains, YUY2 processing is ~30% slower but then again, there's 25% more data to process.

[Chainmax]

So the new chroma limiting LRemoveDust is now this:

function LRemoveDust_planar(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input, grey=true)
rep=Repair(clensed, input, mode=repmode, planar=true, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, planar=true, modeU=-1)
return LimitChange(rg, input, limit)
}

? If TemporalRepair is removed from the script, shouldn't the new default limit mode be 17 instead of 4 so as to avoid unnecesary mistakes?

[Boulder]

Limit is limit, mode is mode. If you want to use mode=17, you could always set it in the function. Besides, that function is not for all purposes, it's for my YUY2 captures;) See the grey=true and modeU=-1 parameters, I only process the luma because I merge the original chroma back after the conversion YUY2->YV12->YUY2.

[Chainmax]

Sorry, I get mixed up with those. I know that the first parameter in LRemoveDust is the RemoveGrain mode, and since everyone seems to be using LRemoveDust(4,x), I assumed that was the default, and since the only default set to 4 is limit I concluded the wrong thing :o.

So, I could just use the old script and call it with LRemoveDust(17,2) then? Could repair cause issues trying to repair stuff that doesn't need repairing when using mode=17? Also, the inofficial plugins are no longer needed, right?

[Boulder]

You could try this for YV12, and (17,2) should be a good one to try:

function LRemoveDust(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input)
rep = Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
return LimitChange(rg, input, limit)
}
You'll need SSETools from the latest RemoveGrain package. What comes to your question regarding Repair, the behaviour hasn't changed since the original function, that is, Repair deals with video that comes from brutal cleaning with Clense.

[Chainmax]

Oh, then it's TemporalRepair that deals with RemoveGrain's mode 4. So, what would be the effects of leaving its line on is using mode 17 instead of mode 4? By the way, aren't you forgetting the planar=true switches in the rep and rg lines?

P.S: from reading the penultimate page, it seems that you and kassandro have different results on wether M17 and M4+TR are really the same or not. Have you got any more tests? Maybe I can try testing it on my X-Men sample...

[Boulder]

Mode 17 should keep the thin lines whereas mode 4 is much more brutal. Putting TR would make the filter a bit less aggressive but I don't know if it would be noticable. The hard limiting already helps so I've left TR out with mode 17.

With YV12 input planar=true is assumed already so no need to put it there.

[kassandro]

Originally posted by Chainmax

P.S: from reading the penultimate page, it seems that you and kassandro have different results on wether M17 and M4+TR are really the same or not. Have you got any more tests? Maybe I can try testing it on my X-Men sample...
M17 and M4+TR are quite different. TR removes any changes made by m4, if there is no motion or temporal noise. This is necessary because m4 destroys thin lines, which the human eye simply doesn't like. M17, on the other hand, should preserve thin lines. Thus there is no need to apply TR. Of course, M17 effects also static areas.

Originally posted by Boulder
Mode 17 should keep the thin lines whereas mode 4 is much more brutal. Putting TR would make the filter a bit less aggressive but I don't know if it would be noticable. The hard limiting already helps so I've left TR out with mode 17.

M17 is remarkably close to M4 as the following script shows:

input=avisource("my.avi")
M17=RemoveGrain(input, mode=17)
M4=RemoveGrain(input, mode=4)
difference(difference(M17, input), M4)

Then for each frame first the difference between M17 and input and then the difference between M17 and M4 is reported in debugview. Thus M17 performs just like M4 as long as thin lines are not affected.
Keeping thin lines seems to be also the basic denoising strategy of digicams. I have two digicams, a Canon A70 and a Ricoh Caplio GX, which are antipodal. The Canon performs very aggressive denoising, while the Ricoh does a very poor denoising job. While the Canon sometimes shows very visable denoising artifacts, it is nevertheless always getting good reviews. The Ricoh, on the other hand, is getting battered for its noise performance in reviews.
Thus one should not minimise the changes with respect to the original, rather it all about deceiving the human eye. Of course, M17 generates also some softness, but it is not the destructive sofftness of M4.
I will perhaps write a scientific paper about the ideal properties of M17, because it involves some mathematical reasoning to actually show that thin lines preserved. This mode would be ideal for digicams.

[Boulder]

Originally posted by kassandro
Of course, M17 generates also some softness, but it is not the destructive sofftness of M4.

What I noticed when trying M17 in RemoveGrain only instead of LRemoveDust, is that it's way too blurry for my taste. In fact, M2 already started softening the image noticably. Since then, I've used M5 for DVD sources, M17 within LRemoveDust for captures and M2 for the rare MPEG4 clips I've processed.

[kassandro]

Originally posted by Boulder
What I noticed when trying M17 in RemoveGrain only instead of LRemoveDust, is that it's way too blurry for my taste. In fact, M2 already started softening the image noticably. Since then, I've used M5 for DVD sources, M17 within LRemoveDust for captures and M2 for the rare MPEG4 clips I've processed.
I'm quite surprised, that you see already softness coming from M2. If you look at the difference with the original, it is relatively small compared with M17. It would be interesting if you could post some pictures (with and without M17). Digicam manufacturers usually do some sharpening after denoising to get rid of softness. I usually turn off sharpening by chosing the softest mode, which also preserves the most details.

[Boulder]

I'll get some screenshots for you a bit later tonight. The difference between 2 and 5 isn't big but 2 appeared to affect the brightness of small objects which then looks like they were softened considerably. I can't explain it better but the screenshots should show the behaviour. The DVD I tested it on was Star Wars Episode 4 which has a sharp and clear video so it'll reveal any excessive detail loss well.

[Boulder]

All right, here are the screenshots. I didn't attach them to the post but posted the links, eight large pictures per post is not nice;)

Scene 1, original (http://www.saunalahti.fi/sam08/norg_1.jpg)
Scene 1, mode=5 (http://www.saunalahti.fi/sam08/mode5_1.jpg)
Scene 1, mode=2 (http://www.saunalahti.fi/sam08/mode2_1.jpg)
Scene 1, mode=17 (http://www.saunalahti.fi/sam08/mode17_1.jpg)
Scene 2, original (http://www.saunalahti.fi/sam08/norg_2.jpg)
Scene 2, mode=5 (http://www.saunalahti.fi/sam08/mode5_2.jpg)
Scene 2, mode=2 (http://www.saunalahti.fi/sam08/mode2_2.jpg)
Scene 2, mode=17 (http://www.saunalahti.fi/sam08/mode17_2.jpg)

The second scene shows that mode=2 affects small details a little too much whereas mode=5 is almost unnoticable.
The script contains only MPEG2Source and cropping the black borders off before RemoveGrain.

[kassandro]

Thanks, Boulder, for the pictures. I will discuss a few details tomorrow. In the first picture clearly the space ship gets softened a lot by M17. You see some slight softening already with M2, but it is much closer to the original then to M17. When I say "thin lines are preserved", then I have to add "but not the end points of these lines" at least for M17. Thus M17 may shorten thin lines by two pixels. If these thin lines are very short as in the fine details of the space ship, then details are lost. M2 even preserves the end points of thin lines. In my tests I always had longer thin lines, hair, for instance, but if a thin line is only 3 pixels long, only the pixel in the middle is safe, while the other two may be erased.

[vcmohan]

Originally posted by Boulder
All right, here are the screenshots.

For experienced users the comparison may be understandable but for me to flit from one photo to other and remembering details is too much. Possibly horizontal stacking original with the intended compare image will be more easily appreciated. For one seeing the images individually I could not appreciate the differences.

[Boulder]

Stacking them doesn't do much good because the images are large in resolution. Use IrfanView for example, or any other program that can do slideshows. That way you'll see the differences very clearly when switching fast from image to another.

[kassandro]

Originally posted by Boulder
Stacking them doesn't do much good because the images are large in resolution. Use IrfanView for example, or any other program that can do slideshows. That way you'll see the differences very clearly when switching fast from image to another.
I have no problem to see the differences. In fact, Irfanview is my only picture viewer. It has the advantage, that one retrieve the exact RGB values of a pixel by clicking on it, which is quite important in order to make a very fine analysis.

Yesterday I mentioned, that M17 may destroy the two endpoints of a thin line. I must say also that Repair(mode=9) restores thin lines - but again without the two endpoints. Thus, if you have a moving white line with a black background, clense erases this line, Repair(Mode=9) restores this line without the two endpoints and M17 erases also the new endpoints. Thus altogether, you may loose 4 pixels with this kind of filtering. TemporalRepair would not help at all, because it restores only static areas and the line is moving. But, of course this is a rather exceptional situation.

However, in my view line shorting is only responsible for the smaller part of the softening. Most of softening of M17 and virtually all of the softening of M2 is due to the YV12 effect. If we apply these filters to the YV12 chroma the aggressiveness of these filters is quadrupling. All the edge softening should be caused by this effect. Already a conversion from RGB to YV12 softens the edges quite a bit.

I have used the lossless jpegcrop to extract some small parts of the original second picture for discussion.
In the first two pics you see how well M17 preserves thin lines. In the second you already see the line shortening effect a little bit, though.
http://home.pages.at/kassandro/pics/Boulder1.jpg http://home.pages.at/kassandro/pics/Boulder2.jpg
In the third, you can see quite a few small details, which are shortened by M17:
http://home.pages.at/kassandro/pics/Boulder3.jpg
Finally, in the fourth pic you a cluster of sprayed black pixels, which is softened strongly by M17, but the human eye will not really see this as a negative.
http://home.pages.at/kassandro/pics/Boulder4.jpg
You can't really do anything here, because these pixels look very much like noise. Any reasonable denoiser will do some destruction here.

[Boulder]

Yes, I made similar observations, and that's why I've decided to stick to M5 followed by TemporalSoften(2,3,4,6,2) with DVD sources as they are usually already quite clean.

For analog captures LRemoveDust with M17 and limit 4 is a very good choice, it leaves just enough noise in the video to make it look good. My eyes stand slight (or even moderate) noise much better than oversmoothed video, this applies especially to human skin which looks horrible when there's even a slightest amount of smearing.

[Chainmax]

kssandro: if Mode5 really is that similar to Mode2 while producing less smoothing, could Mode17 be altered so its secondary mode is based on Mode5 instead?

Boulder: it makes no sense for me to have two almost identical scripts, one for YUY2 and one for YV12, so I will use the following one for both cases:

function LRemoveDust_planar(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input)
rep=Repair(clensed, input, mode=repmode, planar=true)
rg = RemoveGrain(rep, mode=_mode, planar=true)
return LimitChange(rg, input, limit)
}

[Boulder]

The reason why I have two functions is that the one I use for my YUY2 captures deals only with luma and is a little faster because chroma processing is disabled.

My process is to convert to YV12, use LRemoveDust, convert back to YUY2 and merge the original chroma information back. This way I get a lossless YUY2->YV12->YUY2 conversion and faster processing. I've tried feeding YUY2 data directly into LRemoveDust but it's a lot slower because the amount of data to process is 25% larger.

[kassandro]

Boulder,
I initially wanted to release a new version of RemoveGrain, but I postpone to the next weekend to implement some ideas - mostly for Repair - to improve "thin line shortening" at least a little. I still don't understand, why YUY2 is so slow. On the other hand, I can reuse the low level functions for Interleaved2Planar(grey=true), Planar2Interleaved(grey=true), to extract the luma of a YUY2 frame into an YV12 frame with a random chroma and then the way back from an YV12 frame into a YUY2 frame with the chroma set to no color. Both routines should be much faster than ConvertToYUY2 and ConvertToYV12.

[Boulder]

The speed difference is quite likely just because of the 25% of increase in the data that needs to be processed - as it probably means less cache (L1/L2) hits and a larger amount of memory and disk access, it's not surprising the difference can be 35%.

Your plan sounds interesting, hopefully I'll have some time to test it as I'll have a very busy month ahead with my studies as we have to design and implement a database :scared: If it works as planned, it should definitely be much faster since all the chroma processing can be forgotten, all that needs to be done is merging the original chroma back.

[jarthel]

newly installed winxp setup. installed avisynth and run dvd2avi on a vob.

when I tried to load an .avs file with removegrain in it, vdubmod won't load it. if I comment out the "load removegrain" and the removegrain function call lines, I can load the .avs file into vdubmod. any ideas?

here's the script
import("C:\Program Files\NuMenu4U\new.avs\addaudio.avs")
LoadPlugin("C:\Program Files\NuMenu4U\new.dvd2avi\mpeg2dec3dg.dll")
LoadPlugin("c:\downloads\software\windows\videos\DVD authoring\avisynth\plugins25\decomb.dll")
Mpeg2Source("C:\dvd_images\originals\needs_reencoding\ORPHEN_VOLUME_FOUR\D_VTS_01_0\FRAMESERVING\001.d2v" ,idct=7,iPP=true)
FieldDeinterlace()
LoadPlugin("c:\downloads\software\windows\videos\DVD authoring\avisynth\plugins25\removegrain.dll")
LoadPlugin("c:\downloads\software\windows\videos\DVD authoring\avisynth\plugins25\removedirt.dll")
removegrain()
removedirt()
AddAudio()
ConvertToYUY2()

======
update: as I was writing this reply, I was installing norton systemworks. I tried opening up the file again and it works now!!! :|

[kassandro]

I have just uploaded the version 0.9 to the web site (http://www.removegrain.de.tf).
Various new repair modes have been added. Repair modes 3,4 have been improved (they now include the center pixel). I have added a new mode 18, which preserves thin lines and has a better chance than mode 17 to preserve the end points of thin lines as well. However, compression will probably not be comparable with mode 17, which is still my preferred mode. I have added the filters ForwardClense and BackwardClense to RemoveGrain. These filters are primarily for handling sharp scene switches in the RemoveDirt script function (a new version will be released in 1-2 weeks). The speed of the filter Clense, especially the SSE2 version, has been improved substantially.
TemporalRepair has now also the modes smooth=2,3 which are primarily for deinterlacing.
I have noticed a problem with the SSE3 version. If it is used together with avisource VirtualDubMod crashed upon exit. This is probably a problem with the Intel compiler (it seems to handle virtual destructors somewhat differently) and I can't fix it right now.

[Poutnik]

I have encountered troubles in RemoveDust/Dirt scripts with optional parameters of script. I have get "Not defined var(or so)" error in Vdub avs debugging. This disappeared only after changing default(var,value) -> var=default(var,value) Seems like "default(val,value)" assigns value to last, at least not to var.

There are test functions isplanar(), isinterleaved(). So why not use them ? I do conditional planar<->interleved conversion in my scripts.


.....
il=isinterleaved(input)
input = (il==true) ? Interleaved2Planar(input) : input
"Filtering"
return (il==true) ? Planar2Interleaved(last) : last


BTW, I like speed and approach of your filters very much, Kassandro.
What do you think about using strong DCTFilter with TemporalRepair(+Repair?). Something like "poor man" 2D variant of Fizick's FFT3DFilter.

Post Scriptum: Could you put an overview table of filter modes/their usage in your HTML docs ?. I am getting lost as there are so many ones....

[kassandro]

There are test functions isplanar(), isinterleaved(). So why not use them ? I do conditional planar<->interleved conversion in my scripts.

You can't use isplanar(), because planar YUY2 is format, which is not supported by Avisynth. For Avisynth planar YUY2 clips are identical with ordinary interleaved YUY2 clips. RemoveGrain/Repair/RemoveDirt are - so far - the only plugins which use this color space. I have done so, because for using SSE/SSE2/SSE3 one has to convert internally the interleaved format into a planar one. With planar YUY2 this conversion has only to be done once for a script function, which involves only filters from the above three plugins. This saves a lot of time.


I have encountered troubles in RemoveDust/Dirt scripts with optional parameters of script. I have get "Not defined var(or so)" error in Vdub avs debugging. This disappeared only after changing "default(var,value) -> var=default(var,value). Seems like "default(val,value)" assigns value to last, at least not to var.

Thanks, I will correct this. There will be a new version 0.9 of RemoveDirt plugin tonight or tommorrow. This will also correct two unpleasant bugs, which were not destructive enough to be discovered immediately. Also functionality is improved quite a bit especially as far as scene switches are concerned.

[Poutnik]

If supposing usage removedust().removedirt() (without cropping inbetween), are the scripts optimized for this way ?
Or is better to construct a superscript "RemoveDustAndDirt" ?

[kassandro]

Originally posted by Poutnik
If supposing usage removedust().removedirt() (without cropping inbetween), are the scripts optimized for this way ?
Or is better to construct a superscript "RemoveDustAndDirt" ?
No you shouldn't do that. The result will simply be too soft. As you can see from the discussion above, for some people RemoveDust is already too soft and Didee modified RemoveDust by simply limiting the amount of change. Because the chroma is much more sparse than the luma in YV12, RemoveDust is considerably more aggressive on the chroma. On the other hand, the chroma is especially noisy, if it originates from analog capturing. The RemoveDirt script should creat less softness, while maintaining the same or even a slightly better compression ratio, but RemoveDirt bears also some artifact risk depending of course on its configuration. I personally use RemoveDirt now, but I still in the testing phase.

[FredThompson]

When I have any of the 4 variants of the RemoveGrain package in my Avisynth/plugins directory VirtualDubMod will not load. It reports a Visual C++ error that the application requested to terminate in a non-standard manner.

Why would this happen?

[Boulder]

Which VDubMod version do you use? It works on my system with both 1.5.4.1 and 1.5.10.1 just fine.
Which Avisynth version do you have?

[midelic]

The same It happened to me ,same error, but i don't think removegrain is causing this.Any other filter I put in avisynth plugin directory gave me this error.Even Ben's plugin invert.dll gave me the same error.My avisynth plugin directory is full (has removegrain,remove dirt,almost all plugins)but wen i try to load a new one I receive this error.
Avisynth 2.55
Virtualdubmod 1.5.10.1

[FredThompson]

I'm using VirtualDubMod 1.5.10.1, both the base release and the latest update will crash. AviSynth is beta 050505.

[Boulder]

The Avisynth version is the problem I think. It's very fragile as you can read from the topic which contains the discussion around the CVS builds.

[FredThompson]

The AviSynth build doesn't have anything to do with the crashing.

Removing all the filters and testing with different combinations shows the SSE2 and SSE3 builds will cause a crash. SSE3 should be expected, it's not in the Athlon XP. SSE2 should load. This is a minor annoyance, as the 2 simplest vesions will load.

There is some kind of conflict between RemoveGrain, Repair, and SSETools when any of the following are in the plugins directory:

rawsource (200504250, TDeint 1.0 beta 3, TIVTC 0.9.8.5 and warpsharp (2003_11030).

VirtualDubMod will load properly for me with either group 1 or 2, not elements of both. SSE2 should load but it's not. Maybe the Athlon XP model 8 is an incomplete SSE2 or maybe there's something in the Intel CPUs which is actually SSE2+, so to speak.

-- edit: well, that's not entirely true. Oddly enough, it seems the order in which I copied the filters into the filter directory affects crashing. Maybe there is a sequence of filter loading which causes the crashing.

[Wilbert]

Removing all the filters and testing with different combinations shows the SSE2 and SSE3 builds will cause a crash. SSE3 should be expected, it's not in the Athlon XP. SSE2 should load.
Like i said in another forum. Athlon XP doesn't support SSE2, only SSE and iSSE.

Besides, i think we just have to live with this, ie plugins causing crashes in the plugin folder.

[Boulder]

Which Athlon XP has SSE2 instructions support? I thought that was available only for some Semprons and A64-variants.

EDIT: Wilbert was faster..

[FredThompson]

I'm sorry, you are both correct. I thought wcpuid said SSE2 support was present but I was wrong.

Turns out VirtualDub doesn't crash but VirtualDubMod does. Durn...

[midelic]

Just now i observed ,you are right,only virtualdubmod is crashing.

[vigi_lante]

So, Boulder, what is your script at this point ? I guess it's with LRemoveDust(17,2). But what LRemoveDust.avs script are you using now ?

Could you recommend a more strong setting in order to remove even more noise ? Noise is important to me, because with it I can eliminate blocks. But since I'm working with cartoons (analog TV capture), a smooth look is no problem, but if I remove to much noise, blocks will appear.

So, I'm looking for a balance between smooth and noise.

Thanks!

[Boulder]

I use either one of these two, depending on what the original colorspace is:

function LRemoveDust_YUY2(clip input, int clmode, int "limit")
{
clmode=default(clmode,17)
limit=default(limit,2)
repmode = 2
clensed = Clense(input, grey=true)
rep = Repair(clensed, input, mode=repmode, modeU=-1)
rg = RemoveGrain(rep, mode=_mode, modeU=-1)
return LimitChange(rg, input, limit, limitU=255)
}

function LRemoveDust_YV12(clip input, int clmode, int "limit")
{
limit=default(limit,2)
clmode=default(clmode,17)
repmode = 2
clensed = Clense(input)
rep = Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=clmode)
return LimitChange(rg, input, limit)
}

--
My usage for YUY2 sources is
xxxSource()
org=last
ConverttoYV12()
LRemoveDust_YUY2(17,2)
ConverttoYUY2()
MergeChroma(org)

and for YV12

xxxSource()
LRemoveDust_YV12(17,2)
--
If you need more cleaning, you can raise the value for limit, however, there's no magical setting that will clean but leave all the details. I've found out that limit=2 is a good compromise.

[vigi_lante]

In my case, I'm capturing with Huffyuv (YUY2) and my target is to compress it with XviD (YV12). So, which script should I use ?

[Boulder]

Convert to YV12 before LRemoveDust and use LRemoveDust_YV12().

[KillNoise]

First welcome back to this forum, Kassandro !
I have been worried about your disappearance after May 7th with that strange note on http://www.removegrain.de.tf/ saying you were locked out from Doom9 Forum and to use VideoProcessing.Forumer.com (which turned out as dead link for me) neither any response to e-mail at gorw@gmx.de (so was afraid you might have left your excellent work unfinished).
Thank you again for this very usefull, versatile and fast filter set - my favorite for denoising tasks.


Looks like i found two Bugs in RemoveGrain 0.9
(don't trust either my understanding or my CPU/system otherwise)
==> can anyone confirm these bugs on his machine, please ?

Use RemoveGrain latest Ver. 0.9 of 2005-05-01 with AviSynth 2.55 of 2004-09-01 on Pentium 4 CPU (Hyperthreading enabled) under Win XP SP2



1) RemoveGrain(mode=3) Bug in RemoveGrainSSE2.dll
------------------------------------------------------------

RemoveGrain(mode=3) seems to erode bright edges much stronger than it is supposed to; effect is clearly visible at bright letters on dark backgroung (e.g. movie credits).
So please try:

LoadPlugin("RemoveGrainSSE2.dll") # bug is in SSE2 version (RemoveGrain.dll seems to behave properly)
return StackHorizontal( source.RemoveGrain(3), source.RemoveGrain(4) ) # mode 3 supposed to be less agressive

Remarkable: some (about 16 ?) pixels at right and left borders seem to be correctly processed (recognize slight edge in texture ==> probably some different code for border allignment range).

With LoadPlugin("RemoveGrain.dll") everything looks as expected.
(==> guess some simple bug in neighbour pixel rank order adressing of SSE2-optimized code for mode 3; did not check SSE3 build, neither whether any other modes may be affected too)


BTW:
I prefer mode = 3 rather than 4, because mode 4 erases corner pixels and so completly removes 2x2-pixel-objekts (which typically make up important details for natural skin appearence in closeup etc.) Proper mode 3 should preserve these mostly. So it would be fine to have another more sensitive mode added, simmilar mode 17, but based on mode 3 rather than 4, so that it should preserve thin lines as well as corner pixels and 2x2-pixel-objekts. (Maybe that is what mode 18 does, but it was not any explained in RemoveGrain.htm document)





2) Very strange behaviour of RemoveGrain() after Clense()
---------------------------------------------------------------------------
There seems to be something wrong when using RemoveGrain after Clense (Maybe some problem with RemoveGrain and Clense inadmissible sharing static variables or frame cache buffers ?):
Looks somewhat like Clense messes up source frames when followed by RemoveGrain, probably requesting avoidable non-sequential seek operations on source, which would substantially slow down performance and i already experienced that at least DirectShowSource may return different frames on random seek and sequential access (seems to skip frames sometimes, returning previous frame for new number, but this might also be a special problem with my local filter setup involving ffdshow-20041012).

Please try this:

SetMemoryMax(256) # to rule out any memory limit problems

mSrc = DirectShowSource(pathSource, fRate)#.Crop(224, 160, 256, 256) # may optionally cut CPU load for testing
mSource = mSrc.ShowFrameNumber(scroll=true) # burn in frame number for analysis

mDegrain = mSource.RemoveGrain(17) # first remove spatial grain is to improve clensing results (mode 17 eats corner pixels but keeps thin lines)
mClensed = mDegrain.Clense(grey=false) # temporal min/max-clipping removes fluctuation peaks (messing up moving objects)
mRep = mClensed.Repair(mSource, mode=16) # restore clensed details from source (repair mode 16 can restore endpoints)

mTest = mSource
#mTest = mDegrain
#mTest = mClensed
#mTest = mRep

return Subtract( mTest.RemoveGrain(1), mTest.RemoveGrain(1) ).ColorYUV(analyze=true).Levels(116, 1.0, 136, 0, 255)

(ColorYUV(analyze=true) shows frame number & statiscs. Levels() is used to emphasize delta pixels)
Of course there should be no difference at all, which works as expected with mTest = mSource or mTest = mDegrain.
However using mTest = mRep or mTest = mClensed shows large differences between the two same calls of RemoveGrain - at least on my system !!!


Verifying test with UnDot() instead of RemoveGrain()
return Subtract( mTest.UnDot(), mTest.UnDot() ).ColorYUV(analyze=true).Levels(116, 1.0, 136, 0, 255)
==> shows no difference regardless of clip used for mTest, so the bug must be in the combination of Clense together with RemoveGrain


I verified for different combinations of UnDot() and RemoveGrain(), then tried to identify differences with
return StackHorizontal( mTest.RemoveGrain(1), mTest.RemoveGrain(1) )
==> turned out extremly slow with ffdshow-20041012 (much faster after uninstalling ffdshow); sometimes i saw different frame

contents with same(!) ShowFrameNumber burned in. Otherwise no visible difference, but Subtract() uncovers that the two calls of RemoveGrain do not produce same results. (Suspect Clense unnecessarilly recalculates with having differnt neighbour frame contents served on subsequent calls.)


I first stumbled on this strange effect when i looked at changes made by RemoveGrain(1) alone:
return Subtract( mRep.RemoveGrain(mode=1), mRep ).ColorYUV(analyze = true)
it is expected to show some isolated pixels clipped to threir neighbors (as UnDot does) but wondered why i saw connected clusters of pixels.

[kingmob]

NM, my problem was fixed by updating avisynth to the latest beta :o

[jarthel]

any parameter recommendation for anime sources? thanks

[J-Wo]

I'd like to know how to use RemoveGrain on hard-telecined material. I have some episodes of Veronica Mars R1 (season 1 disc 1) which are hard-telecined, i.e. encoded at 30fps but with a 3:2 mixture of progressive/interlaced frames. I'm not too familiar with the bob-weave or seperatefields-weave functions, but is that along the lines of what I should do before using this function? Thanks.

[Boulder]

Do an IVTC to get the progressive frames.

[J-Wo]

Thanks. What about for purely interlaced material, is removegrain a bad choice in this case? The readme leads me to believe so... I've been using Mode=5 with much success on film material (I do a lot of DVD-9 -> DVD-5 using DVD Rebuilder Pro)

[Boulder]

You can't use RemoveGrain directly on interlaced material. There's a lot of threads which discuss processing the issue.

[J-Wo]

Thanks boulder. Do you have any suggested filters I should look up for processing interlaced material similarly to RemoveGrain? I just want to do a little grain removal to aid with compression

[Didée]

any suggested filters I should look up for processing interlaced material similarly to RemoveGrain?
RemoveGrain is, well, sort of a "graded" median filter. There is no other filter of that kind.

The most simple way to use RemoveGrain on interlaced footage is
SeparateFields() .RemoveGrain(x) .Weave()

(Compare e.g. this here (http://forum.doom9.org/showthread.php?p=933541#post933541) - it's not exactly textbook level:o, but covers the basics.)


And regarding

(I do a lot of DVD-9 -> DVD-5 using DVD Rebuilder Pro)
[../..]
I just want to do a little grain removal to aid with compression
you might want to give a try on SPresso. After some playing with limit/bias/RGmode to find settings of your taste, you might get a better ratio of [preserved detail]:[compression gain] than with just using RemoveGrain on its own.

[J-Wo]

thanks for the suggestions Didée! Just wondering, does a filter like RemoveGrain require bobbing on interlaced footage. I'm afraid I'm pretty new to this whole processing interlaced material thing. Also in your script above, can I substitute your SPresso() filter for RemoveGrain()?

[Netuser]

the site http://www.removegrain.de.tf is not working for me, is it down forever ? where can i get the dll and degrain script please ?

Thanks alot :)

[Nightshiver]

It wokrs fine for me.

[thetoof]

Works for me too... but took a while to load.

[Netuser]

Yes finaly it worked for me too, it took hell of time to load lolzz

thanks

[papcom]

I tried to acquire "removegrain" binary via the official link on it's Website but I am landing in the nowhere with an error 404.

Does anyone has an alternative download for this plugin?

[StainlessS]

The Wiki is always a good place to look: http://avisynth.nl/index.php/Removegrain

[colours]

You might want to get RgTools (https://github.com/tp7/RgTools) instead.

Results should be identical to the original RemoveGrain for all modes other than 21 (the original has incorrect rounding) and the convolution modes (modes 11/12/19/20).

[Atlantis]

I just realized they have added TemporalRepair to RgTools v0.98.

So after I use RemoveGrain, what should I use, Repair or TemporalRepair? What's the difference in the results?

[Atlantis]

You are not going to believe this. I searched RemoveGrain and came here to say I just discovered TemporalRepair, how good is it?
Only this time I'm actually testing it! :D