RemoveGrain

[kassandro]

RemoveGrain is a spatial denoiser for progressive video. Because of SSE optimisation, it is fairly fast (above 100 fps on my whimpy 1.3 GHZ Celeron). There are now significantly faster SSE2, SSE3 versions. For more details go to www.RemoveGrain.de.tf (http://www.RemoveGrain.de.tf).
16|07|04 version 0.4 (first public version) released.
17|08|04 version 0.5 released (for a change log see the posting below)
12|09|04 version 0.6 released (for a change log see the posting below)
13|02|05 version 0.7 released (for a change log see the posting below)
09|04|05 version 0.8 released (for a change log see the posting below)
01|05|05 version 0.9 released (for a change log see the posting below)

[Boulder]

Nearly missed this one..I'll test it tomorrow on my P4 system:)

[Boulder]

I can't get the SSE2 version working. I extracted first both RemoveGrain.dll and RemoveGrainSSE2.dll in my plugins folder and then tried removing RemoveGrain.dll. In both cases I got an error message "there is no function named DRemoveGrain".

[ARDA]

Same here; and would like to test this filter

Thanks ARDA

[kassandro]

Sorry, I was a little bit too fast and too tired (after midnight), when I rushed out the plugin. Actually RemoveGrainSSE2.dll was the non-test version, which I wanted to release only after satisfactory testing. I hope that it's correct now. Thanks in advance for testing.
I also made some slight corrections in the documentation.

[Boulder]

OK, I tested the SSE2 version, and there were no differences in the DebugView log. However, the only modes I got working were 5,6,7 and 8, all the others gave an access violation when the script was loaded in VirtualDub. I used a 720x576 MPEG-2 capture as the test material.

Other than that, the filter seems to work very well, mode 8 did an excellent job with my TV caps when followed by RemoveDirt(). Which reminds me, are you planning on adding SSE2 optimizations to RemoveDirt as well?

[kassandro]

First, many thanks for testing.

Originally posted by Boulder
However, the only modes I got working were 5,6,7 and 8, all the others gave an access violation when the script was loaded in VirtualDub. I used a 720x576 MPEG-2 capture as the test material.


Hmmm, was this a read access or a write access violation? Did the access violation also appear with the non-SSE2 version (the only one I could test)? Does the access violation persist if you crop before RemoveGrain with crop(4,4,-4,-4, align=false)? Finally, there may be a bug in the difference filter, though this is extremely unlikely, because it works with the other modes. But nevertheless one should make a try without it. It is enough to test this for the simplest mode=1, where surprisingly the access violation already happens. I expect that you did the tests with an yv12 clip, which is the natural one for mpeg2. The other color spaces are not well tested. In fact, I am stunned about the very poor performance for the non-yv12 color spaces, but I postponed the investigation of this problem for a while.


Other than that, the filter seems to work very well, mode 8 did an excellent job with my TV caps when followed by RemoveDirt(). Which reminds me, are you planning on adding SSE2 optimizations to RemoveDirt as well?
I did some work in this direction, but the performance gain will be quite small as long as the block width is 8. Changing the block width to 16 SSE2 will become very valuable, but then the frame width has to be a multiple of 16. Also unlike RemoveGrain, RemoveDirt heavily uses the psadbw instruction and this instruction behaves differently on 128 bit SSE registers (used with SSE2) and 64 bit mmx registers used (which must be used by the non-SSE2 version). Thus, while I get the RemoveGrain SSE2 version for free (it only requires testing) using smart macros, this is no more the case for RemoveDirt, where I have to make modification at some places, where I use the psadbw instruction. But since Intel has adopted the Athlon64 platform, we all will move to this platform within the next 2-3 years, because with its 16 general purpose registers (14-15 regsiters will then be usable, currently only 6-7 registers can be used) C++ compilers can generate significantly more efficient code even for 32 bit apps on the Athlon64 platform. Also the video resolutions will increase (HDTV). These two reasons more than justify a block width of 16 (the block height will stay at 8), which also has some slight disadvantages.

[Chainmax]

This is the filter that you mentioned works similarly to Undot, right?

[Boulder]

@kassandro: I'll do the tests tomorrow and let you know what happens.

[kassandro]

Originally posted by Chainmax
This is the filter that you mentioned works similarly to Undot, right?
Yes, RemoveGrain(mode=1) = UnDot(). Even the speed of the SSE version (with mode=1, of course) is identical to that of Undot. Like Undot and many other spatial denoisers, also the other eight modes are based on the eight neighbours of a pixel. The details of the algorithms can be found in the documetnation

@Boulder: many thanks already. So far, I can't reproduce these ugly access errors. A thorough inspection of the source code also didn't help.

[Boulder]

Hmmm, was this a read access or a write access violation? Did the access violation also appear with the non-SSE2 version (the only one I could test)?


It gives a read access violation, the exact message is "Avisynth: caught an access violation at 0x012f151a, attempting to read from 0xffffffff". The violation doesn't appear on the non-SSE2 version, all modes are OK.


Does the access violation persist if you crop before RemoveGrain with crop(4,4,-4,-4, align=false)?


Yes, modes 4-8 are the only ones that work.


Finally, there may be a bug in the difference filter, though this is extremely unlikely, because it works with the other modes. But nevertheless one should make a try without it. It is enough to test this for the simplest mode=1, where surprisingly the access violation already happens.


No go, so the problem must be somewhere within RemoveGrain.


I expect that you did the tests with an yv12 clip, which is the natural one for mpeg2. The other color spaces are not well tested. In fact, I am stunned about the very poor performance for the non-yv12 color spaces, but I postponed the investigation of this problem for a while.


Yep, YV12 it is, I don't deal with YUY2 anymore since I bought the PVR-250.

Hope this all helps:)

[kassandro]

Thank you Boulder for all your efforts, but I'll have to withdraw the SSE2 version. I spend quite some time to find the bug(s) using your precise observation, but all I found was one wrong SSE2 macro in for mode 1 and nothing in the other modes. Surprisingly my Celeron with Tualatin P3 design swallows the SSE2 version without any access error (also with 720x576 mpeg2 input). Of course, the output is not quite correct. While with the true SSE2 one can handle 16 pixels at a time, with the P3 SSE2 only the first 8 bytes of the 16 byte SSE register are processed. Thus if chrom processing id disabled (i.e. modeU=0), then after 8 correctly processed pixels there are 8 incorrectly processed pixels with the SSE2 version on Tualatin P3 (I don't know what happens on a Katmai or Coppermine P3 or on the various Athlons). I simply have to get my hands on a P4 to finish the job. Programming for the P4 without having one simply does not work.

[Boulder]

Well, fortunately the speedup isn't that big, I'd estimate somewhere along 8-12% on my system. I suppose that mode 8 (which I also prefer) works OK, at least according to the difference debug log, so the SSE2 version can be used and I probably will.

I don't know if there are any developers who have a P4 system and could help..maybe it would be worth asking around a bit.

Thanks for your efforts anyway:)

[dbzgundam]

Why can't I access the page?

It's loading but I see a blank page. :confused:

[kassandro]

Originally posted by Boulder
Well, fortunately the speedup isn't that big, I'd estimate somewhere along 8-12% on my system. I suppose that mode 8 (which I also prefer) works OK, at least according to the difference debug log, so the SSE2 version can be used and I probably will.

8-12% is quite disappointing. The following formula should hold

SSE processing time - SSE mmemory access time = 2 * (SSE2 processing time - SSE2 mmemory access time)

Now in SSE2 mode, RemoveGrain tries to read and write 16 Bytes in one stroke, while in SSE mode only 8 bytes are read and wrote in one stroke. In 90-100% of all cases RemoveGrain reads from the L1 cache. However, reading and writing is usually unaligned. In SSE3 (Prescott P4) there is even a special instruction for unaligned reading from memory, which may be helpful.


I don't know if there are any developers who have a P4 system and could help..maybe it would be worth asking around a bit.

I think that by the end of the year I will buy a Prescott Celeron. Then I will certainly return to the SSE2 version.

Originally posted by dbzgundam

Why can't I access the page?

It's loading but I see a blank page.

Why are you young guys always so impatient? If you click on the above link, the server from www.AlpenNIC.com (http://www.AlpenNIC.com) maps the domain www.RemoveGrain.de.tf (http://www.RemoveGrain.de.tf) to a path on an Austrian server, where RemoveGrain and all my other plugins are hosted. As this server is advertisment free (however AlpenNIC generates some popups) and probably publicly funded, it is not the fasted (thousands of austrian home pages are hosted there). So you have to be patient. If it doesn't even work patience, I will give you a direct link.

[Boulder]

Originally posted by kassandro
8-12% is quite disappointing. The following formula should hold

SSE processing time - SSE mmemory access time = 2 * (SSE2 processing time - SSE2 mmemory access time)

Now in SSE2 mode, RemoveGrain tries to read and write 16 Bytes in one stroke, while in SSE mode only 8 bytes are read and wrote in one stroke. In 90-100% of all cases RemoveGrain reads from the L1 cache. However, reading and writing is usually unaligned. In SSE3 (Prescott P4) there is even a special instruction for unaligned reading from memory, which may be helpful.


That's probably theory and real life colliding:D The speedup might have (and probably would have) been bigger if RemoveGrain was the only thing eating CPU cycles, there's always other processes and Avisynth and CCE probably cause some loss to the actual gain.

[kassandro]

I just put up version 0.5 to the web site.
There have been substantial internal changes. The code for handling color spaces other than yv12 has changed, however the speed still remains surprisingly poor compared with the very satisfactory speed for yv12. The assembly code for mode=2,3,4 has been optimised. Here is a comparison of the old and new version on my machine (1.3 GHZ Celeron):

mode old version new version
2 149.5 fps 150.5 fps
3 141 145.5
4 142 149.5

I also made a comparison of trbarry's Undot and RemoveGrain(mode=1). It turned out that Undot and RemoveGrain(mode=1) differ for the pixels on the right border. By the very nature of the algorithm border pixels cannot be processed and should therefore be left unchanged. While Undot does just this for the left, the top and the bottom border. it makes a mistake on the right border, where instead of copying the last pixel on the line, it copies the penultimate pixel to the last pixel, which simply doesn't make sense and should be considered as a bug.

[Boulder]

Thanks, RemoveGrain's one of my standard filters for processing analog TV caps:)

EDIT: Has the SSE2 version changed in any way or am I better off using the SSE optimised one? I use mainly mode=8.

[Audionut]

In Vitualdubmod, I get an error,

Unable to load "C:\RemoveGrain.dll"
Same goes for the SSE2 version.

LoadPlugin("C:\PROGRA~1\ARCALC~1\AVS_Plugins\Mpeg2dec3.dll")
LoadPlugin("C:\RemoveGrain.dll")
Mpeg2Source("z:\*.D2V")
RemoveGrain(mode=4,mode=U2)

[Boulder]

Works fine for me.. did you try extracting the dll to the Avisynth plugins directory?

[Audionut]

I was missing a couple of dll's.

Got the same problem with the SSE2 version as Boulder.

[kassandro]

Originally posted by Boulder
Has the SSE2 version changed in any way or am I better off using the SSE optimised one? I use mainly mode=8.
The SSE2 version has changed in the same way as the SSE version (thus only for mode=2,3,4). Actually I use macros for the registers (for SSE I take the 64 bit mmx registers and for SSE2 I take the 128 bit SSE registers) and for the instructions, where SSE and SSE2 differ. Thus all changes effect both versions equally and simultaneously. Unfortunately there must be something wrong with this very economical way of programming.

Originally posted by Audionut
In Vitualdubmod, I get an error,

Unable to load "C:\RemoveGrain.dll"
Same goes for the SSE2 version.

You probably have not installed "msvcr70.dll" properly (see the installation section of www.AvsTimer.de.tf (http://www.AvsTimer.de.tf) for more details). You may also use the staticly linked version RemoveGrainS.dll instead. In fact, this is the only reason, why I provide it.

[kassandro]

Originally posted by Audionut
I was missing a couple of dll's.

Got the same problem with the SSE2 version as Boulder.

My response to your problem in the previous posting is obsolete (your posting was almost simultaneous). Thanks you for your test. I will buy an SSE2 capable cpu (2.66GHZ Prescott Celeron + motherboard with Intel chip set + 512 MB of DDR-RAM is now well below a very affordable 200 Euros) by the end of the year. Then I should be able to track down the problem.

[Audionut]

Originally posted by kassandro
My response to your problem in the previous posting is obsolete (your posting was almost simultaneous). Thanks you for your test. I will buy an SSE2 capable cpu (2.66GHZ Prescott Celeron + motherboard with Intel chip set + 512 MB of DDR-RAM is now well below a very affordable 200 Euros) by the end of the year. Then I should be able to track down the problem.

Thankyou, for this fine filter. Hope you can get the SSE2 version working, without many hassles.

[Audionut]

I love this filter.

6000 frame clip (Clean PAL DVD) single pass, const. quant 2, with no filter's returned 2979kbps bitrate.
With Undot(), returned 2828kbps
With removegrain(), returned 2662kbps bitrate.:D
With removegrain() and unfilter(-5,-5) returned 2328kbps.

Thankyou again.

[ARDA]

@kassandro

First of all thanks for the great work you're doing.
Second I want to make a question about a reference of Undot you've posted.

kassandro wrote:
By the very nature of the algorithm border pixels cannot be processed and should therefore be left unchanged. While Undot does just this for the left, the top and the bottom border. it makes a mistake on the right border,where instead of copying the last pixel on the line, it copies the penultimate pixel to the last pixel,which simply doesn't make sense and should be considered as a bug.
Please would you please be more specific about that bug; some time ago I've made some test with Undot sources and don't remember such a thing. Is it in YV12 or YUY2 colorspace?

Thanks ARDA

[kassandro]

Originally posted by ARDA
Please would you please be more specific about that bug; some time ago I've made some test with Undot sources and don't remember such a thing. Is it in YV12 or YUY2 colorspace?

First of all, the Undot bug is not severe. It doesn't cause a crash and concerns only the last pixel on a line. It is therefore hardly noticable. Nevertheless it is instructive to see how I uncovered the Undot bug. I started with the following script:

input=MPEG2Source("test.d2v")
Undot_clip=Undot(input)
RemoveGrain_clip=RemoveGrain(input, mode=1)
difference(Undot_clip, removegrain_clip)

The difference filter is taken from my AlignFields (http://www.AlignFields.de.tf) plugin. It reports for each frame the SAD difference as well as the number of different pixels to the debugview utility. Each pixel may be counted up to three times (Y, U, V values). Thus for a standard 720x576 clip there may be roughly up to 600000 different pixels for each frame.
Running the script with Vdubmod I was surprised to get the following output from debugview:

[1016] [129] total difference = 1, different Pixels 1
[1016] [130] total difference = 3, different Pixels 3
[1016] [131] total difference = 2, different Pixels 2
[1016] [132] total difference = 1, different Pixels 1
[1016] [133] total difference = 2, different Pixels 2
[1016] [134] total difference = 4, different Pixels 4
[1016] [135] total difference = 5, different Pixels 5
[1016] [136] total difference = 4, different Pixels 4
[1016] [137] total difference = 5, different Pixels 5
[1016] [138] total difference = 3, different Pixels 3

Thus there were differences between Undot and RemoveGrain(mode=1) but only very few. The border pixels were the immediate suspects (Undot and RemoveGrain require 8 adjacent pixels, whence border pixels cannot be processed and should therefore be left unchanged). To prove this I ran the following modified script:

input=MPEG2Source("test.d2v")
Undot_clip=Undot(input)
Undot_clip=crop(Undot_clip, 2,2,-2,-2)
RemoveGrain_clip=RemoveGrain(input, mode=1)
RemoveGrain_clip=crop(RemoveGrain_clip, 2,2,-2,-2)
difference(Undot_clip, removegrain_clip)

and fortunately I now got

[1016] [204] total difference = 0, different Pixels 0
[1016] [205] total difference = 0, different Pixels 0
[1016] [206] total difference = 0, different Pixels 0
[1016] [207] total difference = 0, different Pixels 0
[1016] [208] total difference = 0, different Pixels 0
[1016] [209] total difference = 0, different Pixels 0
[1016] [210] total difference = 0, different Pixels 0
[1016] [211] total difference = 0, different Pixels 0
[1016] [212] total difference = 0, different Pixels 0

Thus the differences between Undot and RemoveGrain(mode=1) are indeed on the border only. Next I asked the question: Are all border pixels handled differently or only a part? Running the following modified script:

input=MPEG2Source("test.d2v")
Undot_clip=Undot(input)
Undot_clip=crop(Undot_clip, 0,0,-4,0)
RemoveGrain_clip=RemoveGrain(input, mode=1)
RemoveGrain_clip=crop(RemoveGrain_clip, 0,0,-4,0)
difference(Undot_clip, removegrain_clip)

I still got the zero difference ouput

[1016] [251] total difference = 0, different Pixels 0
[1016] [252] total difference = 0, different Pixels 0
[1016] [253] total difference = 0, different Pixels 0
[1016] [254] total difference = 0, different Pixels 0
[1016] [255] total difference = 0, different Pixels 0
[1016] [256] total difference = 0, different Pixels 0
[1016] [257] total difference = 0, different Pixels 0
[1016] [258] total difference = 0, different Pixels 0
[1016] [259] total difference = 0, different Pixels 0
[1016] [260] total difference = 0, different Pixels 0
[1016] [261] total difference = 0, different Pixels 0

Thus only the border pixels on the right side are processed differently. Having located the differences I now turned to the Undot source code and found the following cause:

// do last qword
lea esi, [esi+eax-8-BPP] // point at last qword
movq mm0, qword ptr[esi+ebx] // move 1st 4 pixel
movq qword ptr[edi+eax-8], mm0

The comment "point at last qword" is incorrect. Because of -BPP=-1 he doesn't "point at the last qword". Removing "-BPP" from the source code, Undot and RemoveGrain(mode=1) should become identical. Right now, because of -BPP, Undot copies the penultimate pixel of a line of the input frame to the last pixel of the same line of the output frame, which may improve compression a tiny little bit but is simply not correct and was not intended either.

[Boulder]

Heh, that's one nice post about troubleshooting:D Maybe you should notify trbarry so that he might fix the bug once he has the time. I haven't seen him in a while though.

[Fizick]

Tom Barry can be notified by e-mail (recently I inform him about other filter).

Kassandro, may I to use some part of your RemoveGrain code in my DeGrainMedian code ? :)

[ARDA]

@kassandro
Thanks for the long explanation, I think it is very instructive; at least for me.
I've borrowed this function modified a little and moving the slide in virtualdub could see the difference between your plugin and undot on the right side of amplified difference; and as you say it's hardly noticeble.


clip=MPEG2Source("mysource")
v1 =clip.UnDot
v2 =clip.RemoveGrain(mode=1)
sub = v2.subtract(v1)
substrong = sub.levels(122,1,132,0,255)
v3 = StackVertical(StackHorizontal(substrong.subtitle("Difference amplified"),
\v1.subtitle("UnDot")),StackHorizontal(sub.subtitle("Difference"),v2.subtitle("original")))
return v3

Thanks again for this great work.

ARDA

[kassandro]

@Boulder:
Yes, I haven't seen trbarry here for quite a while. He seems to be busy with other things. I will draw his attention to this slight problem, when he returns.

@ARDA:
Thanks for the nice sript. Actually, my input source was not very suitable for the test this morning. It had a black stripe on the right hand side. Thus the penultimate and the last pixel on a line were almost always identical. Only noise did give small differences. Cropping away the black stripe, your script displayed the differences nicely. Actually it shows that the chroma difference effects the last two pixels on the right hand side.
Since trbarry's assembly routine is nearly the same for YUY2, but with BPP=2 instead of BPP=1 for YV12, the same copying mistake is made for YUY2. On the other hand for YUY2, Undot and RemoveGrain(mode=1) cannot be compared, because Undot doesn't process the chroma of YUY2 clips. He could have done it with nearly the same assembly routine and BPP=4, but more attention at the border would have been necessary. Thus Undot and RemoveGrain(mode=1, modeU=0) are nearly identical for YUY2, but for YUY2 Undot is much faster.

[kassandro]

Originally posted by Fizick
Kassandro, may I to use some part of your RemoveGrain code in my DeGrainMedian code ? :)
Sorry, Fizick, for having overlooked your posting yesterday, which I cannot understand, because it was just between Boulder's and ARDA's last posting.
Of course you can use my code. However, as I mentioned earlier, the programming style is very different from trbarry's. Because I have always SSE2 in mind, I use macros where SSE and SSE2 differ, which makes the code more difficult to understand. Understanding is also hampered by my passion to overoptimise. Though the SSE2 code doesn't work currently, I should be able to fix it easily once I have a SSE2 capable cpu. In the end, we all have to switch to SSE2, because the Athlon64 platform, which will abolish the current 32 bit platform within 2 years, because Intel has adopted it, has no mmx registers anymore in 64 bit mode. Rather it has 16 SSE 128 bit registers. It would have been much better to have 8 256 bit registers instead, with which one could easily double dct/idct performance and process 32 pixels instead of 16 simultaneously with various Avisynth filters.

[Fizick]

Kassandro,
Thanks for your permission!
I am quite easy understand your code. It is very well optimized and structured.
I already put it to my filter beta (will be released soon, of course under GPL). But i remove SSE2.

About SSE2. I think its time is not come for the present
(not compatible with Athlon XP, so SSEMMX is standard now). I do not think, that SSE2 speed can be used in every filter. And the speed is not most important thing, but algo and quality (for me). But the next generation of filter-writers will use SSE4 of course! :)

BTW, why you do not like Adobe so much? ;)

[kassandro]

If there are no special restrictions, like in RemoveDirt, where the block size is fixed, then one should be able to produce a SSE and an SSE2 version simultaneously by using some simple macros for registers and some load and store instructions. I just learned that there is one important difference, which I am was completely unaware, when I made RemoveGrain. For 128 bit memory operands of instructions like paddusb, pminub etc. the memory operand must always be aligned (i.e. the address must be a multiple of 16), while 64 bit operands as used for SSE are allowed to be unaligned. I have to blame Intel for my ignorance, because Intel always speaks only of 128 bit memory operands and not of 128 bit aligned memory operands. Only when Intel discusses exceptions, which I never read, because I simply do not expect such exceptions for my programs, then one can read, that a memory alignment error triggers a read access exception, as reported by Boulder. Fortunately I read at least one time the exception stuff and then everything was obvious for me. Now it is my programming style to load into a register, if the data is used more than once (that is a key difference between Undot and RemoveGrain(mode=1)), because only one instruction with a memory operand can be executed at a time (that is the reason why RemoveGrain(mode=1) is about 4 fps faster than Undot). Thus for some modes I don't use at all instructions with a memory operands and these were exactly the modes reported to work by Boulder.
I just uploaded a new SSE test version (it must be called with DRemoveGrain) to the web site. The other parts of the binary archive as well as the source archive have not changed. The SSE2 problem should now be fixed, though I may have overlooked 1 or 2 instructions with a memory operand.

[Boulder]

Great, I'll have a go ASAP!

[Boulder]

OK, I tested the SSE2 version. It looks like all the other modes except number 4 work and produce no differences compared to the SSE version, viewed with DebugViewer. If I use mode=4, VirtualDubMod just closes without giving any error messages whatsoever.

Here's another bug as well, I noticed it earlier but forgot to post:scared: If I use modeU=-1 to disable chroma processing, I first get a screen like this:


http://www.saunalahti.fi/sam08/rg_1.jpg


After scrolling for a few dozen frames in VDubMod, the output turns like this (notice the odd ghosting) :

http://www.saunalahti.fi/sam08/rg_2.jpg

If I use modeU=0, everything's OK. With mode=-1, I also get weird results. This occurs on both SSE and SSE2 version.

[kassandro]

Originally posted by Boulder
OK, I tested the SSE2 version. It looks like all the other modes except number 4 work and produce no differences compared to the SSE version, viewed with DebugViewer. If I use mode=4, VirtualDubMod just closes without giving any error messages whatsoever.

I looked again at the source code and the SSE2 bug seems to be corrected. On the other hand, there have been a lot of changes meanwhile and the current SSE2 binary is only a snapshot. In fact, the next version 0.6 will contain a new plugin, which is derived from the same source file, such that both plugins can be maintained simultaneously. Unfortunately, the technique to creat different plugins from the same source code has one drawback: the source code gets a little bit messy.


Here's another bug as well, I noticed it earlier but forgot to post:scared: If I use modeU=-1 to disable chroma processing, I first get a screen like this:

This is not a bug. If you want to leave the chroma unchanged, you have to use modeU=0. If you want the chroma to be not processed at all (not even copied), which is faster, you should choose modeU=-1. If modeU=-1, then the chroma becomes random (in your case green and then it changes again). Why modeU=-1?. If you have black&video, you can save time if you use modeU=-1 and then at the end you simply use the greyscale command. The chroma is automaticly erased if RemoveDirt is used with grey=true.
If you you use mode=-1, then the luma becomes random as well. Of course, this doesn't make sense even for b&w material.

[Boulder]

Originally posted by kassandro

This is not a bug. If you want to leave the chroma unchanged, you have to use modeU=0. If you want the chroma to be not processed at all (not even copied), which is faster, you should choose modeU=-1. If modeU=-1, then the chroma becomes random (in your case green and then it changes again).

Oh, then it was a mistake on my part. I actually thought that using modeU=-1 ended up in a b&w clip, but that has to be done either with RemoveDirt(grey=true) or Greyscale().

When can we expect the v0.6? :)

[kassandro]

Originally posted by Boulder
Oh, then it was a mistake on my part. I actually thought that using modeU=-1 ended up in a b&w clip, but that has to be done either with RemoveDirt(grey=true) or Greyscale().

From this design you can see that RemoveGrain was designed as a precleaner for RemoveDirt. Unfortunately the very sophisticated RemoveDirt becomes more and more obsolete. The new plugin derived from the RemoveGrain source, currently called Repair, leads to a much more compression efficient way of cleaning: firstly I apply a very simple and very fast temporal clenser (essentially RemoveDirt without any artifact protection), then the Repair plugin compares the clensed with the original frames and instead of removing grain it removes the massive clenser artifacts (unfortunately it restores also medium and big temporal dirt). Finally RemoveGrain is applied to erase some left over. While this filter, called RemoveDust, can never remove medium or big dirt like RemoveGrain, it is much more efficient on dust than RemoveDirt and much more efficent on grain than RemoveGrain. My first tests show very remarkable compression gains with only slight motion blurring. The static parts have the same softness as the corresponding RemoveGrain modes.
Now already, Fizick in his denoiser plugin, made a temporal extension of RemoveGrain mode 5-9, but for mode=9 (trbarry's ST-median) an additional change limitation is necessary to avoid very unpleasant artifacts. While this temporal extension makes a lot of sense, the gain over the purely spatial variants is limited. Like RemoveGrain with mode >=5 it can never remove grain consisting of three equal pixels on a line segment, even if this piece of grain is only on one frame. Thus Fizick can't really exploit the temporal nature of grain either. That will change with RemoveDust.
I will supply various compression comparisons of RemoveDust with RemoveGrain and DeGrainMedian. A comparison with convolution based smoothers doesn't make sense to me.


When can we expect the v0.6? :)
The most difficult work is the documentation and of course a lot of tests are necessary. I hope to release it within one to three weeks.

[Boulder]

Originally posted by kassandro
From this design you can see that RemoveGrain was designed as a precleaner for RemoveDirt. Unfortunately the very sophisticated RemoveDirt becomes more and more obsolete. The new plugin derived from the RemoveGrain source, currently called Repair, leads to a much more compression efficient way of cleaning: firstly I apply a very simple and very fast temporal clenser (essentially RemoveDirt without any artifact protection), then the Repair plugin compares the clensed with the original frames and instead of removing grain it removes the massive clenser artifacts. Finally RemoveGrain is applied to erase some left over. While this filter, called RemoveDust, can never remove medium or big dirt like RemoveGrain, it is much more efficient on dust than RemoveDirt and much more efficent on grain than RemoveGrain. My first tests show very remarkable compression gains with only slight motion blurring. The static parts have the same softness as the corresponding RemoveGrain modes.

Sounds interesting, it will once again be one to try for me. I just hope you'll be able to keep one of the strong points of RemoveDirt - the incredible speed;) Do you have any estimates on when it might be released? I'll have 2-4 analog TV capture clips to encode weekly so I might find the filter very useful for my purposes.

[kassandro]

Originally posted by Boulder
Sounds interesting, it will once again be one to try for me. I just hope you'll be able to keep one of the strong points of RemoveDirt - the incredible speed;) Do you have any estimates on when it might be released? I'll have 2-4 analog TV capture clips to encode weekly so I might find the filter very useful for my purposes.
You may download the current Repair.dll from (it contains also the clenser "Clense") from deleted and use it within the following function

function RemoveDust(clip input, int _mode, bool _grey)
{
repmode=2
clensed = Clense(input, grey=_grey)
rep=Repair(clensed, input, mode=repmode, modeU = _grey ? -1 : repmode)
return RemoveGrain(rep, mode=_mode, modeU = _grey ? -1 : _mode)
}

I currently run my second big test with RemoveDust(4, false) (false= color input). My first big test the movie "Heimatland" broadcasted yesterday on the ARD channel did crunch this movie with xvid 1.0.1 1 pass quality mode with quantiser=4 below 700 Mb, while without RemoveDust it was almost 1200 Mb. It was a dusty digital DVB capture with a lot of grain from 1955. With repmode=1 you get higher compression but motion is blurred more.

Edited: the quantiser was 5 not for 4.

[Fizick]

Kassandro,
It is very interesting evolution of your "Remove..." plugins.

it can never remove grain consisting of three equal pixels on a line segment, even if this piece of grain is only on one frame.
These three equal pixels can be not grain, but some object't edge...:confused:

Thus Fizick can't really exploit the temporal nature of grain either.
Poor Fizick :)

I will supply various compression comparisons of RemoveDust with RemoveGrain and DeGrainMedian.
It is interesting.
But i think, I know results right now: for different sources we must use different plugins.

The more we have, the better.

Good luck!

[kassandro]

Originally posted by Fizick
These three equal pixels can be not grain, but some object't edge...:confused:

Well, that is the question! If you are that conservative, you never can remove scratches. If these three pixels do not appear on one of the two neighbour frames, they are removed by the clenser. Now Repair works as RemoveGrain. However, the center pixel is taken from the clensed frame, while the eight neighbours are taken from the original frame and the center pixel is clipped by one pixel pair of these eight neighbours as in RemoveGrain (well, that's only the almost-truth). It's a very simple but remarkably good artifact removal. It results only in some motion blurring (the amount depends on the repair mode), but the big artifacts removed. Unfortunately, the big dirt is restored too.
Repair can also be used to remove sharpening artifacts of a sharpener or at least makes a sharpened clip more compression friendly. Of course it gets a little bit softer too.



But i think, I know results right now: for different sources we must use different plugins.

That is certainly true. But as a collector of old films my sources are always of the same type. Their origin is old chemical non-anime film, mostly black&white and I receive them over digital sat-tv by recording the transport stream. The most significant difference is, whether these progressive clips are messed up (shifted fields) or not. My denoiser/cleaner results are always for these kinds of sources.

[Audionut]

Mode=4 in sse2 version gives me an avisynth read error: avisynth: caught an access violation at 0x017a14c2, attempting to read from 0xffffffff

Everything else works like a charm.
Thanks again for your work.

[Boulder]

Originally posted by kassandro
It was a dusty digital DVB capture with a lot of grain from 1955. With repmode=1 you get higher compression but motion is blurred more.


I tried it on a small TV capture clip, not particularly noisy but more than enough to annoy. It appears that RemoveDust is quite a bit more aggressive than RemoveDirt, so I hope there will be a mode which would be something between RemoveDirt with its current defaults and repmode=2. I didn't try to watch the result on my TV since I don't have any DVD+RW's to spare at the moment, but on a frame-by-frame comparison, the motion blur looked a little too aggressive. Compressibility-wise RemoveDust wins hands down;)

As I said, it looks very interesting, and I'll be following the development closely.

[kassandro]

Originally posted by Boulder
I tried it on a small TV capture clip, not particularly noisy but more than enough to annoy. It appears that RemoveDust is quite a bit more aggressive than RemoveDirt, so I hope there will be a mode which would be something between RemoveDirt with its current defaults and repmode=2. I didn't try to watch the result on my TV since I don't have any DVD+RW's to spare at the moment, but on a frame-by-frame comparison, the motion blur looked a little too aggressive. Compressibility-wise RemoveDust wins hands down;)

While my first two big tests were quite satisfactory (both were quite soft color movies from fifties), the third big test, which I ran over night was less satisfactory. The source was a very sharp Olsen brother film from the seventies. Knowing that RemoveGrain(mode=4) doesn't like sharp movies, I cautiously took mode=8 instead of mode=4. The output was fairly crisp (RemoveGrain(mode=8) sharpens edges), of course, with some motion blurring, the compression gain was much less impressive (I expected this to a certain extend), but what really hit me hard was the enormous thin line flicker (that certainly worsened compression). Already RemoveGrain(mode=4) not only destroys thin lines like most spatial denoisers but also creats thin line flicker (on one frame a thin line is preserved, on the next it is removed) unlike convolution based denoisers, which however blur more. However, the flicker of RemoveDust(mode=8) was much stronger than that of RemoveGrain(mode=4). RemoveDust seems to strongly amplify the good but also the bad properties of RemoveGrain. Your analog captures are certainly less sharp than my digital ones. Hence the flicker problem should not be such a big issue for you. If you want less motion blurring, then you should try repmode=3. In general, all modes of RemoveGrain are available for Repair. The more aggressive mode is for grain removal the more aggressive it is for artifact removal, but repmode=4 is far too aggressive, it almost completely abolishes the clenser and the output looks like applying RemoveGrain(mode=4) twice, which isn't good either. Also repmode=9 doesn't seem to be a great idea. I have added a new mode 10, which is also available in the recent RemoveGrain SSE2 test version, which you gratefully have tested. It seems to be between mode=1 and mode=2.

The flicker problem forced me to think almost the entire day about a fast, conservative flicker filter and I am coding the first lines right now.

[trbarry]

kassandro -

Nice spotting on the UnDot bug. As others have mentioned I've been away recently and right now don't even have access to my web page, which is going to relocate soon. But trbarry.com is redirected and it should be back up at a new location (with an UnDot and some STMedianFilter fixes) in a couple days.

- Tom

[Fizick]

trbarry,
we will wait your return :)

[kassandro]

Tom,
thank you for your friendly comment. I learned SSE/MMX programming from the source code of your plugins, especially Tomsmocomp. You really can't learn it from the Intel manuals, which can only be used for look up. Before I saw from your plugins how virtuosely it can be used, the integer SSE/MMX didn't make much sense to me.
Having said that, I did found another problem with your STMedian filter, which has also propagated to Fizick's DeGrainMedian. For each output frame you need three input frames, the previous, the current and the subsequent frames and you tacitly make the assumption that all three frames have the same pitch for each plane. On the other hand, your own filter Uncomb may creat a clip, where frames do not all always have the same pitch, if Crop was applied before. In order to improve performance, Uncomb simply passes through frames without combs. On the other hand combed frames are replaced, by frames created with env->NewVideoFrame(vi), which have a smaller pitch, if the child clip was cropped substantially. Now, the reason for assuming that all frames have the same pitch was not negligence, rather it was the register shortage of the Intel architecture. Intel only gives you 6 registers. On the other hand, you have to keep the pointer for the four frames in registers and the pitches for all three source frames. Even, if you decide to reload the pointer of the destination frame each time, you then need one register for reloading. Thus you still cannot keep all the three pitches for the three source frames in registers. Once you start reloading, you have to do even more reloading, which will then have a significant impact on performance. To cope with this ugly problem, I would wrap around a class, say, Homogenate around the child clip, which does the following: if the frame has the standard pitch for each plane, i.e. the same pitch as a frame created by env->NewVideoFrame(vi), then the frame from the child clip is passsed through to STMedian unchanged (this is the usual case). If the frame has not the standard pitch, a new frame is created with env->NewVideoFrame(vi) and the child frame is copied to this new frame, which is then processed by STMedian. Proceeding in this way not only the three source frames have the same pitch, but also the destination frame.

[trbarry]

Yuk! You are right but that is more elaborate a change than I have time to make right now, even though I do have 2 other STMedianFilter bug (and suggested fixes) I have to apply.

I guess a quick workaround would be to precede STMedianFilter by UnDot or UnFilter in the above case. Either runs almost instantly and will copy all the frames causing a constant pitch. UnDot is probably preferable here. Something like UnFilter(-7,-7) instead works best immediately following STMedianFilter() to get better compressibility.

Or I suppose you could just crop after UnComb instead of before. But I'll try to look at some better fix that doesn't involve waiting for Athlons-64's with more registers. ;)

- Tom