RemoveGrain

[Fizick]

Kassandro,
Of course I know about this pitches limitation in Stmedian and degrainmedian. But I forget to note it in doc, sorry all.
Now I use only simplest workaround to prevent possible memory access crash - degrainnmedian uses minimal pitch of 3 frames.
I think it is very rare situation. Nobody found problem in real coding with STMedian for about 1 year.
If somebody will post bug report, i will think about it if have a time.

[kassandro]

Originally posted by Fizick
Kassandro,
Of course I know about this pitches limitation in Stmedian and degrainmedian. But I forget to note it in doc, sorry all.
Now I use only simplest workaround to prevent possible memory access crash - degrainnmedian uses minimal pitch of 3 frames.
I think it is very rare situation. Nobody found problem in real coding with STMedian for about 1 year.
If somebody will post bug report, i will think about it if have a time.
I agree, it is very unlikely that the constant pitch assumptions causes problems, but when it does it will be difficult to spot. Nevertheless one should not reload the pitches all the time. Though I never had the necessity, I will write a class which guarantees frames with standard pitch, i.e. the pitch of frames returned by env->NewVideoFrame(vi). That class should also have the option, to require the frames to be properly aligned. This is particularily imüportant for SSE2, where an even more significant gain if access is aligned.

[kassandro]

I just put up version 0.6 to the web site. As anounced, there is now additional plugin Repair containing the filters Repair and TemporalRepair. I have also added the filter Clense RemoveGrain and there are the new modes 10, 11 for RemoveGrain. Motion blur and flicker of RemoveDust has been reduced substantially using TemporalRepair.

[Audionut]

Thanks kassandro.

Can you give me an example of how to use repair/temporalrepair.
I really do not understand the examples given at the download page.

I use this script.

Mpeg2Source("C:\JEEPER~1\VIDEO_TS\JEEPER~1.D2V")
removegrain(mode=4, modeU=2)
removedirt()
Crop(8,74,-8,-74)
a=trim(0,1769).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
b=trim(1770,141093).Lanczos4Resize(704,288)
c=trim(141094,149818).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
a+b+c
removegrain()
Unfilter(-7,-7)

[kassandro]

Originally posted by Audionut
Thanks kassandro.

Can you give me an example of how to use repair/temporalrepair.
I really do not understand the examples given at the download page.

I use this script.

Mpeg2Source("C:\JEEPER~1\VIDEO_TS\JEEPER~1.D2V")
removegrain(mode=4, modeU=2)
removedirt()
Crop(8,74,-8,-74)
a=trim(0,1769).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
b=trim(1770,141093).Lanczos4Resize(704,288)
c=trim(141094,149818).BilinearResize(704,288).Fluxsmoothst().blur(1.5,1.5)
a+b+c
removegrain()
Unfilter(-7,-7)

Wow, you are doing a lot of blurring. Let me explain TemporalRepair.
Say you have clip input, which is processed by a filter and let's call the resulting clip filtered. Now let's call y(input,n,x,y) the luma of the pixel with coordinates x,y of frame n of clip input and y(filtered, n,x,y) the same pixel but in the filtered clip. Then TemporalRepair first calculates m(input,n,x,y)=min(y(input,n-1,x,y), y(input,n,x,y), y(input,n+1,x,y)) and M(input,n,x,y)=max(y(input,n-1,x,y), y(input,n,x,y), y(input,n+1,x,y)). Then y(filtered,n,x,y) is clipped at m(input,n,x,y) and M(input,n,x,y), .i.e. min(M(input,n,x,y), max(m(input,n,x,y), y(filtered, n,x,y))). In other words, if the filter has changed the luma of the pixel so much such that it got out of the range spanned by m(input,n,x,y) and M(input,n,x,y) it is clipped back into this range. Repair(mode=1) works similar but spatial instead of temporal. Instead of 3 temporally adjacent pixels, one has now 9 spatially adjacent pixels. Though it is not quite true, one should think about Repair the following way: it works just like RemoveGrain, but with two instead of one clip and the center pixel is taken from the first clip, while the neighbour pixels are taken from the second clip. Actually this thought has evolved into a general concept of deriving artifact removing filters from cleaning filters.
I hope this helps. Everybody is wellcome for ideas to illuminate this concept.

[Audionut]

Originally posted by kassandro
I hope this helps.

Not really, unfortunally.
I simply do not understand all that a,b,x,y stuff.:(

If possible, can you or someone else, edit my above script, to include repair.

Thanks.

Originally posted by kassandro
Wow, you are doing a lot of blurring.

The results are quite good at quant 5.
http://www.queenslander.net/users/lioneldavey/test10.jpg

Of course, ffdshow's sharpening and add noise, improve playback quality.

[kassandro]

Originally posted by Audionut
Not really, unfortunally.
I simply do not understand all that a,b,x,y stuff.:(

Hmm, to understand what is going on here, you should first try to understand how trbarry's Undot = RemoveGrain(mode=1) works: the luma of a pixel, called the center pixel in the sequel, is compared with that of its eight neighbours. If it is larger than the minimum of the eight neighbours and smaller than the maximum of the eight neighbour, then the pixel is left unchanged. If it is smaller than the minimum of the eight neighbours than the luma of the center pixel is replaced by the minimum of the eight neighbours. If it is larger than the maximum of the eight neighbours than the luma of the center pixel is replaced by the maximum of the eight neighbours. Quite natural, isn't it? Now, once you have understood this simple idea, we can continue to explain RemoveGrain, Repair, ST-Median, DeGrainMedian, which are all more or less subtle refinements of this basic idea.

[krieger2005]

Hi kassandro,

i thought you want write a mode=10, which is something between 1 and 2? Is this history?

--- EDIT:
I see, your RemoveGrain 0.6 does this... sorry

[kassandro]

Originally posted by krieger2005
Hi kassandro,

i thought you want write a mode=10, which is something between 1 and 2? Is this history?


mode 10 didn't live up to my expectations. When I combined Repair(mode=10, limit=5) with Sharpen(1.0) it did perform much worse than mode=1,2. Fortunately, together with my forthcoming plugin AmplifyDifference it gives a nice, fast, edge friendly sharpener. Thus mode=10 is not completely useless.
To have more choice I will make Repair specific improvements for mode=3 in the next version as I did already for mode 1,2,10. Currently Repair(input, input, mode=3) is identical to RemoveGrain(input, mode=3) because if mode is not 1,2,10 the center pixel of the second clip is not used for artifact removal and this is not optimal.

[kassandro]

"RemoveTemporalGrain" would be a much more appropriate name for the script function "SharpRemoveGrain" in the RemoveGrain. Of course RemoveGrain(mode=4) looks much softer than SharpRemoveGrain(mode=4), because the latter cannot make changes in static areas of the frame. On the hand RemoveDirt/Clense can only remove grain if that grain is absent on both adjacent frames, while SharpRemoveGrain can already remove grain, if it is absent on one adjacent frame. Of course, unlike RemoveDirt/Clense SharpRemoveGrain can only remove very very thin dirt. RemoveDust can remove substantialy thicker grain and dust at the expense of some motion blurring. All in all SharpRemoveGrain is a very conservative denoiser and should only be used with mode=4. It will be renamed "RemoveTemporalGrain" next time.

I also tested the deinterlacer RGDeinterlace. In the compression benchmark that deinterlacer was almost as good as AlignFields(mode=2). However, the ticker tape test was a desaster. Now this test where an interlaced fast moving stock market ticker tape has to be deinterlaced is a tough cookie for any smart deinterlacer. So far any motion adaptive sharp deinterlacer /including the most recent TDeint)shows substantial artifacts under 400% Vdub magnification, but as a blur deinterlacer RGDeinterlace should have had it easier, but it produced the worst results I have ever seen in this test. TemporalRepair simply restores too many pixels which it shouldn't. Without this problem RGDeinterlace would probably beat AlignFields(mode=2) as far as compression is concerned, but at the moment it cannot be recommended for real use.

[kassandro]

I just put up a new version of my RemoveGrain package to the web site (http://www.removegrain.de.tf). The SSE2 version should work fine now (no test version anymore). There are now even SSE3 versions of the plugins. Though Intel added only one new ISSE instruction to SSE3 this single instruction makes a big difference. Though I didn't have time for extensive benchmarks, it is justified to say that SSE3 versions delivers what the SSE2 promised but didn't deliver, namely almost doubling the performance of the SSE version. The performance difference between the SSE3 and the SSE2 version is much larger than between the SSE2 and the standard SSE version.
The remarkably poor performance (already realised by Boulder) of the SSE2 version is a consequence of the movdqu instruction used for unaligned reading of 16 bytes into SSE registers. The instuctions really stalls the cpu. Once it is replaced by lddqu in SSE3 unaligned "almost" sequential reading of 16 byte chunks is almost as fast as alinged reading. In this way Intel really removed a bottlekneck. For Athlon64 and also the Sempron 3100, which also have SSE2, movdqu and whence the SSE2 version may perform much better. The SSE3 version can only be used on Prescott P4s/Celerons.
With my new system I have also upgraded to Visual Studio 2003, which may result in missing DLLs.
The SSE3 version can only be compiled with the Intel Compiler (the Microsoft is simply too old). Because the Intel Compiler generates larger executables than the Microsoft compiler) the SSE3 files are larger than the SSE files.
Various new modes have been added as well. The limit variables available for some modes have been disabled, because they turned out not to be really useful.

[Boulder]

Humm, looks like RemoveGrainSSE2.dll is the same as in v0.6. That is, mode 4 doesn't work and there is no mode 12. Unfortunately I can't test the SSE3 version as I have a Northwood P4.

[kassandro]

Originally posted by Boulder
Humm, looks like RemoveGrainSSE2.dll is the same as in v0.6. That is, mode 4 doesn't work and there is no mode 12.
Sorry, Boulder, but you see it from the date of the SSE2 version, that this was the old version. I did test the SSE3 version extensively since the beginning of February and millions of frames have been processed with various modes of RemoveGrain/Repair. There should be no problem with the SSE2 version because it is almost identical with the SSE2 version. I did finish updating the dicumentation yesterday evening and then I did compile the various version (except the SSE3 version, where I did take just the ones which I have used for quite a while). Unfortunately it was a little bit too late, when I pulled the files into the archive and the old SSE2 simply remained there and got uploaded again. I have now replaced the old version and uploaded it. Modes 1-14 should work now.

Unfortunately I can't test the SSE3 version as I have a Northwood P4.
In general I would say that the Prescott CPU design is the worst, which Intel ever made, but with the SSE3 versions of RemoveGrain/Repair it really shines. I would have loved to buy a Sempron 3100 instead of a Prescott CPU, but it was simply too expensive. A boxed Sempron 3100 did cost 110€ and a suitable main board a further 70€, while I payed only 155€ for a boxed Celeron 330D + P4V88 Mainboard + 512 MB DDR400 RAM. Nevertheless I would be interested how the SSE2 version does relative to the standard version on the various Athlon64 and Sempron 3100 systems. The questions is whether the movdqu is equally poor on AMD systems as it is on Intel systems.

[Boulder]

From what I've heard, SSE2 code runs pretty much slower on AMD's processors in some applications.

I don't know how Prime95 works, but I'd say that program is the best optimized single executable for Intel's SSE2..I do recall saying that the biggest difference is the registers, the amount or the size, don't remember which one.

[kassandro]

I just made a short look the source code of Prime95. It is an assembler monster. However, it seems to use SSE only for FFT type problems. In particular, only floating point SSE seems to be used in Prime95, while we RemoveGrain and some other avs filters use integer SSE. Only DGDecode/Mpeg2Dec3 and Fizick's FFT filters seem to use floating point SSE. SSE3 brings substantial improvements for FFT/DCT/IDCT (most of the new instructions are only useful for this purpose). Thus the above plugins should profit from it, once they specificly use SSE3. I will also release a filter called SSETools (formerly I called it AmplifyDifference), which uses floating point SSE. I changed the name after adding a general color transform filter for YV12 and YUY2, which ccontains all the ColorMatrix transformations as very special cases and should do it more precisely. It will be interesting to compare performance with Tritical's mmx version of the ColorMatrix filter (YV12 only), which uses integer arithmetic only.
However, a new version of RemoveDirt will come first.

[len0x]

Does SSE3 version require any dlls or it is staticly linked? (I have no idea how intel compiler works in this case).

[kassandro]

Originally posted by len0x
Does SSE3 version require any dlls or it is staticly linked? (I have no idea how intel compiler works in this case).
No, it is dynamically link (I hope).

The SSE3 version had to be compiled with the Intel compiler and I don't know this compiler well enough. Unfortunately, the Intel compiler does not integrate into the german Visual Studio GUI.

The library mscvcr71.dll should be enough. Version 0.6 was compiled with the english Visual Studio 2002. It required mscvcr70.dll. Unfortunately each version of Visual Studio requires a different msvcrxx.dll.

[len0x]

Originally posted by kassandro
No, it is dynamically link (I hope).


But if its intel compiler then MSVC libraries should not matter, right? What are the other libraries that might be required by intel compiler?

[kassandro]

Originally posted by len0x
But if its intel compiler then MSVC libraries should not matter, right? What are the other libraries that might be required by intel compiler?
As far as I know, under Windows the Intel compiler uses the Microsoft libraries, under Linux it uses its own libraries. On the other hand, Intel binaries are 50% larger than the Microsoft ones. Perhaps I didn't use the optimal compiler options to get the file size down. At the moment I have not the time to explore the Intel compiler (version 8.1) any further.

[Fizick]

Only DGDecode/Mpeg2Dec3 and Fizick's FFT filters seem to use floating point SSE.
I only use external FFT library with float SSE.
However VagueDenoiser has 3DNow! and float SSE code by Kurosu.

[kassandro]

Originally posted by Fizick
I only use external FFT library with float SSE.

Yes, I thought the it is neccessary to use float SSE in FFT dll, because you can no more use integer approximations (the usual way in Avisynth to avoid floating point arithmetic) in a big FFT, otherwise accumulation of approximation inaccuracies would just give garbage.





However VagueDenoiser has 3DNow! and float SSE code by Kurosu.
I don't know the VagueDenoiser details, but I thought that VagueDenoiser uses a "small" transformation on 8x8 blocks and for these integer approximation should be faster and reasonably accurate. With SSE2 you could process even 8 pixels simultaneously by 16 bit integer approximation.

[tsp]

it seeems as both NASM and MASM support SSE3.
http://www.intel.com/cd/ids/developer/asmo-na/eng/167741.htm?prn=Y

[kassandro]

Originally posted by tsp
it seeems as both NASM and MASM support SSE3.

Thanks, for the information, but I prefer inline assembler over using an external assembler. Firstly, external assemblers require a lot of administrative overhead like segment directives etc. Secondly, with an external assembler using function arguments is difficult and may even become messy, if changes are made afterwards. Similarily, accessing class elements is easy with inline assembler and almost impossible with an external assembler. On the other hand, an external assembler has the big advantage that one can nearly always use the ebp register. Unfortunately, the Microsoft compiler insists on accessing stack variables with the ebp register instead of the esp register for function with inline assembler (the /Oy option is ignored for such functions). Surprisingly, for pure C++ function it can use the ebp register for optimisation and uses the esp register to accesses stack variables, if the /Oy option is used.

[Didée]

Sorry to interrupt the technical talk ...

Thanks for the new version, kassandro! After reeading the doc for a longer while, I think I found all new features ;)

Could you elaborate a little on the "smooth" switch? It sounds interesting, but "considers also fluctuations of neighbor pixels" is a rather vague description. Just a small hint or two, about what is actually compared, figured and done? :)

I ask because some time ago, I happened to play with a similar idea:

(Forgetting about foreground/background motion transitions or such things,) The difference of a pixel to its temporal neighbors should be rather similar for all pixels in a small neighborhood. Thus, the [spatial] difference between the temporal pixel differences should be sort of an indicator for the actual local noise. Hence I tried to even out those temporal differences for all pixels in a 3x3 neighborhood. But the milage was pretty poor: the frame got more blurring than the noise reduction was worth...

[kassandro]

Originally posted by Didée
The difference of a pixel to its temporal neighbors should be rather similar for all pixels in a small neighborhood. Thus, the [spatial] difference between the temporal pixel differences should be sort of an indicator for the actual local noise.

I had the same idea in mind, when I designed TemporalRepair with smooth=1 (actually in the source code, there is also smooth=2, 3 but I disabled it, because it doesn't improve much).
To describe the details, let me first recall the old TemporalRepair(fclip, oclip, smooth=0). Here fclip is the filtered and oclip is the original clip. Let further f(n,x,y) (resp. o(n,x,y)) the luma of the pixel with coordinates x,y of the n-th frame of fclip (resp. oclip). Finally let Min=Min(n,x,y)=min(o(n-1,x,y), o(n,x,y), o(n+1,x,y)) and Max=Max(n,x,y)=max(o(n-1,x,y), o(n,x,y), o(n+1,x,y)). Then in the output of TemporalRepair(fclip, oclip, smooth=0) the pixel f(n,x,y) is clipped at Min and Max, i.e. it is replaced by min(max(f(n,x,y), Min), Max). Now with smooth=1 the interval for clipping is enlarged. To this end let DMin(n,x,y)=o(n,x,y)-Min(n,x,y) and DiffBelow(n,x,y)=max(DMin(n,x,y), DMin(n,x-1,y), DMin(n,x+1,y), DMin(n,x,y-1), DMin(n,x,y+1), DMin(n,x-1,y-1), DMin(n,x-1,y+1), DMin(n,x+1,y-1), DMin(n,x+1,y+1)) and similarily DMax(n,x,y)=Max(n,x,y)-o(n,x,y), DiffAbove(n,x,y)=max(DMax(n,x,y),...). Now f(n,x,y) is clipped at o(n,x,y)-DiffBelow(n,x,y) and o(n,x,y)+DiffAbove(n,x,y).
If smooth=0, then f(n,x,y) is completely replaced by o(n,x,y) if o(n-1,x,y)=o(n,x,y)=o(n+1,x,y). If smooth=1 then the property o(n-1,x,y)=o(n,x,y)=o(n+1,x,y) must hold also for each of its 8 neighbours to ensure that f(n,x,y) is completely replaced by o(n,x,y). Thus if there are no temporal fluctuations in a neighbourhood of a pixel, the spatial cleaning or blurring is removed. If combined with the rather aggressive RemoveGrain(mode=4), then the static parts of a video are not touched at all, while compression is still much better than with the other, more conservative, modes of RemoveGrain.

[kassandro]

Didée,
after one night I now think that TemporalRepair(smooth=1) has nothing do with your idea. However, from time to time I think about a very similar idea, namely to apply spatial filters to the differences of subsequent frames rather than the frames itsself. In this way, spatial filters become interesting filters, which usually preserve static detail.
Why didn't I implement this idea yet?
Firstly, there technical obstacles. The difference of two pixels doesn't fit into a byte anymore, because the sign of the difference is very important. Thus one has to process two difference frames. One for postive and one for negative differences. Furthermore, it is a bad idea to use differences of unprocessed frames. If this is done, then mistakes made at one frame would propagate to all the subsequent frames. Thus one has to take differences between the current unprocessed frame and the previous processed frame. This unfortunately implies that the filter works well only sequentially. However, there is a work around even for this nasty recursion problem. The main argument for rejecting this kind of temporal cleaning is that only two frames are used for temporal cleaning, which makes it much more risky than the three frame approach, which I use in RemoveDirt or RemoveDust.

[Didée]

Huhu, seems I'll need pen, paper and some time to work that up. But sure I'll get through.

Thank you very much for the exact explanation. :)

-----
edit: yes, I also concluded that the idea is not so unproblematic as it might appear on first glance. But still, it has a grain of reasonable ground, somehow.
If its only about speed, let me squeeze it in a script. People are used to my scripts being ridiculously slow ;)
-----

BTW, regarding the "issue" I brought up: yesterday I quickly did this little comparison (http://x4.putfile.com/videos/5905203518.avi) (6 MB) (source/PixieDust(2)/RemoveDust(4)/LRemoveDust(4,2) (http://forum.gleitz.info/showpost.php?p=169704&postcount=32)). Perhaps worth having a look at.

[kassandro]

Originally posted by Didée
Huhu, seems I'll need pen, paper and some time to work that up. But sure I'll get through.

Yes, the matter unfortunately quite technical.

Thank you very much for the exact explanation. :)


BTW, regarding the "issue" I brought up: yesterday I quickly did this little comparison (http://x4.putfile.com/videos/5905203518.avi) (6 MB) (source/PixieDust(2)/RemoveDust(4)/LRemoveDust(4,2) (http://forum.gleitz.info/showpost.php?p=169704&postcount=32)). Perhaps worth having a look at.
Thanks for your tests. However, I must say that your avi is a bit unfair, because it contains virtually no static parts. You even did cut out the sky. This video exhibts the weak points of RemoveDust: if the moving objects have thin details, then they get blurred as if they were shot with a slower shutter speed and also compression deterioates considerably. Under average circumstances, the compression advantage over PixieDust is much higher than just 10% as in your avi. The static parts of the video are even improved. You also have chosen a motion speed, which seems to be slow enough for PixieDust's motion compensation and fast enough to exhibit the weakness of RemoveDust. It is also interesting that your limited version of RemoveDust comes fairly close to the unlimited version as far as compression is concerned, while keeping the video sharp. I have to check whether this also the case under average conditions.

[Valky]

Originally posted by Didée
Huhu, seems I'll need pen, paper and some time to work that up. But sure I'll get through.

Thank you very much for the exact explanation. :)

-----
edit: yes, I also concluded that the idea is not so unproblematic as it might appear on first glance. But still, it has a grain of reasonable ground, somehow.
If its only about speed, let me squeeze it in a script. People are used to my scripts being ridiculously slow ;)
-----

BTW, regarding the "issue" I brought up: yesterday I quickly did this little comparison (http://x4.putfile.com/videos/5905203518.avi) (6 MB) (source/PixieDust(2)/RemoveDust(4)/LRemoveDust(4,2) (http://forum.gleitz.info/showpost.php?p=169704&postcount=32)). Perhaps worth having a look at.

Good looking comparison. What was the speed cpmparing to pixiedust? Default settings in removedust seems to blur lots of details in sleeve. Also would be nice to see similiar comparison with more noisier source like tv-capture etc.

[kassandro]

Didée
what are you doing with the chroma in your script function LRemoveDust? More specifically, what is U=2,V=2 in yv12lutxy(last,input,yexpr=STR,U=2,V=2) good for? I expected values for uexpr and vexpr instead (the same string as for yexpr).

[Didée]

kassandro -
as it is currently written, that LUTxy command performs limiting on the Y plane only. U & V planes are passed through from RemoveDust just as they are. To do the same limiting for chroma, one would specify uexpr & vexpr with the same string as yexpr, and call U=3 & V=3 in the LUTxy comand. However when I tried that, I didn't see any remarkable visual difference. So I took the faster option with just passing chroma untouched.

Granted, that clip snipplet doesn't show all possible aspects. But then, that wasn't my aim. It is so that I really like RemoveDust in general. But there are motion characteristics where I don't like the plain RemoveDust at all, which held me back from using it. The most important one is just this: loss of detail in *slowly* moving areas. If motion is slow, then detail loss is easily noiticeable. In high motion, it's much less of a problem.
And so I just took a random example that showed the problem I'm mostly concerned about. It wasn't meant at all to be a "general" comparison.

Then, one could see such "hard limiting" as a poor technique. In fact it is, because it's just a dumb "cutting-off the effect". But looking at it from another angle, one could say as well like this: Generally, RemoveDust does the right thing. But in certain cases, it just does too much. Therefore, if limiting the effect to rather low values still is able to remove the noise/grain we want to remove, then there is no need to do any more. Or, in other words: The filter is working in the right direction, but is working too diligent. We better tell it where to stop. ;)

Keeping in mind that all of this is working without true motion compensation, I think that this is a really great filter combination for noise and grain removal. And I could imagine that it gets even more impressing when combined with motion compensation, even if the incredible speed is lost then. Alas I've currently not enough free time to look into that. perhaps someone might want to try that.


Valky:

Just try it and see for yourself. Not that difficult, it's just a few lines.
Very roughly from memory: Together with XviD encoding, I got ~5fps for PixieDust and ~20fps for LRemoveDust. LRemoveDust encoding was only marginally slower than without any filtering - XviD was the most limiting factor, there.
For plain rendering without encoding, (L)RemoveDust is in the range of 8~10 times faster than PixieDust. Regarding *that* difference in performance, I think the results are so close to PixieDust, it's almost impertinent :)

[Boulder]

Didée,

would it be possible to make a YUY2 version of LRemoveDust? It seems very interesting but would require converting to YV12 and then back to YUY2 to feed to CCE.

[Didée]

Yes and no. Kassandro's RemoveDust function supports YUY2 natively (in fact, it supports all colorspaces, afaik). But the limiting for LRemoveDust is done through MaskTool's "yv12lutxy" ... the name says it all. Because of that, at least for doing luma limiting there's no way to avoid the yuy2->yv12->yuy2 journey (unless someone codes a plugin for that). However, passing through the untouched "original" yuy2 color planes (as produced by RemoveDust, without limiting) is perfectly possible, with a small speed penalty.

If I made no error (you know ... writing a script without possibility to test it), the following should do it:

function LRemoveDust(clip input, int _mode, int "limit")
{
limit = default(limit,4)
LL = string(limit)
STR = "x "+LL+" + y < y "+LL+" - x "+LL+" - y > y "+LL+" + x ? ?"
repmode = 2
clensed = Clense(input)
rep = Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
trep = TemporalRepair(rg, rep)
trep2 = isyuy2(input)
\ ? trep.converttoyv12()
\ : trep
input2 = isyuy2(input)
\ ? input.converttoyv12()
\ : input
limit!=0 ? yv12lutxy(trep2,input2,yexpr=STR,U=2,V=2) : last
out = isyuy2(input)
\ ? last.converttoyuy2().mergechroma(trep)
\ : last
return out
}

But even if it's correct, still no support for RGB input. Sorry tmpeg users :|

[Boulder]

Thanks a bunch, I'll have to try it on my next capture tomorrow:)

EDIT: A quick test shows it works, at least no errors pop up;)

[Boulder]

I did some tests and found LRemoveDust wonderful! The YUY2 version is quite a bit slower (something like 0.29RT vs. 0.44RT), and I noticed that conversion to YV12 and back to YUY2 (and then back to YV12 in the encoder) is not noticable with TV captures so I ended up using the original function. LRemoveDust keeps details in motion better than RemoveGrain().RemoveDirt() so it looks like it'll become the next allround filter for my analogue captures -- at least till kassandro updates RemoveDirt;)

[kassandro]

Originally posted by Boulder
I did some tests and found LRemoveDust wonderful! The YUY2 version is quite a bit slower (something like 0.29RT vs. 0.44RT)
Yes, the YUY2 and RGB versions are of RemoveGrain etc. are a lot slower. I use a very simple class to separate the components of interleaved color space into planes such that I can apply the same functions as for YV12. Though I have written the interleaved2planar and planar2interleaved routines in C, I am quite surprised about the enormous slow down. I knew this for quite while, but didn't yet investigate the cause. Both conversions shouldn't be much slower than a bitblt operation. I should be able to privide faster assembly routines at least for YUY2. On the other hand, in RemoveDust and LRemoveDust filters from RemoveGrain/Repair are used several times and thus one has several times the interleaved2planar and planar2interleaved routines, which is simply stupid. If there would be planar versions of YUY2, RGB, RGB32 in Avisynth we wouldn't have these problems. We would simply convert to planar at the beginning of the script and back to interleaved at the end. Converting to YV12 and then back to YUY2 is not a good idea, because it blurs the chroma quite a bit. But Boulder, I do not understand, why you need YUY2 output. It could only be for CCE which wants to have YUY2 input although it produces YV12 output. This makes only sense if the higher resolution YUY2 chroma is used for searching motion vectors or a combined luma/chroma motion vector search is employed, but I doubt that CCE is doing that.

When I did my first analog capture recently, I saw that when using RemoveGrain with AVIsource, VdubMod crashes upon exit. I had the same problem with RemoveDirt once. The probable reason is that I didn't deallocate some objects properly and AVISource doesn't like that. This is a bug of AVIsource as well and I recommended to fix it long ago, because under certain circumstances objects cannot be deallocated. This is certainly not a serious problem and it will be fixed with the next version of RemoveGrain in the summer.

YV12LUTxy is a very useful, reasonably fast and also very smart (because of using look up tables) filter, which should be made internal, but in LRemoveDust it can be replaced by the much faster LimitChange. However, when I tested an optimised version two days ago it didn't work anymore, but I hope to post an inofficial version next weekend, which should speed up LRemoveDust a little. It will also come with a modified inofficial version of Clense, which then can be used together with mvtools to provide a slow, motion compensated version of RemoveDust. It should reduce motion blurring considerably but also worsen compression.


-- at least till kassandro updates RemoveDirt;)
Probably by the end of the month. I am testing right now.

[Boulder]

Originally posted by kassandro
Converting to YV12 and then back to YUY2 is not a good idea, because it blurs the chroma quite a bit.

With analogue captures it's OK because there is a substantial amount of chroma noise.

But Boulder, I do not understand, why you need YUY2 output. It could only be for CCE which wants to have YUY2 input although it produces YV12 output. This makes only sense if the higher resolution YUY2 chroma is used for searching motion vectors or a combined luma/chroma motion vector search is employed, but I doubt that CCE is doing that.

I think the main reason is that Cinema Craft doesn't want people screwing up things badly and then say that the program is crap. There's a thread about the whole thing somewhere in the CCE forum. It is a real shame they don't support direct YV12 input.

Probably by the end of the month. I am testing right now.
That's good news:)

[Boulder]

I just had a walk with the dog and this thought came to my mind:

1)source is YUY2
2)convert to YV12
3)use LRemoveDust_YV12, enable greyscale mode for even faster processing
4)convert to YUY2
5)put the original chroma information back
6)denoise the chroma (CNR2)

This way there would be no loss, right? Since luma noise is much more easily seen by the human eye, it wouldn't matter if chroma wasn't processed by LRemoveDust and you can use other plugins like CNR2, which is also fast, for that.

[kassandro]

Originally posted by Boulder
I just had a walk with the dog and this thought came to my mind:

1)source is YUY2
2)convert to YV12
3)use LRemoveDust_YV12, enable greyscale mode for even faster processing
4)convert to YUY2
5)put the original chroma information back
6)denoise the chroma (CNR2)

This way there would be no loss, right? Since luma noise is much more easily seen by the human eye, it wouldn't matter if chroma wasn't processed by LRemoveDust and you can use other plugins like CNR2, which is also fast, for that.

Yes, you can do that.

Actually, I can outwit Avisynth's color space limitations. For Avisynth a planar YUY2 frame will look just like an ordinary interleaved YUY2. However, the information is organized differently on the planar YUY2 frames. Say, you have a 720x576 interleaved YUY2 frame. Then the first byte is the luma of the first pixel, the second byte ist the U value of pixels 1 and 2, the third byte is luma of pixel 2 and the 4th byte is the V value for pixels 1 and 2 and then it goes on with pixels 3 and 4, etc. On a planar YUY2 frame, you have first the 720 luma values of the first line, then the 360 U values of the line and finally the 360 V values of the line. Thus I have only rearranged the values of the pixel lines, but any yv12 filter could handle such frames with only tiny changes just like yv12 frames. However, ordinary spatial YUY2 filters like a horzontal crop would be very destructive (a vertical crop wouldn't hurt) for such frames. Temporal filters, which treat luma and chroma equally could be applied to planar and interleaved YUY2 in the same way.

[kassandro]

Here is my motion compensated RemoveDust:

function MCRemoveDust(clip input, int _smooth, bool _grey)
{
repmode = 2
clmode = 4
quad = Quadruple(input) # each pixel is quadrupled, SSETools are needed
qpel = RemoveGrain(quad, mode=12, modeU=-1) # blur the luma for searching motion vectors
bvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=true)
fvectors = MVAnalyse(qpel, blksize=16, lambda=200, isb=false)
backward = MVCompensate(quad, bvectors)
forward = MVCompensate(quad, fvectors)
clensed = mcclense(quad, backward, forward, grey=_grey).shrinkby2(grey=_grey) #SSETools are needed for shrinking
rep=Repair(clensed, input, mode=repmode, modeU=_grey ? -1 : repmode)
rg = RemoveGrain(rep, mode=clmode, modeU=_grey ? -1 : clmode)
return TemporalRepair(rg, rep, grey=_grey, smooth=_smooth)
}

You need MVTools 0.9.9.1 and the following inofficial plugins (http://home.pages.at/kassandro/inofficial.zip). It contains a new inofficial version of RemoveGrain (a special version of clense for motion compensation has been added) and of SSETools (no documentation). SSETools contains also a filter LimitChange, which can be used for a faster version of Didee's LRemoveDust:

function LRemoveDust(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input)
rep=Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
rd=TemporalRepair(rg, rep)
return LimitChange(rd, input, limit, limitU=255)
}


Here is a version which also "limits" the chroma:

function LRemoveDust(clip input, int _mode, int "limit")
{
limit=default(limit,4)
repmode = 2
clensed = Clense(input)
rep=Repair(clensed, input, mode=repmode)
rg = RemoveGrain(rep, mode=_mode)
rd=TemporalRepair(rg, rep)
return LimitChange(rd, input, limit)
}

[Boulder]

That's a nice one, got another capture coming in later today so I guess it's time to play:)

[kassandro]

Sorry, Boulder. Something must have gone wrong here. I just ran a test and it turned out, that MCRemoveDust gives exactly the same as RemoveDust. There must be misunderstanding of the MVTools on my side.

[kassandro]

Unfortunately, the Filter mcclense, part of a slightly changed RemoveGrain plugin, was working exactly as clense, whence MCRemoveDust did work exactly as the RemoveDust. I have now uploaded a corrected and tested version. The script function remains unchanged. I did a test with a short 5500 frames part of a fairly clean black&white movie from 1952. Motion was definitely sharper as with the original RemoveDust. On the other hand in such old movies motion is always blurred a little. Thus this video was not very suitable for MCRemoveDust. Here are the compression results:

MCRemoveDust 18.896.476 Bytes
RemoveDust 18.791.986 Bytes

Thus with motion compensation one looses only 0.6%, which is very cheap for a significant improvement in quality. Unfortunately, encoding time roughly tripled with MCRemoveDust, where XviD was used with all advanced features except GMC. More detailed results will be posted within the next weeks.

[kassandro]

I just made a comparison with a high quality source from 2002 (63958 frames, 720x576, with ogg vorbis audio):

unfiltered 347.059.657 Bytes
RemoveDust 284.028.286 Bytes
MCRemoveDust 280.028.971 Bytes
LRemoveDust 297.983.847 Bytes

This time MCRemoveDust is ahead of RemoveDust by about 1.5% and contains visibly more detail in moving parts of a frame.
Because the source contained relatively little grain and was virtually dirt free, the compression gain was only 18.2% with RemoveDust and 19.3% with MCRemoveDust, less than half what I experience in the average with my usually much older films.
Encoding time with MCRemoveDust was a little bit more than two times the encoding time with RemoveDust.
Using LRemoveDust (should be identical with Didee's version) the output is 6.4% larger than with MCRemoveDust. The comparison with LRemoveDust is a little bit unfair, because it uses TemporalRepair with smooth=0, the default value, but RemoveDust, MCRemoveDust use smooth=1, but the impact should not be very significant (1-2%), because TemporalRepair only corrects the spatial denoiser RemoveGrain, such that it cannot touch (temporally) static pixels.

[Boulder]

How does LRemoveDust (luma only) compare? Using MCRemoveDust is unfortunately out of my league since the encoding time is already quite long and I can't run the computer overnight. However, it could be used on shorter clips and the result first saved in lossless format to speed up the encoding to the final format.

[kassandro]

Originally posted by Boulder
How does LRemoveDust (luma only) compare? Using MCRemoveDust is unfortunately out of my league since the encoding time is already quite long and I can't run the computer overnight. However, it could be used on shorter clips and the result first saved in lossless format to speed up the encoding to the final format.
I just finished the test with LRemoveDust and added the results to the above posting. I will run one more test with a much grainy source from the early nineties. Here the differences should be bigger.

[vigi_lante]

Sorry, but I can't get LRemoveDust to work. This is my script...

LoadPlugin("c:\filters\decomb521.DLL")
LoadPlugin("c:\filters\RemoveGrainS.dll")
LoadPlugin("c:\filters\RepairS.dll")
Import("c:\filters\LRemoveDust.avs")

AviSource("c:\capture (2005-03-08 at 00-05-53).avi")

Trim(0,7322)

Telecide(order=1,post=0)
Decimate(cycle=5,mode=2)

ConvertToYV12()

LRemoveDust()

The LRemoveDust code is inside LRemoveDust.avs

Is there anything wrong ?

[vigi_lante]

OK. I got LRemoveDust to work with this...

LoadPlugin("c:\filters\decomb521.DLL")
LoadPlugin("c:\filters\RemoveGrainS.dll")
LoadPlugin("c:\filters\RepairS.dll")
LoadPlugin("c:\filters\MaskTools156.dll")
Import("c:\filters\LRemoveDust.avs")

AviSource("c:\capture (2005-03-08 at 00-05-53).avi")

Trim(0,7322)

Telecide(order=1,post=0)
Decimate(cycle=5,mode=2)

ConvertToYV12()

LRemoveDust(4,2)



Just try it and see for yourself. Not that difficult, it's just a few lines.
Very roughly from memory: Together with XviD encoding, I got ~5fps for PixieDust and ~20fps for LRemoveDust. LRemoveDust encoding was only marginally slower than without any filtering - XviD was the most limiting factor, there.
For plain rendering without encoding, (L)RemoveDust is in the range of 8~10 times faster than PixieDust. Regarding *that* difference in performance, I think the results are so close to PixieDust, it's almost impertinent

Well, I using an AthlonXP 2400+ and LRemoveDust speed was not even 2 times faster than PixieDust(5).

I wonder what is wrong...

[Didée]

The question was:Originally posted by vigi_lante
Well, I using an AthlonXP 2400+ and LRemoveDust speed was not even 2 times faster than PixieDust(5).

I wonder what is wrong...

The answer is:Telecide(order=1,post=0)
Decimate(cycle=5,mode=2)

The promised acceleration values for a car don't hold up when the car is pulling a caravan.

[Boulder]

It should be faster, I do it like this:

org=last
ConverttoYV12()
LRemoveDust(4,2)
GreyScale()
ConverttoYUY2()
MergeChroma(org)

and it's still much faster than PixieDust. I have tweaked the function so that only luma is processed by adding the necessary grey=true and modeU=-1 parameters where needed, but the difference shouldn't be too big.

Then again, I have a SSE2 CPU which gives a nice boost thanks to kassandro's optimizations:)

EDIT: Didée's right..if you've tried using PixieDust in your IVTC script, it will show you what slow is;)