VagueDenoiser 0.35.1: a 2D Wavelet denoiser/smoother

[Lefungus]

A 2D wavelet denoiser/smoother.

It transforms each plane into the wavelet domain, then apply one kind of thresholding on the obtained coefficients, to reduce data that should be noise. With a low threshold you'll erase noise, whereas with higher threshold, you'll smooth the picture. It's somewhat effective against blocking too. Judge it by yourself :)

Grab it on my site (http://perso.wanadoo.fr/reservoir/) or on Kurosu site (http://kurosu.inforezo.org/avs/VagueDenoiser/index.html)

See the proper documentation into the package on how to exactly use it.

Required:
Avisynth 2.52
YV12/YUY2 colorspace

[Edited]
Removed obsolete things

[Kurosu]

Originally posted by Lefungus
To be updated soon, Fastsymmlet8 seems to give good results for deblocking ( don't ask me why :) )

Because Symlets are approximately symetric (d'oh) and Daubechies wavelets aren't?

*acts as if he really knew what he was talking of*

Anyway, that's a very good start (awaited in its results by many user), and indeed there can be many things to expect from the update. Is the method used now somehow linked to "waveshrink" (ie zeroing of too small coefficients) ?

[Lefungus]

OK, i've removed the bug
Now symmlet works again

it's quite slow though

For low filtering, begin at 0.5
strong filtering at 1-2

The method i use is soft thresholding, ie same as waveshrink it seems
All coefficients under the threshold are zeroed and those above are reduced to limit artefacts.

[d'Oursse]

hello,

i wanted to try your plugin but it requires libmmd.dll. I think it is the library that comes from the intel compiler. Am I right ?

Another possibillity is that you compile statically your plugin.

thanks !

[Dreassica]

U replaced the old dll with the new one yet? Cuz i am still getting a garbled picture when using the FastSymmlet8 routine (1).

[Lefungus]

Updated to v 0.12

-Precisions problems corrected
-The dll shouldn't need any other dll
-Symmlet8 supported

The source is also released, free.
I have indeed used Daniel Lemire code, the original file is included with the package.

Next step would be to implement other wavelets like Haar and also include other colorspaces.

[SILICON]

What is the Difference between VagueDenoiser and NIc's DCT filter?

Both make DCT and zeroed coefficients under one threshold.

[Kurosu]

The biggest difference (and I assume that's the reason why this filter is good at smoothing blocking artifacts) I see is rather the DCT is done locally, on a 8x8 block with no regard to neighbours in the image, while DWT operates on a whole image at once. Therefore, this introduces in the case of DCT discontinity problems on the 8x8 block' borders.

But maybe Lefungus has chosen an intermediate solution , though processing times are a hint that he's transforming a whole planar (plane?) at once.


So, what about to code Haar wavelet and verify if it is even better?(just kidding)

Actually, it is because symlet8 have 8 vanishing moments. DB4 have only 4. The smoothness of wavelet reconstruction increases with the number of vanishing moments.

Indeed, Haar has only one moment and doesn't have such a good result. Though I would have though wavelets with more than 3 vanishing moments (ie whose recontructed data is at least C² and therefore should smooth out all blocking artifacts) would have been sufficient, hence it wouldn't explain such a difference between the 2 used in this case.

Anyway, this gives me ideas for detecting other kinds of noise...

[edit]
Another (http://infolab.snu.ac.kr/thesis/IJ2.pdf) approach, that tries to identify the part of the decomposition that is due to blocking.

[Lefungus]

Properties of wavelets transformations allow them to be used on the whole picture, so that's what i've done. That's one of the strong point of this filter, i think.

I've played with it a bit, some kind of distorsion appear when the threshold is too high, like vertical or horizontal bars. I'll experiment with hard-thresholding to see if it change anything. But for denoising/smoothing purposes, a better way than thresholds should be found.

In the end i'd like to have a multi-purpose wavelet filter, any help would be appreciated :)

[Acaila]

I've played with it a bit, some kind of distorsion appear when the threshold is too high, like vertical or horizontal bars. Zeroing high frequencies will create ringing artifacts near edges (can easily be seen when you encode a video/image with a low bitrate), is this the kind of artifacts you are refering to? If so then maybe weighting the threshold based on the frequency could reduce this, i.e. a higher threshold for low frequencies and a lower threshold for high frequencies.

Just a thought.

[Lefungus]

Unfortunately, if i have understood correctly wavelets, left coefficients are not low frequencies and high coefficients, high frequencies like in DCT domain. So it's more complex.

Anyway it may be interesting to scale/reduce coefficients instead of zeroing them.

[Kurosu]

It seems to me it's more a result like http://members.optushome.com.au/emikulic/gfx/haar/cat-output.png

[MfA]

As far as ringing is concerned the cause is the Gibb's phenomenon same as with DCT, no changes there.

The horizontal/vertical lines however are not ringing, just typical artifacts of these kinds of denoisers ... I dont really know the cause, but my guess it has to do with the isotropic nature of the basis functions and the aliasing inherent in critically sampled transforms.

Using (pseudo) translation invariant transforms usually helps a lot (in fact with the undecimated wavelet transform for instance you can just use hard thresholding without having to suffer the usual artifacts). By their very nature these transforms have to be expansive though, so the coefficients will outnumber the pixels ... bringing with it higher memory and computation requirements.

[Lefungus]

Version updated to 0.2: lots of changes included, lots of wavelets implemented (15!)
a little faster
See the first post for more informations

There are no artifacts anymore when you select an high threshold, the picture is just highly smoothed
And you can finally select the number of steps, so Haar wavelet with 3 steps should be faster than Antonini with 8 steps.

Sometimes you can have a wrong picture, it means you've used too much steps, so reduce it until it works again.

now i just need to test all this :)

[Lefungus]

In my first tests Brislawn2 wavelet seems to give really good results

[Edited]

On Ghost In The Shell movie, i'm trying this :

VagueDenoiser(threshold=2.5,wavelet=7,method=1,nsteps=4)

It smooths blocks nicely, reduce interlaced artefacts, but still keep details and edges. And noise is killed :)

[Lefungus]

I think i will make some presets like convolution3d.
It definitely seems that the Brislawn 2 function gives the best results. At least it was on my own tests, both on an anime (GITS) and and a movie (Brotherhood of the wolves)

Speed isn't so bad after all, with 6 nsteps on a 640x256 movie, it's ~20 fps on my p4 2.4GHz

So here are my findings:

For a high quality movie:

VagueDenoiser(threshold=0.75,wavelet=7,method=1,nsteps=6)

Threshold can be increased up to 1/1.25, even 2 but you may lose fine details, it's visible a little on skins for example.

For a high quality anime:

VagueDenoiser(threshold=2,wavelet=7,method=1,nsteps=6)

Some may find 2 a little too strong.

On these movies, i've put VagueDenoiser before resizing, if you put it after, you may need to decrease the threshold.

Now i'll seek bad quality movies/anime and try to enhance them :)

[lamer_de]

k, here's a short test result:


avisource("M:\UM Raws\um7.avi",false)
Levels(0,1.000,236,0,255)
VagueDenoiser(threshold=4,wavelet=7,method=1,nsteps=4)


Using Athlon XP 1700+, Avisynth 2.5.2, VDubMod 1.4.13 build14328. I'm encoding to lossless VBLE in YV12 at 2fps with the script above (The levels is just in there cause the source is to dark for my eyes). The same script without VagueDenoiser gives me around 30 fps. The source is an Anime, 640x480, which has some slightly visible blocks in ultra-high motion scenes, only ringing in the rest of the episode. Lower thresholds gave the same speed, but almost no visible difference. With this high threshold, the blocks got slightly smoothed out (but were still visble). Ringing wasn't touched at all in my eyes.

The speed/visible quality gain is too low for me to put this filter in use.

The smoothing effect was better on a lq real life footage video full of blocks, but BlindPP delivers better results with more speed imho (If you have to work with such horrible sources full of blocks)


CU,
lamer_de

[Lefungus]

Well, it's not really designed to remove blocks but noise instead. Maybe it'll be possible later with a fine analysis of the wavelet domain. Wavelets open many possibilities, i've just tried the most simple one.

Anyway, any tests are welcome.

About speed, you're right, it's not usable yet to rip, too much slow. I only use it under vdub now, doing comparisons. i guess what i really need is a fast asm-optimized dwt transform like xvid has for DCT. Now it's pure C code full of memory leaks so that doesn't help :) !

i'll try to improve it, but don't expect much from me on the speed side.

[sh0dan]

There are (fortunately) several ways of optimizing this routine - some that springs to mind.

Edit: Silly me hadn't seen you had already done 1) in 0.2 - I'm leaving it there for understanding. ;)

1) Use floats instead of double. There seem to be (unused?) routines that already does this. It would make assembler optimization possible, and enable better cache efficiency. I don't know if the precision is good enough - Lefungus - what do you say?

2) Assembler int -> float -> int conversion. Since we have a large array this is fairly trivial. I havne't looked at the access patterns in the transform, but it might be a good idea to actually do the conversion there.

3) SSE / SSE2 transforms. This should be able to at least double the speed of the transform/inv transform. Not that trivial, but probably this is where the biggest gain can be found with the current algorithms. Unfortunately none of my current CPU's support SSE, so I might have to get a new MB first. ;)

4) The Intel C++ compiler should be able to squezze out some percent more performance.

If you would try out implementing 1), I wouldn't mind doing 2) and 3).
4) Can be tried now, if anyone has the compiler, and knows how to use it.

We should probably move further dev. talk to the dev. forum.

Edit 2: Did some profiling - the results are here (http://forum.doom9.org/showthread.php?s=&threadid=57084).

[Lefungus]

Version upgraded to 0.21
-The filter doesn't require SSE anymore
-Some improvements from Shodan and Bidoche increase performances up to 30%

[MfA]

Why wouldnt it be true?

[Lefungus]

I've made some comparisons on GITS

the original avisynth script is:

mpeg2source("C:\Vobs\Ghost In The Shell\Movie.d2v",CPU=0,IDCT=5)
telecide(order=1,guide=2)
crop(6,14,708,546)


It's for the first picture

For the second picture, same script but with CPU=6

And third picture, same script, CPU=0, but with VagueDenoiser

You'll see it clean quite well the last interlaced artifacts that have passed through Decomb.

Anyway, results are good too even without interlaced artifacts, i think.

Small comparison (http://perso.wanadoo.fr/reservoir/)

[Lefungus]

Version updated to 0.22:
- YUY2 colorspace support
- Optionnal chroma filtering, (chroma=true/false)
- Cosmetic changes, code cleaned
- Html documentation
- Villa 10/18 filter added

"wavelet" parameter has been renamed to "filter".
The wavelet filters has been sorted.

[unplugged]

Sorry, latest version crashes VDub/Avisynth :(, even with suggested values
I have tried 768x576 and 720x576 res with YUY2 and YV12.

[Lefungus]

I've encountered the "stupid typo error" bug, sorry, i've uploaded version 0.221

[Lefungus]

In order to improve speed, the filter may have to lose flexibility and choose one filter over the 15 today.

My opinion is that Brislawn 10/10 (filter=7) gives the best results.
Haar and any Daubechies filter are always ugly. The others gives fairly similar outputs.

With high tresholds (15-20), Brislawn keep more features than any other filters. That's why i think it could be better. But i use the denoiser with thresholds around 2 so it may be different.

On thresholding, i've found that soft thresholding (method=1) gives a more smoothed result so a lower threshold is required. But the output is more natural, and i've seen some artifacts with hard-tresholding (method=0). So i think to stick with method=0 is a good idea.

More results on various sources will be helpfull, as i can't do as many tests as i'd like to do.

[sh0dan]

Otherwise you could always just call the optimized functions for filter 7. But it seems like a good idea.

Minor stuff:
__inline unsigned char VagueDenoiser::realToChar (float x) {
return (x>255?255:(x<=0?0:(unsigned char)(x+0.5))); };

If you did an unrolled transform/invert of filter 7, you might already get a speedup - and it would be easier for us to assembler optimize it. :)

Edit: Why is there smilies in my [code] section? :(

[Leuf]

I can't seem to get the plugin to load, I just get LoadPlugin: Unable to load vaguedenoiser.dll. Do I need something in addition to 2.5.2?

[Lefungus]

I really don't know :/
i don't even use loadplugin on my computer, i just call it in my script.
And i only use avisynth 2.5.2, nothing exotic installed.

[Lefungus]

Version updated to 0.23:
-Code cleaned, filters class removed.
-Better parameters for compilation. Should really works on every cpu now.
-Removed all filters except Brislawn 10/10, so filter parameter has been removed.(no more filter=7).
-A little speed increase (3-5fps on a 640x256 frame with nsteps=6).

[Lefungus]

Version updated to 0.24:
-Another speed increase, some critical loops unrolled (To infinity and beyond!) :)

I reach 18 fps for a 640x256 YV12 movie. It's still slow but better than 12 fps ! Quality should be unchanged.

[Leuf]

Well, I don't know how to explain it but I was able to compile 221 and it works, my compiled version is 68 kb vs the 96 kb in the download - so we definitely have some different compiler settings. I can't see how that would make it not load though. I have the same problem with 24, and I can't get the source to compile. At least I have something to play with now though.

[Kurosu]

@Leuf
If you have duplicated symbols (namely the float arrays), move them from wavelet.h (Vaguedenoiser doesn't need them) to wavelet.cpp. Another needed modification is replacing
#include "wavelet.cpp"
by
#include "wavelet.h"

If that still doesn't work (it worked for me), use the usual trick

#ifndef __SOMETAG__
#define __SOMETAG__
...
#endif


This way, the .h file doesn't get parsed several times.

To reply to your other concern, the filesize difference is probably due to the compiler trying to do costly optimizations such as loop-unrolling, and maybe padding.

[Lefungus]

There's a bug in latest version, so until it's fixed keep sure res is at least mod16 or even mod32.

[Lefungus]

Version updated to 0.241:
Visual artifacts were seen when clip width or clip height weren't mod4.
So i've added checks at initialisation.

It should be bug-free and crash-free again. And visual output should be exactly the same than with older versions with filter=7

Thanks Kurosu for your trick with "ifdef", it's corrected too.

@Leuf:
Can you try the latest version and tell me if you have problems using it ? Which compiler are you using ?
I use icl 7.1 and checked Blended option. That mean it should works on every cpu on the planet. i don't see any other option that could prevent you to use it. Maybe someone experienced with icl could help there.

[Lefungus]

another version: 0.242 ("The bug wasn't dead")
-I've really removed the nasty bug now. it should be pixel-perfect equal with output given by 0.221, and this with no restrictions on width and height. So no more mod4.
-Some improvements from Kurosu.

Note that YV12 is the favorite mode of this filter, YUY2 will work but slower.

[Leuf]

Your latest compiled dll still doesn't work for me. I'm using MSVC 6 sp5/pp5.

[Lefungus]

Originally posted by Leuf
Your latest compiled dll still doesn't work for me. I'm using MSVC 6 sp5/pp5.

I'm confused, what doesn't work on your computer, my own compiled dll or are you trying to compile it yourself ?

[Leuf]

I can compile the source fine with msvc and it works, but your dll doesn't work for me. I get the error right at LoadPlugin, it just says unable to load the file as if it couldn't be found. The only thing I can think of why it wouldn't work for me but does for others is I'm on win98. I would think the problem would center on avisynthplugininit2() and how it is getting exported. But if it works for everyone else *shrug* I wouldn't worry about it too much.

[Kurosu]

@leuf
What's exactly your CPU ?

[Lefungus]

version updated to 0.243:
-Code cleaned

No more versions from me for a little while. I've squeezed speed as much as possible with pure C.
Compilation parameters should be almost optimal too. It could be made a little faster for p4, but it'll require two dlls.

Time to get some holidays now :) !

[Kurosu]

I finally managed to get some optimization done into this filter... Using 3DNow! (ie only AMD processors will be able to use it, sorry, SSE2/P4 enthusiasts), I got an mind-breaking speed improvement of... 17% for 0.241, on which it's based.

For people really using that filter, and after a speed improvement, even the tiniest, get the filter here (http://kurosu.inforezo.org/avs/VagueDenoiser.zip). And for those that can't run the filter for whatever reason, a VC6 compile is included...

[Lefungus]

Version updated to 0.25:

-some speed increase (profile & vectorization)
-defaults added (threshold=2,method=1,nsteps=3,chroma=true)
-a visual.net dll is also included for compatibility purposes. It's a lot slower though


3Dnow optimisations from Kurosu aren't included yet, i'll try to include it later.

Output could be slightly different from previous versions with height and width not mod32. But it's not noticeable and have no impact on compressibility. (I have 97db instead of 103 db on my compared clips when i use threshold=0)

I have now 22fps on a 640x256 YV12 movie with these settings
Vaguedenoiser(threshold=2.5,nsteps=6)
It's above realtime when i don't process chroma.

[morsa]

As Leuf said: "I get the error right at LoadPlugin, it just says unable to load the file as if it couldn't be found. The only thing I can think of why it wouldn't work for me but does for others is I'm on win98."

Well the same applies to my situation, WIN98SE.
No way to load it, and I'm the most stupid guy in the world for compiling soft.

[Lefungus]

Originally posted by morsa

No way to load it

Could you try this version (http://perso.wanadoo.fr/reservoir/dl/VagueDenoiser-0.251.rar), and tell me if it works better ?

[morsa]

Well managed to compile it myself and....It works now!!!
Don't know why but it is functioning perfectly and is very fast for me.
I'll try your new version tonight.
Thank you

[Boulder]

What's the status of this fine filter, any new versions coming out?

[Lefungus]

0.26 was released a while ago in avisynth development forum. And 0.27 is coming too with nice improvements, thanks to Kurosu.

[Kurosu]

And here is 0.27 (http://kurosu.inforezo.org/avs/VagueDenoiser/index.html)

This time, I'd like some report. I suspect 0.26 could crash easily, yet nobody reported it...

@Lefungus
Feel free to mirror. FU2 Development thread.

[edit]
Even better maybe, could you edit your first topic so as to link to it (mine or the one you would host)?

[Mr_Khyron]

can VagueDenoiser be used on interlaced videos or do i have to deinterlace?
:confused:

[Lefungus]

It has no special case for interlaced videos. Interlaced videos introduce too many headaches amongst too many devs, so i've just skipped it :D

About 0.27:
- faster
- even if it's not icl compiled, it's faster for pentium 4 too !
- Qian thresholding that should keep more details than soft, but still be more effective than hard.
- experimental adaptive thresholding mode, that compute a threshold for each levels of the wavelet decomposition
- debug mode that show wavelets coefficients

Feedback would be cool, for the new adaptive mode, or the Qian thresholding.

[Kurosu]

Fixed YUY2 problem, added RGB24/32 support while I was at it.
Link as above, or at the start of this thread when Lefungus has edited it.

[unplugged]

Originally posted by Mr_Khyron
can VagueDenoiser be used on interlaced videos or do i have to deinterlace?
:confused:
Discussed many times.
Of course if you don't want deinterlace, like me and others:

# Fast method for non-temporal cleaners (beware, more aggressive vertically!)
SeparateFields()
AnySpatialCleanerYouLike()
Weave()
...
# Optimal method for spatial/temporal cleaners (must for temporal)
ComplementParity().Bob(0,0.5)
AnySpatialCleanerYouLike()
AnyTemporalCleanerYouLike()
SeparateFields().SelectEvery(4,1,2).Weave()
Voilà! ;)

[Piper]

@Kurosu

Thanks for the quick fix, only it appears as though the zip at VagueDenoiser (http://kurosu.inforezo.org/avs/VagueDenoiser/index.html) contains the 0.27.0.0 dll version.

[Kurosu]

It should work anyway for the matter you encountered, as I just forgot to edit the version file.

I however found something rather fishy. While fixing the bug increase by 50% the filter's speed, there are still strange result for lower resolutions.

[Piper]

Originally posted by Kurosu
It should work anyway for the matter you encountered, as I just forgot to edit the version file.

It did. Thanks again.

[Shayne]

Like to say this is a super denoiser right out of the box. I have finally found the replacement for my old DNR2.

I will need to play with the settings some. Anyone been tweaking and have some settings to share?

[cipher]

I've just started using this filter rencently, and it gave really good results!
This idea of using wavelet is quite interesting and creative.
Nice work, Lefungus!

edit:
btw, would it be possible in the future it'll be working in RGB colorspace?:)
thx!

[Mango Madness]

chroma option with method 2 crashes

[Chainmax]

Awesome, I have been looking for a replacement to Dust to be used on my DVD rips. Can anyone tell me what settings would make VagueDenoiser act closest to Spacedust()?

[Lefungus]

Dust is temporal (motion involved) whereas Vaguedenoiser is spatial only, so i don't think you could clone dust behaviour with this filter

@Mango Madness:
I don't think i'll improve this filter soon, i'll try to look at it, and i may just disable method 2

@Cipher
Rgb filtering has already been added by Kurosu in 0.28. I haven't tried it, as it's already slow as hell in yv12, but it should work

[Dali Lama]

LeFungus,

Chainmax was talking about replicating "SpaceDust" which is a spatial only filter. It is a setting of the Dust filter.

bye,

Dali

[Chainmax]

Yeah, that's right. I am about to rip my Simpsons DVDs and would like to apply some slight noise reduction before IVTCing/Deinterlacing. Since I'd like to avoid colorspace conversions [(Space)Dust works in YUY2], I want to try VagueDenoiser instead.

[Chainmax]

I am trying VagueDenoiser (in the Simpsons rip I mentioned earlier) for slight denoising before IVTCing. If I use

SeparateFields()
VagueDenoiser(threshold=1.2,method=1,nsteps=6,chroma=true)
Weave()

VDubMOD crashes when loading the avs and WMP reports an error in the VagueDenoiser settings line. However, if I use

ComplementParity().Bob(0,0.5)
VagueDenoiser(threshold=1.2,method=1,nsteps=6,chroma=true)
SeparateFields().SelectEvery(4,1,2).Weave()

like unplugged suggested, the problem is gone. Can anyone tell me why? I'm using Avisynth 2.53 on an 850Mhz Athlon with 160Mb Ram and W2k.

[Chainmax]

Well?

[Lefungus]

Try to reduce nsteps, it means how much you decompose the original picture. If the original picture is too small, the filter won't be able to do n passes on it, so it may bork. nsteps=3 may give good enough results, and you'll gain speed.
If it's not an nsteps issue, then i don't know..

[Chainmax]

I'm not resizing, so frame size is 720x480. My computer has been acting weird lately and a formatting is long overdue anyway. I'm going to try again with SeparateFields()+Weave() after resintalling the OS and report back.

[RadicalEd]

Well, seperatefields() still feeds vaguedenoiser a 720x240 image.

[Chainmax]

I wouldn't call 720x240 a small image, it's the same number of pixels than 480x360...
Oh well, as I said I'm going to try using separatefields+weave without lowering the nsteps after I format my machine. If that doesn't work, then I'll try your alternative. In any case, I'll tell you guys how it turns out. Thanks for the suggestions :).

[Xndo]

so what would you suggest for a 512x384 res.

[Chainmax]

Well, that is a ~14% bigger frame size than 720x240. Try using nsteps=6, the worst that can happen is that VDubMod will shutdown and you'll have to lower the setting.

[Xndo]

well I'm aleady haveing thread deadlock issues with vdub right now, so I don't think if it errors out on that setting, isn't going to matter much ;P

[Chainmax]

Be sure to use version 1.5.4.1 of VDubMod. I read somewhere in here that later version are severely bug-ridden.

[Xndo]

i just went back to using the 1.76 project file from d2v, seems to work just fine. Also i didn't have any errors on using nsteps=6 or nsteps=3, though it did add another hour to per pass, on my encoding time =(... o well, at least it looks better

[Lefungus]

Vaguedenoiser 0.29:

Updated version from Kurosu
- Lots of changes in asm parts, now under NASM
- Many bugs fixed, in particular with colorspaces and interlaced mode


You shouldn't see any more version anytime soon from me or Kurosu as we have both quite dropped its development.

Kurosu have also done some work on a similar Vaguedenoiser filter for Gimp. if anyone is interested, you know his email :)

[P0l1m0rph1c]

@Lefungus: are you sure you updated your page? I can only see links to download Vaguedenoiser 0.28 there.

But anyway, i edited the link myself (http://perso.wanadoo.fr/reservoir/dl/VagueDenoiser-0.29.rar), and in the readme it says it's version 0.28.2.

I supose you forgot to update something, right?

BTW, I can't reach Kurosu's page, which is in the first post of this thread. It gives a 404.

[ARDA]

First of all thanks for the great work both have been doing with this filter.

with the following script

mpeg2source("mysource.d2v")
crop(12,0,704,476,align=true)
LanczosResize(672,384)
SeparateFields()
VagueDenoiser(threshold=0.8,method=1,nsteps=3,chroma= true)

I've got access violation or an image with distortion
I've tested with divx, xvid and ffdshow decoder,in any case
I don't get it to work correctly.Other resolution after
SeparateFields also gives problems.
Maybe I should use interlaced=true and apply SeparateFields afterwards
So I don't know if it is a bug or filter limitation that should be
mentioned.
Thanks again.ARDA

[P0l1m0rph1c]

Originally posted by P0l1m0rph1c
@Lefungus: are you sure you updated your page? I can only see links to download Vaguedenoiser 0.28 there.
Stupid me, I was reading Lefungus' page from cache :confused:

But anyway, i edited the link myself (http://perso.wanadoo.fr/reservoir/dl/VagueDenoiser-0.29.rar), and in the readme it says it's version 0.28.2.
It seems readme has been updated too now. And it seems also Kurosu removed his page.

I would like to thank you all for your great work in this filter
:)

[Lefungus]

This filter is sensitive to resolution, so i would advise using separatefields() after it. If the source is interlaced, maybe using interlaced=true is enough.
I'm not sure what separatefields exactly do. With a resolution of 672x192, vaguedenoiser works fine, but not after separatefields, i'll try to find why, and correct this at least.

[killerhis]

Avisynth open failure:
LoadPlugin: unbable to load "C:\blabla\vaguedenoiser.dll"

WTF is this... just reinstalled my pc.. wanna try the new vaguedenoiser, and I get this error, what am I missing, by the way, path is right... copy/pasted it and check it 5 times! If I remove it, then my script works perfectly.. but I want vague :devil:

who can help me?

[serbersan]

Yes I have had the same problem with the last version, it seems a problem of .29 version, I've downloaded the version 0.27 from downloads page of doom9 and it works ok¡¡

[DarkNite]

I confirm that unable to load error. Version .29 will not load on any of the pc's here.

Previous versions are working though. Just revert to a previous version until it all gets worked out.

[killerhis]

asd;ljaslkdjas :angry: :angry: :angry:

I hate this, please fix this in new release.. meanwhile I'll go back to old version :angry:

[Sigmatador]

Originally posted by killerhis
asd;ljaslkdjas :angry: :angry: :angry:

I hate this, please fix this in new release.. meanwhile I'll go back to old version :angry:

...

[GrofLuigi]

If I have VagueDenoiser.dll present in my AVISYNTH plugins directory, VirtualDubMod throws the following error on START:

"VirtualDubMod.exe - Unable To Locate Component

This application has failed to start because libmmd.dll was not found. Re-installing the application may fix this problem."

and then it continues to work normally. It is reproducable, if I remove Vague Denoiser, all is well. If I put it back in... you get the idea. Again, this is on startup of VirtualDubMod, even before I get the chance to open ANY file.

BTW, libmmd.dll is present in some Adobe products.

GL

[celtic_druid]

libmmd.dll is an Intel compiler library... Simply means that the plugin was compiled with ICL.

[Valky]

Originally posted by d'Oursse
hello,

i wanted to try your plugin but it requires libmmd.dll. I think it is the library that comes from the intel compiler. Am I right ?

Another possibillity is that you compile statically your plugin.

thanks !

So version 0.29 needs this dll in my machine too (Athlon XP). Any chance to get this filter work without it?

[Valky]

Originally posted by serbersan
Yes I have had the same problem with the last version, it seems a problem of .29 version, I've downloaded the version 0.27 from downloads page of doom9 and it works ok¡¡

Couldn't find it there..could you help a little bit? In which section it is suppose to be there?

[DarkNite]

You can always try warpenterprises (http://www.avisynth.org/warpenterprises/).

[Soulhunter]

Link... (http://forum.doom9.org/showthread.php?s=&postid=473342#post473342) ;)


Bye

[Lefungus]

I haven't had time yet to fix the 0.29 issue, but for those who need it, my site still host some old vd versions, just change the version number of the last link. (dll only 0.25/0.251/0.27/0.28/0.29, with sources 0.221/0.25/0.27/0.28/0.29 ).

[ProTech]

Originally posted by Valky
So version 0.29 needs this dll in my machine too (Athlon XP). Any chance to get this filter work without it?
http://www.rarewares.org/files/libmmd7.zip

[Alxemi]

Hi I´m trying to use VagueDenoiser because i´ve read very good things about it :)
But i must confess that get lost a little with the settings.. have you think about code some presets like 3cd/mipsmooth has?
Or maye someone in the forum that could suggest settings for typical confs (moviehq, movielq, animehq, etc)

Thanks a million!!

[Soulhunter]

I like this configs...

# Slight noise

Clip=last
clip1=clip.VagueDenoiser( threshold=0.25, method=1, nsteps=6, chroma= false )
clip2=clip.VagueDenoiser( threshold=0.50, method=1, nsteps=6, chroma= true )
clip1.mergechroma(clip2)


# Normal noise

Clip=last
clip1=clip.VagueDenoiser( threshold=0.50, method=1, nsteps=6, chroma= false )
clip2=clip.VagueDenoiser( threshold=1.00, method=1, nsteps=6, chroma= true )
clip1.mergechroma(clip2)

For stronger noise I also add a temporal de-noiser, but for anime mainly Deen !!!

[Lefungus]

Just to be sure:
chroma=false apply filtering on luma only
chroma=true apply filtering both on luma AND chroma

So i don't understand why you're filtering like this

[Soulhunter]

To use separate thresholds for luma and chroma denoising... :rolleyes:


Bye

[Fizick]

I tryed to tweak Vaguedenoiser for my grainy clip without great success, but probably tweaked version will be helpful to somebody else. However it is quite experimental (beta).
So, Ver. 0.30 from Fizick.
Added (restored) Brislawn 9/7 wavelet (C version only) and wavelet selection option.
Added WienerChop two-pass denoising mode using two different wavelet basises.
Replaced boolean "chroma" parameter to float "chromaT" as threshold value for chroma planes.
Added automatic (adaptive) threshold estimation .
3DNow mode of hard thesholding temporary replaced by SSE or C versions.
Fixed possible memory leakage bug for non YV12 modes.

http://bag.hotmail.ru

[Kurosu]

Originally posted by Fizick
Added (restored) Brislawn 9/7 wavelet (C version only) and wavelet selection option.
Restored? Or did you make the option available. AFAIK, I had hidden it because only one wlt was present. Btw, I guess that's not a 9/7 Brislawn wavelet but a Daubechies one. I don't know if the code/framework is compatable, but you have optimized GPL code for that here (http://sourceforge.net/project/showfiles.php?group_id=38123&package_id=79794)


Added WienerChop two-pass denoising mode using two different wavelet basises.
Very easily optimizable, but I guess it's coming.

Added automatic (adaptive) threshold estimation.
You should contact Lefungus, he had something up his sleeve too.

3DNow mode of hard thesholding temporary replaced by SSE or C versions.
Why is that? Crash/unexpected result?

Fixed possible memory leakage bug for non YV12 modes.Didn't know about this one, so, well done.

[Fizick]

Kurosu! I am glad to see you !
1. I say about restoring similar option from old version 0.221.
2. I use Brislawn 9/7 wavelet from v.0.221 source.
There are many 9/7 wavelets: Antonini, Villa1, Odegard, etc (but not Daubechies).

EDITED: Seems, Brislawn 9/7 is popular CDF 9/7 wavelet (A. Cohen, I. Daubechies, and J. C. Feauveau)


I do not know what is used in "Wavelet 9/7 codec". Thanks for link.
It is possibly, that some new wavelet types can give better results (coiflets etc). It is still researched.

3. Very easily optimizable, but I guess it's coming.
May be later.
But seems, I have not any difference in speed not with 3dnow not with SSE (probably due to slow SDR memory for Athlon XP 2400).
Anybody can post some speed results with 0.29 and 0.30 (for various settings)?

4. 3DNow mode of hard thesholding simly not works (null transform) for my compiled source of original version 0.29. I try VC6 and NASM 098. Released binary 0.29 works O.K.
You must look to it (not my 0.30 version, but enough original 0.29).
May be released 0.29 source (from Lefungus homepage) is not last one?
It is probably true, for example the main part of code was:

void VagueDenoiser::filterBlockPlane(int h, int w, const unsigned char* src,
unsigned char* dst, int srcPitch, int dstPitch)
{
if (!itlc)
{
/* Most usual case, always this one for YUY2 and RGBx */
if (w != srcPitch) byte2float(h, w, src, srcPitch, block);
else byte2float1D(w*h, src, block);

//Apply filtering

wavelet->transform2d(block,w,h,nsteps);
//if (method != -1) filter(block, w*h, thresh, w, h);
if (!debug)
wavelet->invert2d(block,w,h,nsteps);



Yes, "filter(...) was a comment! I spend a lot of time until see it.:confused:
Probably it was your "anti-hacker protection"? :D

[Fizick]

Seems, Qian method=3 do not work with wavelet=1 on Athlons?

[Fizick]

V0.31
Added noise ratio parameter for Wiener pass.
Fixed interlaced mode. Seems it now work.
3DNow mode of Qian thesholding temporary replaced by SSE or C versions.

[Lefungus]

Thanks for your updates Fizick !

About my adaptative thresholding part, i dropped it due to unsatisfying results.

[Fizick]

My implementation of automatic thhsholding is also simply experimental.

[Kurosu]

Originally posted by Fizick
Kurosu! I am glad to see you !
Thanks :)

EDITED: Seems, Brislawn 9/7 is popular CDF 9/7 wavelet (A. Cohen, I. Daubechies, and J. C. Feauveau)
I do not know what is used in "Wavelet 9/7 codec". Thanks for link.
Yes, that's the one used in the lossy part of JPEG-2000.

It is possibly, that some new wavelet types can give better results (coiflets etc). It is still researched.
There are a bunch of wavelets to choose from, depending on their abilities. I don't know what wavelet that would work in the current frame work. For instance, the "wavelet transform modulus maxima" method is well-known for its ability to separate details, but can't be used directly here (and I would say, not easily).

But seems, I have not any difference in speed not with 3dnow not with SSE (probably due to slow SDR memory for Athlon XP 2400).
Anybody can post some speed results with 0.29 and 0.30 (for various settings)?
I always found better results with 3DNow! than SSE with my Athlon Barton 2800 (DDR333). I even found that in some situations, the SSE part was horribly slower, maybe due to a mix of bad pairing and lack of proper prefetching.
Overall though, videos that are MOD64 (like 512 or 640) takes a huge stall somewhere in the filter. I don' recall well where, but that's why I suggested to filter before resizing. Maybe it decreases the speed difference in your case.

4. 3DNow mode of hard thesholding simly not works (null transform) for my compiled source of original version 0.29. I try VC6 and NASM 098. Released binary 0.29 works O.K.
You must look to it (not my 0.30 version, but enough original 0.29).
*must* ? ;)

Yes, "filter(...) was a comment! I spend a lot of time until see it.:confused:
Well, releasing the source code is really not a hacker protection :p
No, I simply guess that was left from my in-between experiment on localized/in-place transform (the actual code is really bandwidth-hungry, I guess it's now the main limitation).

Anyway, I *did* fix all modes you reported as broken. I also fixed an annoying warning with the rodata section of the assembly objects. You can have get the fixed asm files from here (http://kurosu.inforezo.org/avs/filter.zip)

[Fizick]

Anyway, I *did* fix all modes you reported as broken. I also fixed an annoying warning with the rodata section of the assembly objects. You can have get the fixed asm files from here
O.K., one of us *must* ;) incorporate it to new version.

the actual code is really bandwidth-hungry, I guess it's now the main limitation
There are a bunch of wavelets to choose from, depending on their abilities
What about speed, MAY BE more fast would be using wavelets with integer (rational) coefficients (5/3 etc) with scaling, and using not float, but integer calculation. But it is full rewriting of the code.

And the last note (to all). Vaguedenoiser uses fast decimated wavelet transform. It is not shift-invariant. Therefore, if we shift image and apply filter, and re-shift backward, the result we be different from
sipmly apllying th filter. More precisely, the main features will be same, but some small details will be different. In particular, some artefactes will be also different. It may be used for improving the result by combining two (or more) images with Layer command.
It will be partially equivalent to undecimated wavelet transform.

Similar method is used by Didée for deblocking of Dust results.

AVIsource("h:\makar.avi")
loadplugin("masktools.dll")
loadplugin("vaguedenoiser.dll")

// some border mirror function motivated by Didée.

Function MirrorBottom( clip clp, int "v2" )
{
v2 = default( v2, 8 )
bot=clp.crop(0,clp.height-v2,-0,-0)
stackvertical( clp, bot.flipvertical() )
return last
}

Function MirrorTop( clip clp, int "v1" )
{
v1 = default( v1, 8 )
top=clp.crop(0,0,-0,v1)
stackvertical( top.flipvertical(),clp )
return last
}

Function MirrorRight( clip clp, int "h2" )
{
h2 = default( h2, 8 )
right=clp.crop(clp.width-h2,0,h2,-0)
stackhorizontal(clp,right.fliphorizontal())
return last
}

Function MirrorLeft( clip clp, int "h1" )
{
h1 = default( h1, 8 )
left=clp.crop(0,0,h1,-0)
stackhorizontal(left.fliphorizontal(),clp)
return last
}

converttoYV12()
input=crop(320,8,240,512) # for debug

# Shift image 2 pixels to the right and bottom
shifted=Crop(input,0,0,-2,-2).MirrorLeft(2).MirrorTop(2)
first=VagueDenoiser(input,threshold=27,method=0,nsteps=5,wavelet=1)
second=VagueDenoiser(shifted,threshold=27,method=0,nsteps=5,wavelet=1)
reshifted=Crop(second,2,2,-0,-0).MirrorRight(2).MirrorBottom(2)
sum=YV12layer(first,reshifted,"add",level=128)
stackhorizontal(input,first,reshifted,sum) # to see all


The summary result (and speed :( ) is similar to Wiener method.
See example.

http://bag.hotmail.ru/vague/makarl.png

[Fizick]

It seems, it is crached in SSE mode, if there are two or more Vaguedenoisers in script.
Anybody is here?
I also still wait speed report before releasing next ver.

[MfA]

Slow already? You still have to repeat the denoising 4^nsteps - 2 more times to become shift invariant :) (For true shift invariant transform based denoising you are better off just using a block DCT ... faster and easier.)

[d'Oursse]

Originally posted by Fizick
What about speed, MAY BE more fast would be using wavelets with integer (rational) coefficients (5/3 etc) with scaling, and using not float, but integer calculation. But it is full rewriting of the code.

i don't know if yo a re aware of algo too compute such wavelets (at least, scale functions), but you could use an induction algorithm to compute the scale function on the points k/2^j (j>=0, k \in Z). It's quite fast and i think quite optimizable, too

regards

[Fizick]

MfA, I do'nt want to become shift-invarint! :)
I simply use that i am not.

d'Oursse, interger wavelet well known (it used for example in loss-less jpeg2000).
I found a lot of them now. Fo example, 5/3 wavelet coef is:
h =-1/8 1/4 3/4 1/4 -1/8
g = 1/4 -1/2 1/4 0 0

In current code it is possible simply to change float to integer, and add integer divider (scale.)

But as Kurosu says:
the actual code is really bandwidth-hungry, I guess it's now the main limitation
The mostly time spend to copy with pitch (transpose colunm to lines etc). I checked it.
More over, I not found any speed increasing with assembler 3Dnow and SSE in wavelet multiplication part.

So, now i am a pessimist about speed in current framework. May be with lifting method?

[Soulhunter]

Originally posted by Fizick
I also still wait speed report before releasing next ver. I could do, if you give me a list of settings I should use for the test... ;)


Bye

[Lefungus]

Originally posted by Fizick
So, now i am a pessimist about speed in current framework. May be with lifting method?

I tried to add integer lifting methods (http://perso.wanadoo.fr/polyvalens/clemens/lifting/lifting.html) too without great success, but i still think it's the way to follow. The whole plugin would have to be changed though.

[Fizick]

Lefungus, it seems we stil will live with your plugin. :) :(

Soulhunter or somebody, please check speed of v.0.29 and 0.30 (or above) with wavelet=2 without Wiener, method=0, especially for Athlon:
Vaguedenoiser(threshold=2, method=0, wavelet=2)
Also it is interestly to compare speed for different wavelets .

But sizes must not be mod 64 (i.e. NOT 640)!
As Kurosu noted, there are big slowdown for such sizes. As I found, it is due to 2 COPY functions, mostly following (pitch2 is frame width or height):
_inline void Wavelet::copy(float register*p1, float register*p2, int pitch2, int length)
{ // very-very slow for pitch2 is mod64 (cache ?)
for(int i=length;i!=0;i--)
{
//*p2=p1[i];
*p2=*(p1++);
p2+=pitch2;
}
}

Very simple code (may be too). :rolleyes: Any processor and assembler guru here?

By the way, I prepare v.0.32.
Part of work was done by Kurosu!
Added Villasenor-Belzer-Liao 6/10 wavelet (not optimized C version only)
Changed estimation wavelets to more optimal pairs for WienerChop mode.
(Brislawn wavelet has too different thresholds and sharpeness).
3DNow optimized version of WienerChop and AutoThreshold - thanks to Kurosu. :)
3DNow mode of hard and Qian thesholding still have a some bugs and are not used. :( - Hard method produce some vertical lines, and Qian work more similar to hard than Qian.

I can not release it now due to some problem with www server.:confused: Later.
Probably I will have time to get report from Soulhunter. :)

[DarkNite]

Testbed

Athlon XP 2400+, 512MB PC2700, Windows XP SP1a

HelixYV12 AVI-> AviSynth 170604-> AVS2AVI 1.39-> HelixYV12 AVI

Script
loadplugin("vaguedenoiser29.dll")
#loadplugin("vaguedenoiser31.dll")
#loadplugin("vaguedenoiser32.dll")
avisource("intro_test.avi")
vaguedenoiser(threshold=2, method=0)
#vaguedenoiser(threshold=2, method=1)
#vaguedenoiser(threshold=2, method=3)
#vaguedenoiser(threshold=2, method=0, wavelet=1)
#vaguedenoiser(threshold=2, method=0, wavelet=2)
#vaguedenoiser(threshold=2, method=0, wavelet=3)
#vaguedenoiser(threshold=2, method=1, wavelet=1)
#vaguedenoiser(threshold=2, method=1, wavelet=2)
#vaguedenoiser(threshold=2, method=1, wavelet=3)
#vaguedenoiser(threshold=2, method=3, wavelet=1)
#vaguedenoiser(threshold=2, method=3, wavelet=2)
#vaguedenoiser(threshold=2, method=3, wavelet=3)

Raw
Average FPS = 80.99

VagueDenoiser 0.29

Method=0
Average FPS = 26.57

Method=1
Average FPS = 25.83

Method=3
Average FPS = 26.25

VagueDenoiser 0.31

Method=0
Wavelet 1: Average FPS = 24.43
Wavelet 2: Average FPS = 26.19

Method=1
Wavelet 1: Average FPS = 23.52
Wavelet 2: Average FPS = 26.12

Method=3
Wavelet 1: Average FPS = 23.54
Wavelet 2: Average FPS = 26.30

VagueDenoiser 0.32

Method=0
Wavelet 1: Average FPS = 23.46
Wavelet 2: Average FPS = 25.86
Wavelet 3: Average FPS = 23.13

Method=1
Wavelet 1: Average FPS = 23.39
Wavelet 2: Average FPS = 26.05
Wavelet 3: Average FPS = 23.14

Method=3
Wavelet 1: Average FPS = 23.50
Wavelet 2: Average FPS = 25.52
Wavelet 3: Average FPS = 22.75

Hmm... not much of a difference as far as I'm concerned.

[EDIT]: More results posted. It's been a long day. :)

[Fizick]

Thanks for report.
Hmm... not much of a difference as far as I'm concerned.
The most interesting thing is that v.0.29 use 3DNow! (Athlon) optimized thresholding filter function (and probably must be more fast), but v.0.31 use SSE instead of it. I have similar results with my old SDR PC133 DRAM memory.

What were test clip width and height?

And what about speed of new wavelet=1?

[DarkNite]

The test clip was 632x480. As for wavelet=1 test... I have 3 renders running now (that actually have a deadline) so that'll have to wait until I get some free cpu cycles tomorrow.

[Soulhunter]

Gladiator - Chapter No. 19 / XviD v.1.0.1 - Q2 - MPEG - VHQ1



Unprocessed:


Speed: 15.1 fps @ XP 3000+

Average bitrate: 5561.7 kbit/s



V.0.28 Results:


VagueDenoiser(threshold=1, method=0)

Speed: 8.8 fps @ XP 3000+

Average bitrate: 5486.2 kbit/s


VagueDenoiser(threshold=2, method=0)

Speed: 8.7 fps @ XP 3000+

Average bitrate: 5369.6 kbit/s



V.0.30 Results:


Vaguedenoiser(threshold=1, method=0, wavelet=2)

Speed: 9.0 fps @ XP 3000+

Average bitrate: 5486.5 kbit/s


Vaguedenoiser(threshold=2, method=0, wavelet=2)

Speed: 9.0 fps @ XP 3000+

Average bitrate: 5369.6 kbit/s



Now with some more noise...



Unprocessed:


Speed: 13.1 fps @ XP 3000+

Average bitrate: 8196.3 kbit/s



V.0.28 Result:


VagueDenoiser(threshold=3, method=0)

Speed: 8.1 fps @ XP 3000+

Average bitrate: 7499.5 kbit



V.0.30 Result:


Vaguedenoiser(threshold=3, method=0, wavelet=2)

Speed: 8.3 fps @ XP 3000+

Average bitrate: 7504.6 kbit/s



Bye

[Fizick]

Thanks for report. It seems , that 3DNow opimization is not better than SSE.

So, I released new version 0.32.
Addition to notes in my above message:
3DNow mode of hard and Qian thesholding re-enabled after some bugs were fixed by Kurosu and (at last ?) by Fizick.
Small speed increasing mainly due to copy reverse order (but big slowdown for mod64 still exist).
Ported to NASM memcopy assembly function (no more non-NASM assembly).
(SSE float-byte transforms still replaced by C).

Question - is optimization will be better for wavelet transform function ? (now C code for new wavelet 1 and 3).
I wait reports for different wavelets.

[DarkNite]

More results posted. See previous report.

[EDIT]
It's been such a long day I'm even considering doing a mod64 comparison later. :D

[Kurosu]

Originally posted by Fizick
[B]Lefungus, it seems we stil will live with your plugin. :) :(
But sizes must not be mod 64 (i.e. NOT 640)!
As Kurosu noted, there are big slowdown for such sizes. As I found, it is due to 2 COPY functions, mostly following (pitch2 is frame width or height):
_inline void Wavelet::copy(float register*p1, float register*p2, int pitch2, int length)
{ // very-very slow for pitch2 is mod64 (cache ?)
for(int i=length;i!=0;i--)
{
//*p2=p1[i];
*p2=*(p1++);
p2+=pitch2;
}
}

Very simple code (may be too). :rolleyes: Any processor and assembler guru here?

Ah yes, I recall why now. What happens is that all reads on p2 pointer are from different cache line size. ie, the processor caches 63 bytes for nothing. For K6 and P3, the same should happen with mod32 pitches. This is unavoidable with the current code, and that's why very specific rewriting of the code for vertical transform would be needed.

The lifting scheme has this advantage that there are well known in place transforms, which would help quite a bit with this.

3DNow optimized version of WienerChop and AutoThreshold - thanks to Kurosu. :)Did you test them? I didn't, and that's why I didn't talk about them.

3DNow mode of hard and Qian thesholding still have a some bugs and are not used. :( - Hard method produce some vertical lines, and Qian work more similar to hard than Qian.
Ah, I saw in later posts that this was fixed. What had I forgotten?

And that one:
It seems , that 3DNow opimization is not better than SSE.
On my system (Athlon XP 2800+/512 L2 cache and DDR266), I noticed a rather important difference (around 20%); at that time, I really thought the SSE code was a waste of my time considering it's slower and I don't need it.

[MfA]

Yet another paper (http://dali.korea.ac.kr/publication/int_jour/paper/IntJour53.pdf) on how to do the transform better ... this is a variation on the local wavelet transform I think.

Marco

PS. I dont quite see why strides of a multiple of the cacheline size would cause a slow down. Also on my Athlon stride 259 accesses are about as fast as stride 67, but the slow down of stride 256 is 3 times greater than the slowdown of stride 64 ... so 64 isnt the only magic number. If you thought 640 was bad, try 768/1024.

I modified a test program from blitzwave a bit, see if you can make heads or tales of the results ...


#include "stdafx.h"
#include <iostream>
#include <time.h>
int _tmain(int argc, _TCHAR* argv[])
{
using namespace std;
int j = 0;
for (int e=5; e<=11; ++e) {
for (int stride=(1<<e)-3; stride<=(1<<e)+3; ++stride) {
int n = 345;
int *data = new int[n*stride];
double timer = clock();
for (int dummy=0; dummy<100000; ++dummy) {
for (int i=0; i<n; ++i) {
j += data[stride*i];
}
}
cout << "n: " << n << "\tstride: " << stride
<< "\ttime: " << (clock() - timer)/10000 << endl;
delete [] data;
}
cout << endl;
}
cout << j;
return 0;
}

[Fizick]

DarkNite: More results posted. See previous report.
Thanks. We see that not-optimized wavelet 1 is slower (a litle).

Kurosu, now for AthlonXP2400 I found a really big slowdown not for mod64, but only for width=256, 512 and 768 (i.e mod256):
256 - 9 fps
260 - 22 fps
384 - 16 fps
388 - 20 fps
512 - 4 fps
516 - 15 fps
640 - 11 fps
644 - 13 fps
704 - 11 fps
708 - 11 fps
764 - 11 fps
768 - 2.5 fps
(PAL)
Pentium 3 (512 MB cache) have not such big slowdown.

may be it is effect of cache Ll or L2 miss.
But we have not reading with pitch, but writing with pitch in this copy fuction. Reeading by 64 blocks is O.K. But probably writing also use the SAME cache line as reading block, and data will be replaced, and must be read again.
This is unavoidable with the current code
Another copy function have pitch for reading. Probably it can be MMX optimized (write 8 byte without write-cache?)
...this was fixed. What had I forgotten?
Nothing interesting (see source comments). But thanks to your bugs I began to understand a MMX code a little. :D

MfA, thanks for link! Very useful, as always from you.
Seems, it is almost the same method which I have in my head yesterday (but not in code).

For the present, bug-fixed version 0.32.1 is released.
Fixed bug in copy function, introduced in v.0.32.
Re-enabled SSE optimized float-byte and byte-float conversion after fixing some bugs.

[MfA]

The only thing I can think of is that the code might be trashing the TLB.

[Fizick]

I do not understand you. :confused:

[MfA]

Try

http://crd.lbl.gov/~dhbailey/dhbpapers/cache.pdf

and

http://citeseer.ist.psu.edu/cache/papers/cs/13015/http:zSzzSzwww.research.ibm.comzSzpeoplezSzdzSzdfbzSzpaperszSzBacon94Framework.pdf/bacon94compiler.pdf

Cant say I have my head around it entirely, but Id say the speed hit is a combination of both cache and tlb trashing. Because with 16 way associativity and 256 kb cache I dont think you would be hitting just cache sets often enough to explain the slowdown (unless you used a huge vertical resolution). Might be wrong though :) Not that it matters, some padding will fix both.

[Fizick]

I do not know what is "TLB" :confused:,
but I know (but not very sure) that Athlon have only 2-way cache level 1 (and exclusive 16-way cache level 2, yes).

[Fizick]

"Padded" version 0.33 released.
Fixed slowdown for mod64 width by padding (thanks to MfA)
Add partial denoising mode (by blending with source)
Fixed bug with AutoThreshold for 3DNow.
AutoThreshold is now also dependent from "wratio" parameter.
Added messages for Debugview utility.
Change some parameters default values to more optimal (for me?):
thresh=0 (auto), method=3, nsteps=4, wavelet=1, percent=75.

It is all. The bigger improvements are still shelved. :(

[MfA]

TLB == translation lookaside buffer, which maintains the mapping between virtual and physical memory addresses. At any time only part of the mapping is known to the processor, ie. cached ... and this cache can be trashed just as the data cache. With the same type of results.

How did you pad BTW? Personally Id try detecting multiples of 64 and pad with 16 bytes (being aligned on 16 byte boundaries is nice SSE parallelization).

[Fizick]

I use 2 point right pad (i.e. 8 byte float) if current plane width is mod64.
And I use mirror symmetry for this points to prevent border effects.

Id try detecting... BTW, for what (plugin ?)

[Soulhunter]

As promised, some more tests... ;)


VagueDenoiser v.0.33 test No. 1


Source:

Gladiator (PAL/R2) - Chapter 19


Settings:

XviD v.1.0.1 - Fixed Q2 / VHQ1 / MPEG / No extra stuff


My box:

Athlon XP2600 @ XP3000 / 1024MB DDR400 CL2 RAM / WindowsXP


Results:

Mpeg2source("C:\blah.d2v").Trim(1,1500)

Size = 45.35 MB / Speed = 14.9 fps


VagueDenoiser(0.75,3,6,0,false,false,1,true,0.5,75)

Size = 44.73 MB / Speed = 4.3 fps


VagueDenoiser(0.75,3,6,0,false,false,1,false,0.5,75)

Size = 44.07 MB / Speed = 7.1 fps


VagueDenoiser(1,3,6,0,false,false,1,true,0.5,75)

Size = 44.38 MB / Speed = 4.2 fps


VagueDenoiser(1,3,6,0,false,false,1,false,0.5,75)

Size = 43.55 MB / Speed = 7.1 fps


VagueDenoiser(2,3,6,0,false,false,1,true,0.5,75)

Size = 42.68 MB / Speed = 4.3 fps


VagueDenoiser(2,3,6,0,false,false,1,false,0.5,75)

Size = 41.59 MB / Speed = 7.2 fps


VagueDenoiser(0.75,3,6,1,false,false,1,true,0.5,75)

Size = 45.00 MB / Speed = 4.3 fps


VagueDenoiser(0.75,3,6,1,false,false,1,false,0.5,75)

Size = 44.29 MB / Speed = 7.1 fps


VagueDenoiser(1,3,6,2,false,false,1,true,0.75,75)

Size = 43.79 MB / Speed = 4.3 fps


VagueDenoiser(1,3,6,2,false,false,1,false,0.75,75)

Size = 43.45 MB / Speed = 7.1 fps


VagueDenoiser(1.25,3,6,0,false,false,1,true,0.5,75)

Size = 43.96 MB / Speed = 4.3 fps



VagueDenoiser v.0.33 test No. 2


Source:

Matrix Revolutions trailer (E)


Settings:

XviD v.1.0.1 - Fixed Q2 / VHQ1 / MPEG / No extra stuff


My box:

Athlon XP2600 @ XP3000 / 1024MB DDR400 CL2 RAM / WindowsXP


Results:

Mpeg2source("C:\blah2.d2v").trim(1,2500)

Size = 61.95 MB / Speed = 15.2 fps


VagueDenoiser(0.75,3,6,0,false,false,1,true,0.5,75)

Size = 61.04 MB / Speed = 4.3 fps


VagueDenoiser(0.75,3,6,0,false,false,1,false,0.5,75)

Size = 60.31 MB / Speed = 7.2 fps


VagueDenoiser(1,3,6,0,false,false,1,true,0.5,75)

Size = 60.69 MB / Speed = 4.3 fps


VagueDenoiser(1,3,6,0,false,false,1,false,0.5,75)

Size = 59.79 MB / Speed = 7.2 fps


VagueDenoiser(2,3,6,0,false,false,1,true,0.5,75)

Size = 59.18 MB / Speed = 4.3 fps


VagueDenoiser(2,3,6,0,false,false,1,false,0.5,75)

Size = 58.08 MB / Speed = 7.2 fps


VagueDenoiser(0.75,3,6,1,false,false,1,true,0.5,75)

Size = 61.42 MB / Speed = 4.3 fps


VagueDenoiser(0.75,3,6,1,false,false,1,false,0.5,75)

Size = 60.62 MB / Speed = 7.2 fps


VagueDenoiser(1,3,6,2,false,false,1,true,0.75,75)

Size = 60.10 MB / Speed = 4.3 fps


VagueDenoiser(1,3,6,2,false,false,1,false,0.75,75)

Size = 59.72 MB / Speed = 7.2 fps


VagueDenoiser(1.25,3,6,0,false,false,1,true,0.5,75)

Size = 60.31 MB / Speed = 4.3 fps


VagueDenoiser(1.25,3,6,0,false,false,1,true,0.5,100)

Size = 59.30 MB / Speed = 4.3 fps


VagueDenoiser(2,3,6,3,false,false,1,true,0.5,50)

Size = 59.06 MB / Speed = 4.2 fps


VagueDenoiser(1,3,6,2,false,false,1,true,0.5,50)

Size = 60.72 MB / Speed = 4.3 fps

VagueDenoiser(1.25,3,6,0,false,false,1,true,0.5,100)

Size = 43.05 MB / Speed = 4.4 fps


VagueDenoiser(2,3,6,3,false,false,1,true,0.5,50)

Size = 42.68 MB / Speed = 4.3 fps


VagueDenoiser(1,3,6,2,false,false,1,true,0.5,50)

Size = 44.37 MB / Speed = 4.3 fps



Bye

[SILICON]

Originally posted by Fizick
"Padded" version 0.33 released.

Change some parameters default values to more optimal (for me?):
thresh=0 (auto), method=3, nsteps=4, wavelet=1, percent=75.

You have a bug in this change.

If your use VagueDenoiser() the Avisynth give a ERROR: "Invalid Theshold"

[Fizick]

Confirmed. :(
Bug fixed in 0.33.2

Did anybody try Vague on Pentium 3 or 4? (for width=512)

[cheburashka]

@Fizick

I've tried it(0.33.1) for "38 popugaev" on P4 2.4. Result is pretty good and fps ~ 10-11. Clean enough with keeping details.

Default options and width 512.

[Fizick]

cheburashka, thanks for info.

(but i think (but not try), that "38 popugaev" original source is restored and very good without any filter)

To all (who uses YUY2).
YUY2 dont work since v.0.32.1 :(
Bug fixed in 0.33.3 (released today)

[krieger2005]

"38 popugaev" original source is restored (on DVD). But maybe cheburashka have this from TV-Capture or VHS-Capture...

I tried VagueDenoiser and it make the picture clean without bluring... But it can't remove this "blobs" (like in the example from you Fizick where the background is so green).

I tried for this Problem this:

sauber=VagueDenoiser(threshold=3,method=3,nsteps=6,chromaT=0.8, wiener=true, wratio=5, interlaced=false).
\UnDot.
\UnFilter(30,30)

a=ConvertToRGB32._2DClean(0,10,2,2).ConvertToYV12.
\DideeBlur(sauber.Blur(1),25).UnDot()
b=ConvertToRGB32._2DClean(0,10,2,2).ConvertToYV12
m=YV12subtract(a,b).
\YV12LUT(YExpr="x 129 > x 255 < & x 122 < x 0 > & | 255 0 ?")
MaskedMerge(sauber,m,Y=3,V=3,U=3,useMMX=true)
UnDot
UnFilter(40,40)


"sauber" should be clip without this "blobs" (it can be unsharp).
"a" should be tuned (i mean the value "25"), till the Blubs are away from the Picture.
"b" is a picture with Blubs
"m" here you can decide wich parts from "sauber" should be used. 129-255 are the dark blobs, 0-122 are the bright blobs. If the values are about 127/128 the function removes more... try it!

[Fizick]

New version 0.33.4 - August 23, 2004.
Fixed bug with 4 rigthmost mirrorred padded pixels.

[Soulhunter]

Nice... :)

Btw, could you maybe add a link (http://bag.hotmail.ru/vague/vaguedenoiser.dhtml) in your signature ???

Would be easier than searching it every time a new version is out... ;)


Bye

[Leak]

Originally posted by Soulhunter
Btw, could you maybe add a link (http://bag.hotmail.ru/vague/vaguedenoiser.dhtml) in your signature ???

Well, the WWW button under his posts leads to bag.hotmail.ru, where you'll get links to all his filters - isn't that close enough?

np: The Modernist - Prozac Europe (Kangmei)

[Soulhunter]

Originally posted by Leak
Well, the WWW button under his posts leads to bag.hotmail.ru, where you'll get links to all his filters - isn't that close enough?
... :o

[Fizick]

:cool:

[OBcecado]

Hi, sorry for bugging you, but isn't it possible to update the thread's subject ?
It'll be easier to find out when there are updates of this great filter.


All the best.

[Fizick]

But what do you want from update? :)

[celtic_druid]

I guess they just want the current version in the title when it is released.

[Wilbert]

For some reason unknown to me I can't update the title of this thread. Perhaps Sh0dan is so kind to update it each time :)

[Teegedeck]

Doesn't changing the title of the first post in the thread always do the trick?

[Wilbert]

Nope, as you can see I changed it to "VagueDenoiser 0.33: a 2D Wavelet denoiser/smoother", but the thread title remains the same.

[Fizick]

May be we must ask Lefungus? :)

[celtic_druid]

You just select edit thread from the admin tools drop down thingy and change it.

[Wilbert]

I guess one is never too old to learn something. Thanks!

[Lefungus]

Last time I tried to change the topic title for the new Fizick version, It was impossible. I guess once a mod change it, the first poster isn't able to change it anymore

[LigH]

A user not registered here (but in the german board) asked me to post a bug report here:

'interlaced=true' produces an access violation with 'Wiener=true'

Configuration:
Athlon Thunderbird 800 MHz
256 MB RAM
Windows XP Professional (no SPs, but all critical updates)

Tools:
AviSynth 2.55 (final 2004-08-31)
DGIndex 1.0.12 & DGDecode
VagueDenoiser 0.33.4 from http://bag.hotmail.ru/
VirtualDubMod 1.5.10.1 build 2439

Script:
LoadPlugin("DGDecode.dll")
LoadPlugin("VagueDenoiser.dll")
MPEG2Source("HeavyNoise_interlaced_PAL_DVD.d2v",iPP=true)
VagueDenoiser(interlaced=true,Wiener=true)

Procedure:
Open script in VirtualDubMod: "Avisynth read error: Avisynth: caught an access violation at 0x########, attempting to read from 0x########"

[Fizick]

LigH,
confirmed :(
thanks :)
I released v.0.33.5 today.
Fixed bug with AutoThreshold for Interlaced Wiener mode.
I also used newest avisynth.h for compiling,
and made spell check of documentation. Now it has only 2 mistakes :)

P.S. What about update of the thread name? Anybody still can do it ? :D

[LigH]

@ Fizick:

Please thank 'Viperzahn' in your changelog for his original report in the german doom9 / Gleitz board; I just quoted it here.

[Fizick]

Done.

[Viperzahn]

There is another bug in the combo of nsteps, chromaT and interlaced:
MPEG2Source("Interlaced_NTSC_DVD.d2v")
VagueDenoiser(nsteps=6,chromaT=0,interlaced=true)
produces a green touch in the upper left corner.
'nsteps=6,chromaT=0,interlaced=false', 'nsteps=6,interlaced=true', 'nsteps=5,chromaT=0,interlaced=true' works fine.

[Fizick]

Probably it is not bug, but feature :)

Too big nstep for interlaced YV12 chroma plane (height=120).

[Viperzahn]

Ok :cool:,
but any chance to get an warning message instead of an ugly picture?

[Fizick]

Viperzahn,
You may consider Lefungus note in doc and my post above as warning message. :)

But any chance to get "Thanks for your updates" message?
(Lefungus was only who post it).
:rolleyes:

[Viperzahn]

You may consider Lefungus note in doc and my post above as warning message.
I find nothing about this in the manual, and sometimes, you dont see this error on the first preview. It is really hard to get a worse result after a ~20h encoding.



But any chance to get "Thanks for your updates" message?
(Lefungus was only who post it).
I'm sorry - it was very late yesterday :(
Thanks for your work.

[Fizick]

New Version 0.33.6 - October 13, 2004
Nsteps parameter now is auto-limited to max admisible value if input too big or =0 (don't worry, Viperzahn :)

Some improving of exception handling (try-catch-throw).

Try!

[Viperzahn]

Thank you! :)




EDIT
hm, same problem with 'nsteps=0,chromaT=0,interlaced=true' and 'nsteps=6,chromaT=0,interlaced=true' is still selectable and green :(

[Fizick]

It must be selectable. But why green?:confused:

please give more info about CPU, source and script and program.
And how with other source? I have no NSTC.

Anybody else can confirm?

[Viperzahn]

please give more info about CPU, ... script and program.
Configuration:
Athlon Thunderbird 800 MHz
256 MB RAM
Windows XP Professional (no SPs, but all critical updates)

Tools:
AviSynth 2.55 (final 2004-08-31)
DGIndex 1.0.12 & DGDecode
VagueDenoiser 0.33.6 from http://bag.hotmail.ru/
VirtualDubMod 1.5.10.1 build 2439

Script:
LoadPlugin("DGDecode.dll")
LoadPlugin("VagueDenoiser.dll")
MPEG2Source("HeavyNoise_interlaced_NTSC_DVD.d2v",iPP=true)
VagueDenoiser(nsteps=0,chromaT=0,interlaced=true)

Procedure:
Open script in VirtualDubMod: green colouring in the upper left part of the image



please give more info about ... source ...
And how with other source? I have no NSTC.
Check this, please:
LoadPlugin("VagueDenoiser.dll")
BlankClip(length=300,width=720,height=480,pixel_type="YV12",fps=29.970,color_yuv=$0000EB)
VagueDenoiser(nsteps=0,chromaT=0,interlaced=true)
You may test it with nsteps=0\6\5,chromaT=0,interlaced=true\false

720x576 seems to work (in this script) and i tested with two different NTSC DVD sources (MPEG-2 Video MP@ML,YV12,29.970 FPS, 720x480 pixels,interlaced,TFF,pic-structure=frame).

[Fizick]

Viperzahn,

I tried your last test script.
I have got red frame without any green corner.

So i still wait reports from others.
May be old Athlon?
BTW, what YV12 decoder does your use? I use Helix.

However, my red frame is not pure red, but with some dark left upper zone.

I found more sharp spots with "height=512".
So, some bug (or feature) really exist.

May be, some mirror (symmetry) code missed.
I do not know.
May be, i simply decrease the max admissible nsteps in next version.

Thanks for report!

[Viperzahn]

>May be old Athlon?
CPU supports MMX, integer SSE, 3DNow!, 3DNow! 2

>BTW, what YV12 decoder does your use?
XviD Decoder (Koepi's binaries - 1.0.2-29082004 _Final Release_)



I have got red frame without any green corner.
.
However, my red frame is not pure red, but with some dark left upper zone.
Yes, that's what i mean, but in a natural picture is this dark area green.



Please compare this:
A=BlankClip(length=300,width=720,height=480,pixel_type="YV12",color_yuv=$0000EB)
\.VagueDenoiser(nsteps=0,chromaT=0,interlaced=true)
B=BlankClip(length=300,width=720,height=480,pixel_type="YV12",color_yuv=$0000EB)
Subtract(A,B)
and
A=BlankClip(length=300,width=720,height=576,pixel_type="YV12",color_yuv=$0000EB)
\.VagueDenoiser(nsteps=0,chromaT=0,interlaced=true)
B=BlankClip(length=300,width=720,height=576,pixel_type="YV12",color_yuv=$0000EB)
Subtract(A,B)
Something is false with 480 height and nsteps,chroma=0,interlaced=true

[Fizick]

V.0.33.7 - October 17, 2004 (Fizick)
Decreased max admissible value of nsteps to fix some internal bug (or feature)
(thanks to Viperzahn for insistent report :-)

I am really wonder, why about 30 peoples downloaded last bugged version without any test and report (for collection may be):angry:

[Viperzahn]

It seems to work now, in my fast test - Thank you Fizick :)
I make a harder testing in the next days, when i found more time :(


BTW, sorry for my bad english.

[Fizick]

V.0.34 - November 24, 2004
Added auxiliary (some prefiltered) clip for first pass of Wiener mode.

Add Parameter auxClip: auxiliary clip for first pass of Wiener mode (default=none)

Prefiltered source clip, preferably with temporal or spatial-temporal filter (such as DeGrainMedian) for best denoising and artifacts suppression.

Sample usage:

aux=degrainmedian(mode=0)
vaguedenoiser(wiener=true,auxclip=aux)


Now Vaguedenoiser is not pure spatial filter. :)

[DarkFoon]

I've looked high and low, but I can't seem to find anywhere if VagueDenoiser requires an SSE capable CPU.
This means it's probably right under my nose.
so, Does vaguedenoiser work w/o SSE?
this may seem like a stupid question, because all modern CPU's have SSE in some form. But I have a pentium II I fancy as an encoder, so I can use my other computer (a PIII) instead of waiting for it to get done.
I get a green-screen output with vague in my script(on the PII, it works just fine on my PIII), should I take this to mean that it requires SSE?
if it does... i'm screwed.
is there anyway to make a script that has optimizations, but can also work w/o them? I don't know programming, so those may be opposing ideas....


I hope this makes sense, I'm tired and hungry, and frustrated:angry:
thanks for your time/help

[Fizick]

Hmm...
Vaguedenoiser must work without SSE.
It contains pure C (not-optimized part) part.
But it use some MMX part for mem_set,
and use some internal Avisint functions for copy (BITBLT)...

OK. Now i made some fix to mem_set (change prefetcht0 to prefetchw).
Try new version 0.34.1
It must work, but I can not test it.

[DarkFoon]

Thank's for your help.
just FYI, it's a PII MMX using Slot 1 to connect to mobo.
it's a total junker, but since it isn't a PI, i can use it for encoding.

I'll test the new version. I should tell you what I saw when I used the filter: green... The whole frame was 100% green. You know, that YV12 green color.

If it happens with the new code, I'll post a screen shot and my script.

Thanks again.

[DarkFoon]

I've just tested it on my PII, bad news.

When I open up my script in VDubMod instead of getting a video frame, I now get:
"Avisynth Read Error: Avisynth: Illegal instruction at 0x01ef9400"
in the little bottum line where the menu descriptions appear.
This happens for all video frames, and when I go to
File>File Information
Vdub crashes. Somthing like an out-of-bounds access violation, I can't get it to dump the crash info, because of stupid win98SE and the way it crashes...

Now, I shall post my script, maybe that can offer you some insight into this strange problem...

[Begin script post]

loadplugin("C:\plugins\dgdecode.dll")
loadplugin("c:\plugins\MPASource.dll")
loadplugin("C:\plugins\guavacomb.dll")
loadplugin("C:\plugins\peachsmoother.dll")
loadplugin("C:\plugins\decomb521.dll")
loadplugin("C:\plugins\kerneldeint140.dll")
loadplugin("C:\plugins\mipsmooth.dll")
loadplugin("C:\plugins\msmooth.dll")
loadplugin("C:\plugins\vaguedenoiser.dll")
loadplugin("C:\plugins\fluxsmooth.dll")
v=mpeg2source("sad2.d2v",cpu=0,idct=4,ipp=true,moderate_h=20 ,moderate_v=40 ,showq=false,fastmc=false,cpu2=" ")
a=MPASource("sad2.mpa", normalize=false)
AudioDub(V,A)
trim(0,-1)++trim(2918,0)
trim(0, 18002)++trim(23109, 0)
trim(0, 39543)++trim(39844,0)
trim(1,39543)
crop(0,62,0,-66,align=true)
telecide(order=1, guide=1, post=0, back=0, nt=15, gthresh=15, vthresh=25, bthresh=90.0, show=false, hints=true)
decimate(mode=2,threshold=1.0, threshold2=3.0, quality=3, show=false)
msmooth(threshold=3, strength=3, mask=false)
VagueDenoiser(threshold=1.5,method=1,nsteps=6,chromaT=0,interlaced=false,wavelet=1,wiener=false,wratio=0.5,percent=100)
#tweak(sat=1.50, coring=false)
#Levels(0, 0.9, 255,0, 255, coring=false)

[End script post]

That's the whole thing, I hope it is helpful. I can also give you my system stats. (those of the PII, because that's where the problem is)

Pentium II MMX 300MHz (possibly overclocked?)
233 MHz system bus
128 MB of SDRAM (I think it's 133Mhz ram)
Intel 440LX chipset/motherboard
some other crap

Thanks for all your help. And your time. I do hope that this helps pinpoint the problem

[LigH]

If I remember correctly, "everything in green" happened when the 'nsteps' parameter as a power for 2 was bigger than the video size.

But in general: If something strange happens, try to comment out everything which is not really necessary, start at the most simple script (MPEG2Source, Crop/Resize - nothing else). If the video is already green here, you may have encrypted video. If not, then add the VagueDenoiser: If this results in green, then there is a general problem with the VagueDenoiser parameters. If you see some content there, the filter chain causes the problem.

[DarkFoon]

Indeed LigH, I have tried all these that you have mentioned. As a matter of fact, it all works/looks fine, until I un-comment out VagueDenoiser, and then I get green. But I know my parameters are correct, because they work on my other computer. I can run the same script on my PIII (the computer I am using to write this post) and it will work fine. The two systems are similarly configured too. They both use Koepi's Xvid build to decode YV12. If there is a better(faster) YV12 decoder than that, I would love to hear it.
But back to the subject at hand.
What I should have said in my last post, was that the newest version, 34.1, which fizick made to correct my first, "green" problem, now just plain doesn't work (on my PII). I could try 33.7, or 34.0 again with my PII, but I doubt there will be anything different.
*sigh*
Old tech...
a love/hate relationship...

[Fizick]

So, 0x01ef9400 is prefetch address.
OK, i remove it from code.
Try new release of modifyed version 0.34.1 (i forget change version number):)

DarkFoon, please test with simplest script
(only vague).

[Fizick]

DarkFoon wrote:
@Fizick This is completely off topic, but version 0.34.1 of Vaguedenoiser crashes all of my computers: not only my P2, but also my P3. Just thought you'd like to know (especially since I complained about the P2 problem in the first place ) Yeah, yeah... I know there's a thread for that filter, but I'm too lazy to find it (I have this thread bookmarked, however)

With lLatest version fom my site?

Anybody else have a problem?

[Kurosu]

Originally posted by DarkFoon
Thank's for your help.
just FYI, it's a PII MMX using Slot 1 to connect to mobo.
Thanks again.
Strange, it should be rejected, as it doesn't have neither SSE nor 3DNow! support. There's absolutely no way VD will ever run on that CPU, unless someone implements a new wavelet and code an integer-based lifting scheme.
But the problem with the P3 is even more strange. I thought I had taken care of those prefetch/prefetchw/prefetchnt{a,0,1,2}

Btw, some speed could be save by merging thresholding and float2byte conversion (I can't remember if it is possible). I guess the current code is more memory-bound that CPU bound. Anyone to test with different memory timings/frequencies but at the same CPU clock ?

[Fizick]

Kurosu,
You probably forget, that there is "plane C" code branch in Vague, so P2 CPU can use it. :)

I removed prefetch from mem_set in last version.
BTW, there was 2 different versions of 0.34.1.

I really wonder why DarkFoon (or somebody else) did not report so long about this problem. :confused:

My AthlonXP works well.

[DarkFoon]

@Kurosu
There was no problem with the P3, only the P2; sorry for the confusion.
I haven't had the time to re-test the new 0.34.1 version on my P2 (I have been making the P2 a multiboot recently) I will do that shortly.

@Fizick(and all VD developers)
Maybe there should be two versions of VagueDenoiser: a SSE optimized version (for all newer CPUs) and a "legacy" MMX-only optimized version for crappy P2 and lower CPU. Just a crazy thought. (or maybe the filter should figure out which optimizations are present and pick the ones to use? like 3Dnow! on Athlon, SSE2 on P4, SSE on P3, and MMX on P2) <-[to a non-programmer like myself, that sounds difficult to implement, but I have seen it before...]

[Kurosu]

Originally posted by Fizick
Kurosu,
You probably forget, that there is "plane C" code branch in Vague, so P2 CPU can use it. :)
Indeed, I had totally forgotten. So much for my "absolutely". As for mem_set, I doubt it improves much anything.

Originally posted by DarkFoon
Maybe there should be two versions of VagueDenoiser: a SSE optimized version (for all newer CPUs) and a "legacy" MMX-only optimized version for crappy P2 and lower CPU. Just a crazy thought. (or maybe the filter should figure out which optimizations are present and pick the ones to use? like 3Dnow! on Athlon, SSE2 on P4, SSE on P3, and MMX on P2) <-[to a non-programmer like myself, that sounds difficult to implement, but I have seen it before...]
SSE2/SSE/MMX as you see them used are for integers. VD uses floats. So, back to my "code an integer-based lifting scheme."

[DarkFoon]

@Kurosu
I know nothing about optimizations, but I remember reading somewhere something about float vectors or such in SSE. But those are vectors, not plain ol math ^^U, my bad.
What about a GPU version of VD, is that possible? Kind of like the FFT3DGPU. I mean, the reason FFT3D has a GPU bersion is because of parallel floating-point math (or something like that, the programmer said).
Anyways, like I said, I don't know hardly enough about this subject to even comment, but here I am ;)
So, take my words "cum granum salis".

[Fizick]

Maybe there should be two versions of VagueDenoiser: a SSE optimized version (for all newer CPUs) and a "legacy" MMX-only optimized version for crappy P2 and lower CPU. Just a crazy thought. (or maybe the filter should figure out which optimizations are present and pick the ones to use? like 3Dnow! on Athlon, SSE2 on P4, SSE on P3, and MMX on P2) <-[to a non-programmer like myself, that sounds difficult to implement, but I have seen it before...]

Darkfoo,
You may be wondered, but VagueDenoiser HAS such automatic CPU selection :)

So, when you make new test on P2?
Kurosu and I have some other work too.

[DarkFoon]

Fizick asks me
So, when you make new test on P2?
Kurosu and I have some other work too.

And I reply: right now. ;)
Here are the results...

With this script (below) and YV12 source, I get a solid black frame... :(

loadplugin("c:\plugins\vaguedenoiser.dll")

avisource("Ghost in the Shell SAD 3.avi")
VagueDenoiser(threshold=4, method=3, nsteps=8, chromaT=-1, wavelet=3, percent=100)


With this other script (below) and YV12 source, I get a solid green (0,135,0 = RGB) frame... :(

loadplugin("c:\plugins\vaguedenoiser.dll")

avisource("Ghost in the Shell SAD 3.avi")
VagueDenoiser(threshold=4, method=3, nsteps=8, chromaT=0, wavelet=3, percent=100)


Video informations:
720x352
YV12
23.976 FPS Progressive (it has been IVTC'ed)
XviD Codec (you want the version number? ;))

Computer informations:
CPU: Pentium II MMX 300Mhz Slot 1 type
RAM: 128MB PC133
M/B: Some AOpen piece of crap
GFX: 32MB(?) TNT2

I hope this helps....

[Fizick]

I checked a code but did not found any reason.

Kurosu, can we use _mm_alloc (_aligned_malloc) with old P2 CPU?

[somail]

could someone post a link to .29 I have not been able to find a link to a binary all night. (EDIT: Never mind I'm an idiot.)

Also I am currently using .27, and I have an odd issue with the filter. It seems to only be filtering 50% of the picture (the left side). I havn't found anything on this issue, so I am wondering if I have a possible system .dll not installed or a setting wrong. Hopefully going to .29 will help.

thanks

[Kurosu]

Originally posted by Fizick
Kurosu, can we use _mm_alloc (_aligned_malloc) with old P2 CPU?
Yes, this is platform dependent (Windows only), not CPU-dependent. A green buffer is typically all values set to 0, so there's probably one or more missing copy somewhere. Or maybe all values are thresholded to 0.

[Fizick]

I will wait the problem confirmation from somebody with old Pentium2 CPU.

[DarkFoon]

I doubt there will be any confirmation from anybody else with an old Pentium2. I think I am the only person who tries to do video on a Pentium2 in this whole forum. ;)

I wonder if it's just my lame computer...

[Fizick]

Darkfoon,
what speicific Avisynth version do you use?
2.55 ?

[LigH]

I have an AMD Duron 800 (MMX, ISSE, basic 3Dnow!). If that helps, then I may check for issues, too - but I'm unsure which constraints I have to follow, this thread starts to confuse me the more I read it. Would you collect the constraints again, please?

[Fizick]

LigH,
thanks, but Duron is not so old CPU. :)
We need in P2.

[DarkFoon]

I think i am using avisynth 2.53 on the P2.
and 2.55 on my P3.

[paviko]

Hi.
Thank you for great denoiser.
I've got strange behaviour when chromaT parameters is greater or equal to 0 and video is in YV12 mode. Then I see in the upper-left corner small, blue rectangle. It doesn't appear in YUY2 mode. It's not a big deal because I'm mostly using YUY2 mode.
My system: Athlon XP (Barton), Win XP SP2.

Thank you,
Regards

[Fizick]

V.0.34.2 - March 11, 2005 (Fizick)
Fixed small bug (blue dot) for chromaT>=0 in YV12 for Athlon (Thanks to Pavico for report). Added pitch for internal buffers.

[paviko]

Thank you :)
You are the most active plugin developer :cool:

Regards

[Fizick]

I found, that AutoThreshold mode (threshold=0) gives overestimated threshold values, and found a bug in algo.
So, new version: V.0.34.3 - September 11, 2005 (Fizick)
Fixed AutoThreshold algorithm

[Fizick]

Nobody interested? O.K., new version.

V.0.35 - September 17, 2005
Improving soft method=1 - do not filter lowest level anymore, mean picture intensity is not changed now .
Re-enabled method=2 of multilevel subband adaptive thresholding, implemented as NormalShrink method;
Added method=4 uniformly smooth shrinking function;
Replaced partial denoising blend mode by limited shrinking of small coefficients for all thresholding functions;
Changed default percent=85;
Implemented new SSE versions for method=0,3,4 and C versions for rest (3DNow disabled with 4% speed decreasing);
Added 3DNow optimized versions for transform of wavelet 1 and 3 (10% speed).

[DarkFoon]

yay!
thanks Fizick

[Soulhunter]

Yeah, thx... ^^


Bye

[LigH]

3Dnow! speedups and algorithm improvements? http://cosgan.de/images/smilie/liebe/f020.gif

[Fizick]

Pentium4 users, please compare the speed of Brislawn wavelet=2 with version 0.343:
Vaguedenoiser(threshold=2,wavelet=2,percent=100)
I disabled part of SSE code here (it is slower on Athlon) then pure C.

[Fizick]

No any Pentium4 user?

New v.0.35.1 - september 26,2005 - Added YUY2 and RGB format for auxclip

[Kisa_AG]

Hello Fizick!

No any Pentium4 user?

I'm a Pentium 4 user!
Here are informaton about my computer:
Intel Pentium 4 (Willamette), 1.70GHz, MMX, SSE, SSE2, Front Side Bus Frequency 100.2 MHz, Bus Speed 400.9 MHz

I tried both filter versions but didn't find any speed improvements. I used DV source with 53 seconds duration. Here is my AVS:

LoadPlugin("C:\Program Files\Multimedia_Soft\Video\AVS_Plugins\vaguedenoiser0343\VagueDenoiser.dll")
#LoadPlugin("C:\Program Files\Multimedia_Soft\Video\AVS_Plugins\vaguedenoiser0351\VagueDenoiser.dll")

AVISource("D:\Multimedia\Part11.avi")
ConvertToYV12
Vaguedenoiser(threshold=2,wavelet=2,percent=100)

I loadad this AVS into VirtualDub 1.6.10 and saved as AVI with DV compression. Both versions of your filter gave me the same result
3:25 - VirtualDub
2:47 - CPU time in Task Manager.

[Fizick]

Greeting to new Russian doom9 user!
So, SSE optimization of old version was really not effective.
(why you do not overclock your PC ? :)

[Kisa_AG]

Fizick.

Greeting to new Russian doom9 user!

Thanks!

why you do not overclock your PC ?

It's difficult question, I don't know... I have two PCs, P4-1700 and P4-2660, and both work with the original CPU clock. Maybe I prefer stability over speed improvment.

[onesoul]

Does anyone have latest version (v0.35.1) of vaguedenoiser?

[Wilbert]

I attached VagueDenoiser v0.35, and v0.35.1:

[onesoul]

Thank you nonetheless Wilbert :).

[ARDA]

Why do not follow Fizick's links just some posts up

My plugins are at my site http://bag.hotmail.ru and its mirror at http://www.avisynth.org/fizick

Regards ARDA

[DarkFoon]

@ARDA
because now he is charging for his filters, so you cannot get them from those links anymore.

[ARDA]

My apologizes, I stay corrected I didn't check to download before posting

Regards ARDA

[Fizick]

because now he is charging for his filters, so you cannot get them from those links anymore.
It is not quite true.
1. I temporary removed them. And wait some donations. Read more info in my site.
2. Now I got some donations and begin to restore my site (partially).

[royia]

How does it compare to other spatial denoisers?

Is there a 64bit version?

[StainlessS]

Hi Royia

Nobody else seems to wanna answer U so, all I can say is
VagueDenoiser is the ONLY purely spacial denoiser I ever use,
not terribly fast but it is very good at what it does.
Not an in-depth appraisal, maybe there are better ones, but
when I like summick, I stick with it.

[Usedocne]

How does it compare to other spatial denoisers?

Is there a 64bit version?

Spatial denoiser comparison (bit old though):
http://www.aquilinestudios.org/avsfilters/spatial.html

I don't think there's a 64bit version. Maybe someone has compiled one though (for personal use)?

[royia]

I see.
There are Edge Preserving Wavelets which should result in a great Spatial Denoiser.
I'll give this one a try.

If I'm not mistaken the developer is still active in the forum, maybe he might provide a 64bit version.

[Magik Mark]

Can somebody help us compile a 64bit version? It has been awhile

[FranceBB]

@Magik Mark... no luck trying to compile the x86 version either, but I don't get why...

-------------- Build: Win32 Release in VagueDenoiser (compiler: Microsoft Visual C++ 2010)---------------

[ 6.7%] cl.exe /nologo /I"C:\Programmi\Microsoft SDKs\Windows\v7.0A\include" /I"C:\Programmi\Microsoft Visual Studio 10.0\VC\include" /c Brislawn.cpp /FoRelease\Brislawn.o
Brislawn.cpp
Brislawn.cpp(234) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(234) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(234) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(237) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(238) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(239) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(240) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(241) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(242) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(243) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(244) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(245) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(246) : error C2065: 'i' : undeclared identifier
Brislawn.cpp(247) : error C2065: 'i' : undeclared identifier
Process terminated with status 2 (0 minute(s), 2 second(s))
14 error(s), 0 warning(s) (0 minute(s), 2 second(s))



And GCC as well...


-------------- Build: Win32 Release in VagueDenoiser (compiler: GNU GCC Compiler)---------------

[ 6.7%] mingw32-g++.exe -m32 -c "C:\Documents and Settings\l\Desktop\src\Brislawn.cpp" -o Release\Brislawn.o
In file included from C:\Documents and Settings\l\Desktop\src\wavelet.h:21:0,
from C:\Documents and Settings\l\Desktop\src\brislawn.h:4,
from C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:1:
C:\Documents and Settings\l\Desktop\src\avisynth.h: In member function 'BYTE* VideoFrame::GetWritePtr() const':
C:\Documents and Settings\l\Desktop\src\avisynth.h:347:20: error: '_ASSERT' was not declared in this scope
_ASSERT(FALSE);
^
C:\Documents and Settings\l\Desktop\src\avisynth.h: In member function 'BYTE* VideoFrame::GetWritePtr(int) const':
C:\Documents and Settings\l\Desktop\src\avisynth.h:356:22: error: '_ASSERT' was not declared in this scope
_ASSERT(FALSE);
^
C:\Documents and Settings\l\Desktop\src\avisynth.h: At global scope:
C:\Documents and Settings\l\Desktop\src\avisynth.h:540:7: error: conflicting type attributes specified for 'virtual GenericVideoFilter::~GenericVideoFilter()'
class GenericVideoFilter : public IClip {
^
C:\Documents and Settings\l\Desktop\src\avisynth.h:392:21: error: overriding 'virtual IClip::~IClip()'
virtual __stdcall ~IClip() {}
^
In file included from C:\Documents and Settings\l\Desktop\src\brislawn.h:4:0,
from C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:1:
C:\Documents and Settings\l\Desktop\src\wavelet.h:85:41: warning: 'align' attribute directive ignored [-Wattributes]
__declspec(align(16)) transformFuncPtr Transform;
^
C:\Documents and Settings\l\Desktop\src\wavelet.h:86:41: warning: 'align' attribute directive ignored [-Wattributes]
__declspec(align(16)) invertlowFuncPtr InvertLow;
^
C:\Documents and Settings\l\Desktop\src\wavelet.h:87:42: warning: 'align' attribute directive ignored [-Wattributes]
__declspec(align(16)) inverthighFuncPtr InvertHigh;
^
C:\Documents and Settings\l\Desktop\src\wavelet.h:90:2: error: '_inline' does not name a type
_inline void Wavelet::copy(float register*p1, float register*p2, int length)
^
C:\Documents and Settings\l\Desktop\src\wavelet.h:95:2: error: '_inline' does not name a type
_inline void Wavelet::copy(float register*p1, int pitch1, float register*p2, int length)
^
C:\Documents and Settings\l\Desktop\src\wavelet.h:116:2: error: '_inline' does not name a type
_inline void Wavelet::copy(float register*p1, float register*p2, int pitch2, int length)
^
C:\Documents and Settings\l\Desktop\src\wavelet.h: In destructor 'Wavelet::~Wavelet()':
C:\Documents and Settings\l\Desktop\src\wavelet.h:73:20: error: '_mm_free' was not declared in this scope
_mm_free(temp_out);
^
In file included from C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:1:0:
C:\Documents and Settings\l\Desktop\src\brislawn.h: At global scope:
C:\Documents and Settings\l\Desktop\src\brislawn.h:19:2: error: '_inline' does not name a type
_inline void transform2d (float *input, int hsize, int vsize, int nsteps);
^
C:\Documents and Settings\l\Desktop\src\brislawn.h:20:2: error: '_inline' does not name a type
_inline void invert2d (float *input, int hsize, int vsize, int nsteps);
^
C:\Documents and Settings\l\Desktop\src\brislawn.h:21:2: error: '_inline' does not name a type
_inline void invert_step(float register* input, float register* output,
^
C:\Documents and Settings\l\Desktop\src\brislawn.h:23:2: error: '_inline' does not name a type
_inline void transform_step(float register* input, float register* output,
^
C:\Documents and Settings\l\Desktop\src\brislawn.h:25:2: error: '_inline' does not name a type
_inline void symmetric_extension(float *output, int size, int right_ext);
^
C:\Documents and Settings\l\Desktop\src\brislawn.h:26:2: error: '_inline' does not name a type
_inline void antisymmetric_extension(float *output, int size, int right_ext);
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp: In constructor 'Brislawn::Brislawn(int, int, int)':
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:22:63: error: '_mm_malloc' was not declared in this scope
temp_in = (float*) _mm_malloc((32+max(x,y))*sizeof(float),128);
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp: At global scope:
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:29:74: error: no 'void Brislawn::transform2d(float*, int, int, int)' member function declared in class 'Brislawn'
void Brislawn::transform2d(float *input, int hsize, int vsize, int nsteps)
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:66:31: error: no 'void Brislawn::transform_step(float*, float*, int, int)' member function declared in class 'Brislawn'
int size, int lowSize)
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:72:71: error: no 'void Brislawn::invert2d(float*, int, int, int)' member function declared in class 'Brislawn'
void Brislawn::invert2d(float *input, int hsize, int vsize, int nsteps)
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:112:65: error: no 'void Brislawn::invert_step(float*, float*, int)' member function declared in class 'Brislawn'
void Brislawn::invert_step(float* input, float* output, int size)
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:152:75: error: no 'void Brislawn::symmetric_extension(float*, int, int)' member function declared in class 'Brislawn'
void Brislawn::symmetric_extension (float *output, int size, int right_ext)
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:177:79: error: no 'void Brislawn::antisymmetric_extension(float*, int, int)' member function declared in class 'Brislawn'
void Brislawn::antisymmetric_extension (float *output, int size, int right_ext)
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:204:43: warning: 'align' attribute directive ignored [-Wattributes]
__declspec(align(32)) float analysisLow[10] = {
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:207:44: warning: 'align' attribute directive ignored [-Wattributes]
__declspec(align(32)) float analysisHigh[10] = {
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:210:44: warning: 'align' attribute directive ignored [-Wattributes]
__declspec(align(32)) float synthesisLow[10] = {
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:213:45: warning: 'align' attribute directive ignored [-Wattributes]
__declspec(align(32)) float synthesisHigh[10] = {
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp: In function 'void transform_c(float*, float*, unsigned int)':
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:234:7: error: name lookup of 'i' changed for ISO 'for' scoping [-fpermissive]
for (i=10;i<10+lowSize;i++)
^
C:\Documents and Settings\l\Desktop\src\Brislawn.cpp:234:7: note: (if you use '-fpermissive' G++ will accept your code)
Process terminated with status 1 (0 minute(s), 3 second(s))
22 error(s), 7 warning(s) (0 minute(s), 3 second(s))

If I use -fpermissive it says "use obsolete binding at "i"", which means that it only exits "inside" something and something else is trying to call it from "outside" and fails to compile as well.
This is kinda bad as I had in mind to use the new avisynth.h and compile it with new assembly optimizations... :(

[Groucho2004]

Can somebody help us compile a 64bit version? It has been awhile
Try this one (https://www.dropbox.com/sh/6kb3723po5oqd4b/AADbP8gIJ3YrHVoLU3joqJDma?dl=0).

[Groucho2004]

no luck trying to compile the x86 version either, but I don't get why...
Your C2065 errors are caused by scope problems. Either fix the the problems in the source file or use the "/Zc:forScope-" switch. This is a very common issue with "older" code since compilers like VC6 weren't very strict about scope rules.

If you want to use GCC for Avisynth plugins you'll have to re-write the code using the Avisynth C-API first.

I suggest you learn at least some basics about programming and compilers before attempting to port 32 bit code to 64 bit.

[Magik Mark]

Try this one (https://www.dropbox.com/s/ifbkorpaequejim/VagueDenoiser0351_64.7z?dl=0).



Thanks a lot! so far so good. Quality so far is similar to knlmeans without the strain to the gpu


Sent from my iPhone using Tapatalk

[feisty2]

They are not even on the same level of quality, "similar"?

[real.finder]

Try this one (https://www.dropbox.com/s/ifbkorpaequejim/VagueDenoiser0351_64.7z?dl=0).

the link is dead

[Groucho2004]

the link is dead
Sorry, moved here (https://www.dropbox.com/sh/6kb3723po5oqd4b/AADbP8gIJ3YrHVoLU3joqJDma?dl=0).

[romangal]

Sorry but just had a test of Neo_VagueDenoiser. Ive discovered it doesnt work correctly. First: If i add BlankClip in avisynth neo_VD doesnt work with this error message: "One Clip has audio and the other doesnt" Other filters work ok the same settings. Second: Audio is missing if i use Neo_VD. I dont know it is strange but the audio is disappearing.

[StainlessS]

Well I dont know what you are doing (And I think the Neo_vagueDenoiser might be in devs forum), but maybe something like this


C = Avisource("D:\...\...\my_file.avi") # whatever

Blank = C.BlankClip() # Use C clip as template where blankclip args are omitted [ie Blank same type audio as C clip, but silent audio]

#do whatever you are doing with both identical properties clips.


EDIT: If you still have problems then you must post your script. [guessin' is fun but takes a helluva lot longer to solve]

EDIT: [Neo] VagueDenoiser:- http://forum.doom9.org/showthread.php?t=181310

[romangal]

Well I dont know what you are doing (And I think the Neo_vagueDenoiser might be in devs forum), but maybe something like this[/url]

I have video VHS capture standard 720[576. Interlaced.
I think i understand what is happening. My script is Deinterlace - Neo_VD - Trim(... ...)++BlankClip()++Trim(...) And right after an error appears trying to open script: "One clip has audio other doesnt."
But without BlankClip script is opening ok, but with no audio. I suppose, thats why this error occured: Trim - has no audio but BlankClip has audio(channels and audio_rate)
Anyway, The question is why after neo_vd enabled the clip opened ok but with no audio. Other filters(ttempsmooth, fluxsmooth, _2dCleanYuy2 work with no problems):confused:

[MeteorRain]

There's a dual synth platform issue underneath the code that would kill the audio during processing. I shall release a quick fix to that.

[real.finder]

There's a dual synth platform issue underneath the code that would kill the audio during processing. I shall release a quick fix to that.

what about this? https://forum.doom9.org/showthread.php?p=1910272#post1910272

[StainlessS]

Deinterlace - Neo_VD - Trim(... ...)++BlankClip()++Trim(...)
While above gives a couple of clues, it is hardly a script.


Deinterlace
Neo_VD
Blank = Last.BlankClip(Length=SomeNumber) # using last clip as template for properties, ie audio
Trim(... ...) ++ Blank ++ Trim(...)

Using your 'clues', new clip Blank adopts same audio properties as the then current Last clip. [Otherwise Blank would recieve default audio properties, probably not same as required].

anyways, according to MeteorRain and real.Finder, you have other non self inflicted problems, when they are sorted out, then you need use BlankClip as above.

[MeteorRain]

what about this? https://forum.doom9.org/showthread.php?p=1910272#post1910272

Not sure which route I should go. Truncating gives correct but inaccurate results, while rounding gives "incorrect" but accurate results.

[real.finder]

Not sure which route I should go. Truncating gives correct but inaccurate results, while rounding gives "incorrect" but accurate results.

maybe adding new parameter for that, and make it do Truncating by default (user can change it to rounding it he like it)

[MeteorRain]

maybe adding new parameter for that, and make it do Truncating by default (user can change it to rounding it he like it)

Today I examined your code again and I noticed the problem. The problem is you did a convertbits(8) which I think is the cause of the problem. If you rounded it down to 8, the difference would have been much smaller.

In other words, because you truncated LSB, any fractional part between 0.0 and 1.0 (out of 8-bit) was wiped down to 0, while they were correctly rounded down in a 8-bit VD.

[real.finder]

Today I examined your code again and I noticed the problem. The problem is you did a convertbits(8) which I think is the cause of the problem. If you rounded it down to 8, the difference would have been much smaller.

In other words, because you truncated LSB, any fractional part between 0.0 and 1.0 (out of 8-bit) was wiped down to 0, while they were correctly rounded down in a 8-bit VD.

ok, so I did this

function luma_histogram(clip c)
{
VersionNumber() < 2.60 ? c. mt_lut (expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128) : c.mt_lut(expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128, scale_inputs="allf")
}
ColorBars
converttoyv12
AddGrainC
Interleave(neo_vd.convertbits(16).luma_histogram(),convertbits(16).neo_vd().luma_histogram())

it's still big! edit: but it's better than before!

[real.finder]

even with this

function luma_histogram(clip c)
{
VersionNumber() < 2.60 ? c. mt_lut (expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128) : c.mt_lut(expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128, scale_inputs="allf")
}
ColorBars
converttoyv12
AddGrainC
Interleave(neo_vd.convertbits(16).luma_histogram(),convertbits(16).neo_vd().luma_histogram())
ConvertToStacked()
DitherPost(mode=-1) #-1 mean round down to 8bit

but this did it!

function luma_histogram(clip c)
{
VersionNumber() < 2.60 ? c. mt_lut (expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128) : c.mt_lut(expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128, scale_inputs="allf")
}
ColorBars
converttoyv12
AddGrainC
Interleave(neo_vd.luma_histogram(),convertbits(16).neo_vd().ConvertToStacked().DitherPost(mode=-1).luma_histogram())


so it's bug in convertbits(8)? convertbits(8,dither=1) seems ok, so I think -1 (default) is not ok, pinterf :)

edit: maybe make -1 like DitherPost one (round down to 8bit) and new -2 like how -1 in convertbits do now
edit2: -1 in convertbits now is like ConvertToStacked().Dither_get_msb() which is bad! very bad

[MeteorRain]

> neo_vd().convertbits(16)
and
> convertbits(16).neo_vd()
are different things. 8-bit VD has much less accuracy than 16-bit VD so the difference will still be big.

> neo_vd()
and
> convertbits(16).neo_vd()."round-down-to-8"
will give similar result.

ConvertBits, according to the manual (wiki), is to do bit-shifting by default. It's only meant to be used when lsb IS 0. So I think you simply used the wrong function, or used the function in the wrong way. And I don't think it makes sense to change the default behavior of convertbits.

[real.finder]

with convertbits(8,fulls=true) it kinda similar in case of noise but there are big difference

[romangal]

Using your 'clues', new clip Blank adopts same audio properties as the then current Last clip. [Otherwise Blank would recieve default audio properties, probably not same as required].

anyways, according to MeteorRain and real.Finder, you have other non self inflicted problems, when they are sorted out, then you need use BlankClip as above.
Ok, I understand. Thanks! :thanks:

[MeteorRain]

I have uploaded r2 which should fix the audio issue. Please try it, and let me know any issues.

[romangal]

I have uploaded r2 which should fix the audio issue. Please try it, and let me know any issues.
I have tried. The sound is playing ok! There is no problem now. But i have to notice that it is quite unstable now. I dont know what exactly happens, but sometimes(not always), when i open script with mediaplayerclassic or virtualdub (appcrash) or avspmod, neo_vd is hanging all of them. I use neo_vd msvc x86.
Thank you - Works nice mostly and fast that is incredible for such a strong and difficult filter as Vague Denoiser.:thanks:

[MeteorRain]

That doesn't sound good. Do you have a minimal script that triggers the issue? Or does it happen regardless?

[romangal]

That doesn't sound good. Do you have a minimal script that triggers the issue? Or does it happen regardless?

This is a CrashInfo.txt. It occurs when i twice open the same script with F2 -reopen video(the first one is ok).
The name of module with mistake is ntdll.dll But if i disable NEO_VD no error is appearing.