MC noise removal script

[Spuds]

Well in an effort to first learn more about avisynth scripting capabilities and secondly to learn more about what you can do with mvtools I created a MC noise reduction script. Nothing really new here but I figured I'd post it since others may find it useful.

To use it you import it just like any other script and call it with its various parameters. To make it easy I have presets which set lots of things so you don't have to. You can use a preset plus override part of it by specifying the individual parameter as well.

Those more familiar with scripting and mvtools and some of other functions that I used, please let me know what mistakes I'm making since I'm trying to learn.

The basic calling syntax is: mc_spuds(frames=(1-4),strength=(1-6))
The higher the strength and frames the stronger the denoising effect. As you increase strength you change block size and overlap in mvanalyse and fft3dfilter, plus turn on pre and post processing. The settings are in the script.

A good starting place for a noisy capture is mc_spuds(frames=2,strength=3or4or5). You have to be insane to use frames=4,strength=6 but its there if you have the time. Use debug=1-7 to see some of the settings and masks being used. I use mc_spuds(frames=2,strength=3) on some pretty bad vhs clips with very good results.

The full syntax is a bit more complex :)

mc+spuds(clip clp, int "frames", int "strength", int "blocksize", int "overlap", int "thsad", int "dct", int "ml", bool "chro", bool "postprocess", bool "preprocess", bool "aggressive",int "debug",bool "edgeclean")

What the script does:
- optional pre process for motion vector detection (supersample and denoise for search)
- Gets the motion vectors
- For frames = 1 or 2 use the motion vectors to mvdegrain, for frames = 3 to 4 use the vectors to mvflow.
- Optional for frames 3-4 is to first mvdegrain as if frames = 1-2 then do an mvflow.
- create a motion mask for the new degrained frame
- apply removegrain to the motion mask
- apply limitedsharpenfaster to the inverse (static) mask
- compute an edgemask and dering
- optional post process

The pre process does some simple cleaning and up sizing of the clip to help with the motion vector calculation. The post process is a bit of a jumble of remove dirt and dust scripts which needs improvement and/or simplification but seems to work. With post I was really trying to limit what changed since at that point the clip been worked on quite a bit.

There are also various debug views (debug=1-5) I like 4 and 5 the most.

Script will be in the next post, give it a try, tell me whats not working and what would work better.

[Spuds]

Old script removed, get the latest here (http://www.rcboataholic.com/MC_Spuds.zip)

[Chipstra]

*Spuds:
- mt_masktools ------>> version ??
- removegrain ------>> version ??
- mvtools ------>> version ??

and where can I find the correct version?

error: mvanalyse does not have the named argument "sharp" - line 116

[Spuds]

I'll update the header in the script in a bit, but for now here is what you need to have installed to have this run.

required filters:

- mt_masktools -- version: 2.0.32 or later
code: http://manao4.free.fr/masktools-v2.0a32.zip
discussion: http://forum.doom9.org/showthread.php?t=98985

- removegrain -- version: 0.9 or later
code: http://www.removegrain.de.tf/
discussion: http://forum.doom9.org/showthread.php?s=&threadid=79751

- removedirt -- version 0.9 or later
code: http://www.removedirt.de.tf/
discussion: http://forum.doom9.org/showthread.php?s=&threadid=70856

- mvtools -- version: 1.8.4 or later
code: http://avisynth.org.ru/fizick.html
discussion: http://forum.doom9.org/showthread.php?t=84770

- limitedsharpenfaster -- in mt_masktools

- ff3dfilter -- version 2.1.1 or later
code: http://avisynth.org.ru/fizick.html
discussion: http://forum.doom9.org/showthread.php?t=85790

- ttempsmooth -- version: 0.8.4 or later
code: http://bengal.missouri.edu/~kes25c/
discussion: http://forum.doom9.org/showthread.php?t=77856

- deblock -- version 1.2 or later
code http://avisynth.org.ru/fizick.html
discussion: http://forum.doom9.org/showthread.php?t=110352

[Didée]

This definetly qualifies for the "complex script" category, congratulations. :)

Small bug found in line 78:
# make a mask based on the new motion vectors
idx = idx + 1 should be: global idx = idx + 1


Apart of that, I looked close if there are any "idx" mistakes in the script - could happen easily, with the multitude of different options and possible combinations. For a minute, I thought you trapped in with the "preprocess" thingy, then I saw you escaped with "supersample". Cheeese.

AFAICS, all idx usage seem to be correct.


But one last thing:
global idx = rand()Funny idea, but not really good, IMO. Do not randomize that. Even if it's unlikely, it might give idx collisions if someone combines your script with other MVTools-based functions. (Often a silent(!) collision without error message, only a sub-optimal endresult, that is.)

[EuropeanMan]

^ any ideas on the speeds for this?

i'll assume this simplifies things? -scriptwise...for example:

setmemmax...
dgdecode...
colormatrix...
deinterlacing...
cropping...
resizing...
mc_spuds(....)

that'll do the denoising plus sharpening? sort of like integrated imaging p?

[Didée]

Bug for frames=3|4:

To my understanding, the implementation for frames=3 and 4 is flawed. In a nutshell, it does not do temporal processing over +/- 3|4 frames, but only over +/-1 (when frames=3) or over +/-2 (when frames=4). Also, maskmotion/maskstatic cannot yield to the wished result in this case.

IF frames > 2:

1) clp2 is firstly created as a long interleaved sequence of MVFlow compensations. (For frames=1|2, at this stage MVDegrain1|2 is applied. For frames=3|4, it is not. Only the interleave is done.)

2) Next, f34_phase2 is called on the long interleaved clip. Here, new vectors are calculated on the long interleaved clip. This means that the already-compensated (by MVFlow) wing frames are searched once more against the original center frame. Then, MVDegrain1|2 is applied to the long interleaved clip.

3) Now, a new vector search is done to build maskmotion. Since the working clip is still the long interleaved clip (with already compensated, and now even denoised wing frames), there should be virtually no motion to be detected around the center frame: maskmotion becomes useless.

4) Lastly, the center frame is Select()'ed out of the long interleaved clip again.


Suggestions: (for frames=3|4)

- the steps 3) and 4) are swapped. You first have to Select() the center out of the interleave, and then build maskmotion.

- Step 1 would have to be split in step 1a and step 1b: First do a MVDegrain over 2 frames, then call flow() on that degrained clip, then MVDegrain once more.
(Or something similar ... doing such "long distance" processing by means of MVDegrain is tricky.)


However, the denoising power of this function is really good, already with frames=2. Good work!



^ any ideas on the speeds for this?
Pretty slow. That's impossible to avoid when doing advanced MVTools stuff. If you're interested in speed, rather stay with more simple filters.

[Fizick]

about frames=3.
It is quite possible that you will see MVdegrain3 soon...
;)

[Didée]

OT - if you're at it, maybe consider to give MVDegrain two thresholds: thSAD_low and thSAD_high. thSAD_high the same as thSAD is now, and thSAD_low defining a lower thresh, allowing for full averaging strength if block SAD is below that. (I.e. the way like ttempsmooth is working.)
Just an idea, if it's not too difficult to do.

[Spuds]

This definetly qualifies for the "complex script" category, congratulations :) I was quite impressed with what you could do with the scripting language in avisynth, pretty powerful. I did a couple of things that were not good "practice" but was trying to test out what I could do.

Small bug found in line 78: ah yes, thanks for catching that, i'll make the update. I'll also change the idx=rand initialization and use some number that should be safe like 25. That way its at least deterministic.

On your analysis of the frames 3-4, I think I understand what you are saying. I was looking for a way to integrate mvdegrain with mvflow and went a little off track. Its working but I'm not getting all of the benefit out of the cpu time ... let me work with your suggestions and see what I can come up with. Greatly appreciate the analysis.


However, the denoising power of this function is really good, already with frames=2. Good work! :thanks:

[Spuds]

I've made some changes over the last couple of days, v0.2b has been posted in post number 2.

The changes are:

a couple of bug fixes with global idx as discussed above.
change with idx in the main function to increment in case mc_spuds was called more than once in the same script to ensure no collisions.
changed the pre filter noise function from remove grain to fft3dfilter to allow the noise vector prefilter to respect the chroma search flag.
changed the blocksize of smoothed mask noise filter (increased) so it was not as aggressive. Makes increasing of strength more linear (less of a jump between 3 and 4).
changed frames = 3 or 4 function to first do mvdegrain then mvflow and hold that flowed clip through mask and post functions.
fixed dct not being used on user input, only default was being used. Improved default selection values.

[TheRyuu]

I get an error when trying to use strength=4-6 (4 through 6).

I forgot the exact wording.

[Spuds]

Thats not much of a hint to go work on :)

Are you saying all other modes worked or those are the only ones you tested and you received an error?

The most obvious error that could get thrown is if the video is not in yv12 colorspace or if you are missing one of the required plugins. If you could post the error I can help a lot more.

For a vhs clip you should have something like this:

load_stdcall_plugin(plugin + "yadif.dll")
Import("mc_spuds.avs") # MC noise removal
AVISource("somevideo.avi")
ConvertToYV12(interlaced=true)
AssumebFF()
yadif(mode=1,order=-1)
mc_spuds(frames=2,strength=4,debug=0)
AssumebFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

[Spuds]

I updated post number 2 with rev 0.3 of the script. I had to change the code a bit to get proper results out of frames=3 or 4. Didee first pointed out the error in my logic and it took me some time to wrap my head around the basic issue.

Anyway I had to add an option to the frames =3 or =4 call and its called aggressive. MC_spuds(frames=3,strength=2,aggressive=true)

With aggressive off the script will first mvflow a clip, then apply fft3d and temporalsoften to the now long flowed clip, then pull out the denoised frame and use that frame back in the original clip sequence to generate the masks.

If you turn aggressive on it will first get motion vectors and process the clip with mvdegrain2, then get new motion vectors and do a mvflow on the mv denoised frame. That will then be temporalsoftened and then the center frame pulled back out and used in the original sequence and mv masks generated.

With aggressive on you can use a low strength such as 2 and get quite good denoising. I'll post some example clips with various calls in a bit.

[Spuds]

Some examples of what the script can do:

Original image vs mc_spuds(frames=1,strength=4)
http://img138.imageshack.us/img138/8395/frame1eq7.th.jpg (http://img138.imageshack.us/my.php?image=frame1eq7.jpg)

Original image vs mc_spuds(frames=2,strength=4)
http://img138.imageshack.us/img138/1526/frame2ue3.th.jpg (http://img138.imageshack.us/my.php?image=frame2ue3.jpg)

Original image vs mc_spuds(frames=3,strength=4)
http://img138.imageshack.us/img138/421/frames3vw6.th.jpg (http://img138.imageshack.us/my.php?image=frames3vw6.jpg)

Original image vs mc_spuds(frames=4,strength=4)
http://img138.imageshack.us/img138/9053/frames4cu2.th.jpg (http://img138.imageshack.us/my.php?image=frames4cu2.jpg)

Not 'clean' enough, then try aggressive=true with frames= 3 or 4

Original image vs mc_spuds(frames=3,strength=3,aggressive=true)
http://img138.imageshack.us/img138/3827/frames3aso2.th.jpg (http://img138.imageshack.us/my.php?image=frames3aso2.jpg)

insane processing with full frames and strength and aggressive.
Original image vs mc_spuds(frames=4,strength=6,aggressive=true)
http://img138.imageshack.us/img138/1723/frames46aaq7.th.jpg (http://img138.imageshack.us/my.php?image=frames46aaq7.jpg)

Example overrides, normally post process is set to false with strength=3, override like this

original vs mc_spuds(frames=3,strength=3,aggressive=true,postprocess=true)
http://img138.imageshack.us/img138/1019/frames33apptgw4.th.jpg (http://img138.imageshack.us/my.php?image=frames33apptgw4.jpg)

Or like this
original vs mc_spuds(frames=2,strength=3,postprocess=true)
http://img138.imageshack.us/img138/3631/frames23pptnc7.th.jpg (http://img138.imageshack.us/my.php?image=frames23pptnc7.jpg)

[Adub]

I have to say, I am impressed. That is a really crappy source, not just because of noise, but it looks like aliasing has some affect. Take a look at the white horse ring poles.

[foxyshadis]

Spuds, this is a cool filter =D But I was wondering, is there a way to work with the filter without the global blankclip? Because that seems to disable setmtmode when mc_spuds is used as an avsi. (Loaded before the script.)

[tsp]

foxyshadis: try moving it inside FUNCTION MC_Spuds.
As it is global it should work inside the other functions but are only evaluated when MC_spuds is actually used

[Spuds]

@foxyshadis: I updated the function to rev 0.3b, its in the second post of this thread. I removed the global nullclp and now set nullclp in the first function and pass it as need to the other sub functions. Let me know if this solves the issue you were having using it as an avsi and setmt.

Granted doing it this way there is a tiny overhead of setting the nullclp on each call to mc_spuds vs just once with global, but I don't think that will be noticed given all the calls to mvtools :rolleyes:

@Merlin7777 Thanks. Hard to believe, or maybe not, but that was a commercially produced video. Goes to show what low light, poor optics and wrong film choice can do for you.

[salehin]

Thanks a lot for your hard work and for sharing it with us :)

Seeing those screens, I'm motivated to try mc_spuds.avsi with some really old films (now on DVD) i got :)

[Alain2]

Result on your screenshots seems good, I will try your function when I've got the occasion :)

btw to improve the readability of the presets in your code, maybe you could do something like in the hybridFuPP (http://forum.doom9.org/showthread.php?t=92061&highlight=hybridfupp) script ?
# HybridFuPP 0.97a by FuPP

Function HybridFuPP(clip input, int "width", int "height", bool "Fast", bool "Dering", bool "Deblock", int "MP_Mode",
\ int "EM_Mode", int "DB_Q", int "DB_Off_a" , int "DB_Off_b", int "M_Thr", int "M_SCD", int "E_Thr",
\ int "D_Thr", int "B_Thr", int "Resizer", int "S_Str", int "T_Str", int "S_Radius", float "S_Dist",
\ int "C_Str", int "M_Str", int "E_Str_X", int "E_Str_Y", int "E_Str_B", int "LP_Str", int "DR_Str" ,
\ int "DR_Radius", string "show", string "N1", string "N2", string "M", string "E1", string "E2" ,
\ string "LP", string "DR", string "Preset", bool "Skin", int "Sp_Str", bool "ARAdjust", int "Skin_Type")
{
width = Default(width , width(input))
height = Default(height, height(input))
Preset = Default(Preset, "high")
Show = Default(Show , "")

preset_num = -1

preset_num = Preset == "medium" ? 0 : preset_num
preset_num = Preset == "high" ? 1 : preset_num
preset_num = Preset == "very high" ? 2 : preset_num
preset_num = Preset == "low" ? 3 : preset_num
preset_num = Preset == "very low" ? 4 : preset_num
preset_num = Preset == "anime1" ? 5 : preset_num
preset_num = Preset == "anime2" ? 6 : preset_num
preset_num = Preset == "HybridQ" ? 7 : preset_num

Assert(preset_num >=0 ? true : false, chr(10) + "This preset does not exist !" + chr(10))

_Resizer = Select(preset_num, 3 , 4 , 5 , 2 , 0 , 2 , 0 , 3 )
_Fast = Select(preset_num, true , true , true , true , true , true , true , true )
_S_Str = Select(preset_num, 10 , 5 , 7 , 12 , 12 , 7 , 20 , 12 )
_S_Radius = Select(preset_num, 1 , 1 , 1 , 1 , 1 , 1 , 2 , 1 )
_S_Dist = Select(preset_num, 0.1 , 0.1 , 0.1 , 0.1 , 0.1 , 0.1 , 0.1 , 0.1 )
_T_Str = Select(preset_num, 3 , 3 , 2 , 5 , 5 , 6 , 15 , 5 )
_C_Str = Select(preset_num, 14 , 14 , 14 , 14 , 14 , 15 , 15 , 14 )
_MP_Mode = Select(preset_num, 2 , 2 , 2 , 2 , 2 , 2 , 2 , 2 )
_M_Thr = Select(preset_num, 10 , 10 , 10 , 10 , 10 , 10 , 10 , 10 )
_M_SCD = Select(preset_num, 260 , 260 , 260 , 260 , 260 , 260 , 260 , 260 )
_M_Str = Select(preset_num, 120 , 120 , 80 , 120 , 120 , 0 , 60 , 120 )
_EM_Mode = Select(preset_num, 1 , 1 , 1 , 1 , 1 , 3 , 3 , 3 )
_E_Thr = Select(preset_num, 12 , 10 , 10 , 14 , 14 , 8 , 6 , 14 )
_E_Str_X = Select(preset_num, 0 , 0 , 0 , 0 , 0 , 46 , 46 , 1 )
_E_Str_Y = Select(preset_num, 0 , 0 , 0 , 0 , 0 , 46 , 46 , 0 )
_E_Str_B = Select(preset_num, 0 , 0 , 0 , 0 , 0 , -2 , -2 , 0 )
_Dering = Select(preset_num, false, false, false, false, false, true , false, false)
_DR_Radius = Select(preset_num, 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 )
_DR_Str = Select(preset_num, 30 , 30 , 30 , 30 , 30 , 60 , 60 , 30 )
_D_Thr = Select(preset_num, 35 , 32 , 31 , 37 , 39 , 30 , 30 , 30 )
_B_Thr = Select(preset_num, 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 )
_LP_Str = Select(preset_num, 30 , 30 , 30 , 30 , 30 , 40 , 40 , 30 )
_Deblock = Select(preset_num, false, false, false, false, false, false, false, false)
_DB_Q = Select(preset_num, 25 , 25 , 25 , 25 , 25 , 25 , 25 , 25 )
_DB_Off_a = Select(preset_num, 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 )
_DB_Off_b = Select(preset_num, 0 , 0 , 0 , 0 , 0 , 0 , 0 , 0 )
_Skin = Select(preset_num, false, false, false, false, false, false, false, true )
_SP_str = Select(preset_num, 0 , 0 , 0 , 0 , 0 , 0 , 0 , 2 )
_ARAdjust = Select(preset_num, false, false, false, false, false, false, false, true )
_Skin_Type = Select(preset_num, 1 , 1 , 1 , 1 , 1 , 1 , 1 , 1 )


Resizer = Default(Resizer, _Resizer)
Fast = Default(Fast, _Fast)

S_Str = Default(S_Str, _S_Str)
S_Radius = Default(S_Radius, _S_Radius)
S_Dist = Default(S_Dist, _S_Dist)
T_Str = Default(T_Str, _T_Str)
C_Str = Default(C_Str, _C_Str)

MP_Mode = Default(MP_Mode, _MP_Mode)
M_Thr = Default(M_Thr, _M_Thr)
M_SCD = Default(M_SCD, _M_SCD)
M_Str = Default(M_Str, _M_Str)

D_Thr = Default(D_Thr, _D_Thr)
B_Thr = Default(B_Thr, _B_Thr)
LP_Str = Default(LP_Str, _LP_Str)

[....]

[Spuds]

@Alain2 .. Thank you for the pointer to that script layout, very cool. I'll see what improvements I can make to the readability of the presets section.

I can't do exactly what he did since I'm allowing you to override a single part of a preset which allows you to take most of preset has and just tweak it vs start from ground zero and specify everything should you just want one change, like thsad for example.

Anyway it gives me some stuff to look into!

[Alain2]

Actually in fupp layout it is the same, you use
Preset = Default(Preset, "high")
to define a default preset, and this preset then select the value for each of the parameters, for instance the resizer:
_Resizer = Select(preset_num, 3 , 4 , 5 , 2 , 0 , 2 , 0 , 3 )
(note that it starts with _, so it's a different variable than the parameter Resizer from the function header)

And all these parameters are still overridable, as define after with the Default function, for instance for the resizer (_Resizer from preset is overriden by Resizer if it exists/is defined when calling the function)
Resizer = Default(Resizer, _Resizer)

[Spuds]

Doh! missed the _ on the variable :o Perhaps the occasional use of a comment would have helped me :D

Its certainly one approach to the problem but I think I'm going to leave this in the if-then-else format that I currently have. I'm going to go back and do better alignment of the lines to improve readability though.

I'm also intrigued with the string definitions for the functions in one place, makes for easy function changes, may take a shot at that as well.

Again thanks for the pointer, good stuff!

[Spuds]

Rev 0.4 posted.

I made some changes mostly aimed at speed. Even for a motion compensated script this was starting to crawl !

I did some benchmarking of functions for speed (function vs function and function vs parameters) to arrive at some new items in the script.

The major changes are:
1) changed resizer for the upsampled clip, used to help search for motion vectors. Was LanczosResize now BicubicResize, small speed increase
2) changed noise filter for pre-filtered clip, used to help search for motion vectors. Was fft3dfilter now removegrain + temporalrepair. Large speed increase. Made add config option to choose in the future if there is interest.
3) removed last call to ttempsmoothf, seemed redundant and that is a very slow function. Good speed increase.
4) Improved code readability
5) Improved debug views
6) Added additional lite edge cleaning via masking
7) Changed some of the block and overlap values in the presets to help with speed

Overall the speed increased 2-3X from where it was, still slow but now comparable to what you expect from a motion compensated noise script.

Let me know if you find any errors in the script or my logic.

Thanks

[Zanejin]

Be sure to add that Unfilter is a necessary filter and that the latest package of RemoveGrain (http://videoprocessing.11.forumer.com/viewtopic.php?p=593#593) should not be used with this script (except for SSETools, which is not in the pre-release (http://videoprocessing.11.forumer.com/viewtopic.php?t=9) package). Also, line 82 calls the function "inpand"; should it instead be "mt_inpand"?

[Spuds]

@Zanejin -- good catch on the mt_inpand(), thanks for that! ... I often find myself reading both the V1 anv V2 documentation for masktools since they both have good information. Kind of need to read V1 to understand the V2 sequel but that leads to my mt_ mistakes :)

I'll make the header update to show unfilter as a req, not sure what to do about the new release of removegrain, for now I just was more explicit via date code in the header of what I've tested.

Rev 0.4a posted

[2Bdecided]

In addition to the filters you listed, I also had to grab unfilter from here:
http://avisynth.org/warpenterprises/

Is that right, or should it have come from somewhere else?

Cheers,
David.

[DeathAngelBR]

Thought I should give an opinion on this filter: it rocks my socks!

Some screens (frame 17790) for comparison purpose.

Original:
http://img152.imageshack.us/img152/6694/17790001originalrj6.th.png (http://img152.imageshack.us/img152/6694/17790001originalrj6.png)

Denoise + sharpening script 1:

degrainmedian(mode=0)
degrainmedian(mode=3)
ttempsmooth()
fastlinedarkenmod(thinning=0, strength=25)
limitedsharpenfaster(ss_x=1.0, ss_y=1.0, smode=4, strength=20)

http://img444.imageshack.us/img444/3884/17790002lsfhs1.th.png (http://img444.imageshack.us/img444/3884/17790002lsfhs1.png)

Denoise + sharpening script 2:

fft3dfilter()
ttempsmooth()
fastlinedarkenmod(thinning=0, strength=25)
limitedsharpenfaster(ss_x=1.0, ss_y=1.0, smode=4, strength=20)

http://img152.imageshack.us/img152/2286/17790002fft3dka9.th.png (http://img152.imageshack.us/img152/2286/17790002fft3dka9.png)

Denoise + sharpening script 3:

mc_spuds(frames=2,strength=4)
ttempsmooth()
fastlinedarkenmod(thinning=0, strength=25)
limitedsharpenfaster(ss_x=1.0, ss_y=1.0, smode=4, strength=20)

http://img218.imageshack.us/img218/2517/17790002mcspudstr4wa6.th.png (http://img218.imageshack.us/img218/2517/17790002mcspudstr4wa6.png)

I don't even need to use LSF on this. Here's one without it.
http://img218.imageshack.us/img218/6307/17790003mcspudstr4nolsfhu7.th.png (http://img218.imageshack.us/img218/6307/17790003mcspudstr4nolsfhu7.png)

[Fizick]

spuds,
why bicubic upsampled instead internal interpolation?

[2Bdecided]

Is that version of RemoveDirt on-line anywhere? I have a copy (RemoveGrain 1.0.rar) already, but can't find anywhere for anyone else to grab it, which isn't very helpful.

Cheers,
David.

[Reuf Toc]

The URL to get this plugin is in the script :)

RemoveDirt 0.9 (http://home.arcor.de/kassandro/RemoveDirt/RemoveDirt.zip)

[ankurs]

http://i10.tinypic.com/72prl35.jpg

??

also i only called it into my script and just used it as

mc_spuds(frames=3,strength=4)

my avisynth version is 2.5.8 latest update ..

P.S : everything required is there including all the versions of the filters you stated in your first post ..

[Leak]

P.S : everything required is there including all the versions of the filters you stated in your first post ..
So what dimensions does the video you're feeding into MC_Spuds have? Did you actually check if the error message might not, just for once, be right? If either it's width or height cannot be divided by 8 without a remainder it's not going to work...

np: Bola - AMNiON (Soup)

[Spuds]

I don't even need to use LSF on this. Here's one without it. The script makes use of LSF on the static parts of the frame after it does its initial denoising, you should not need to call it again. Glad you are enjoying the scripts results !

why bicubic upsampled instead internal interpolation? Thats a great question, wish I had a great answer :) ... I assume since I'm using pel=2 I could just use sharp=1 to get a pretty similar result. My intention was to utilize the b/c parameters of bicubic to explore their impact and see if anything of interest came out. Does mvtools do a straight b=c=1/3 bicubic for sharp=1 ?

mc_spuds(frames=3,strength=4) Just like Leak said the script checks to see if the clip size (Width & Height) is divisible by 8 before it will run. I should update the logic so it only throws that error when asking for a frames=3 or 4 since that invokes a call to deblock which is what needs mod8.

I have some small updates in the works, I'll post a new version in the next couple of weeks. I've got another notably bad videocam clip that I'm working on to help make further improvements, mostly edge enhancements.

[DeathAngelBR]

The script makes use of LSF on the static parts of the frame after it does its initial denoising, you should not need to call it again. Glad you are enjoying the scripts results !

Unfortunely I just found out it's causing a lot of blocking, and it blurs too much the next frame, even lowering the strength value. Other than that it could have been the perfect denoiser IMO.

The blocking seems to be common among filters using mvanalyse or any mvtools thing.

[Spuds]

Do you have an example video clip you could post and a still shot of the issue? I'd like to see if it could be avoided.

Blocking to my understanding is an artifact caused by video compression. If you are getting it after you compress the scripts output then we might be able to do a couple of things to the to help avoid that (trick the compression engine to allocate more bandwidth into the sections that are blocking).

[DeathAngelBR]

All of those appear while seeking in virtualdub.


setmemorymax(768)

movie = DGDecode_mpeg2source("G:\Gundam Wing - Endless Waltz\VTS_01_1.d2v",cpu=4,info=3)

movied2v = "G:\Gundam Wing - Endless Waltz\VTS_01_1.d2v"

movie = tfm(movie,d2v=movied2v,order=1).tdecimate(mode=1)
movie = ColorMatrix(movie,hints=true)

last = movie

crop( 4, 0, -4, 0)

Lanczos4Resize(640,480)

mc_spuds(frames=2, strength=2)


Frame 1412 blured.
http://img267.imageshack.us/img267/9287/1412blurkp5.th.png (http://img267.imageshack.us/img267/9287/1412blurkp5.png)

Frame 1413 NOT blured.
http://img204.imageshack.us/img204/2973/1413normalvx8.th.png (http://img204.imageshack.us/img204/2973/1413normalvx8.png)

Frame 1438 blured.
http://img204.imageshack.us/img204/3708/1438blurhq5.th.png (http://img204.imageshack.us/img204/3708/1438blurhq5.png)

Frame 1439 NOT blured.
http://img211.imageshack.us/img211/4908/1439normalhl1.th.png (http://img211.imageshack.us/img211/4908/1439normalhl1.png)

Blocks on frame 1459.
http://img211.imageshack.us/img211/6397/1459blockshr1.th.png (http://img211.imageshack.us/img211/6397/1459blockshr1.png)

Will post an uncompressed vid as soon as I can.

[ankurs]

So what dimensions does the video you're feeding into MC_Spuds have? Did you actually check if the error message might not, just for once, be right? If either it's width or height cannot be divided by 8 without a remainder it's not going to work...

np: Bola - AMNiON (Soup)

my source is 720x576 ..
also it isnt woking on a 720x480 source .. also i am tryin to get this one to 640x272 ..

[foxyshadis]

my source is 720x576 ..
also it isnt woking on a 720x480 source .. also i am tryin to get this one to 640x272 ..

You're absolutely sure you have no cropping or resizing beforehand? Replace mc_spuds with Info() to verify. If it's the case, post your full script.

[Terranigma]

The blocking seems to be common among filters using mvanalyse or any mvtools thing.
LSF doesn't cause blocking. If your source has blocks, it'd just pronounce it. Your analysis here is correct though, and this script allows you to avoid it by increasing the overlap value. That's one of the reasons why it was added (Although it'd give you better denoising as well.) Try using a blocksize of 8, and overlapping value of 8/2, or 16 16/2, or 4 4/2. Though if you do decide to do this, expect the processing speed to drop tremendously.

[DeathAngelBR]

Try using a blocksize of 8, and overlapping value of 8/2, or 16 16/2, or 4 4/2. Though if you do decide to do this, expect the processing speed to drop tremendously.

That didn't work. There's still a *lot* of blocking and bluring. :(

[Spuds]

@DeathAngelBR -- Thanks for the examples, good stuff to work on, look forward to the video clip link so I can do some checking. I'm curious on the blurred frames if they are like that straight from tfm.tdecimate or not.

On the blocking as Terranigma pointed out you can override the default blocksize and overlap to try and deal with them, but be prepared to board the speed pain train :eek: I do have a couple of script ideas I'd like to try out on them to see if they would be effective or not. I'll get to trying those when I get the clip :)

[Terranigma]

Will post an uncompressed vid as soon as I can.
Well, i'll be waiting for the sample. :)

[DeathAngelBR]

http://www.sendspace.com/file/06xiuy
(134MB)

I don't know how to split it into smaller VOBs, so if any of you knows a software to do this I'm all ears.

:thanks:

[Terranigma]

Downloaded and tested on the exact same frame as you did (after forcing film since it's 95% film.)

Original:
http://img405.imageshack.us/img405/683/originalvu9.png

MC_Spuds Default:
http://img233.imageshack.us/img233/6813/mcspudssv7.png

MC_Spuds, my suggestion (mc_spuds(blocksize=8,overlap=8/2)):
http://img233.imageshack.us/img233/694/overlapfv9.png

So it's obvious that you weren't doing what I said correctly. :D

[DeathAngelBR]

Downloaded and tested on the exact same frame as you did (after forcing film since it's 95% film.)

Original:
http://img405.imageshack.us/img405/683/originalvu9.png

MC_Spuds Default:
http://img233.imageshack.us/img233/6813/mcspudssv7.png

MC_Spuds, my suggestion (mc_spuds(blocksize=8,overlap=8/2)):
http://img233.imageshack.us/img233/694/overlapfv9.png

So it's obvious that you weren't doing what I said correctly. :D

What about the frames 1459 and 1603 that have blocks all over?

Also there's this transition from blury frame to better, sharpened looking frame:

Blury:
http://img441.imageshack.us/img441/350/1313blurmk1.png

NOT blury:
http://img441.imageshack.us/img441/108/1314normalva8.png

[Terranigma]

What about the frames 1459 and 1603 that have blocks all over?

Well I just tried the one frame.
I deleted your sample, so i'll redownload and try it once more for the other troubled frames.

[Vesi]

any idea, why i get error. there is no function "unfilter" when i load my script with avsp

[Terranigma]

What about the frames 1459 and 1603 that have blocks all over?


mc_spuds(blocksize=8,overlap=8/2)
Frame 1459:
http://img223.imageshack.us/img223/7844/frame1459zz2.png

Frame 1603:
http://img223.imageshack.us/img223/6050/frame1603vi2.png

Also there's this transition from blury frame to better, sharpened looking frame:

Blury:
http://img441.imageshack.us/img441/350/1313blurmk1.png

NOT blury:
http://img441.imageshack.us/img441/108/1314normalva8.png

Your concerns about this qualm is a valid one, as I too noticed the transition from sharp to blurry.

mcspuds (Frame 1313):
http://img443.imageshack.us/img443/2184/frame1313spudscl2.png

mcspuds (Frame 1314):
http://img142.imageshack.us/img142/6628/frame1314spudslc6.png

s.w.t. + linedarken (Frame 1313):
http://img118.imageshack.us/img118/231/frame1313urlsdhc5.png

s.w.t. + linedarken (Frame 1314):
http://img510.imageshack.us/img510/60/frame1314urlsdeg2.png

s.w.t. (Frame 1313):
http://img510.imageshack.us/img510/378/frame1313urlsd2sh5.png

s.w.t. (Frame 1314):
http://img111.imageshack.us/img111/3381/frame1314urlsd2ku7.png

[DeathAngelBR]

s.w.t stands for...?

[Terranigma]

s.w.t stands for...?

something whipped together, but only for testing purposes.

[ankurs]

You're absolutely sure you have no cropping or resizing beforehand? Replace mc_spuds with Info() to verify. If it's the case, post your full script.

oops didnt know no resizing or cropping is not before it required .. well putting it before them i got a different error :mad:

http://i11.tinypic.com/6y3bekw.jpg

and this is what i got with info()

http://i9.tinypic.com/8dyzgbl.png

and idea ?? :rolleyes:

P.S Will post my script shortly ..

[Dreassica]

You need Unfilter plugin.

HERE (http://avisynth.org/warpenterprises/files/unfilter_25_dll_20030116.zip)

[2Bdecided]

Sometimes the motion compensation seems to "stick" on colours, leaving trails like this...

http://www.david.robinson.org/pics/mcspuds_problem.jpg

source clip:
http://www.david.robinson.org/pics/mcspuds_problem.avi

code for that example:
directshowsource("mcspuds_problem.avi")
tweak(sat=1.3)
bob()
a=last
converttoyv12(interlaced=false)
b=mc_spuds().ConvertToYUY2()
stackvertical(a,b)

Any suggestions?

(Apart from this, I think the denoising is working very well here)

Cheers,
David.

[Didée]

Call MC_Spuds with "chro=true". The script's default is to not use chroma during motion search -- the benefit is speed, the drawback is what you have just shown.

[2Bdecided]

That should have been what I was looking for Didée, but it didn't solve it at all. :(

Cheers,
David.

[Terranigma]

That should have been what I was looking for Didée, but it didn't solve it at all. :(

Cheers,
David.

It seems as if the motion search analysis is somewhat flawed with this script. The same thing is visible with the images I posted above; It looks like it's adding things from the previous frame, or the frame ahead. Sorta like the artifacts easily witnessed near scene changes if you use the default thsad value for mvdegrain and dct 1, or if you use an absurd high thsad value such as 2000.

[Didée]

Okay, 1st guess was wrong.

2nd guess: Are you using "RepairSSE2.dll"? If so, try this: don't import RepairSSE2, but instead the plain Repair.dll or RepairC.dll.

There have been reports of issues with the filter "TemporalRepair()" in some versions, and IIRC it was related to the ~SSE2 versions.
Sorry I can't test this myself right now: although my CPU supports SSE2, every script that uses RepairSSE2 crashes at latest on the 2nd frame ... O_o

OTOH, using the vanilla Repair.dll, I can *not* reproduce the issue on 2Bdecided's sample. I get a clean result without chroma artefacts. That's indeed a hint that the problem lies with TemporalRepair().

[2Bdecided]

That's it Didée! Deleting RepairSSE2.dll and dropping in Repair.dll solves the problem, both in that clip, and in other sections I've seen it in.

Thank you so much for your help. You'll get a credit on my finished video restoration (which will never be finished, never mind watched, but it's fun!)

Cheers,
David.

[Terranigma]

Ok Didée, what about this problem:
http://img443.imageshack.us/img443/2184/frame1313spudscl2.png

You can see parts of another frame (like ghosting) to your left.

Seems to be an underlying problem in buried in mc_spuds somewhere.
(This's what I was talking about :p)

[Spuds]

@Didée .. Awesome call on the TemporalRepair() artifacts, how you can remember all those versions and various issues is beyond me! I was about to dig in and see if my chro=true flow was boinked.

@Terranigma .. what call did you make to the function? I can see frames 3/4 causing some of what you are seeing since its builds a new flowed clip and then temporalsoftens it. With the anime stuff the frames change unnaturally and/or inorganically which causes all sorts of trouble. Once I know how it was called I can at least trace it.

@DeathAngelBR .. When I worked on the script I did testing it on my box of old vhs tapes (movies, camcorder, tv shows), never ran it on anime stuff. I have some updates that will help now with anime, or $%^#&# anime as I've been calling it the last couple of days :) I added some logic that will help remove blocking when using blocksize=16, its pretty good but not as good as going to a smaller blocksize, the benefit is a bit of speed. I could remove more but it starts to come at the cost of blurring.

I also added a option called starfield which will restore stars that are being removed since they look just like noise. You will get some noise back with this option but I think its a better tradeoff.

Now I'm off to see what might be causing the occasional blur frame that was reported. I'll post a link to the updated script later today.

[Terranigma]

@Terranigma .. what call did you make to the function? I can see frames 3/4 causing some of what you are seeing since its builds a new flowed clip and then temporalsoftens it. With the anime stuff the frames change unnaturally and/or inorganically which causes all sorts of trouble. Once I know how it was called I can at least trace it.

mc_spuds(blocksize=8,overlap=8/2)

[DeathAngelBR]

@DeathAngelBR
...
I have some updates that will help now with anime, or $%^#&# anime as I've been calling it the last couple of days :)
...
I also added a option called starfield which will restore stars that are being removed since they look just like noise. You will get some noise back with this option but I think its a better tradeoff.
...

Awesome!

May I suggest adding options for the ss_x, ss_y and strength (you could call it lsfstr or something) values of LSF? Yours "strength=80" and LSF's default for smode=4 is slightly haloing some edges here and there. Not noticeable when watching it, but is clearly seen when looking frame by frame.

Also you might wanna try ttempsmooth. I don't know how good/bad it is compared to temporalsoften at denoising, but it doesn't seem to introduce data from previous frames.

:thanks:

[Vesi]

i think not for me, i use this script

DGDecode_mpeg2source("C:\Documents and Settings\vesi\Desktop\Test\test.d2v")
AssumeTFF().Telecide(guide=1).Decimate()
crop( 2, 54, -2, -50)

Spline36Resize(640,272) # Spline36 (Neutral)
dull=last
sharp = dull.LimitedSharpenfaster( ss_x=1.25, ss_y=1.25, smode=4,strength=150, overshoot=1 )
Soothe(sharp,dull,45)
mc_spuds(frames=2,strength=3)

and i get fbps 1.50 for first pass and 2nd pass .50
also when i add mc_spuds before sharping, that doesn't work.
any idea on how to get good speed for denoising on good source
with less noise?

[Didée]

The ghosting-alike artefacts reported by Terranigma seem to be a bug in MVTools. Using DeathAngelBR's sample, those ghosts appear when dct is not zero. dct=2/3/4 are similar 2eo, with dct=1 it's even worse.
This is not caused by MC_Spuds() doing anything wrong - I traced it with just MVDegrain2(). Tried with MVTools 1.8.4 and 1.8.5.1 - same issue.
(edit: well okay, one could blame MC_Spuds for automatically setting dct=3 when blocksize=8 ... :p )

Spuds, a thought about the default blocksize values:
Is there a particular reason why you ordered the blocksizes from bs=16 for strength=1 down to bs=4 for strength=6?
Because of the way how different blocksizes are reacting to weak or strong noise, the usual rule of thumb is "small blocksize for small noise, big blocksize for big noise" ...

[Terranigma]

I traced it with just MVDegrain2(). Tried with MVTools 1.8.4 and 1.8.5.1 - same issue.

Ahh yes, so I assume that for the blocksize I specified, it automatically switches to another dct mode. I told Fizick about this issue in a pm, and he told me to try the "limit" parameter, but with dct-1, it was still there unless I changed limit to 0 (which doesn't do anything.) The only way to avoid this, is to by 1; use only dct 0 with the default thsad value, or use a thsad value < 100.

[Spuds]

Here (see later post) is a beta copy of the latest changes. I'll try and cover all the changes but I'm a bit tired right now so who knows what my poor brain will forget. Test it out, let me know whats working and whats not and whats just dumb :) ... Remember its beta!

The big changes are the following flags
Starfield = true (false by default). This is my first pass at adding a mask which 'saves' stars in the sky and adds them back after denoising. Really for anime but may be useful for other space shots. Frame 2623 us shown for an example.

With: http://img401.imageshack.us/img401/9012/2623atstmc3.th.png (http://img401.imageshack.us/my.php?image=2623atstmc3.png) Without: http://img401.imageshack.us/img401/2463/2623afsfbf3.th.png (http://img401.imageshack.us/my.php?image=2623afsfbf3.png)

Anime = true (false by default). This changes the dct to 3, lowers the scene detection threshold, turns on removeblocks which helps with frames <3 and blocksize > 8 and changes the secondary motion cleaning to a value which will be kinder to thin lines.

The blurring problem that was reported will be less with anime=true, it was caused by frame1=motion frame2=still, frame3=motion, frame4=still, etc. You would get one frame ID'ed correctly as motion and denoised as motion and then the very next frame it was all still so no additional denosing. This jerky motion on/off/on/off effect is a problem.

Frame 1313 was of interest as it showed a scene change problem where the frame 1312 was completely different from 1313 but was not flagged as a scene change due to the amount of similar black in both backgrounds. May need some more tweaking of thscd2 here.

With: http://img401.imageshack.us/img401/6459/1313nobleedma1.th.png (http://img401.imageshack.us/my.php?image=1313nobleedma1.png) Without: http://img401.imageshack.us/img401/9668/1313bleedqa9.th.png (http://img401.imageshack.us/my.php?image=1313bleedqa9.png)

Changed dct to 0 for all but anime=true which defaults to 3. It was / can cause some artifacts and it can remove / prevent others, bit of a tough nut :) Anime has some rapid luma changes which dct=3 seems to help with but has its issues as well. Frame 1603 is shown as an example:

DCT=3: http://img401.imageshack.us/img401/9027/1603dct3zi4.th.png (http://img401.imageshack.us/my.php?image=1603dct3zi4.png) DCT=0: http://img401.imageshack.us/img401/3497/1603dct0vc2.th.png (http://img401.imageshack.us/my.php?image=1603dct0vc2.png)

Added ability to set ss_x, ss_y and lsfstr values for LSF. Uses smode=4 values and strength = 80 by default.

Removeblocks = true (false by default). Helps break up some blocks, most useful with frames < 3 and blocksize =16. This is a trade off so decide by what you like. Blocks can be fixed several ways, 1) a smaller blocksize=8 and overlap=2 Also effective is use of frames=3 which does not use mvdegrain so it is not block prone but can have other problems with anime since frames are not naturally flowing (lots of quick scene changes and luma changes can bleed with frames=3) and lastly by turning on removeblocks which improves blocks but not as well as options 1 and 2 in all cases.
Frame 1461 is shown for example:

With: http://img401.imageshack.us/img401/5350/1461removeblocksos3.th.png (http://img401.imageshack.us/my.php?image=1461removeblocksos3.png) Without: http://img401.imageshack.us/img401/7057/1461blocksyr8.th.png (http://img401.imageshack.us/my.php?image=1461blocksyr8.png)

Ability to set thscd1 & thscd2 for mvdegrain1/2.

Basic call is still mc_spuds(frames=2, strength=3) or mc_spuds(frames=2, strength=3, anime=true, starfield=true)

[Spuds]

a thought about the default blocksize values:
Is there a particular reason why you ordered the blocksizes from bs=16 for strength=1 down to bs=4 for strength=6?
Because of the way how different blocksizes are reacting to weak or strong noise, the usual rule of thumb is "small blocksize for small noise, big blocksize for big noise" ...

Hummm ... I thought that the smaller block was a more complete search for noise and was most effective for all noise. So for lite noise you would use a blocksize of 4 vs 16? Bit of a bummer that the one that needs the least removal would take the longest to process, guess thats the cost of protecting details wiith a smaller block?

I'm not sure I'm going to leave blocksize=4 as a preset, if someone really wants that they can use an override. Instead I think another variation of blocksize 8 would be more useful.

[DeathAngelBR]

:eek:
I'm sooo going to be a test whore. :p

:thanks:

[Didée]

Hummm ... I thought that the smaller block was a more complete search for noise and was most effective for all noise. So for lite noise you would use a blocksize of 4 vs 16? Bit of a bummer that the one that needs the least removal would take the longest to process, guess thats the cost of protecting details wiith a smaller block?
Take a look at this example, showing original vs. 3 versions of MVDegrain2 with blocksizes 4 / 8 / 16:

Sample: Rapidshare (http://rapidshare.com/files/72611142/blocksizes.rar.html) -or- MediaFire (http://www.mediafire.com/?fdhnfl90txi) (8MB)

It's very obvious that blocksizes 8 and 16 are much more robust against the source's noise (generally, 16 even more than 8, though in this sample it's not a big difference).
With a blocksize of 4, the denoising is rather poor.
This is because with such a small blocksize, the ME engine is able to (partially) compensate the grain, which results in much less effective denoising.

Hence you are correct with
I'm not sure I'm going to leave blocksize=4 as a preset, if someone really wants that they can use an override.
For purposes of denoising, a blocksize of 4 most times isn't a good idea. Usually, blocksize 8 is fully sufficient. And overlap, albeit slow, from a "quality" point of view is a must ...

[Spuds]

@Didée .. Thanks for that example, it was very useful and eye opening. Not at all what I expected from the various blocksizes. I've made some changes to the presets based on this and your quality / overlap comments, '4' is gone :)

I've posted the latest beta HERE (http://www.rcboataholic.com/MC_Spuds.zip) with the following updates:

1) tweaked the presets
2) additional anime=true tweaks in the edge noise (ringing) area, basically lowered it as the edge blur was more noticeable in anime.
3) added addnoise=true setting which adds some lite grain in the dark areas, useful in some cases
4) fixed an oops in the mvdegrain level that was left behind from testing Doh!
5) Some startfield improvements

Give it a try and let me know what you find and if I broke anything else :)

[Terranigma]

Sorry to be bothering you, but I just can't seem to get the latter 2 beta versions to work.
With the call of just mc_spuds(), i'm getting this message:
Evaluate: operands of `<' and friends must be string or numeric
(, line 38)

line 38
: Assert((width(clp)%4 == 0 && height(clp)%4 == 0 ),"Width and height must be divisible by 4 for MC_Spuds")

[Spuds]

Not a bother at all, you have found some fun stuff for me to work on.

The reason for the error message is simple, its because I'm an dolt sometimes :o

I'm doing a validation check before I set up my default parameters. In this case with a call of mc_spuds() 'frames' is not preset set so the assert fails ungraciously.

I need to move the assert lines later in the script so frames at a minimum is 'set' before the assert call. For now a just call it as mc_spuds(frames=2) and you should be OK.

[Spuds]

Got a few moments to update a few areas and make some fixes based on some error reports.

1) fixed the function failing if called with no parameters
2) add ability to override limit=xxx, used in mvdegrain12
3) added threshold for starfield determination (thStar=float)
4) set removeblocks on for anime and blocksize=16, above that its off by default for anime and off for non anime.
5) added logic to starfield, now it will only apply stars back to the clip if the created mask is above threshold. Prior to this 'bad' spots were added back into the clip. With the default thStar frames with a low number of spots (stars), say 5, will not get added back, it needs to see a larger number of dots before it kicks in.
6) improved (maybe) the logic on removeblocks for anime and blocksize=16. Now its only called when the filter sees those large luma (prev/next) changes which is mainly where (not always though) the blocks were occurring.
7) fixed error in the addnoise (grain) line.

Get it HERE (http://www.rcboataholic.com/MC_Spuds.zip)

[Vesi]

Can we use MeGUI's MPEG2 Deblocker , like cpu=4 etc or any other deblocker?
and what does mean postprocess and (debug=1-5)?

[Spuds]

Can we use MeGUI's MPEG2 Deblocker , like cpu=4 etc or any other deblocker?

Yes, since we are tackeling two different block issues. Deblocking an Mpeg source is needed when a video is created without enough bandwidth in an effort to save size. This can create blocking artifacts that mpeg2source cpu=4, blindpp, deblock etc can fix.

The blocking that mc_spuds is using is aimed at artifacts that get produced from mvtools during denoising. These may occur when a large blocksize is used in the motion estimation.

what does mean postprocess and (debug=1-5)
Postprocess is a routine in mc_spuds that will do some limited further cleansing on a clip, its an extra final step that may produce results that are more desirable for some clips. It will remove some of the sharpening that was done in previous steps. The final image produced is softer and may look better for some videos.

Debug is now 1-9 :D It will produce various visualizations which can help in deciding what settings to use. Debug=1 just shows the settings the script is using. Debug=3 shows a side by side original vs final, nice to see the effects of various settings to help you choose whats best. Debug=4 shows a map of the changed pixels, the settings of 5 through 9 show the various masks that are used by the script to apply sharpening, denoising, etc ... I used them for tunning the masks and left them in there since they were fun.

[Vesi]

why i get error message when i use postprocess=true, error:there is no function named "LimitChange"

[Spuds]

error:there is no function named "LimitChange"

LimitChange() is in SSETools.dll which is in (v0.9) of the RemoveGrain (http://www.removegrain.de.tf/) package

[Vesi]

thanks alot,that works

[Spuds]

Well I've made some changes to the script over the last few weeks, the latest version is available in the link in post #2.

There have been numerous changes and I'm sure I'll forget to mention some of them in this post.

1) First for some reason I had a mt_binarize on my sharp mask, must have been left over from some debugging. That has been removed so the sharpening is applied smoothly.

2) First there is a new parameter sharpp=x which can be 1,2 or 3. 1 will use a Didée function called contra-sharpening, applied with a mask. 2 is limited sharpen faster, and 3 is none so you can apply your own like seesaw. The default is one.

3) I changed the edge mask, this is used to help dering edges. The old mask contained many more edges, really to many, then the new one does. I feel the new one works much better for what its intended. There is also one specific for anime in addition to the standard one.

4) frames=3 now will call mvdegrain3, if you don't have it you have three options. 1) wait and don't use frames=3, 2) make a donation to Fizick :) 3) use the flow=true parameter.

By default frames=3 will try to use mvdegrain3, you can use the previous revisions frames=3 function by specifying flow=true ie mc_spuds(frames=3,strength=4,flow=true). frames=4 only uses the flowed function. MVFlowed denoising can provide more pleasing results on some videos.

5) Changed, yet again, the denoised clip that gets used by mvanalyze for motion vector searches. I've tried removegrain, fft3dfilter, dfttest, hqdn3d and combinations of them. Its been a speed quality trade off and I think I now have a good balance and one that follows the strength parameter more appropriately.

6) Removed the dependency on limitchange by replacing it with a mt_lutxy call. One less DLL needed :)

7) Various changes, optimizations and code structure changes.

todo: Update the script header with the needed filters, write a help file

[Terranigma]

Now I can reascend my earlier post (which I deleted a short while ago); Thanks for the update, been testing it out for some time and i'm really taking a liking to it. In fact, i'm thinking about replacing your script with the one I concocted, albeit with a few changes. ;)

[Vesi]

6) Removed the dependency on limitchange by replacing it with a mt_lutxy call. One less DLL needed is mt_lutxy is defrent dll file or it includes mt_masktool? if defrent any link?

[foxyshadis]

Not really that hard to use search, but yes, all the mt_ filters are from the same dll.

[TheRyuu]

With focus=true I get a green screen.

I tracked the problem down to removegrain.

Called removegrain(20,-1) gives me a green screen.
Was working fine (don't know if anything was changed) then I reformatted. Now it's giving me this.

Any ideas?
Thanks.

The complete call is:
MC_Spuds(frames=2,strength=2,anime=true,focus=true)

[Piper]

@Spuds: I just want to thank you for this script, really fantastic work! Thanks so much! :thanks:

[signal]

Just started testing this and I'm getting about ~ 0.28 fps (frames=1, strength=4) and curious what speed other folks are getting.

Expected it to be slow but wondering if I have something setup wrong.
CPU is an Intel E6850 Core Duo 3Ghz.

[Spuds]

@wizboy11 .. Thanks for the bug report, I have a good idea why its doing that :) ... the focus stuff was just some stuff I was playing around with so I had not done any testing in the last update. I see you were looking at the script to make the call !

@piper .. Thanks!

@signal .. WOW that is slow, could you post your entire script? I run mc_spuds it all the time with setmtmode(2,2) (I've tried to ensure that everything in the script is setmtmode OK). With frames = 2 strength= 4 or 5 I typically get 5-6fps and my PC is a core 2 duo 2.4Ghz. I've seen it run slow on some toons, but never that slow. With a 3Ghz system and frames = 1 you should be smoking ( well for MV stuff that is :) )

[Boulder]

@signal: For the heck of it, try adding the line Crop(0,0,-0,-0,true) before the MC_Spuds line.

[signal]

@Spuds

Mind you I'm just experimenting but it simply crawls with frames=1. With frames=2 I get just over 5fps.

The script is pretty basic:import("..\MC_Spuds.avs")
MPEG2Source("my.d2v", cpu=4, iPP=false).ConvertToYV12()

mc_spuds(frames=1, strength=2)
The source is an 80's anime and using Force Film in DGIndex. Aside from 1 problem frame it is 23.976 progressive.

I'm not using the mt version of AVISynth (probably should be?).

@Boulder

I'll give it a shot tonight.

[IanB]

@Boulder,

Crop(0,0,-0,-0,true) will be a no-op in all version prior to the current 2.5.8. Previous version only acted based on the left edge crop parameter of the call. The current CVS version checks the actual data pointer alignment of each frame and then copies to a new aligned frame if it is unaligned.

[Boulder]

@Boulder,

Crop(0,0,-0,-0,true) will be a no-op in all version prior to the current 2.5.8. Previous version only acted based on the left edge crop parameter of the call. The current CVS version checks the actual data pointer alignment of each frame and then copies to a new aligned frame if it is unaligned.That's odd - I remember that at least one time I needed to add that line to fix a major slowdown with the LRemoveDust function:confused:

[IanB]

@Boulder,

Almost certainly you were doing a real crop with the left edge not mod 16. i.e. crop(6, ...

[signal]

Confirm I'm not seeing anything different with the crop command.

I'm using standard AVISynth 2.5.7 and will try giving the MT version a shot.

Is the general consensus that frames=1 should be faster than frames=2 or 3?

My general assumptions (should never assume) was that frames=1 was doing less analysis but it's obviously working harder than frames=2 or above.

[Spuds]

@signal,

Yes frames=1 should be faster as its only looking one frame back and one forward when computing the motion vectors. I just ran it on my laptop with an anime and was getting ~8fps so I'm still at a bit of a loss to understand why its so slow for you on frames=1 but normal with frames=2.

The only thing that is unique to frames=1, in terms of what functions get utilized, is a call to mvdegrain1. Try installing the latest version of mvtools again and see if that fixes the issue.

[Caroliano]

Off-topic:

Is it possible to create an standalone funcion that works like Soothe() for preserving stars? I'm filtering an anime and using another filterchain that unfortunately also kill stars. I will not use MC_Spuds because I think it is an overkill, and I don't have knowledge to extract only the starmask/starfield part from your complex script.

If you can help me, it is better start an new topic for it. :thanks:

[Spuds]

I've posted a new version of the script, the link is still in post number 2.

I've added a first and maybe final pass of a help file so folks can sort though all the parameters. There are a lot of 'buttons' but they are really for those who like to tinker.

The script changes are:
1) Removeblocks now calls deblock_qed. There are default values set for quant1, quant2 and uv but you can supply your own as well. If you use this you will need to import the script yourself. Thanks to Terranigma for the suggestion and testing.

2) Ability to pass the search parameter to mvanalyse, previously it was set to 2, now you can set it to what you want.

3) Ability to pass the pel parameter to mvanalyse, previously it was set to 2

4) Colorbleed correction via colorbleed=true. Thanks again to Terranigma for the suggestion and testing.

5) Added back the ability to use aggressive with frames=3 and flow=true.

6) fixed a few things that caused slowdowns in certain areas under certain conditions. No I can't be more vague about that :)

[Malcolm]

@spuds,
thanks a lot for the new script.:thanks:

However, when comparing the version from 22nd of december to the current one, i see a problem with the 'preprocess' parameter.
In brief: 'preprocess' now leads to blurry results (at least with my material
Please look at the following code snippets and the corresponding images.
left part is the result of using the december version of MC_Spuds(), middle part is the result the new january version. On the right you can see the amplified difference of the 2 clips.
left = MC_Spuds_JAN(strength=4, frames=3, flow=false, focus=false, preprocess=false, postprocess=false)
middle = MC_Spuds_DEC(strength=4, frames=3, flow=false, focus=false, preprocess=false, postprocess=false)
right = subtract(left, middle).levels(...)http://img118.imageshack.us/img118/4355/mcspudscomparejanpreprozs3.th.png (http://img118.imageshack.us/my.php?image=mcspudscomparejanpreprozs3.png)

left = MC_Spuds_JAN(strength=4, frames=3, flow=false, focus=false, preprocess=false, postprocess=false)
middle = MC_Spuds_DEC(strength=4, frames=3, flow=false, focus=false, preprocess=true, postprocess=false)
right = subtract(left, middle).levels(...)http://img118.imageshack.us/img118/2707/mcspudscomparejanpreprofj3.th.png (http://img118.imageshack.us/my.php?image=mcspudscomparejanpreprofj3.png)

left = MC_Spuds_JAN(strength=4, frames=3, flow=false, focus=false, preprocess=true, postprocess=false)
middle = MC_Spuds_DEC(strength=4, frames=3, flow=false, focus=false, preprocess=true, postprocess=false)
right = subtract(left, middle).levels(...)http://img118.imageshack.us/img118/3784/mcspudscomparejanpreproch5.th.png (http://img118.imageshack.us/my.php?image=mcspudscomparejanpreproch5.png)

By switching between image 1 and 2 and looking at the left side, you can compare the results of enabling / disabling the 'preprocess' feature in the december version of MC_Spuds().
You can see that enabling preprocessing does not degrade image quality at all.

By switching between image 2 and 3 and looking at the middle part, you can compare the results of enabling / disabling the 'preprocess' feature in the january version of MC_Spuds().
You can see that enabling preprocessing does degrade the image quality a lot! It is blurred by a hefty degree. (Please check the girls faces.)
You can also see this by looking at the right part where the difference of the december version and january version is visualized (preprocessing enabled in both versions). The contours of the faces are clearly visible.

Since 'preprocess' is supposed to help mvdegrain by distinguishing movement from noise, it should improve the denoising result.
When judging by the given examples it looks like enabling preprocessing leads to worse results with the january version of MC_Spuds. Whereas enabling preprocessing in the december version of MC_Spuds does not degrade image quality.

Maybe you have an explanation or solution for me.
Besides of that i hope this helps you to further optimize MC_Spuds().

Best regards,
Malcolm

[Spuds]

Malcolm,

Thanks for the detailed finding! :thanks:

I went back through the script and found what I think was the cause of the problem, one that seemed to be causing the biggest delta between versions I was surprised by.

Anyway I posted an update at the link in post number 2, please give it a try and let me know if fixes it for you as well and if not I'll do some more research and head scratching.

Update: Now that I see there is a MVtools V 1.9.2 and looking at the change log I'm no longer surprised about the big delta! Anyway get the new script and new mvtools.

[Malcolm]

@Spuds,

thanks! I just made a quick check.. works great now! :)
When i'm at home i can send a screenshot to show the new result.

Edit:
Sorry, i can't figure out how the script from above looked exactly (there were others filters in it too). so a comparison isn't meaningful. therefore i won't post a pic.

greetings,
Malcolm

[Adub]

Adding this to the wiki.

Edit: Added to the wiki.
Link: http://avisynth.org/mediawiki/MC_Spuds#Links

[Spuds]

Found a fixed a error in the final output of the script, nothing major but I was not applying one of the transforms.

Minor tweaks to the starfield function when its used on non anime sources.

The link in post 2 has been updated as well as the wiki.

[halsboss]

Which warpsharp DLLs and xsharpen should I use ?

There's 2 at http://avisynth.org/warpenterprises/ with one 24k the other larger ... does it give aWarpsharp and xsharpen or are there more dlls ?

Confused ...
http://forum.doom9.org/showthread.php?p=1047273#post1047273 mentions aWarpsharp.dll and http://forum.doom9.org/showthread.php?p=1047300#post1047300 mentions xsharpen ...

[Spuds]

xsharpen is contained in the warpsharp.dll, get it here (http://avisynth.org/warpenterprises/...l_20031103.zip)

Alternativly there is a modified version of the warpsharp.dll at this site (http://seraphy.fam.cx/~seraphy/program/WarpSharp/index.html), if you have an Intel processor use the ICC version else the VC version.

[Vesi]

I get this error when I want to use debug.
Script error: Invalid arguments to function "debug_view"

[Spuds]

I found one error in the debug view function when you used strength=1 or set edgeclean=false when you call the function. If you were doing either of these then change the bold text below in line 153 as follows:

maskedge = (edgeclean) ? dering_mask(clp4, anime, strength, edm_hi, edm_lo) : nop()

to:

maskedge = (edgeclean) ? dering_mask(clp4, anime, strength, edm_hi, edm_lo) : nullclp

If this does not fix the issue please let me know and also post how you called mc_spuds. I'm working on a couple of other improvements and when done I'll post an update with this fix.

[Vesi]

Problem fixed, thanks alot spuds.
and yes I was using strength=1

[halsboss]

Hi Spuds, I needed a dummies simplified strong denoiser just for my own purpose. Thanks to you I was able to base it on your function (uses less filters and I didn't understand the frames=3 vs frames=4 logic). Hope that's OK. Thanks again.
Could you clarify the limit default setting code is right as it wasn't quite the same as the other code around it ?

limit = (defined(limit)) ? limit : (defined(limit)) ? select(limit , 96 , 96 , 102 , 128 , 156 , 225 , 255) : 128 # threshold for the motion mask inclusion, lower includes more

Anyway here's my cut-down script based on your terrific effort. Query: the IDXing doesn't seem right ... is it ?
FUNCTION DCW_STRONG_DENOISE(clip clp, int "strength", bool "fixcolorbleed", bool "edgeclean", bool "fixfocus", bool "addnoise") {
#
# clp MUST BE IN YV12 and mod 8 size
# Strength (1-6), default 3, more=stronger denoising
# fixcolorbleed (true or false) default false, remove color bleeding
# edgeclean (true or false) default false, help any edge ringing left behind by adding a mild blur to object edges
# fixfocus (true or false) default false, can help with edges that are blurry, this is from IIP script
# addnoise (true or false) default false, add some grain in dark areas, help with compression and detail illusion
#
##### VERY STRONG motion compensated denoiser, based on MVDegrain3
#
#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# Derived from MC_Spuds()
# Motion compensated noise removal with sharpening -- version: BETA - 29.Feb.2008
# http://forum.doom9.org/showthread.php?p=1101306#post1101306
# Thanks to Didee for his comments, scripts and posts from which I learned much and borrowed some from :)
# Required filters:
# mt_masktools: Ver 2.0.32+
# Code http://manao4.free.fr/masktools-v2.0a32.zip : Discussion http://forum.doom9.org/showthread.php?t=98985
# removegrain: Ver 1.0 (8/2005) & Ver 0.9 (5/2005) v1.0 has an issue with repairsse3, so use the repairsse3.dll one from V0.9
# Code http://www.removegrain.de.tf : Discussion http://forum.doom9.org/showthread.php?s=&threadid=79751
# removedirt: Ver 0.9+
# Code http://www.removedirt.de.tf : Discussion http://forum.doom9.org/showthread.php?s=&threadid=70856
# mvtools: Ver 1.9.1+
# Code http://avisynth.org.ru/fizick.html : Discussion http://forum.doom9.org/showthread.php?t=84770
# unfilter: Ver 0.0.1.5+
# Code http://www6.impacthosting.com/trbarry/downloads.htm
# fft3dfilter: ver 2.1.1+
# Code http://avisynth.org.ru/fizick.html
# addgrain: Ver 1.4+
# Code http://foxyshadis.slightlydark.com/random/AddGrainC.zip
# awarpsharp and xsharpen versions mentioned at
# http://forum.doom9.org/showthread.php?p=1107734#post1107734
# awarpsharp from awarpsharp.dll
# http://forum.doom9.org/showthread.php?p=1107379#post1107379 and
# xsharpen from warpsharp.dll
# http://forum.doom9.org/showthread.php?p=1107379#post1107379
#++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\fft3dfilter.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\WarpSharp.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\aWarpSharp.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\MaskTools.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\Unfilter.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\AddGrainC.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\mt_masktools.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\RemoveGrainSSE2.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\RepairSSE2.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins-dcw\RemoveDirtSSE2.dll")
#Import("C:\Program Files\AviSynth 2.5\LimitedSharpenFaster.avsi")
#Import("C:\Program Files\AviSynth 2.5\Deblock_QED_MT2.avs")
#
#
global idx_pointer = 24

# set up our parameters based on what the user input

strength = default( strength , 3 ) # how aggressive to remove noise / grain, sets predefined defaults
fixcolorbleed = default( fixcolorbleed , false ) # Special mode to remove color bleeding
edgeclean = default( edgeclean , false ) # edge clip for extra noise removal?
fixfocus = default( fixfocus , false ) # special mode to try and crisp up edges of blurry / soft video, code from IIP
addnoise = default( addnoise , false ) # Special mode to add some grain in dark areas, help with compression and detail illusion

shadow_l = 24 # Low limit for dark grain mask
shadow_h = 42 # high limit for dark grain mask
edm_lo = string( 55 ) # edge detection lo threshold
edm_hi = string( 255) # edge detection hi thresold
ls_x = 1.25 # LSF Smode=4 default
ls_y = 1.25 # LSF Smode=4 default
lsfstr = 100 # Strength parameter in LSF
quant1 = 0 # quant 1 for deblock_qed
quant2 = 45 # quant 2 for deblock_qed
uv = 1 # uv for deblock_qed
pel = 2 # pel for MVAnalyse
search = 2 # search mode for mvanalysis

blocksize = (defined(strength)) ? select(strength , 16 , 8 , 8 , 16 , 16 , 16 , 16) : 16 # blocksize for motion search
overlap = (defined(strength)) ? select(strength , 2 , 0 , 2 , 4 , 4 , 8 , 8) : 2 # overlap of the block for motion search
pnew = (defined(strength)) ? select(strength , 32 , 32 , 32 , 32 , 32 , 32 , 32) : 32 # prevent replacing of an alrady good predictor by one thats just a little better
thsad = (defined(strength)) ? select(strength , 400 , 300 , 400 , 415 , 450 , 450 , 500) : 400 # threshold to motion, increase for more grain removal ie 500
thscd1 = (defined(strength)) ? select(strength , 400 , 400 , 400 , 400 , 400 , 400 , 400) : 400 # threshold to motion comared to SAD
thscd2 = (defined(strength)) ? select(strength , 130 , 130 , 130 , 130 , 130 , 130 , 130) : 130 # Scene detection threshold, how much must a frame must change to be a scene change
ml = (defined(strength)) ? select(strength , 100 , 120 , 100 , 95 , 90 , 90 , 80) : 100 # threshold for the motion mask inclusion, lower includes more
limit = (defined(strength)) ? select(strength , 96 , 96 , 102 , 128 , 156 , 225 , 255) : 128 # threshold for the motion mask inclusion, lower includes more

# initilize clip-dependent constants
clp_width = clp.width()
clp_height = clp.height()
clp_width_2x = 2 * clp_width
clp_height_2x = 2 * clp_height
nullclp = blankclip(clp)

# Remove color bleeding
clp = (fixcolorbleed) ? clp.removegrain(0,11) : clp

# Do the primary noise removal
#+++
# pre-filter a clip for motion estimation, then upsample it too
vector_prefilter1 = clp.fft3dfilter(sigma=strength*4.,sigma2=strength*3.,sigma3=strength*2.,sigma4=strength*1.,bt=1,bw=16,bh=16,ow=4,oh=4,plane=4)
upsampled1 = vector_prefilter1.BicubicResize(clp_width_2x,clp_height_2x,b=0.0,c=0.5,src_left=0.25, src_top=0.25)
# compute the motion vectors
global idx_pointer = idx_pointer + 1
bv3 = vector_prefilter1.MvAnalyse(isb=true, delta=3, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled1,pnew=pnew)
bv2 = vector_prefilter1.MvAnalyse(isb=true, delta=2, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled1,pnew=pnew)
bv1 = vector_prefilter1.MvAnalyse(isb=true, delta=1, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled1,pnew=pnew)
fv1 = vector_prefilter1.MvAnalyse(isb=false, delta=1, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled1,pnew=pnew)
fv2 = vector_prefilter1.MvAnalyse(isb=false, delta=2, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled1,pnew=pnew)
fv3 = vector_prefilter1.MvAnalyse(isb=false, delta=3, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled1,pnew=pnew)
# mvdegrain the resulting frame based on the motion vectors
global idx_pointer = idx_pointer + 1
clp2 = clp.MVDegrain3(bv1,fv1,bv2,fv2,bv3,fv3,thSAD=thsad,IDX=idx_pointer,limit=limit,thscd1=thscd1,thscd2=thscd2)
# make a mask based on our denoised framed, we need new vectors since the key frame changed
vector_prefilter2 = clp2
upsampled2 = vector_prefilter2.BicubicResize(clp_width_2x,clp_height_2x,b=0.0,c=0.5,src_left=0.25,src_top=0.25)
global idx_pointer = idx_pointer + 1
bv1 = vector_prefilter2.MvAnalyse(isb=true, delta=1, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled2,pnew=pnew)
fv1 = vector_prefilter2.MvAnalyse(isb=false, delta=1, idx=idx_pointer, truemotion=true,blksize=blocksize,overlap=overlap,pel=pel,sharp=0,search=search,chroma=true,pelclip=upsampled2,pnew=pnew)
# use the vectors to create masks, we apply stage two filtering to masked areas.
maskmotion = mt_average(clp2.mvmask(kind=1, vectors=bv1, ml=ml),clp2.mvmask(kind=1, vectors=fv1, ml=ml)).UtoY().BicubicResize(clp2.width(),clp2.height())
maskstatic = mt_invert(maskmotion).mt_lut("x 48 < 0 x ?")
# create our filtered clips to apply via the masks, don't go to heavy on smoothed since we have already degrained.
smoothed = clp2.RemoveGrain(mode=17)
sharp = clp2.limitedsharpenfaster(smode=4, ss_x=ls_x, ss_y=ls_y, strength=lsfstr)
# apply smooth and sharp clips with the masks
clp3 = mt_merge(clp2, smoothed, maskmotion)
clp4 = mt_merge(clp3, sharp, maskstatic)
#+++

# There could be some edge ringing left behind, this will help it by adding a mild blur to object edges.
maskedge = (edgeclean) ? dering_mask(clp4, edm_hi, edm_lo) : nullclp
dering = (edgeclean) ? clp4.bicubicResize((clp_width/2),(clp_height/2)).bicubicResize(clp_width,clp_height,b=0.8,c=0.1).unfilter(-5,-5) : nullclp
clp4 = (edgeclean) ? mt_merge(clp4,dering,maskedge) : clp4

# apply focus function if requested, can help with edges that are blurry, this is from IIP script.
clp4 = (fixfocus) ? focus(clp4, clp_width, clp_height) : clp4

# add grain to dark areas so encoders dont block them up on a low bitrate, also can give detail illusion
maskdark = (addnoise) ? clp4.mt_lut("x " + th_lo + " < 255 x " + th_hi + " > 0 x " + th_lo + " - " + th_hi + " " + th_lo + " - / 255 * ? ? "): nop()
clp5 = (addnoise) ? mt_merge(clp4,clp4.sharpen(0.25).addgrain(2,0,0),maskdark,luma=true): clp4

RETURN(clp5)
}

#----------------------
FUNCTION RemoveDustDirt(clip input, int repmode, int clmode, int limitY, int limitUV)
{
repmode = default(repmode, 16)
clmode = default(clmode, 5)
limitY = default(limitY, 5)
limitUV = default(limitUV, 255)
clensed = Clense(input)
repaired = Repair(clensed, input, mode=repmode)
degrained = RemoveGrain(repaired, mode=clmode)
degrained = mt_lutxy(degrained,input, \
expr="x y - abs " + string(limitUV + 1) + " < x x y - 128 + 128 < y " + string(limitUV) + " - y " + string(limitUV) + " + ? ?", \
yexpr="x y - abs " + string(limity + 1) + " < x x y - 128 + 128 < y " + string(limity) + " - y " + string(limity) + " + ? ?",y=3,u=3,v=3)
RETURN (degrained)
}

#----------------------
FUNCTION MinBlur(clip input, int r, int "uv")
{
# Taken from MCBob.avs:
uv = default(uv,3)

# process chroma if uv==3, otherwise just luma
uv2 = (uv==2) ? 1 : uv
rg4 = (uv==3) ? 4 : -1
rg11 = (uv==3) ? 11 : -1
rg20 = (uv==3) ? 20 : -1
medf = (uv==3) ? 1 : -200

# make our blur clips, r controls amount ...keep the best blur pixel from each
RG11D = (r==1) ? mt_makediff(input,input.removegrain(11, rg11),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(input,input.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
\ : mt_makediff(input,input.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
RG4D = (r==1) ? mt_makediff(input,input.removegrain(4,rg4),U=uv2,V=uv2)
\ : (r==2) ? mt_makediff(input,input.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
\ : mt_makediff(input,input.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
DD = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
RETURN (input.mt_makediff(DD,U=uv,V=uv))
}

#----------------------
FUNCTION dering_mask(clip input, string edm_hi, string edm_lo)
{
nmask = input.mt_edge(mode="min/max",thy1=0,thy2=255,thC1=0,thC2=255,Y=3,V=1,U=1)
amask = nmask.mt_lut("x 70 > 255 x ?").removegrain(11,modeU=0,modeV=0)
tmask = amask.mt_expand.mt_inflate
maskedge = mt_lutxy(amask.mt_invert(), tmask, expr="x y * 255 /").mt_lut("x " + edm_hi + " > 255 x " + edm_lo + " < 0 x " + edm_lo + " - " + edm_hi + " " + edm_lo + " - / 255 * ? ? ")
RETURN (maskedge)
}

#----------------------
FUNCTION Focus(clip input, int clp_width, int clp_height)
{
# May help some edge blurred sources by defining the edges
ssx = 3.5
ssy = 3.5

# Super-duper-size the clip ensuring its mod16
xx_ss2 = int(clp_width * ssx / 16 + 0.5) * 16
yy_ss2 = int(clp_height * ssy / 16 + 0.5) * 16
input = input.LanczosResize(xx_ss2,yy_ss2)

# apply our transfors to the edges (blur, warpsharp (only to luma), and xsharpen)
input = input.removegrain(20)
input = input.awarpsharp(depth=10.5, thresh=0.5, blurlevel=2, cm=0)
input = input.xsharpen(255,255)

# return the clip in its original size, this will just have enhanced edges
RETURN input.Lanczos4Resize(clp_width,clp_height).RemoveGrain(mode=1)
}

#----------------------
FUNCTION blockbreaker(clip input, clip bv1,clip bv2, clip fv1, clip fv2, clip clp12, int frames)
{
# create a blockmask
blockmask = mvmask(input,bv1,kind=1,ml=6,gamma=3).mt_invert.mt_binarize.mt_expand.mt_inflate

# creatre a flowed and stabilized clip
clpnb = (frames==2) ? interleave(input.mvflow(bv2,IDX=idx_pointer),input.mvflow(bv1,IDX=idx_pointer),input,input.mvflow(fv1,IDX=idx_pointer),input.mvflow(fv2,IDX=idx_pointer)) \
: interleave(input.mvflow(bv1,IDX=idx_pointer),input,input.mvflow(fv1,IDX=idx_pointer))
clpnb = (frames==2) ? clpnb.temporalsoften(2,1,1,32,2) : clpnb.temporalsoften(1,1,1,32,2)
clpnb = (frames==2) ? selectevery(clpnb,5,2) : selectevery(clpnb,3,1)

# merge the clips together to smooth the blocks
clp12 = mt_merge(clp12,clpnb,blockmask)
RETURN (clp12)
}

[foxyshadis]

IanB's going to be very annoyed at you. :p

You never actually do anything with the vector_prefilter2.MvAnalyse(.... clips, but I assume that's intentional. You should probably include "global idx_pointer = idx_pointer + 1" in blockbreaker, or it will use the wrong idx if called after DCW_STRONG_DENOISE.

Note that isn't even necessary anymore; every thread internally gets its own idx offset or something like that in the latest releases of mvtools, so it's completely safe to just use a hardcoded number even with multithreading.

Just don't use the same hardcoded number as some other script! A random higher idx or a global idx like you use is much safer when mixing mvtools functions.

Oh yeah, and your idx initialization needs to be outside the function or it'll be the same for every thread (but will work fine with newer mvtools anyway). The file also has to be imported before the setmtmode/mt.

[halsboss]

Whoa ! That's the last thing anyone needs. :)

Thanks for the explanation.

I attempted to use the "logical flow" of MC_SPUDS's code and function calls, and that's what I thought it ended up to be... off to review that, then.

[Spuds]

@halsboss

First no problem taking the function and doing what you want to it, its how we learn! Share, reuse, tweak, learn :)

limit = (defined(limit)) ? limit : (defined(limit)) ? select(limit , 96 ......
Yeah thats wrong, must have forgotten to take my medication that day. The way its written unless you passed the function a limit value it would always be 128, no real harm just not as intended. The forth/fifth limits should be strengths. Thanks for catching that!

foxyshadis already gave you a few pointers on that script. I believe the blockbreaker comment is due to the fact that you don't call it from within DCW_STRONG_DENOISE, so if you intend to call it outside that function you need to be careful.


OK onto some new stuff, I have posted a new version of the script with several important changes. I have not updated the wiki so for now you can only get in at the link in post #2. This is a test version, thats why I have not updated the wiki.

The changes are as follows:
1) It no longer externally upscales the frame, instead it lets mvtools do the work. I have gone back and forth on this point and tried many different resize functions but in the end I could not see any visual difference that really pointed me in one direction or the other, perhaps even a detriment doing it externally. This was further complicated by allowing pel=4 in the last update :)

2) Significant changes to the flow=true functions. They now work on frames 1 through 4 where in the past it was only on 3 and 4. They also now use mvcompensate in place of mvflow and use ff3dfilter or dfttest for denoising (instead of removegrain and temporalsoften) on the long temporal clip that flow=true creates. I feel its a significant improvement from the old routine, give it a try !

3) Aggressive=true will work on frames 1-4 instead of just frames 3 and 4. It basically does a two pass through the denoising, a lite first pass and and a second normal for lack of a better description.

4) Various areas cleaned up

[Terranigma]

You're going to overload your brain one day. =P
Anyways, thanks for the update. I'll test it a.s.a.p. :)

[talen9]

Spuds, there's a slight typo (I think) in line 73 of the recently modified MC_Spuds.avs (from your .zip archive, but's there in the wiki's version too).

My source was "mod 4" (but not "mod 8") both by width and by height, but then, on the bicubicresize done in line 144, when the clip's width and heigth is halved, AviSynth outputs the error "YV12 width must be multiple of 4".

In the check you're doing in line 73, you're checking for "not-mod 8" sources, but in the comment line above you wrote:
"# baseline clip color size error check, mod 8 and yv12 colorspace"
.

Changing the mod check value in line 73 to "8" has the effect of correctly failing the assert ... that's the way to go, or am i missing something?


:thanks: for you precious work on this function!

[Spuds]

@talen9

Thats a bit strange ... the function should work with mod4 clips.

Originally I had restricted it to mod8, that was the assert comment line that you found. That restriction caused some a bit of anguish so I opened it up to allow mod4. Ideally you should be mod8 or preferably mod16 for efficiency but mod4 *should* work.

If it was throwing the error on the resize line (part of the final motion mask) it narrows it down a bit. Could you please post the call you made to mc_spuds as well as the clip dimensions so I can try and reproduce the error.

Thanks!

[talen9]

The source is a PAL R2 DVD, 720x576, cropped to 716x424 by "crop(2,76,-2,-76,align=true)".

The call to MC_Spuds is

MC_Spuds(frames=2, strength=3, thSAD=100)

As i (awfully) tried to explain in the previous post, i think the culprit is the first BicubicResize on line 144:

dering = (edgeclean && anime && strength < 5) ? minblur(clp4,1,1) \
: (edgeclean && anime && strength > 4) ? minblur(clp4,2,1) \
: (edgeclean) ? clp4.bicubicResize((clp_width/2),(clp_height/2)).bicubicResize(clp_width,clp_height,b=0.8,c=0.1).unfilter(-5,-5) : nullclp

The complete error line is:

Resize: YV12 width must be mutiple of 4.
(C:\<path>\MC_Spuds.avs, line 144)
(<calling script>, line 95)


EDIT: the fact that I'm calling MC_Spuds in an "SetMTMode(2,2)"-d script might have to do something with the anomaly? I thought of that only just now ...

EDIT2: erase the previous edit, it behaves the same even if I remove the SetMTMode().

[Spuds]

Now I got it :)

Nothing like doing some division in a script and then not validating that the results are valid, doh! I've posted a new version via the link in post #2, it should fix that error and allow it to be mod4.

There are a few new items in that script as well that you are welcome to try out.

1) temporal=true .. I'm sure everyone can guess what that is, but if not it uses the temporal de-graining method that Didee introduced some weeks back. Since the script already had 95% of the data needed it seemed like a natural extension to add. Try it with frames 1-4 or flow=true to see if you like its results.

2) prefiltering now follows the strength parameter, this will speed up (well .5fps is a speed up in mc stuff !) processing for lower strengths. You can override with premax=true which will turn on full pre-filtering of the clip for motion vector searches.

3) added ability to specify the sharp parameter to mvanalyze via sharp = (0-2) should someone just want to tweak and play.

[Nikos]

Spuds thanks for your work, just a correction, in my opinion.
From mvtools doc:

mt_merge

mt_merge : clip clip1, clip clip2, clip mask, bool "luma"(false)

* It's the backbone of the framework. It merges two clips according to the mask. The bigger the mask value, the more the second clip will be taken into account ( the actual formula is y = ((256 - m) * x1 + m * x2 + 128) / 256 )

* luma is a special mode, where only the luma plane of the mask is used to process all three channels.

* u and v are defaulted to 2 (that way, the resulting clip contains the chroma of clip1, and looks right).


From your function:

# add grain to dark areas so encoders dont block them up on a low bitrate, also can give detail illusion
maskdark = (addnoise) ? clp4.mt_lut("x " + th_lo + " < 255 x " + th_hi + " > 0 x " + th_lo + " - " + th_hi + " " + th_lo + " - / 255 * ? ? "): nop()
clp5 = (addnoise && chro) ? mt_merge(clp4,clp4.sharpen(0.25).addgrain(2,0,0),maskdark,luma=true) \
: (addnoise) ? mt_merge(clp4,clp4.sharpen(0.25).addgrain(2,0,0),maskdark): clp4

I thing, this is the correct syntax to add grain in luma and chroma. Because U=2 and V=2 are defaults, unless luma=true override them.

# add grain to dark areas so encoders dont block them up on a low bitrate, also can give detail illusion
maskdark = (addnoise) ? clp4.mt_lut("x " + th_lo + " < 255 x " + th_hi + " > 0 x " + th_lo + " - " + th_hi + " " + th_lo + " - / 255 * ? ? "): nop()
clp5 = (addnoise && chro) ? mt_merge(clp4,clp4.sharpen(0.25).addgrain(2,0,0),maskdark, luma=true, U=3, V=3) \
: (addnoise) ? mt_merge(clp4,clp4.sharpen(0.25).addgrain(2,0,0),maskdark): clp4

[talen9]

Now I got it :)

Nothing like doing some division in a script and then not validating that the results are valid, doh! I've posted a new version via the link in post #2, it should fix that error and allow it to be mod4.


:thanks:

[McCauley]

Hi Spuds,

first i have to thank you for your hard work on this aweseome script, i tried some other MC denoise scripts, but none of them were so easy to use/understand like this, plus it's fast (okay, that was a lie, but its speed is pretty decent for MC stuff).

I tried it on some DV footage from my Canon XM2 and it looked great.

After all the praise, may i make some suggestions?
I had to examine this script a bit to filter my camcorder footage and found a few things that should be quite useful not only for me. :-)
Since this is becoming a total picture conversion script, i would like to propose some features to bloat it up a bit more. :-)


I dont quite understand the syntax of MVtools, nor the function in your script, but wouldn't it be useful to use only every second frame but the same amount of frames forwards and backwards for anime, because often frames are doubled?
If this is already integrated or doesn't make much sense for some reason i don't understand -> ignore it, i just had this idea while going through the documentation of MVtools.

To apply this script to cleaner sources, weaker modes would be very nice, since mode1 already kills to much detail on good sources like DVD, HD broadcast or disc streams. Additionally a strength=0 to deactivate the denoising would be nice, so one can only use the other useful functions in the script.

A dest_x/dest_y parameter, so it's not needed to call LSF anymore. This parameter should also be applicable to the ContraSharpening and if no sharpening is applied.(Others may not share my opinion, but i find Lanczos for upsizing and Spline36 for downsizing very pleasing.)

Is sharpening only for frames (or even better areas) with little motion like mentioned here (http://forum.doom9.org/showthread.php?t=135194&highlight=frames) already implemented? There is something written in the script, but as far as i understand it it's not what i mean. With the already determined motion vectors it should be possible to write a function that can do that, that should be a really efficient bitrate saver.

The sharpp parameter should be IMHO off at the smallest value e.g zero instead of 3.

From my own humble experience i found a deblocking/debanding function for very dark areas tremendously useful on most sources. A function like i posted here (http://forum.doom9.org/showpost.php?p=1096407&postcount=4) would be nice, since this is basically the same function like the addnoise in dark areas it should be easy to implement in the part of the script. A bool value and a strength value for the deblocking should be sufficient, the gradfun2db can be raised by 0.1 points for every five quant points, a separate value doesn't seem to be necessary to me. As defaults i suggest "darkdb(deblock=24, gradfun2db=1.2)" and "ddbquant(24)" for real life footage, for anime higher values should be usefual by default, but i'm not very familiar with that topic.

Similar to the function mentioned above, i would like to propose a function to apply stronger degrainig for very light areas, since grain is more disctracting in these areas and they normally contain less detail than areas with average brightness. A separate deblocking for these areas doesn't seem necessary to me.

An internal cropping parameter would necessary to use this as an AllInOneScript, since deblock_qed needs mod16 cropping in the begining to to find the blocks, This should apply to every output (only denoised, denoised/resized, denoised/resized/sharpened), so the picture don't has to be cropped/resized to the final size.

I also have some suggestions that are not directly related to the script:
You forgot to mention HQN3D as a necessary filter in the .html.
A version history with the changes in the .html file would be nice. This is just out of curiosity. Additionally links to the older versions would be nice, also out of curiosity.:)

Smaller jumps in the version numbers, i really don't want to you reach a 1.0, because i'm afraid you could take a break after that. :p

I hope all the suggestions e.g implementation wishes are not too impertinent, because i really appreciate you work on that script!:thanks:
I also hope it was not too much bla bla to read, but i wasn't able to compress my text more, i already denoised it with MC_Spuds. :p
I would like to help you more with code, but unfortunatelly my suggestions are the best i can do, since my scripting skills are not that great.

Regards
McCauley

[halsboss]

I tried it on some DV footage from my Canon XM2 and it looked great.

Hi, I want to apply this to my DV camera footage too. As I understand it this function processes progressive frames ... How did you (and I shall) deal with adapting it for PAL interlaced frames ? :thanks:

I note http://forum.doom9.org/showthread.php?p=1061218#post1061218 indicates something the code below should work, however I'm not sure of the effect of strong noise on TDEINT or YADIF. One of these should result in interlaced BFF output - any suggestions for a PAL BFF "strongly noised" DV camcorder input shot with a lot of low-light (dusk/night) scenes?


AVISource("somevideo.avi")
ConvertToYV12(interlaced=true)
AssumeBFF()
yadif(mode=1,order=0) # mode=0=deinterlace mode=1=doubleframerate, order=0=BFF order=1=TFF
mc_spuds(frames=4,strength=5)
AssumeBFF().SeparateFields().SelectEvery(4, 0, 3).Weave()



AVISource("somevideo.avi")
ConvertToYV12(interlaced=true)
AssumeBFF()
TDeint(mode=1,order=0) # mode=0=deinterlace mode=1=doubleframerate, order=0=BFF order=1=TFF
mc_spuds(frames=4,strength=5)
AssumeBFF().SeparateFields().SelectEvery(4, 0, 3).Weave()



AVISource("somevideo.avi")
ConvertToYV12(interlaced=true)
AssumeBFF()
interp = separatefields().eedi2(field=0)
tdeint(mode=1,order=0,edeint=interp) # mode=0=deinterlace mode=1=doubleframerate, order=0=BFF order=1=TFF
mc_spuds(frames=4,strength=5)
AssumeBFF().SeparateFields().SelectEvery(4, 0, 3).Weave()

[McCauley]

Hi halsboss,

i shot in the frame mode of the XM2 which outputs a progressive DV picture, so i unfortunatelly cannot help you, sorry!

Regards
McCauley

[Terranigma]

How did you (and I shall) deal with adapting it for the interlaced frames issue ? :thanks:
By first progressifying the frames through deinterlacing or restorative methods?

[halsboss]

Thanks Terranigma, in between posts I had edited that post http://forum.doom9.org/showthread.php?p=1110600#post1110600 with some example deinterlacing, seeking suggestions (not sure of the effect of strong noise on TDEINT or eedi2 or YADIF).

May I also ask what "restorative" means ?

[Spuds]

@Nikos, technically you are correct, the U=3,V=3 should be added, great catch. I do believe that the luma=true processing implies u=3,v=3 and I ran a short comparison of luma=true vs luma=true,u=w,v=3 and did not see a delta but ....


@McCauley .. wow thats a lot to digest! Let me try to address your points / requests as best I can.

but wouldn't it be useful to use only every second frame but the same amount of frames forwards and backwards for anime, because often frames are doubled?
You should not have a lot of duplicate frames unless you had and interlaced source that you deinterlaced while doubling the frame rate. Consider doing a same frame rate deinterlacing and feeding that to the script.

weaker modes would be very nice, since mode1 already kills to much detail on good sources like DVD, HD broadcast or disc streams. Additionally a strength=0
I'll add a strength=0 that is a low touch mode, thats easy. You can also specify things like thsad=100, ml=255 which would lower the noise removal quite a bit.

A dest_x/dest_y parameter I hear ya on that but there are just SO many ways to do that it would really need its own script to handle it properly. I've already got my hands full with the functions today !

Is sharpening only for frames (or even better areas) with little motion it has that today and its adjustable with the ml parameter. ml is the motion limit value, the higher the value the more something has to be moving to be seen as motion, the lower the less it has to move. ml=255 would basically sharpen everything while ml=0 would be similar to sharpp=3, ie nothing.

The sharpp parameter should be IMHO off at the smallest value e.g zero instead of 3 Purist ! .. I'll add that in the next rev :) sharpp=3 will also remain to keep compatibility.

deblocking/debanding I'll have to spend some time on these points. The script has the light/dark functionality today so implementing the concepts is not difficult, but will not do anything to improve speed either :) I do like the ideas so I'm going to spend some time on this to see what I can come up with.

Yeah the revisions, history and required parts are well lacking to say the least. Never thought this would be a 'project' :p

[Terranigma]

May I also ask what "restorative" means ?

It means to return in it's original state.
In this case, i'd be talking about through ivtc.

[zilog jones]

This is a very impressive script - it's working very well on anything noisy I throw at it (analogue cable recordings, old VHS tapes etc.). Good stuff!

However, it seems to be cropping every side by 8 pixels for some reason. Line 161 seems to be doing this - is the crop function supposed to be there? I'm using the latest version from your site (updated on the 16th) and I followed the instructions regarding which plugins to use (I think I have everything). I'm getting no errors. I've tried both 768x576 and 352x240 videos and it's cropping both in the same way.

[Spuds]

@zilog jones

I *think* you caught me in the middle of an update that caused that problem. Please download the script again, I just finished uploading it to the link in post #2 as well as the wiki.

Latest version 0.9, so close to 1.0 :)

Version 0.9 'finalizes' the changes from the last few betas. The full list of changes are:
V0.9
1. Added temporal=true which uses the temporal de-graining limit from Didee. Since the script already had 95% of the data needed to do this it seemed like a natural extension to include.
2. Prefiltering now follows the strength parameter, this will speed up processing for lower strengths.
3. Added premax=true which will turn on full strength pre-filtering of the clip for motion vector searches (overrides strength settings on pre filtering levels).
4. Added ability to specify the sharp parameter to mvanalyze via sharp = (0-2)
5. Added strength=0 (actually redefined) so its very low touch for higher quality input videos
6. Updated documentation so that sharpp=0 is no sharpening
7. Fixed mod4 error in resize logic
8. Deringing strength now follows the strength parameter. Changed deringing from simple bilinear blur to fft3d for non anime sources.
9. Added mod16 blank borders to improve edge processing and keep clip mod16 for processing. Borders are removed at the end to return the original sized clip.

Enjoy and as always le me know what I broke this time so I can get 0.9a uploaded :p

[zilog jones]

Seems to be working OK now - thanks!
It is quite slow, but it seems to do a better job than what I was using before (RemoveHighNoiseMC) - it's apparently less destructive regarding removing small shiny things appearing for a short time (i.e. things incorrectly removed/moved for being noise). Lots of settings to mess around with too!

[archaeo]

Can MC_spuds be used with MT such as:

MT("MC_spuds") ?

I did a couple of quick short clip tests using this, and there didn't appear to be any problems, (it increased fps significantly) but I'm not sure if it would eventually run into problems.

[canuckerfan]

might be a bit of an obvious question but how well does this retain detail as opposed to mvdegrain3? (i'd imagine it's the same since mc_spuds is also MC'd)

[Spuds]

@archaeo .. I run it with setmtmode 2 and have not seen any issues as long as you are using the latest version of mvtools. I get a nice speedup as well. I'm not as sure about running it with MT() and my hesitation is how the overall block and overlap processing occurs on the seam that MT() creates. May be fine I'm just not sure :confused: I'd look through the MT thread and see what others have determined about dfttest, fft3dfilter and mvtools.

@canuckerfan .. Well if you call it with frames=3 it will use mvdegrain3 as part of its processing, the strength of it is set via strength=0-6. Lots and lots of options, check the docs for the complete list and set any special modes based on your video and personal preferences of what looks good.

[Malcolm]

@Spuds,
i just switched from BETA 29.Jan.08 to your latest version 0.9 from 17.Mar.08 and noticed, that the new version seems to reduce noise (esp. color noise) _much less_ than the january version.

Here are 2 screenshots:
First one with the january version (left: unprocessed frame, middle: mcspuds, right: difference 16x amplified)
Second one with the current version 0.9 (left: unprocessed frame, middle: mcspuds, right: difference 16x amplified)
http://img379.imageshack.us/img379/2706/mcspudsjanhq6.th.jpg (http://img379.imageshack.us/my.php?image=mcspudsjanhq6.jpg)
http://img135.imageshack.us/img135/6184/mcspudsmarchmp4.th.jpg (http://img135.imageshack.us/my.php?image=mcspudsmarchmp4.jpg)

I have other clips (but not available at the moment) where the difference between the 2 versions of mcspuds is even bigger. I compared the two scripts using a diff tool but there are so many changes i'm unable to judge why the visual differences are so huge. Maybe you know..?!

I have used mcspuds with the following parameters:
MC_Spuds(strength=6, frames=3, flow=false, focus=false, preprocess=true, postprocess=true)
Altering parameters like 'flow' and 'temporal' only changed the results marginally.

Is it by intention that the new version of mcspuds filters (so much) less?? Or is it a fault?? How can i get the same amount of filtering with the new version of mcspuds?

greetings,
Malcolm

[Spuds]

@Malcolm

That source looks as bad as what I get off my cable and thats a fact!

I think this issue was a consequence of me tunning the chro=true vs false behavior ... basically increasing how rigorous I was about those intended behaviors.

I believe I know what causes what you demonstrated so well in your post (thank you!). I have posted a .9a Beta for testing in post number 2, please try that version and let me know if it corrects the issue.

[Malcolm]

@Spuds

Great!
.9a brings back the filtering of this ugly color noise. :) Here's an updated screenshot with the new script.
http://img156.imageshack.us/img156/5031/mcspudsmarch9aiw9.th.jpg (http://img156.imageshack.us/my.php?image=mcspudsmarch9aiw9.jpg)

If you compare it with the first screenshot from my last post, you can see that the large color bandings (violet / green) are removed again (look at the amplified difference). Thanks a lot! :thanks:

On the other hand: if you look at the luma values & finer details in the amplified part, you can also see that with the .9a version there is less 'stuff' removed from the original than with the january version. I'm uncertain if your new script removes less noise or just preserves more details/edges/etc.
(actually you can already see in the screenshots that the new version preserves more details and is also sharper than the old version). But still the question remains: Is it still as effective to noise than before?

Just to get me right: I don't want to criticize your changes in the script, i just want to understand how the behaviour changed. Maybe this is all by intention.

Greetings,
Malcolm

P.S. The screenshots are from very old (10+ years) VHS captures of a famous (or 'cult') german comedy show ('Oliver Kalkofe') which i try restore & preserve on DVD. Actually if you could see a clip instead of just a single screenshot you'd be shocked how bad the flickering colors, noise all over the screen and moving radio interference is. But thanks to your ingenious script, i can fight it effectively! ;)

[archaeo]

@archaeo .. I run it with setmtmode 2 and have not seen any issues as long as you are using the latest version of mvtools. I get a nice speedup as well. I'm not as sure about running it with MT() and my hesitation is how the overall block and overlap processing occurs on the seam that MT() creates. May be fine I'm just not sure :confused:

Just a quick update:
I went ahead and ran a full encode with MT("MC_spuds"), and found that it did create problems with CCE... I noticed it hung up the program a couple of times over the course of the encode. It may be due to a moderate overclock that I have on my E6700 (OC'd to 3.0Ghz), but I'm not so sure. After finishing, I noticed that the playback would freeze at a number of locations. At this point, although it seems to run, it appears that setmtmode is the most reliable method to use MT.

[ChrisW77]

I can't get setmode to actually do anything, whereas MT pushed my core2 to it's limits.

For instance

SetMTMode(2,4)
MC_Spuds(strength=4)

stays around the same 3-4 fps I'm getting without SetMT.

Yet

MT("MC_Spuds(strength=4)",2,2)

I get around 8-10 fps. But, I notice a lot of random dots appearing at various times, and the odd corruption now and then.
Still, great script.

[archaeo]

...I notice a lot of random dots appearing at various times...



Funny you mention that, I have also noticed something similar too ... Do they appear to you as sort of white pixel flashes? They would be what I would describe as a 'sparkle'. I originally thought they may be on the source, but a quick check didn't show them. I'd have to run another encode to see if I can repeat it. I may have even seen this on an encode where I did not use MT("MC_spuds").

[ChrisW77]

Do they appear to you as sort of white pixel flashes? They would be what I would describe as a 'sparkle'

Kind of, although most are just random pixels across the screen. A bit like lots of dots, some light coloured, some grey-ish.

[Spuds]

@Malcolm .. I've uploaded a .9b to the link in post number 2. To answer your questions, yes the latest version I intended to protect details more than the previous versions for lower strength settings.

That said I noted two issues that were causing diminished processing strength that were not intended. One was I used the number of frames instead of the strength value when setting up the prefilter, this left its effect weaker then it should have been. The other issue was well :o I basically nullified the prefilter effect on the motion search. The new version should be much more effective!

@archaeo .. Interesting finding on CCE, sounds like one of the filters is not playing nice with MT processing. Overall I would really expect some form of distortion / artifacts (like a entire bad frame now and then or odd seam in the middle of a frame) vs a stuttering or freezing.

I've run lots of setmtmode runs and they have been artifact free (since mvtools 1.9.1 I think), earlier versions of mvtools and MT did not play nice together.

On thing with setmtmode is that you need to set it as the first line in your script, prior to any processing or loading of videos, or you will not get any speedup. I also tend to go to mode 5 prior to loading a video or any changes in colorspace and then back to mode 2, for example

setmtmode(2,4)
Import("mc_spuds.avs")
setmemorymax(256)
setmtmode(5)
AVISource("some video")
ConvertToYv12()
setmtmode(2,4)
mc_spuds(frames=3,strength=4)

[ChrisW77]

Using what Spuds posted, This worked well, and so far I haven't seen any artifacts.

SetMTMode(2)
Import("C:\Program Files\AviSynth\plugins\MC_Spuds.avs")
SetMemoryMax(256)
SetMTMode(5)
AviSource("D:\VHS\test1.avi")
ConvertToYV12(interlaced=false)
SetMTMode(2)
mc_spuds(frames=3,strength=4,sharpp=2)

It seems, at least with the VHS material I'm working with, using only 'Mode' values, and not 'Thread' values, I get no problems and great speeds (now seeing 11-12 fps on 720x576 caps).
Using threads like SetMTMode(2,4), there are instances of overlaid pink or green garbage, and even white noise in the sound.

[elguaxo]

A really great noise removal script! thanks Spuds!!

Now I'm using this on a source that was previosly resized to 640x480

mc_spuds(frames=3,strength=5,debug=4)

All denoising parameters seem to work as expected, but I decided to try the debug option and the result looks like this:

http://img182.imageshack.us/img182/2632/debugzt0.th.png (http://img182.imageshack.us/img182/2632/debugzt0.png)

it's the first frame, which is black, but all the debugging text has been cropped. I'm using this version of your script:
# MC_Spuds()
# Motion compensated noise removal with sharpening -- version: 0.9c BETA - 23.Mar.08

Any hints? Thanks in advance!

[Spuds]

@elguaxo .. Thanks for complements on the script :)

Regarding the cropping of the debug info, yup its a bug that got introduced in the .9 version. I have a 0.9d that has a couple of small updates just about ready and I'll add a fix for the error you found. Just give me a couple of days!

[McCauley]

Hi Spuds,

before you post the 0.9d could you please send me 0.8 via PM, i deleted it accidently. With 0.9c colours are oversaturated after denoising (it looks like sat=1.05 or something like that). I'll see if can trace it back to the latest revision.


Regards
McCauley

PS: I don't know if you remember it, but i suggested strong deblocking for dark areas, here (http://forum.doom9.org/showthread.php?t=135739)'s what i came up with. maybe it is of use for your script, or you want to make some suggestions how to improve the function.

[Spuds]

@McCauley,

You can get the old versions of the script at the wiki (http://avisynth.org/mediawiki/Image:MC_Spuds.avs)
The .8 was the 2-28 one, the various versions of .9 are also available there. Please let me know what you find on the color saturation, seems odd.

I'll take a look at the stronger deblocking in dark areas.

Thanks

[salawalas]

Beautiful job, Spuds! Thank You.

[Infrid]

hi,
great script spuds, just what i need, but i don't understand if your script can handle an interlaced source. i have made some test and your nise remover looks better.

http://img86.imageshack.us/img86/4512/testmcnd4.jpg

the last question, i got 1pfs with virtualdub mod, is it normal? (with MT at last 3fps)

here my script, is for PAL VHS, i keep the movie interlaced

LoadPlugin("fft3dfilter.dll")
LoadPlugin("deblock.dll")
LoadPlugin("mt_masktools-26.dll")
LoadPlugin("mvtools.dll")
#LoadPlugin("RemoveDirt.dll")
LoadPlugin("RemoveDirtSSE2.dll")
#LoadPlugin("RemoveGrain.dll")
LoadPlugin("RemoveGrainSSE3.dll")
LoadPlugin("TTempSmooth.dll")
LoadPlugin("RepairSSE3.dll")
loadplugin("ChromaShift.dll")

SetMTMode(2)
import("mc_spuds.avs")

SetMTMode(5)

Avisource("rca_oro.avi")
trim(11861, 14650)
ConvertToYV12(interlaced=true)

ChromaShift(C=2, L=-2)

#FFT3Dfilter(plane = 0,bw=64, bh=64, ow = 32, oh = 32,bt=3, sigma=3.5, sharpen = 0.5)
SetMTMode(2)
mc_spuds(frames=2,strength=4)


thanks
Infrid

[elguaxo]

i don't understand if your script can handle an interlaced source
you must deinterlace before applying mc_spuds

the last question, i got 1pfs with virtualdub mod, is it normal? (with MT at last 3fps)

yes :p

[Spuds]

@Infrid

As elguaxo pointed out, you must deinterlace before your run the script. The basic reason for this is that when you consider a frame of video the denoiser expects that the frame is a single moment in time, ie all pixels are spatially related. An interlaced source is a interleave of two moments in time, so the pixels of an interlaced frame are temporally related which is a problem. The script will run on this but the results are not optimal.

re: speed ... yeah motion compensated stuff is slow, and just wait till you deinterlace first ! I will say that 3fps on a dual core for frames=2 seems a bit weak, I would get 4.x with the call you have on my laptop, but processor speed, memory, other programs, etc etc etc

One last thing, do not use RepairSSE3.dll, it has some issues with color corruption, just use plain old Repair.dll

[Infrid]

thanks guys.

Spuds, i have a strange problem, this is my script (denoiser.avs)


LoadPlugin("fft3dfilter.dll")
LoadPlugin("deblock.dll")
LoadPlugin("mt_masktools-26.dll")
LoadPlugin("mvtools.dll")
LoadPlugin("RemoveDirtSSE2.dll")
#LoadPlugin("RemoveGrain.dll")
LoadPlugin("RemoveGrainSSE3.dll")
LoadPlugin("TTempSmooth.dll")
LoadPlugin("Repair.dll")
loadplugin("ChromaShift.dll")
loadplugin("decomb.dll")

SetMTMode(2)
import("mc_spuds.avs")
SetMTMode(5)
Avisource("L:\primi_dinosauri.avi")
ConvertToYV12(interlaced=true)

ChromaShift(C=2, L=-2)

SeparateFields()
even_frame = SelectEven()
odd_frame = SelectOdd()

SetMTMode(4)
even_frame = mc_spuds(even_frame,frames=2,strength=4)
odd_frame = mc_spuds(odd_frame,frames=2,strength=4)

Interleave(even_frame, odd_frame)
Weave()


when i change strength=2 i get this error, why? :D

http://img167.imageshack.us/img167/2797/errornk7.png

i have an intel dual core E4500, and i still get 2.0fps even with this script that should be slower

[45tripp]

check mc_spuds dependencies,

unfilter:
http://www6.impacthosting.com/trbarry/downloads.htm

[Infrid]

right, it's my fault, sorry

from the README
unfilter: Ver 0.0.1.5 Unfilter

[McCauley]

Hi Spuds,

i tried getting the 0.8b version from the wiki, but when i go back to march 28 the link to the avs still points to the latest revision.
I think understand what is causing the (over-)saturation, the denoising seems to replace noise pixels with pixels of the same colour of their neighbours, so the overall saturation increases, because the "grey" noise is removed and the whole area becomes more intense in his colours. I don't know if this description is an accurate one to describe the phenomenon.
I'll make some screens and will send you a sample in a few days.
I have no idea to counter that, everything i have in mind will slow down the script more.

As we're talking of slowing the script, i have some ideas to improve the computed output. :)
Since this script is oftne used to restore old movies we have to fight the jitter that i caused by loose reels during the film scanning process, this messes up the motion vector search and will worsen the noise removal.
You maybe want have a look at this (http://forum.doom9.org/showpost.php?p=1114432&postcount=2) post from g-force, it helped me to stablize my source very effieciently.

Another thing is the deblocking.
Since we already computed a motion and a static mask,why not use stronger deblocking for the motion mask? Areas with motion tend to be blockier than areas with low motion (guess why :) )
Maybe a third "high" motion mask for the stronger deblocking would be appropiate?!

As always looking for forward what you will come up with. :-)

Regards
McCauley

[talen9]

Hi Spuds,

i tried getting the 0.8b version from the wiki, but when i go back to march 28 the link to the avs still points to the latest revision.

Ahem ... http://avisynth.org/mediawiki/upload/archive/2/20/20080318015002%21MC_Spuds.avs ;)

You just have to click on the link marked "15:15, 28 February 2008" in the wiki page linked in the previous post from Spuds :)

[Spuds]

@Infrid Looks like you got your answer on the unfilter error. A couple of comments on your script. First using setmtmode(4) before mc_spuds is sure to slow it down, use setmtmode(2) instead. Also I'd recommend doing a bob on the script vs the even/odd processing you have ... I'd do something like

SetMTMode(2)
import("mc_spuds.avs")
SetMTMode(5)
Avisource("L:\primi_dinosauri.avi")
ConvertToYV12(interlaced=true)
ChromaShift(C=2, L=-2)
SetMTMode(2)
Assume?ff # set to tff or bff depending on what you have
bob()
mc_spuds(frames=2,strength=4)
Assume?ff() #set the fieldorder again (must be the SAME as above!)
separatefields().selectevery(4,0,3).weave()

[2Bdecided]

bob(0.0, 1.0) can be better than bob()

IIRC there is no requirement for the second assume?ff to match the first - you can happily swap the field order at this point if you want to. However, the quality will be reduced, since all the "kept" lines will be interpolated, unless you use a better bobber (e.g. mcbob) - in which case it's a perfectly OK thing to do (if you want to!).

Unless I've misunderstood?

Cheers,
David.

[McCauley]

Ahem ... http://avisynth.org/mediawiki/upload/archive/2/20/20080318015002%21MC_Spuds.avs ;)
Shame on me...:rolleyes:

Regards
McCauley

[Infrid]

thanks spuds, but i lose detail with your script, download this file and look yourself.

http://mihd.net/jpncdg6
file list:
[sample_original.avi] the original video with noise
[sample_denoised.avi] noise removed, strength=3 *
[spuds_script.avi] processed with your script (bob), strength=3

i'd like to keep an interlaced video without a deinterlacing process (like bob), because i want to keep the maximum detail as possible.

i will not use MT anymore, after 9 hours of encoding there are some nice green frames and withe noise in the aiudio track... :D

used ChromaShift(C= 4, V=-2, L=-2, U=-2 ) and FixChromaBleeding()

[Spuds]

@2Bdecided
I'll probably get the wording wrong here but after something runs through bob avisynth resets the parity flag. So if you did a assumetff.bob.info it would say bottom field video.

After bobbing its progressive so the parity is irrelevant so the flag is reset (my guess). Bob is not the only deinterlacer that will do this, so do some don't. In this case, especially with dumb bob, we want the original lines and not the reconstructed lines, hence the need to state it again (well for a tff video).

@Infrid
I'm a bit surprised by this result. Bob is a relatively dumb filter, using it only creates a double frame rate, very fluid video to denoise.

After denoising we reconstructed the interlaced video with the original and now denoised data and discarded the bob data. As the above discusses you must set and reset the correct field order, in your case its tff (based on the video you uploaded), if you get it wrong you could get the bob data and thats not what you want.

If you already did all this correctly then choose what your eyes like, in the end thats whats most important.

[2Bdecided]

@2Bdecided
I'll probably get the wording wrong here but after something runs through bob avisynth resets the parity flag. So if you did a assumetff.bob.info it would say bottom field video.

After bobbing its progressive so the parity is irrelevant so the flag is reset (my guess). Bob is not the only deinterlacer that will do this, so do some don't. In this case, especially with dumb bob, we want the original lines and not the reconstructed lines, hence the need to state it again (well for a tff video).Yes, I know - I didn't disagree with you in my previous post. I was just pointing out that if you want to swap parity, here is a place you can do it, if the bob is a good one. Otherwise, of course not (like I said).

btw, I don't think I've ever said thank you for pulling together such a great script - thank you!

Cheers,
David.

[Spuds]

@David,

I figured you knew that, looking back I just misread your post a bit, thought it was stating a no need for the second vs what it really says of no need for it to match. Doh!
-----------------

I have made some updates to ver 0.9d, consider this a beta and as so its only on the link in post number 2 and not released to the wiki just yet.

The changes in this update are as follows:

Fixed error in the debug output, it was cropping the top line
Improved the overall output of the parameters debug view (debug=1). Many more are now listed.
Moved the location of the colorbleed function to post mv-denoise vs pre mv-denoise
Added parameters to allow setting of additional mvanalyse options, including truemotion (bool), lamda (int), lsad (int), plevel (0,1) and mvglobal (bool)
Added fs1,fs2,fs3,fs4 parameters so tweakers can pass sigma values to the flow function instead of using the scripts values. For flow and frames>2 only fs1 is required.
Increased search radius of contra-sharp for higher strength settings.
Added new prefast option to speed up processing, prefast=true/false. This is a alternative faster function to clean the video for motion vector searches. Not as robust but adequate for many sources. (beta)
Changed edgeclean function from using fft3dfilter to hqdn3d, uses same mask. (beta) This increases speed but may not be as effective.


I'm considering making the prefast=true the default for strength values 0,1,2 and maybe 3. The edge cleaning is a test, the current function worked well but speed increases could be found with other functions hence the beta test.

[Vesi]

http://img329.imageshack.us/img329/8205/53271555gx2.png (http://imageshack.us)

I'm getting color problems with latest Script
mc_spuds(frames=2,strength=1)

Edit: And I found that, the problem exist in beta V with .9 everything is ok

[Spuds]

@vesi

Was this the version on the wiki or the version from the link in post number 2 (the beta).
If the beta what is the date in the script header? There was a version that had the color processing wrong, although with the call you made you should not have seen that.
If its the version from the wiki make sure you did not get repairsse# installed, only use repair.
If none of the above then you know the drill, the script and short clip with the error please.

Thanks!

[Vesi]

version from the post number 2. And I use repair.
Date 26.04.08 14:34

Edit: I will send you the sample along with script and test result soon by pm.

[Spuds]

@Vesi

I updated the file in post number 2, this should take care of the color issue you were seeing. The defect was interesting, it was basically a few color blocks from forward looking frames coming through into the current frame, occurring 1-2 frames ahead of a scene change.

Frankly I was cheating things a bit and using luma motion vectors on all the planes for mvdenoise (speed but not really proper) and this tripped up the scene change detection. I could eliminate your example by lowering thscd2 to 90 but in the end decided to change the processing to be chroma proper.

Give the new script a try and let me know what you think.

[Vesi]

Everything is fine now Spuds. And thanks for updating the script.

[Avenger007]

First, let me say thanks a lot for an impressive noise removal script. :)

Second, an observation about debug view...
In function pre_filter, bt and ov are:
bt = (premax && frames > 1) ? 3 : (premax) ? 3 : (strength < 5) ? select(frames , 1 , 1 , 1 , 2 , 2) : select(frames , 1 ,1 , 2 , 2 , 3)
ov = (premax) ? 8 : select(strength , 2 , 4 , 4 , 4 , 8 , 8 , 8)
But in function debug_view, bt and ov are redefined:
bt = (premax && frames > 1) ? 5 : (premax) ? 3 : (strength < 4) ? select(frames , 0 , 1 , 2 , 3 , 3) : select(frames , 0 ,2 , 3 , 4 , 5)
ov = (premax) ? 8 : select(strength , 2 , 4 , 4 , 4 , 8 , 8 , 8)
So when I experimented with bt and ov (like bt=3, ov=8) in pre_filter, the debug view list remained the same. It's no big deal really, I just thought I would point it out. ;)

Third, is dering_clp still being tested?
Is there an updated version for it or for the MC_Spuds script in general?

:thanks: again for a great script.

[Spuds]

Second, an observation about debug view...
In function pre_filter, bt and ov are:

Good catch! ... The way I'm doing that in debug_view was just lazy on my part, bt and ov should be passed in the functions or made global but not defined twice ... but when it came to the debug view function I was kind of lazy :)


is dering_clp still being tested Yes and no ... overall I'm satisfied with the processing in the last beta (post #2 and I think its the same as the last official on the wiki but would have to double check) and I'm satisfied with the edge mask creation. In the script you some of the ways I've worked on the edges, but like everything its a speed, effectiveness, detail retention tradeoff.

All of those deringing noise functions are applied via an edge mask which is just as important as the denoising function itself. You will get more deringing by changing the edm_lo and edm_hi values (use debug views to see how those add or removed edges to the mask).

All that said I'm always looking for and trying new things.

[Dreassica]

Using starfield = true in 0.9d results in an: The named argument "threshhold' to mt_binarize had the wrong type, line 104 and line 524 in mcspuds avs.

[Spuds]

Dreassica .. thanks for the bug report in 0.9d, looks like you had starfield and anime set to true. I placed a new beta version at the link in post#2, its still 0.9d but now has todays date in the header.

When using starfield remember to do some tunning of the thstar and lumathres values.

lumathres will allow more spots to be considered in the mask, lower includes more spots (don't go below 130). Its basically the luma value of the stars in question, if they are all very bright increase the value to avoid false hits.

thstar will look at the mask that is created with the lumathres value and decide if there are enough spots that what we have is indeed a starfield and not just a few spots or dots that are really noise.

use debug=9 to see the starmask to help in tunning. I have used starmask to retain stars, bubbles, plankton, and lots of other swarming fields in video that would have normally been removed or diminished by tunning the values.

[hartford]

You should add something like this to your doc:

# requires: addgrain v1.4
# awarpsharp
# despot v3.4.1
# dfttest v1.1
# fft3dfilter v2.1.1
# hqdn3d v0.11
# mt_masktools v2.0.32
# mvtools v1.9.2
# removedirt v0.9
# removegrain v1.0
# removegrainT v1.0
# Repair v1.0.dll
# undot v0.0.1.1
# unfilter v0.0.1.5
# warpsharp 03Nov03.dll

# Scripts:
#
# deblock_qed.avs
# LimitedSharpenFaster.avs



Xsharpen is in the old warpsharp as dated above.
Clense is in the latest RemoveGrainT from Kassandro

<a href="http://home.arcor.de/kassandro/prerelease/RemoveGrainT-1.0.rar
">RemovegrainT</a>

<a href="http://avisynth.org/warpenterprises/files/warpsharppackage_25_dll_20031103.zip">Old Warpsharp</a>

[McCauley]

Hi Spuds,

sorry, i forgot to give an update to my last post. The saturation thingy was caused by another function i really didn't take
into account (and i still don't understand why it does it...).

May i make a few suggestions again? :D

How about implementing MVDegrain4/5/6 (http://forum.doom9.org/showthread.php?p=1141506#post1141506)?

While you're at it you could implement , as far as i understand it, the speed impact should be almost zero with everything already
computed. MoComp'ed ContraSharpening (http://forum.doom9.org/showthread.php?p=1104251#post1104251)

Since this script should be used most of the time to denoise/degrain older movies, why not integrate a real scratch removal function? It already does a
decent job on scratch removal, but i want it to be better. :devil:

How about offering support for fft3dGPU like in UDegrain (http://forum.doom9.org/showthread.php?t=139766)?

Another cosmetic change i want to suggest would be to change prefilter,prefast and premax to a single prefilter call with integer values:
0 off
1 prefast
2 strength based
3 premax
That would make the usage a bit prettier IMHO.


Looking forward to further development of your outstanding script.

Regards
McCauley

[45tripp]

While you're at it you could implement , as far as i understand it, the speed impact should be almost zero with everything already
computed. MoComp'ed ContraSharpening (http://forum.doom9.org/showthread.php?p=1104251#post1104251)

+1


How about offering support for fft3dGPU like in UDegrain (http://forum.doom9.org/showthread.php?t=139766)?


+1


ty
tripp

[Spuds]

Had a bit of free time so decided to play around a bit and add some of the features based on the latest suggestions. The latest 1.0 Beta is available in the link (http://www.rcboataholic.com/MC_Spuds.zip) in post number 2. Consider it a test version to check things out and let me know if there are any problems.

The changes are:
1. Added mvdegrain4 capability for frames=4. Could have added more but since there are also with temporal and aggressive processing options the script has enough noise removal power.

2. Added gpu=true which will utilize fft3dgpu in place of fft3dfilter throughout the script.

3. Added prefilter = 0-4, 0-none, 1-fast, 2-strength based, 3-max. The old premax, prefast, etc also are still available.

4. Added third sharpening option, contra with MC determined limits. Use via sharpp=3 (0=none, 1=contra, 2=limitedsharpenfaster, 3=MC contra). Have not had enough time to fully test this so give it a try and let me know how you like the results.

5. Fixed / added some debug information in the debug=1 screen

[elguaxo]

thanks!

[McCauley]

Awesome!
Thank you very much!

Regards
McCauley

[Lux Delux]

Wooo your effort is greatly appreciated! This scripty does wonders on Chuck Norris :D

[Lux Delux]

Hmmm I just noticed sharpp=3 introduces 2px white borders around my image. Any ideas? :eek:

EDIT: actually it seems it appears with removeblocks=true. Even with sharpp=1 I got the lines but then they were sort of grayish. Removing removeblocks made it dissapear with sharpp=1 but the problem is still there it seems with 3.

[g-force]

Spuds,

your MCcontra routine is totally hosed. You're applying the clamp to the difference signal, it should be applied to the un-differenced end result.

something like:

pshrp = last
pmax = source.MT_Logic(bw1,"max").MT_Logic(fw1,"max")
pmin = source.MT_Logic(bw1,"min").MT_Logic(fw1,"min")
rg11d = MT_MakeDiff(pshrp,pshrp.RemoveGrain(12,-1))
MT_MakeDiff(pshrp,pshrp.RemoveGrain(4,-1))
MT_LUTxy(rg11d,last,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?")
MT_MakeDiff(pshrp,last)
MT_MakeDiff(last,last.RemoveGrain(12,-1))
MT_LUTxy(pshrp,last,"y 128 - 1.5 * x +",chroma="copy first")
MT_Clamp(last,pmax,pmin,1,1,chroma="copy first") #or better yet, have a variable overshoot amount

-G

[Didée]

Yeah, the "MCsharpening()" routine is a bit off. Clamping the diff instead of the result, as g-force already found ... one or two mt_LutXY not making much sense ... oh, and a closing parenthesis at the wrong place, which in effect reduces the sharpening strength for strength >=4, where it should increase. Indeed, just a little bit broken. ;)

Corrected version:

FUNCTION MCSharpening(clip denoised, clip original, int frames, int strength, bool flow, int thsad)
{
# apply sharpening to the denoised clip
# sharp = yv12lutxy( denoised, denoised.minblur(1,1).removegrain(11,-1), "x y = x x x y - abs 16.25 / 0.25 ^ 16.25 * 1.5 * x y - 2 ^ x y - 2 ^ 4.5 + / * x y - x y - abs / * 1 x y - abs 20 / 4 ^ + / + ?",U=2,V=2)
s = denoised.minblur(1,1) # Damp down remaining spots of the denoised clip.

# The difference of a simple kernel blur. Use a larger radius for stronger strength values
ssD = (strength < 4) ? mt_makediff(s,s.removegrain(11,-1)) \
: mt_makediff(s,s.removegrain(11,-1).removegrain(20,-1))

# determine our clamping limits based of our original clip and a simple MVcompensate
global idx_pointer = (flow) ? idx_pointer : idx_pointer + 1
comp_bw1 = (flow) ? sv_cb1 : original.MVCompensate(sv_b1,thsad=thsad,idx=idx_pointer)
comp_fw1 = (flow) ? sv_cf1 : original.MVCompensate(sv_f1,thsad=thsad,idx=idx_pointer)
pmax = original.mt_logic(comp_bw1,"max").mt_logic(comp_fw1,"max")
pmin = original.mt_logic(comp_bw1,"min").mt_logic(comp_fw1,"min")

denoised.mt_adddiff(ssD,U=2,V=2) # Apply the blur difference. (Sharpening is just inverse blurring.)
last.mt_clamp(pmax,pmin,0,0,U=2,V=2) # limit the sharpened clip to not exceed the original temporal neighborhood

RETURN (last)
}

[Lux Delux]

Great, no more problems with sharpp=3 now :)

And I like it btw, I'm starting to use mcspuds as a all in one denoiser/sharpener/deblocker :D

[Seraphic-]

Wanted to try this out, but I'm having a problem as it keeps crashing VirtualDub.
Could anyone show me what is wrong with my code?
To be clear, my source isn't Anime, but 3D rendered graphics.

On a side note, could this be used to denoise, sharpen and/or smooth along with helping with anti-aliasing to remove jagged edges?

AviSource("C:\Documents and Settings\Desktop\test.avi")
ConvertToYV12()

import("C:\Documents and Settings\Desktop\AVISynth\Filters\avs\mc_spuds.avs")
import("C:\Documents and Settings\Desktop\AVISynth\Filters\avs\LimitedSharpenFaster.avs")
import("C:\Documents and Settings\Desktop\AVISynth\Filters\avs\FastLineDarken_1.3.avs")

LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\mt_masktools-26.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\RemoveGrainSSE3.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\deblock.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\FFT3DFilter.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\mvtools.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\RemoveDirtSSE2.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\TTempSmooth.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\hqdn3d.dll")
LoadPlugin("C:\Documents and Settings\Desktop\AVISynth\Filters\dll\RepairSSE3.dll")

mc_spuds(frames=2,strength=4)
ttempsmooth()
fastlinedarkenmod(thinning=0, strength=25)
limitedsharpenfaster(ss_x=1.0, ss_y=1.0, smode=4, strength=20)

[McCauley]

Hey Spuds,

maybe worth a try?
MC'ed Soothe (http://forum.doom9.org/showthread.php?p=1166238#post1166238).

What do you say to MV based scratch removal?

Regards
McCauley

[Lux Delux]

@Seraphic, what Mvtools version you have? I know it crashed my vdub also when I had the new multi-threaded build. With the older one it worked fine.

[talen9]

@Seraphic, avoid using *SSE3 version of Kassandro's plugins: their presence in autoload folders or in explicit "LoadPlugin()" instructions has been proved in the past to be a source of crashes/problems .... use the *SSE2 versions (they're almost as fast), delete the *SSE3 from your system and try again ;)

[Didée]

maybe worth a try?
MC'ed Soothe (http://forum.doom9.org/showthread.php?p=1166238#post1166238).
Generally, not a bad idea. ;)
However: Within a script that does MC noise reduction, doing an explicit MC-Soothe either adds unnecessary processing overhead, and/or is suboptimally efficient. Within an MC'ed noise reduction script, I prefer to do that step inherently :

http://img181.imageshack.us/img181/4940/prob2piccd1.th.jpg (http://img181.imageshack.us/my.php?image=prob2piccd1.jpg) http://img440.imageshack.us/img440/3530/prob1picww0.th.jpg (http://img440.imageshack.us/my.php?image=prob1picww0.jpg)

http://img261.imageshack.us/img261/7952/ok2picnm3.th.jpg (http://img261.imageshack.us/my.php?image=ok2picnm3.jpg) http://img181.imageshack.us/img181/8733/ok1picaj4.th.jpg (http://img181.imageshack.us/my.php?image=ok1picaj4.jpg)

http://img181.imageshack.us/img181/716/hmm02picpi7.th.jpg (http://img181.imageshack.us/my.php?image=hmm02picpi7.jpg) http://img410.imageshack.us/img410/2915/hmm01picvi8.th.jpg (http://img410.imageshack.us/my.php?image=hmm01picvi8.jpg)

Just re-mixing the same old methods over and over again is a bit of yawn. We've like twenty different MC denoisers, and ALL of them have to make the same decision of compromise at some certain points. Above, MC_spuds is just a random example. Exchange it with TemporalDegrain, GTDegrain, "Ultimate"Degrain, or whatever ... the result looks more or less the same.

The old vegetable soup tastes vapid by now. Time to use new spicery.

[Boulder]

The old vegetable soup tastes vapid by now. Time to use new spicery.Chef Didée, would you like to show us the spice you apparently discovered on your vacation? :D

[Nikos]

Thanks Didie for the MC'ed Soothe. Here (http://forum.doom9.org/showpost.php?p=992576&postcount=446) is one of your old post about soothe:

In "standalone" Soothe, in a temporally mood I had decided to make the parameter "keep", i.e. how much of the original sharpening to keep at least. Hence, 0=maximum soothing (no minimum sharpening to keep), 100=minimum soothing (keep all sharpening).

When building SeeSaw, I found it's a bit unintuitive this way, and reversed the meaning to "strength-of-soothing". Hence, in SeeSaw, sootheT=0 means minimum soothing (do no soothing, keep all sharpening), sootheT=100 means maximum soothing (no minimum sharpening to keep).

The meaning of "reduc" in the MC'ed Soothe is the same as "keep" in standalone soothe function or the opposite as the SeeSaw internal sootheT?

I want to be sure :D

[Didée]

No, "reduc" and "keep" are not the same. Old Soothe() does not even ask for "reduc", it silently uses reduc=1.0. Then it uses the quoted procedure on top of that.
These both versions of soothing really are facing pretty different conditions. Old Soothe is completely un-aware of motion, hence it is tweaked to behave reasonably no matter whether there is motion or not. OTOH, for the shown MC-soothing something like "motion" doesn't exist, it is living in a static no-motion world [created by MVDegrain, which also takes care of most possible errors]. Therefore no need for a "keep" constraint.


@ Boulder: Application for admission has been filed, but you know, sometimes it takes years until the FDA approves. Especially in case of suspicious spice, they're paranoid about possible spillovers.

[Seraphic-]

Alright have the script loading at least, but there is quite a bit of delay when seeking other parts of the video, even for my system.

@Seraphic, what Mvtools version you have? I know it crashed my vdub also when I had the new multi-threaded build. With the older one it worked fine.

I was using mvtools V1.9.6.0_6, but changing to mvtools-v1.9.5.7 and it seems to work now.

@Seraphic, avoid using *SSE3 version of Kassandro's plugins: their presence in autoload folders or in explicit "LoadPlugin()" instructions has been proved in the past to be a source of crashes/problems .... use the *SSE2 versions (they're almost as fast), delete the *SSE3 from your system and try again ;)

Went to SSE2 versions when I changed to the older mvtool version. After it loaded without crash, I tried the two SSE3s again and it still worked.

[Nikos]

Thanks Didie, for explanation that "reduc" and "keep" are not the same.
Now the question is:
Big "reduc" number mean that we will have big reduction in sharpening effect or the opposite?
or in simple words what's the meaning of "reduc"?

[Boulder]

OT: The SSE3 problems with the RemoveGrain plugin might be fixed with this version: http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar

[Spuds]

Thanks for finding and providing some fixes for the error. I updated the script and its available at the same old link as always.

@Didée .. do you have a link to those source samples so others can play? ;)

[Adub]

Thanks for the link Boulder! That version fixes a few issues that I myself was having with removegrainsse3. I'll update the avisynth wiki to point to this release.

[Didée]

do you have a link to those source samples so others can play? ;)
Here you go. Five snipplets (total 82MB) from the original footage, with plenty of problems for grain removal filters:

(DL from MediaFire) (http://www.mediafire.com/download.php?x0wlihyfyse) / (DL from MegaUpload) (http://www.megaupload.com/?d=INSIH6AK)

Goal is to get the strong-grained static sections really calm, but to not seriously harm faint textures in the moving parts.

I'm mostly aiming at a basic problem, and at pre-filtering.
OTOH: If a too weak pre-filter leaves leftover flicker behind, then the ME tends to follow the flicker, leading to flicker in the end result.
OTOH: If a too coarse pre-filter removes too much of faint-but-important texture in motion parts, then there's not enough left over to make the ME follow that motion, which leads to also loosing that texture in the end result.

So, using pre-filtering does not solve a problem, it mostly just relocates it. A different method of making decisions is what is needed.

[Nikos]

Spuds this is your MCSharpening function:

FUNCTION MCSharpening(clip denoised, clip original, int frames, int strength, bool flow, int thsad)
{
# apply sharpening to the denoised clip
s = denoised.minblur(1,1) # Damp down remaining spots of the denoised clip.

# The difference of a simple kernel blur. Use a larger radius for stronger strength values
ssD = (strength < 4) ? mt_makediff(s,s.removegrain(11,-1)) \
: mt_makediff(s,s.removegrain(11,-1).removegrain(20,-1))

# determine our clamping limits based of our original clip and a simple MVcompensate
global idx_pointer = (flow) ? idx_pointer : idx_pointer + 1
comp_bw1 = (flow) ? sv_cb1 : original.MVCompensate(sv_b1,thsad=thsad,idx=idx_pointer)
comp_fw1 = (flow) ? sv_cf1 : original.MVCompensate(sv_f1,thsad=thsad,idx=idx_pointer)
pmax = original.mt_logic(comp_bw1,"max").mt_logic(comp_fw1,"max")
pmin = original.mt_logic(comp_bw1,"min").mt_logic(comp_fw1,"min")

denoised.mt_adddiff(ssD,U=2,V=2) # Apply the blur difference. (Sharpening is just inverse blurring.)
last.mt_clamp(pmax,pmin,0,0,U=2,V=2) # limit the sharpened clip to not exceed the original temporal neighborhood

RETURN (last)
}

I think, this is better:

FUNCTION MCSharpening(clip denoised, clip original, int frames, int strength, bool flow, int thsad, int sharpen_strength)
{

# determine our clamping limits based of our original clip and a simple MVcompensate
global idx_pointer = (flow) ? idx_pointer : idx_pointer + 1
comp_bw1 = (flow) ? sv_cb1 : original.MVCompensate(sv_b1,thsad=thsad,idx=idx_pointer)
comp_fw1 = (flow) ? sv_cf1 : original.MVCompensate(sv_f1,thsad=thsad,idx=idx_pointer)
pmax = original.mt_logic(comp_bw1,"max").mt_logic(comp_fw1,"max")
pmin = original.mt_logic(comp_bw1,"min").mt_logic(comp_fw1,"min")

# non linear sharpener with variable strength, from Didie's LSF, smode=4
Str=string(float(sharpen_strength)/100.0)
mt_lutxy(denoised,denoised.RemoveGrain(11,-1),"x y == x x x y - abs 16 / 1 2 / ^ 16 * "+Str+
\ " * x y - 2 ^ x y - 2 ^ "+Str+" 100 * 25 / + / * x y - x y - abs / * + ?")

# limit the sharpened clip to not exceed the original temporal neighborhood
last.mt_clamp(pmax,pmin,0,0,U=2,V=2)

RETURN (last)
}

Didie if you wish, explain to us the meaning of "reduc" in the MC'ed Soothe. (Please!!!)

[Spuds]

@Didée .. thank you very much for the clips, they are great (or can be :) ) Do you think its worth exploring creating a basic motion mask and then applying a stronger prefilter to the static areas and weaker to the moving areas and then to use that merged prefiltered clip in the primary mvanalysis?

@Nikos .. I think that does look a bit nicer as well, need to do some more testing. Original I had explored several other sharpening techniques but was not happy with the results, of course I now now that's in part because I had applied the clamp incorrectly so I need to go back and play.

[Didée]

Do you think its worth exploring creating a basic motion mask and then applying a stronger prefilter to the static areas and weaker to the moving areas and then to use that merged prefiltered clip in the primary mvanalysis?
Exploring is always good. Try and tell. :)

(When being out in a nutshell on stormy seas, it might become difficult to travel on a straight line ...)


@Nikos:
The sharpen-differences of several frames are temporalsoften'ed together. Then:
"reduc=0.0" -> take that average as-is. (a pixel's sharp-diff can be either decreased or increased by soothing)
"reduc=1.0" -> the average may only decrease a pixel's sharp-diff, not increase it. (a pixel can only get less sharpening by soothing, never more).

[Nikos]

Thanks Didie for the explanation!!!
From my understanding, with reduc=0.4:
if abs original sharpen diff. <abs MC'ed sharpen diff. then we apply a mix 40% of original sharpen diff. and 60% of MC'ed sharpen diff. else we apply 100% of MC'ed sharpen diff.

In simple words, the result with reduc<1 is unknown unlike normal soothe where we only get less sharpening.

It's so simple :D

[Spuds]

I've made some changes to the script, its available here (http://www.rcboataholic.com/MC_Spuds_mvt.zip)

There are a few important updates in this test version.

First is that it has been changed to run with the version of mvtools that josey_wells created, so the function is now called mc_spuds_mvt. It will not run with the last official version of mv_tools, you need the 1.9.6.1+ branch

Second is that I added motion masking to the prefilter phase. This has the effect of better retaining details in the high motion areas that were normally heavily tamped down during the preprocess. Probably needs some more tweaking but its good enough now to get some feedback on.

Third is to make it a bit easier to use for the non tweakers I added several mode options mc_spuds_mvt(mode="medium") for example, this sets processing options suitable for that mode of noise removal. The modes are "very low", "low", "medium", "high", "very high", "anime low", "anime high". You can use that and then override with any of the myriad of options :)

Last I found an error with temporal=true with certain options enabled, thats been fixed.

I have not folded the last three items back into mc_spuds, I'll do that sometime over the next few days and post that update as well.

[Ranguvar]

Excellent update, Spuds! This is becoming an easy to use do-it-all cleanup kit, which is fantastic when I don't have the time to fine-tune everything for my source =] Keep up the great work.

[Nightshiver]

I get this error: "There is no function named 'MVAnalyseMulti' "
I have all the latest plugins.

[talen9]

Did you read the announcement of this version?

First is that it has been changed to run with the version of mvtools that josey_wells created, so the function is now called mc_spuds_mvt. It will not run with the last official version of mv_tools, you need the 1.9.6.1+ branch.

You'll find it in the MVTools thread, look at the last posts, not the first one.

[Nightshiver]

what does the +branch mean? On page 49 of the discussion, there are several links to several different versions of 1.9.6.1, but I think the correct one is the 1.9.6.1 final, right?

[Ranguvar]

Get the last posted one by josey_wells ;)

[moviefan]

When I use MC_Spuds_mvt, I still have dancing pixels left in static flat areas... Even when I enable postprocessing, there is movement left and the picture gets blurred more... Is there a way to remove this dithering even more in FLAT AREAS without touching other parts of the picture and harming sharpness in detailed areas?

By the way: I use prefilter=3, frames=1-3 (tried all of them), sharpp=3, blocksize=16, overlap=4-8 any my source is 720x576p

[Ranguvar]

Try adding a FluxSmoothT. If there's already one in MC_Spuds, use higher settings for it.

[Spuds]

I still have dancing pixels left in static flat areas

You can try various increased settings but you run the risk of killing detail in other areas ... everything is a tradeoff :) Also various issues are still being worked out on the threaded version of mvtools so be careful.

To further increase cleaning try things like:
(aggressive=true) .. This is a two pass mvdegrain option
(temporal=true) .. Another form of two pass mvdegraining but with special limiting
(flow=true) .. This sometimes helps, I think it hurts more often, but depending on the source you may like its results. It uses mvcompensations in place of mvdegrains
(sharpp=0) .. temporarily turn off sharpening to see if that is what is making the pixels stand out.
(strength=4 or strength=5) with prefilter=2, increasing strength changes several items that may help catch those pixels.

You can also just try (mode="high") or (mode="very high") which does much of the above expect for the sharpp=0

Other options are (postprocess=true) for a final 'dusting' or (chro=true) for full luma and chroma plane processing

If none of that works, post a short clip, it might just need some more specific mvanalysis parameters.

[Spuds]

I made some minor tweaks to the _mvt version of the script and rolled all of those updates (mode, motion compensated pre filter, see changelog for full list) into the original script. I also added a "medium high" mode since the delta from medium to high modes was to big.

Both versions are available here (http://www.rcboataholic.com/MC_Spuds.zip) along with the updated help file and change log.

Edit: Updated file at link, had one with the old block/overlap defaults

[mozzle]

First, thanks a lot for your work on this very cool script!

It's effective and it's very fun to play with. Thanks especially for modifying for the new, faster version of MVTools.

I was reviewing the documentation and I believe there might be a typo in the thSAD description. The value range is listed as 0-255.


Thanks again for the update!

[ankurs]

been trying to make mcsharpening() work , i get an error saying invalid arguements to the function mcsharpening() whilst i am putting in the denoised clip , any help ? tips on usage ?t :

edit : i am using the latest mvtools , but i am not denoising from mc_spuds , using udegrain /fft3dgpu and taking the clip from there ..

[Lux Delux]

Awesome Spuds, with the latest MT mvtools my FPS increased by 4 :eek: without raising CPU usage (using medium_high with tweaked thsad, which is awesome btw)

only thing, when using sharpp=2 I still get this weird 2px border like thingie in my video, example:

http://img228.imageshack.us/img228/2050/borduk8.th.png (http://img228.imageshack.us/my.php?image=borduk8.png)

look at the very top zoomed. When using limitedsharpenfaster outside of the script it doesn't happen. Any ideas?

[Didée]

@ ankurs: The way mcsharpening() is layed out, you can not use it "externally" for arbitrary post-processing. It works only for internal usage in MC_Spuds.

(You need either motion vectors or MV-compensated frames for this operation, but you don't have them available when you're starting with source.AnyOfTheMCDenoisers(). Therefore, it's needed to modify the "AnyOfTheMCDenoisers()" script itself.)


@ Lux Delux: Hmm ... that could be caused by some plugin filters that silently don't process the outmost border pixels. Such behaviour can cause a slight "edge" (processed pixels beneath not-processed pixels), which subsequently might get amplified by upfollowing filtering operations.
Just a guess, needs further investigation at which stage that border thingy gets introduced.

[Nikos]

Didée for the contra mcsharpening which is the most suited sharpening routine.
a. mt_lutxy(clp,clp.RemoveGrain(11,-1), "x y == x x x y - abs 16 / 1 2 / ^ 16 * "+Str+"
\ * x y - 2 ^ x y - 2 ^ "+Str+" 100 * 25 / + / * x y - x y - abs / * + ?"), with Srt in the range 0.8-1.5

b. mt_lutxy(clp, clp.RemoveGrain(11,-1), "x x y - "+Str+" * +"), with Srt~1.5

c. the above with Srt=1, mt_lutxy(clp, clp.RemoveGrain(11,-1), "x x y - +")

d. Something else... for sharpening on several frequency ranges, not just a simple 3x3 kernel.

In my test without temporal limiting, the sharpening effect of a is very strong. With temporal limiting, the results are almost identical.

[ankurs]

@ ankurs: The way mcsharpening() is layed out, you can not use it "externally" for arbitrary post-processing. It works only for internal usage in MC_Spuds.

(You need either motion vectors or MV-compensated frames for this operation, but you don't have them available when you're starting with source.AnyOfTheMCDenoisers(). Therefore, it's needed to modify the "AnyOfTheMCDenoisers()" script itself.)




so i am supposed to denoise using mc spuds before using mcsharpening ?

noob here , dont know how to internally put mcsharpening in mc_spuds.avs and use it :confused:

which mc denoiser :confused: i assume ur asking me to edit mc spuds or mc spuds mvt isnt it ?

edit : so dumb of me , got it :p thanks a lot didee !

[Spuds]

@Lux Delux ... Glad you are seeing the speed increase, the new mvtools is a nice improvement. I'm not sure what would be causing that top line error, could you please let me know how you were calling the script and the clip size? Also does this appear on all clips you process or was that one somehow a special case?

@ankurs ... mcsharpening is inside of mc_spuds and is on by default, you can turn it off with sharpp=0 or change it to contra with sharpp=1 or LSF with sharpp=2. Its not a standalone function as it needs some value information that is calculated from within mc_spuds, notably the motion vectors. Could you make it standalone, yes, is it worth it if not already doing mc work, most likely not.

@Nikos ... Not sure if one is more suited or not, its the old do you like smode=4 or smode=3? I have made a small update to the routine so that it performs the sharpening to an upsized clip, more in full spirit of LSF so the results are now different.

The files at the old link have been updated if you want to give it a try.

[Lux Delux]

Hey Spuds :), here's an example where it happens with an NTSC source

mpeg2source("D:\EXTREME_LIMITS\d2v\VTS_01_1.d2v",info=3)
ColorMatrix(hints=true,interlaced=true)

TFM(order=1)
TDecimate()
vinverse()
crop(8,12,-8,-12)

mc_spuds_mvt(mode="medium_high",edgeclean=true,edm_lo=25,edm_hi=25,thsad=200,sharpp=2)

or a different PAL

mpeg2source("D:\BATMAN_1966\d2v\VTS_01_1.d2v",info=3)
ColorMatrix(hints=true)

crop(4,14,-4,-14)

mc_spuds_mvt(mode="high",thsad=170,sharpp=2)

BTW is no one else experiencing this? So I know whether it's perhaps a .dll that screws up with the rest.

[Didée]

Aahh ... the script is adding black borders to pad the input clip to mod16 resolutions. Black borders introduce a hard edge, that can cause unwanted effects.
Instead, try padding by repeating the border pixels:

# calculate the appropriate borders so its width and height are mod 16, these will be removed later
x_temp = ab16(clp.width)
x1 = round(x_temp / 2.0)
x0 = int(x_temp - x1)
y_temp = ab16(clp.height)
y1 = round(y_temp / 2.0)
y0 = int(y_temp - y1)

# add blank borders (to improve frame edge processsing and make the clip mod 16), we will remove these at the end.
# clp = clp.addborders(x0,y0,x1,y1)
clp = clp.pointresize( clp.width()+x_temp, clp.height()+y_temp, -x0, -y0, clp.width()+x_temp+.001, clp.height()+y_temp+.001 )
nullclp = blankclip(clp) ("+.001" is there for compatibility with Avisynth 2.5.6, not needed for 2.5.8)

This should improve on the issue given that the borders of the input clip are clean. However, if there're any dark pixels left over from original borders, YMMV.

[Lux Delux]

:thanks: I'll try it out once a current encode using MC is finished :)

[Didée]

Oops, there was an error in the suggested modification (I misinterpreted what ab16() is calculating).

Code above has been corrected.

[Lux Delux]

Can't find any borders for now so it seems that fixed it. Thanks again Didee :)

[tengo6dedos]

help!

im getting this error

mc_spuds_mvt.avs, Line 66, Evaluate: operands of `==' and `!=' must be comparable


thanks

[Spuds]

@tengo6dedos ... Download the script again, I made the fix for that error plus updated border section with Didées suggestion. Your error was because I had not initialized sharpp to its default value before using it in a logic statement.

@Didée ... Thanks so much for the find and suggestion. Quite the kick in the pants as I added the extra borders just to try and avoid border artifacts from sharpening, doh!

As always updates are here (http://www.rcboataholic.com/MC_Spuds.zip)

[ChrisW77]

Nice, as always, but I can't seem to get anything out of gpu=true.

setmemorymax(512)
LoadPlugin("C:\Program Files\AviSynth\plugins\DGDecode.dll")
Import("C:\Program Files\AviSynth\plugins\MC_Spuds_mvt.avs")

mpeg2source("D:\DVB-T-Caps\WildChina\china1.d2v", cpu=4, iPP=true, idct=0)
ConvertToYV12(interlaced=true)

AssumeTFF()
LeakKernelDeint(order=1,threshold=12,sharp=false)

Crop(4,4,-4,-4,align=true)
Spline36Resize(640,352)

MC_Spuds_mvt(frames=2,strength=4,sharpp=2,flow=false,focus=false,preprocess=false,postprocess=false,gpu=true)

Simple script, gets me about 12-14fps on a E7200, 4Gb RAM, ATI 4850 512Mb, with GPU=false. Change GPU=true, and it stays the same. :confused:
Would have thought I would at least see a small increase, especially for a 4850.

[Spuds]

I can't seem to get anything out of gpu=true. The use of fft3dXXX is located in the prefilter stage and in some areas of flow processing, both of which you have turned off in your call to the script, hence no change. You should notice a change if you set preprocess=true, what that would be I don't know as my graphics card is a lamer.

[ChrisW77]

Cheers, I'll give that a try.

[tengo6dedos]

Download the script again, I made the fix for that error

yes, now that error is gone but now when i try to use "aggressive" or goin above frames=2 or strengh=2 when having anime=true i get an error saying "dont know what bv2 means"

using mc_spuds_mvt

maybe i have a plugin missing?

LoadPlugin("D:\video\plugins\Unfilter.dll")
LoadPlugin("D:\video\plugins\Undot.dll")
LoadPlugin("D:\video\plugins\RemoveGrain.dll")
LoadPlugin("D:\video\plugins\Repair.dll")
LoadPlugin("D:\video\plugins\SSETools.dll")
LoadPlugin("D:\video\plugins\RemoveDirt.dll")
LoadPlugin("D:\video\plugins\hqdn3d.dll")
LoadPlugin("D:\video\plugins\despot.dll")
LoadPlugin("D:\video\plugins\AddGrainC.dll")
LoadPlugin("D:\video\plugins\dfttest.dll")
LoadPlugin("D:\video\plugins\fft3dfilter.dll")
LoadPlugin("D:\video\plugins\aWarpSharp.dll")
LoadPlugin("D:\video\plugins\WarpSharp.dll")
LoadPlugin("D:\video\plugins\mt_masktools.dll")
LoadPlugin("D:\video\plugins\mvtools-v1.9.6.7.dll")
LoadPlugin("D:\video\plugins\deblock.dll")
LoadPlugin("D:\video\plugins\dctfilter.dll")
LoadPlugin("D:\video\plugins\DGDecode.dll")


Import("D:\video\plugins\LimitedSharpenFaster.avsi")
Import("D:\video\plugins\Deblock_QED.avs")
Import("D:\video\plugins\MC_Spuds_mvt.avs")




thanks

[Adub]

It's probably a spelling error. Make sure that you are importing your d2v correctly.

If you are, then it is somewhere in the script. Just use "Find".

[talen9]

"bv2" is actually a clip variable used into the script ... I suspect a little bug caused by the change to the MVMulti functions.

@tengo6dedos: the complete error message should report the line number where the error happens ... if you can write it here, it should simplify the debugging operations ;)

[tengo6dedos]

sure,

"dont know what bv2 means: mc_spuds_mvt Line 257, Line 176"

[Spuds]

"dont know what bv2 means: mc_spuds_mvt Line 257, Line 176"

Thanks for the bug report, yup another little bug from the conversion to the threaded version of mvtools. I forgot that I used bv2/fv2 frames for a blocking check with the anime flag that can occur during large luma swings over just a few frames. I made the changes and updated the files at the old link.

Looks like more changes will be required based on the very latest version of mvtools just posted, the current script will work but will probably need some more updates to be optimal.

[Adub]

It will be best if you wait till that MVtools has been finalized. It's going through alot of changes, and with more to come as well.

[elguaxo]

Is there a way to avoid these spots that are being introduced by MC_Spuds:

http://img403.imageshack.us/img403/9020/spotsdu8.png

comparison - previous | current | next
http://img123.imageshack.us/img123/928/prevhw6.th.png (http://img123.imageshack.us/img123/928/prevhw6.png) http://img527.imageshack.us/img527/5018/currentju1.th.png (http://img527.imageshack.us/img527/5018/currentju1.png) http://img527.imageshack.us/img527/5696/nextqw5.th.png (http://img527.imageshack.us/img527/5696/nextqw5.png)

VOB Sample: http://www.mediafire.com/?e3utfwxdxki

I'm using Avisynth 2.57, MC_Spuds 1.1 BETA - 21.AUG.08 (non multithreaded version) and MVTools 1.10.2.1. And my script is this:

crop( 24, 0, -24, 0)
mc_spuds(mode="low")

Any hints? TIA.

edit: I'm seeing more frames that show this problem. I think it's always during a scene change.

[Terranigma]

Those look like the chroma artifacts I was referring to when thsad in mvdegrain was raised to a very high amount, like, 800 back when i was trying to get thsadc implemented.
Lower thsadc and see what happens. ;-)

[elguaxo]

mode="low" is not using a very high thsad, it's using Thsad=400 and the default value for chro is false, so it shouldn't process it. I lowered Thscd1 from 500 (default for mode="low") to 400 and this solved the problem completely:

http://img363.imageshack.us/img363/6259/nospotsfi6.png

:)

[Terranigma]

and the default value for chro is false

Yea that's right. i've forgotten that I personally edited mc_spuds so that chroma is set to true by default (which is why I recommended that you try altering that parameter).

I'm glad you managed to figure out what the problem was though. :)

[elguaxo]

One question regarding Thscd2. I'm using v1.1 BETA - 21.AUG.08. The manual says:

Strength Settings

Strength 0 1 2 3 4 5 6
Thscd2 Regular/Anime 105/130 105/130 105/130 105/130 105/130 105/130 105/130

but lines 89-90 are:

thscd2 = (defined(thscd2)) ? thscd2 : (anime) ? select(strength , 85 , 85 , 85 , 85 , 85 , 85 , 85) \
: select(strength , 115 , 115 , 115 , 115 , 115 , 115 ,115)

so, the defaults are 85 for anime and 115 for the rest? It's a long script and I get a little http://img355.imageshack.us/img355/5836/orccrazy21ub5.gif trying to understand it. Thanks! :)

[Adub]

Bug report:

I am receiving some weird artifacts while working through my Heroes dvd's (on which MC_Spuds does wonders by the way). I have zipped everything together for testing, so see if you can replicate my issue.

I am using MVTools 1.11.4.4. and MC spuds 1.1 Beta 21.Aug.08.

http://www.mediafire.com/download.php?mlj0zyzhmll

Please confirm.

Also, Debug=1,5, etc seems to be broken. Something about "fv1".

[Spuds]

Thanks for the full bug report and files !

I see the defects you are referring to ... I believe the issue is that with the medium setting its using a blocksize of 16 and an overlap of 4 which can in some cases lead to block development.

You can try leaving it as medium and override the blocksize with blocksize=8 and see if that takes care of the problem. Likewise leaving the blocksize alone and setting overlap=8 may also cure the issue.

Another way to approach the issue is with changing the thscd1 value to something pretty low like thSCD1=50 (with the blocksize=16). The moving background is like mini scene changes and things are getting tripped up with the larger blocksize. I don't know if lowering it that much will cause other issues.

The computer that I'm on at the moment does not have a good screen so I can't tell if that will take care of all the problems. Let me know what you find with the above suggestions.

Oh I'll post an update to fix the fv1 thing, I remember fixing that but never posted the update.

[Adub]

Yes, changing the blocksize or the overlap does fix the problem. The problem is that decreasing the block size also slows the script down a lot. The overlap adjust fixes most of the artifacts, but they are still there a little bit. But it is a lot faster. Hmm. I might just use MCTemporalDenoise on this one. It destroys a little more facial details, but overall it cleans up the content a little bit better. Plus it's dehaloing is working pretty well with the burned in subtitles.

[Sagekilla]

Well, blksize for MVAnalyse determines performance rather directly -- In terms of speed (generally speaking) it's as follows: 4x4 < 8x8 < 16x16. Where 4x4 is slower than 8x8 etc..

[Adub]

Yeah, I understand that. The block size directly specifies the search window. Another thing I might try is using mvcompensate. Just to see how it works.

[Adub]

Another bug report:

When using "Flow=true" with mode=medium, I get an error saying "Valid modes for bt are (-1,0,1,2,3,4)". Line 343 of spuds has fft3dgpu using bt=5. It seems to be caused by the use of frames==2. The bug is contained in the f14_flow function.

[Spuds]

Thanks for the report, I did not know that fft3dgpu did not allow bt=5 since its allowed in fft3dfilter. I'll make the changes and post and update soon.

[Adub]

Huh, that is weird. You would think fft3dgpu would be able to do it, but I guess not.

Edit: and is it just me or is line 337 weird?

ov = (temporal) ? 8 : 8

[Spuds]

I posted a quick update which should fix the bt=5 issue with fft3dgpu and the fv1 issue during debug views (actually forgot to check the fv1 thing but I remember fixing it some time ago). Anyway the update is available at the link in post #2.

Edit: and is it just me or is line 337 weird? I'll follow the forum rules and say its not you :) Really for me that's a common practice when I'm trying different things. I put in the flexibility in for playing. At some point I was trying different overlay values with temporal processing but for now have left it to be the same.

[Adub]

Okay, I just thought I would check. I will download the latest script and keep using/testing ;)


Also, is there a reason why the flow=true function causes excessive banding unless i really turn down the sigma value's? It's kind of insane. I like the flow functionality, but it seems kind of odd that it will always cause insane banding.

Edit: So are you holding off on updating the Multithreaded version for some reason? I have just been testing the single threaded version so far, but I am considering switching to the Multithreaded version soon.

Edit2: Okay, I am going to stick with single threaded for now. The multithreaded version of mvtools is too unstable. Plus there are a few bugs in the MC_Spuds_mvt script.

For example, using the settings:
MC_Spuds_mvt(mode="low", edgeclean=true, gpu=true, sharpp=2, postprocess=true)
I get an error about "MVMultiExtract: Index must be between 0 and number of elements in array minus 1".
Then it complains about line numbers 246, and 176.

Now, I think I understand the error with mode="low", as is only uses 1 frame, and yet you are using the statement "frames - 2" in the multiextract line.

[Spuds]

I'm probably not going to update the mutli version, I did it mainly as an experiment. I'll update the script at some point to use the new official 2.0 version of mvtools.

Oh yes I am aware of the banding issue with flow=true, its always been a problem and I think some of that is inherent in how it works. That said I have not worked on that function in some time so I'm sure there are improvements that can be had!

[canuckerfan]

I'm trying to get mc_spuds working here. i'm calling this: MC_Spuds(frames=3,strength=1,chro=true,edgeclean=true)

but it keeps saying there is no function named "Clense". but I have RemoveGrain and RemoveGrainT in my plugins folder. here's what I have: http://img95.imageshack.us/my.php?image=plugswa7.png

running avisynth 2.5.7

edit: nevermind. i've figured it out. had an old removegrain version.

[Vesdaris]

Im kinda lost here.
DOwnloaded the latest version.
Checked what plugins i need to get it started
Then tried to call this( just to check if it works and if there are any problems) mc_spuds(frames=2,strength=3)
and i got this
http://img121.imagevenue.com/aAfkjfp01fo1i-28051/loc205/55081_27.01_122_205lo.jpg
does it have smth to do with masktools or mvtools?

[Leak]

i got this
http://img121.imagevenue.com/aAfkjfp01fo1i-28051/loc205/55081_27.01_122_205lo.jpg
You might want to check that link:

Not Found

The requested URL /aAfkjfp01fo1i-28051/loc205/55081_27.01_122_205lo.jpg was not found on this server.
np: The Bug - Insane (London Zoo)

[Vesdaris]

Oh. strange. It worked fine
here it is
http://img502.imageshack.us/img502/3739/3001gm1.th.jpg (http://img502.imageshack.us/my.php?image=3001gm1.jpg)

[thetoof]

mt_something = masktools2
Do you have it in your plugins folder? Else, make sure you explicitely load it with loadplugin("path\mt_masktools-25.dll")

[JnZ]

I tested MC last couple of days and I must only say: very nice job Spuds.

For strong noise I used only Neat Video, which was capable remove very hard noise, last few days I found mc_spuds and tested. Results are very good, almost unbeliveable. :)

Original:
http://img4.imageshack.us/img4/7808/originalhp8.th.png (http://img4.imageshack.us/my.php?image=originalhp8.png)

mc_spuds (fr3, str6, edge, chroma):
http://img23.imageshack.us/img23/6281/mcfr3st6chroedgeyv8.th.png (http://img23.imageshack.us/my.php?image=mcfr3st6chroedgeyv8.png)

I'm remastering DVD SG, it's really crappy source...

Thanks for this beutiful filter Spuds, and keep it going...:thanks:

[rkalwaitis]

salehin

It worked well on some old films I had. Very impressed.

[rkalwaitis]

Anyone,

I cant use focus=true as I receive an error of invalid use or RemoveGrain mode of 20. Does anyone know what Im doing wrong? I loaded all of the appropriate .dlls

Why cant I use temporal and aggressive at the same time? Is this taboo?

I also can not get debug to work. What is the correct syntax to use?

I can not use sharpp=3 it gives me the error or the invalid use of remove grain mode 19.

What are the advantages of using preprocessing and postprocessing as the obvious disadvantage is fps loss?

Sorry for so many questions, try to get the most of this excellant script.

Thanks K

[Adub]

Make sure that you have the latest version of RemoveGrain.

See here:
http://avisynth.org/mediawiki/Removegrain

[rkalwaitis]

Merlin7777, that seemed to help. To many versions of a wide variety of goodies to keep up with :) Ill play more and try to ask worthy questions.

[rkalwaitis]

I also tried FuPPs Hybridq with preset of Hybridq and gave very similar results if not a tad better (at least to my eye). The kicker FuPPs is much faster.

http://forum.doom9.org/showthread.php?t=131920&page=2

[fjhdavid]

I read all the thread. Right now I use fft3dfilter alone and I would like to test mc-spuds.

I found the last mc_spuds script and I read the mc_spuds.htm which list all the needed filters (great documentation).

I found some of the filters, but some links are broken (MVtools, deblock,...)...

Could you post somewhere a link to an archive "all in one package" .rar with all the filters?

It would be very helpful!

thanks
Francois

[Yobbo]

Could you post somewhere a link to an archive "all in one package" .rar with all the filters?

It would be very helpful!

thanks
Francois

Agreed.

[fjhdavid]

more simple for the moment,

Could someone of MC_spuds user on this forum create an archive of their avisynth\plugins directory and post it attached here?

thanks
Francois

[Yobbo]

(Here (http://www.zshare.net/download/5688818253373915/), fjhdavid.)

When using "temporal=true" I get "don't know what 'tlimit' means"? I am using 1.1 beta. :confused:

[Vesdaris]

Great script. Thank you.
btw i was wondering why the speed is faster with strength=3 than with strength=2
(MC_Spuds(strength=2,frames=4,search=3,sharpp=2) is slower than the same but with str=3

[Dominiko]

About Mc_Spuds(Temporal = true[,aggressive=false]), is it equal to Mc_Spuds(aggressive=false).TemporalDegrain() or something like? This Script is too complicated for me :(

[rkalwaitis]

Spuds, I havent seen any updates in a while. Is there a plan by you to update your script for the new version of dft?

K

[rkalwaitis]

Which version of the removegrain packages must I use to enable the postprocess option. I get an error stating clense is not available.

[Didée]

Noted the list (http://forum.doom9.org/showthread.php?p=1060208#post1060208)?

[rkalwaitis]

Thanks Didee, I had the right Removegrain/Dirt packages, but I didnt get rid of the other dlls that I did not need. After rereading the instructions there was only to be one of them and not the SSE or S. This seemed to fix the problem as not post process works. Thank you again.

[rkalwaitis]

Now I can not use the LSF option with MC_spuds. The postprocessor works but not sharpening. Contrasharpening works. LSF Does not. I get a RemoveGrain: Invalid mode 19. Same situation when trying to use LSFMod with it.

Does anyone know how to fix this? Thanks K

I did not mean to make Lato so upset,

But the read me at the beginning is confusing,

this is from another source and Didee said, "You need to get the "v1.0b "beta" release", and don't use the ~SSE3 versions." I downloaded the link Didee gave there and it seems to work. So the v1.0b "beta" release is so far working.

(The official v0.9 is too old, the intermediate pre-v1.0 exist in different versions with different bugs, and the actual v1.0 changed the internal framework by so much that it's quite incompatible with the old versions for which most scripts are layed out...)

[LaTo]

Always, always the same thing... SEARCH!

[rkalwaitis]

Nevermind, I see how this is going to go. Sorry for any inconvienances.

[Didée]

Admitted, there is lots of confusion with RemoveGrain versions, and finding the "fitting" one is not trivial. To make things worse, you can't just go with the very-latest version, since 99% of all scripts are not adopted to the final version v1.0.
(And yes, as far as my scripts are concerned, I'm part of the misery, too.)

Hint: this one (http://forum.doom9.org/showthread.php?p=976573&highlight=hurdy-gurdy#post976573) should work without problems, in almost all cases. Just note the SSE3-fix is *not* in there. Use the SSE2 versions, they're fine.

[rkalwaitis]

thanks Didee.

[Dogway]

MC_Spuds(mode="high") freezes with srestore. Dont know why.

[rkalwaitis]

Is it possible to utilize all of the bells and whistles of MC_Spuds, but use dfttest as the main denoiser? I know it will be slow.

[Dogway]

Im still wondering how dfttest is any good, I would like to see tests...

[rkalwaitis]

I think DFTTest is a very good denoiser, all though a bit on the slow side. There is a script on this site for a motion compensated dfttest, check it out for yourself.

[popper]

Old script removed, get the latest here (http://www.rcboataholic.com/MC_Spuds.zip)

your link from the second post no longer works, its now a parked site apparently, is there other more current noise removal scripts available for download ?

[Usedocne]

@popper

Maybe try here (http://avisynth.org/mediawiki/MC_Spuds).

[popper]

thanks

[javlak]

Apologies for necroing this thread, but I was porting this for MVTools2 and while browsing through the code I noticed a small typo:

line 292: global idx_poiner = idx_pointer + 1

This should clearly be:

global idx_pointer = idx_pointer + 1

So to anyone still using this script and to the maintainers of it, I would advise changing the particular line.

[diimaan]

when using this in my mc_spuds

removeblocks=true


I am getting the error "There is no function named Deblock_QED"

but I have the deblock_qed avs in my plugin's directory! what could be wrong here?

[Gavino]

but I have the deblock_qed avs in my plugin's directory!
If you want it loaded automatically from the plugin directory, you will need to rename it as .avsi.

[althor1138]

I'm a sci-fi fan and often find myself using avisynth to denoise stuff with starfields in it and they always seem "diminished" afterwards. If I understand correctly the function "starmask" in MC_Spuds preserves the stars so I've been trying to extract it from the script so I can use it as a standalone solution to my problem. The problem is I've gotten it to where I think it is correct however it doesn't really seem to do anything so I wonder if anybody has any input on where I've gone wrong. I'm not really sure what I'm doing so please be kind :).

FUNCTION Starmask(clip input,int thStar,int lthresh)
{

low = eval("input.greyscale.fft3dfilter(sigma=4,bt=1)")
high = eval("input.greyscale.fft3dfilter(sigma=12,bt=1).removegrain(17)")

maskstar = mt_makediff(low,high).mt_lut("x " + string(lthresh) + " < 0 x ?")
maskclip = maskstar.removegrain(4)
nullclip=blankclip(input)
check=Conditionalfilter(maskclip,nullclip.mt_binarize(upper=false),maskstar.mt_inflate().mt_inflate,"AverageLuma()","<",string(thStar),false)
output=mt_merge(input,check,maskstar,luma=false)
RETURN output
}

[Jenyok]

Didee
.
I am very sorry.
Why do you use MVxxxx (MVAnalyse, MVFlow and so on) functions in MC SPUDS script ?
Why do you NOT use Mxxxx (MAnalyse, MSuper and so on) functions in MC SPUDS script ?
.
May be you will rewrite script MC SPUDS to new MVTools functions (MSuper, MAnalyse and so on) ?
New version of MC SPUDS script.

[Tempter57]

Jenyok
MC_Spudsmod.avsi (http://multi-up.com/756728) with mvtools2.dll

[Jenyok]

Tempter57
.
Thanks.

[Vesdaris]

I once used it ages ago and had no problems running it. Idecided to give it another go yesterday and couldnt get it working.
i downloaded MC_Spudsmod.avsi, all dlls that are needed to run it
but im getting this error "there is no function named clense"
im on win 7 64

here is what i have in my plugin directory
http://i47.tinypic.com/35lsxzm.jpg


any ideas would be greatly appreciated, guys)

[cobo]

Using RemoveGrain v1.0 pre-release would give that error. The original requirement (http://forum.doom9.org/showthread.php?p=1060208#post1060208) was for RemoveGrain v0.9 - I didn't look through this thread to see if it had been updated though. I think RemoveGrain v1.0b "beta" release might work too.

Links to the different versions are here:
http://avisynth.org/mediawiki/Removegrain

[Vesdaris]

hmm i tried using removegrain 0.9 then v1.0b "beta" release and now it says "there is no function named "removegrain" .instead of there is no function named clense.. im lost..
oh and there is a mistake on a picture above. instead of rsharpen it was of course RemoveGrainTSSE2.

[Jenyok]

Corrected version of script.
.

#############################################
# Motion Compensated dfttest #
# Really Really Really Slow #
# #
# For best results, use a mcradius equal to #
# (tbsize-1)/2 #
# #
# MDegrain will also be run if mdg == true #
# #
# mcradius is hardlimited at 5 #
# #
#############################################



LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\MVTOOLS-V2_5_11_3\mvtools2.dll")
LoadPlugIn("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\DFTTEST-1_9_2\dfttest.dll")
LoadPlugIn("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\DEGRAINMEDIAN_20061008\degrainmedian.dll")



function dfttestMC (clip input, bool "Y", bool "U", bool "V", int "ftype", float "sigma", float "sigma2",
\ float "pmin", float "pmax",
\ int "sbsize", int "smode", int "sosize", int "tbsize", int "tmode", int "tosize",
\ int "swin", int "twin", float "sbeta",
\ float "tbeta", bool "zmean", float "f0beta", int "mcradius", bool "mdg", int "pp", int "ppstr",
\ int "thSAD", int "mdgthSAD", int "thSCD1", int "thSCD2",
\ int "blksize", int "pel", int "overlap", int "dct", int "search")
{
o = input

# dfttest-related options
Y = default(Y, true)
U = default(U, true)
V = default(V, true)
ftype = default(ftype, 0)
sigma = default(sigma, 16.0)
sigma2 = default(sigma2, 16.0)
pmin = default(pmin, 0.0)
pmax = default(pmax, 500.0)
sbsize = default(sbsize, 12)
smode = default(smode, 1)
sosize = default(sosize, 9)
tbsize = default(tbsize, 5)
tmode = default(tmode, 0)
tosize = default(tosize, 0)
swin = default(swin, 0)
twin = default(twin, 7)
sbeta = default(sbeta, 2.5)
tbeta = default(tbeta, 2.5)
zmean = default(zmean, true)
f0beta = default(f0beta, 1.0)

# mvtools-related options
mcradius = default(mcradius, 2)
mcradius = (mcradius > 5) ? 5 : (mcradius < 1) ? 1 : mcradius
mdg = default(mdg, false)
pp = default(pp, 1)
ppstr = default(ppstr, (pp >= 2) ? 16 : (pp == 1) ? 1 : 9001)
mdgthSAD = default(thSAD, 400)
thSAD = default(thSAD, 10000)
thSCD1 = default(thSCD1, 400)
thSCD2 = default(thSCD2, 130)
blksize = default(blksize, 8)
pel = default(pel, 2)
overlap = default(overlap, 2)
dct = default(dct, 0)
search = default(search, 5)

# Pre-ME denoising
pp = (pp >= 2) ? o.dfttest(sigma=ppstr) : (pp == 1) ? o.DeGrainMedian(mode=ppstr) : o

# MSuper
#pp_super = pp.MSuper(pel=pel)
pp_super = pp.MSuper(pel=pel, sharp=2)

# Motion vector search
b5vec = (mcradius >= 5) ? MAnalyse(pp_super, isb=true, search=search, delta=5, overlap=overlap, blksize=blksize, dct=dct) : NOP
b4vec = (mcradius >= 4) ? MAnalyse(pp_super, isb=true, search=search, delta=4, overlap=overlap, blksize=blksize, dct=dct) : NOP
b3vec = (mcradius >= 3) ? MAnalyse(pp_super, isb=true, search=search, delta=3, overlap=overlap, blksize=blksize, dct=dct) : NOP
b2vec = (mcradius >= 2) ? MAnalyse(pp_super, isb=true, search=search, delta=2, overlap=overlap, blksize=blksize, dct=dct) : NOP
b1vec = MAnalyse(pp_super, isb=true, search=search, delta=1, overlap=overlap, blksize=blksize, dct=dct)
f1vec = MAnalyse(pp_super, isb=false, search=search, delta=1, overlap=overlap, blksize=blksize, dct=dct)
f2vec = (mcradius >= 2) ? MAnalyse(pp_super, isb=false, search=search, delta=2, overlap=overlap, blksize=blksize, dct=dct) : NOP
f3vec = (mcradius >= 3) ? MAnalyse(pp_super, isb=false, search=search, delta=3, overlap=overlap, blksize=blksize, dct=dct) : NOP
f4vec = (mcradius >= 4) ? MAnalyse(pp_super, isb=false, search=search, delta=4, overlap=overlap, blksize=blksize, dct=dct) : NOP
f5vec = (mcradius >= 5) ? MAnalyse(pp_super, isb=false, search=search, delta=5, overlap=overlap, blksize=blksize, dct=dct) : NOP

# Optional MDegrain
#o_super = mdg ? o.MSuper(pel=pel, levels=1) : o
o_super = mdg ? o.MSuper(pel=pel, levels=1, sharp=2) : o
mdegrained =
\ (mcradius >= 3 && mdg) ? o.MDegrain3(o_super, b1vec, f1vec, b2vec, f2vec, b3vec, f3vec, thSAD=mdgthSAD, thSCD1=thSCD1, thSCD2=thSCD2) :
\ (mcradius == 2 && mdg) ? o.MDegrain2(o_super, b1vec, f1vec, b2vec, f2vec, thSAD=mdgthSAD, thSCD1=thSCD1, thSCD2=thSCD2) :
\ (mdg) ? o.MDegrain1(o_super, b1vec, f1vec, thSAD=mdgthSAD, thSCD1=thSCD1, thSCD2=thSCD2) : o

degrained = (mdg) ? mdegrained : o

# Motion Compensation
#degrained_super = degrained.MSuper(pel=pel,levels=1)
degrained_super = degrained.MSuper(pel=pel, levels=1, sharp=2)
b5clip = (mcradius >= 5) ? degrained.MCompensate(degrained_super, b5vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP
b4clip = (mcradius >= 4) ? degrained.MCompensate(degrained_super, b4vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP
b3clip = (mcradius >= 3) ? degrained.MCompensate(degrained_super, b3vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP
b2clip = (mcradius >= 2) ? degrained.MCompensate(degrained_super, b2vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP
b1clip = degrained.MCompensate(degrained_super, b1vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)
f1clip = degrained.MCompensate(degrained_super, f1vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2)
f2clip = (mcradius >= 2) ? degrained.MCompensate(degrained_super, f2vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP
f3clip = (mcradius >= 3) ? degrained.MCompensate(degrained_super, f3vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP
f4clip = (mcradius >= 4) ? degrained.MCompensate(degrained_super, f4vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP
f5clip = (mcradius >= 5) ? degrained.MCompensate(degrained_super, f5vec, thSAD=thSAD, thSCD1=thSCD1, thSCD2=thSCD2) : NOP

# Create compensated clip
interleaved =
\ (mcradius >= 5) ? Interleave(f5clip, f4clip, f3clip, f2clip, f1clip, degrained, b1clip, b2clip, b3clip, b4clip, b5clip) :
\ (mcradius == 4) ? Interleave(f4clip, f3clip, f2clip, f1clip, degrained, b1clip, b2clip, b3clip, b4clip) :
\ (mcradius == 3) ? Interleave(f3clip, f2clip, f1clip, degrained, b1clip, b2clip, b3clip) :
\ (mcradius == 2) ? Interleave(f2clip, f1clip, degrained, b1clip, b2clip) :
\ Interleave(f1clip, degrained, b1clip)

# Perform dfttest
filtered = interleaved.dfttest(Y=Y, U=U, V=V, ftype=ftype, sigma=sigma, sigma2=sigma2, pmin=pmin, pmax=pmax,
\ sbsize=sbsize, smode=smode, sosize=sosize,
\ tbsize=tbsize, tmode=tmode, tosize=tosize, swin=swin, twin=twin,
\ sbeta=sbeta, tbeta=tbeta, zmean=zmean, f0beta=f0beta)

output = filtered.SelectEvery(mcradius * 2 + 1, mcradius)

return (output)
}

[STJAM]

I was hoping that MC_spudsmod.avsi could be mirrored on a site were membership or a download program is not required - possibly mediafire.
Thanks.

[sinz718]

MC_spudsmod.avsi Script Mirror @Request (http://www.4shared.com/file/2JTXlP3G/MC_SpudsMod.html)

[STJAM]

Thanks, but no thanks. I neither want a 4Shared account or desktop.

[qwerty1983]

Could someone be kind enough to post MC_spudsmod.avsi ?
Thank you.

[GMJCZP]

Could someone be kind enough to post MC_spudsmod.avsi ?
Thank you.

I did not get it, the Russians seem to be the last who have used it.

[StainlessS]

I spent abut 20/30 mins too, could not find.

[udobroemme]

I've found it: :)

[STJAM]

Script error: Invalid argument to function "debug_view". One of the things I really liked about MC_Spuds were the debug outputs.

[qwerty1983]

use
MC_Spudsmod(debug=x) # 0 >= x < 10 , default is debug=0

# 1 = show settings
# 2 = show input/ouput what changed frame
# 3 = side by side input and output
# 4 = side by side, final view and grayscale difference map
# 5 = side by side motion mask and static mask
# 6 = side by side motion mask and motion vectors
# 7 = side by side edge mask and inverse edge view
# 8 = side by side motion mask and edge mask
# 9 = 4view of motion,static,edge and star masks


BUT there is a PROBLEM

the debug_view arguments are in the wrong order.
you have to replace lines 228-233 of the modified script with this :
(debug > 0 && debug < 10) ? debug_view(last, clp, debug, blocksize, overlap, thsad, ml, frames, strength, \
preprocess, postprocess, chroma, dct, maskmotion, maskstatic, maskedge, \
aggressive, bv1, dering, edgeclean, focus, anime, thscd1, thscd2,pel, sharp, rfilter, \
ls_x, ls_y, lsfstr, starfield, addnoise, maskstar, unblock, search, \
premax, sharpp, flow, temporal, edm_hi, edm_lo, thstar, lumathres, \
quant1, quant2, uv, truemotion, thsad, lambda, lsad, pnew, plevel, \
th_lo, th_hi, limit, mvglobal, fs1, fs2, fs3, fs4, prefast, premc) : last

[STJAM]

Thanks qwerty1983, I can't wait to evaluate MC_Spudsmod.

[caiyuwei]

I'm playing around with MC_Spudsmod and I don't understand why can't it use with QTGMC?

My script is

QTGMC(Preset="Slower", EZDenoise=0)
AssumeFPS("ntsc_film")
MC_Spudsmod()


I checked with AvsPmod and no error but when i encode, there's error output stream.
But for IVTC, it's sill ok? :confused: and my source is YV12

Is there anyway for it working with QTGMC?

Thank you

[manono]

QTGMC(Preset="Slower", EZDenoise=0)
AssumeFPS("ntsc_film")
MC_Spudsmod()
I'm curious about the script. Are you creating a slow motion video on purpose? Do you understand what QTGMC does? If you're trying to use it as an IVTC, that's not how.

I suspect your encode is crashing because you have two complex and slow filters in the same script. Maybe try dividing it into two encodes, the QTGMC part first to a lossless AVI and then the MC_Spudsmod.

Or, if you're using multi-threading, don't.

[caiyuwei]

I'm curious about the script. Are you creating a slow motion video on purpose? Do you understand what QTGMC does? If you're trying to use it as an IVTC, that's not how.

I suspect your encode is crashing because you have two complex and slow filters in the same script. Maybe try dividing it into two encodes, the QTGMC part first to a lossless AVI and then the MC_Spudsmod.

Or, if you're using a multi-threading, don't.

Thank you for your interesting :)
I just do deinterlace for the video. The scripts uses yadif as default. I just replace yadift by QTGMC.
I tried changing QTGMC ("Slow") to QTGMC ("Medium") but my encode is still crashing...

But for another videos using ITVC to interlace, it works well :confused:

[ajk]

You might be running out of memory, and also in general I haven't always had great luck with running several MVTools intensive filters at the same time.

If nothing else helps, just run QTGMC() separately to an intermediate, lossless AVI file, and then do the noise removal afterwards in another script.