MC_Spuds.avs (Jan. 08): Does anyone have this version?

[moviefan]

Hi guys,

I have used MC_Spuds() yesterday for a test encode with the version from January 08, which I had downloaded a couple of months ago, and found out, it was very good and pretty fast. Today, I thought, I could update to the latest one and was shocked by the extremely slow filtering speed. Unfortunately, I replaced the old by the new version and now don't have the nice fast one anymore. Does anyone have the MC_Spuds.avs from January 08? If yes, could you please post the code?

Regards

[talen9]

I don't have it ... but, looking at how things went in this post (http://forum.doom9.org/showthread.php?p=1093464#post1093464) from the official thread, something surely wasn't working the way it was supposed to be, in MC_spuds.avs pre-dating February '08 ... maybe your issue with speed is somewhat linked to the bug? :confused:

[2Bdecided]

I have a version I saved on 22nd November 2007. Maybe mc_spuds himself archived all the versions.

Cheers,
David.

[K0zi]

This one?

http://www.mediafire.com/?gz5gdbjogzb

[moviefan]

That's the one, thanks! Now, can anyone tell me, why it's so damn fast?

[Didée]

Because that version is (halfway) broken.

In script line#171, there's "return(clp2)", which should be "return(last)". In effect, the script does nothing of all the nifty additional stuff that it's supposed to do: no softening for motion areas, no sharpening for static areas, no post-processing, no edgecleaning, no nothing. Except for "starfield", which does go into "clp2" early enough.
Subsequently, the result of that version should be quite similar to what was suggested here (http://forum.doom9.org/showthread.php?p=1125396#post1125396). :p

BTW, MCSpuds is a great script, but ... it's a bit of a jungle, when you're in search for a specific issue with(in) it ...

[moviefan]

Sh*t, I already suspected, there would be something wrong with super filtering in super time...

Edit: Now that I've accepted, I had a bad script version, I looked closer at the current version and I think I'll have to upgrade my system with a quad core CPU... The filtering takes ages right now with my P4 2,66 GHz, that's completely unacceptable, but the quality is amazing.

[thetoof]

Having a Quad-core doesn't necessarily means that you'll have a big speed boost when filtering... that's what I have and I only use 1 thread with TemporalDegrain because if you use MT("filter"), it'll split the images, which is bad for motion vectors calculation (that is also done in mc_spuds) because a vector can't go from a part to another without having a very high overlap, which will be very slow anyways. The SetMTmode, even if the documentation says it'll work with it, can produce weird bugs where frames appear 3-4 frames before when they should... maybe it can be solved with a SetCache() call or something like that, I don't know. Also:
If you should go on to re-design mvtools, would you also implement true multithreading?
The current issue with mt is that the frame needs to be split up into several slices, which prevent motion-vectors to go from one slice to another.
Probably you say about MT() plugin.
Sorry, currently I do not plan to redesign MVTools in general and for M.T. specifically.

[Didée]

Those problems when using SetMTMode probably are due to too huge memory requirements. Using only one thread, the memory requirements for denoising scripts with nested MVTools operations can easily be in range of 0.5 to 1 GB. When you use sth like SetMTMode(TheFullVeryComplexScript,threads=many), you're (basically) running the script many times in parallel, wich means that memory requirements should scale somewhat linear with the number of threads.
First, this means that *lots* of RAM is needed.
Second, Win32 has a 2GB process limit ... :D

For scripts with "complex" MVxxx-filtering, it would be most beneficial to fiddle the mt-stuff into the guts of the script.

Basic example:

function foo( clip source)
{
[vector search...]
filter1 = source .MVDegrain3( [vectors...] )
filter2 = filter1 .MVDegrain3( vectors...])
return( filter2 )
}

When putting such a script as-is into SetMTMode, RAM requirements grow very fast, and you might end with no speedup or erroneous results due to limited RAM. What happens is that each thread on its own does the nested MVDegrain().MVDegrain() - operation. That doesn't seem efficient already at first sight ...

Better would be to split the work within the script: one thread for the 1st MVDegrain, cache the result, one thread for the 2nd MVDegrain, cache the result, one thread for the rest of the work that's in the script.
That's a very rough outline (I've zero experience with multithreading) - but it should give the idea:

When there's lots of work to do, it's not sufficient to just set 20 persons at it. You also need a plan for how to distribute the work among the 20, so that they can work efficiently.

[thetoof]

Second, Win32 has a 2GB process limit ... :D Oh that's why! My RAM usage was extremely high, but I thought it was fine since I have 4GB... meh...:angry:windows:angry:
Better would be to split the work within the script: one thread for the 1st MVDegrain, cache the result, one thread for the 2nd MVDegrain, cache the result, one thread for the rest of the work that's in the script.
1st question that comes to my mind is how?? Setcache, like you used here (http://forum.doom9.org/showthread.php?p=1123904#post1123904)?

Or, if you don't know since you have no MT experience, does anyone out there know how? tsp, any hints?

edit: oh well, seems like I have a talent for taking moviefan's threads a little bit off topic... sorry 'bout that. :p

[Spuds]

Because that version is (halfway) broken. Oh I think we can call that one fully broken :) I was looking at the starfield function at that point in time so thats what I was outputting, then stupid me forgot to turn everything back on!

The SetMTmode, even if the documentation says it'll work with it, can produce weird bugs where frames appear 3-4 frames before when they should I personally have not seen this but all my computers have 2G+ so that may have something to do with it, but who knows for sure. You can also cause yourself significant grief with wrong idx usage and setmtmode. In my usage it generally takes 5-6 hours to clean an hour of std def video when in setmtmode(2,2)

it's a bit of a jungle, when you're in search for a specific issue Now that is a fact that I will back up! I keep compartmentalizing things in every release (fancy way of saying adding task specific subroutines) to help with this task, not to mention all those debug modes :p

Better would be to split the work within the script That is something to ponder ... not sure how to accomplish this with the current set of tools.

[Zep]

Oh that's why! My RAM usage was extremely high, but I thought it was fine since I have 4GB... meh...:angry:windows:angry:

1st question that comes to my mind is how?? Setcache, like you used here (http://forum.doom9.org/showthread.php?p=1123904#post1123904)?

Or, if you don't know since you have no MT experience, does anyone out there know how? tsp, any hints?

edit: oh well, seems like I have a talent for taking moviefan's threads a little bit off topic... sorry 'bout that. :p

did you ever figure this out? get it into the guts of the script like Didee said and if so what happened? I need to do this also :)

[thetoof]

Nope, still waiting for the answer...

[Zep]

Nope, still waiting for the answer...

look at the MT thread I found a work around that works great in just 5 minutes using google :)