Dust Filter and Quad Core...

[Gannjunior]

Hi guys,

few days ago i did my first VHS encoding with my new quadcore.

I used one of my pluri-tested script for old VHS that obviously included the Goldust for the space denoise.
In this years i never found neither (for specific use in VHS recover) better than dust, for what concern the union of cleaning/'reduced' deterioration of image and easy use of its setting. (obviously, you know, it always been preistorically slow! :cool: ).

So, which is my problem? If goldust(also pixiedust etc) is in the avs script, the system uses only max 40% of cpu. tried with divx, xvid and nero recode.
I'm sure that the problem is with dust, because I built the script adding line after line and testing it: in each combination of the script without dust the cpu utilization was at about 75-80% that is what you expect by multicore cpu. When i introduced goldust it dropped to 40%.
That is a nosense. Obviously the dust has to slow the encoding but not the cpu utilization!
You can understand my frustration: i passed years to encode with my old system at 3-4 fps in my vhs recovered by dust, and now with new multicore cpu i wanted to see more human encoding time, but...

Uh, tested both in xp32 and Vista64, the same result...

any suggest to solve this problem or alternatives to dust (powerful or better than it) ?

TIA

ciao!

[FlimsyFeet]

The exapmle at the bottom of the first post in this thread may help you:
http://forum.doom9.org/showpost.php?p=660247&postcount=1

[Fizick]

The only known way to use Dust is two source, two scripts, two encode at once.

Alternative? Use search :)

[Gannjunior]

thanks flimsyfeet.
just tried the mt.dll, but my system uses always only one core if dust is inside the script...

fzick yes i tried to search many others filters but never satisfies me as dust. any indication on the fly?:)

[cestfait]

try fizick's stuff, in particular mvtools.

[tsp]

Gannjunior: Dust wouldn't work with mt.dll/setmtmode but running two separate encodes at the same time would work. You just need to splice the result together afterwards.

[FlimsyFeet]

RemoveNoiseMC (http://forum.doom9.org/showthread.php?t=110078) is a script function written as an alternative to Dust. I've used it on VHS captures and think it gives good results.

... but it uses MVtools, so will not run multithreaded either (see posts below).

[Didée]

Note that all functions based on MVTools do not work correctly with multithreading, either ...

For those you have to the same route as with Dust: Splicing in pieces, load & encode separately (as to use 'multithreading'), lastly put the splices together again.

[IanB]

@Didée,

TSP and Fizick have been waving their magic wands and an MT safe version of MVTools is pretty close.

[Didée]

I've heard the rumours, yep. But it's not there as of now, and consequentially the above recommendations about using MVTools-Denoisers are misrecommendations. Right now, it won't work.

[Fizick]

Didée, right now MVTools v1.8.4.2 should work with MT. Nobody reported about bugs.
Currently we try to decrease memory usage, but it is other story.
And we need in more beta-testers
:)

[Didée]

Oops, I was not aware about that version, sorry.
(Was it announced at all on this forum ... ?)

But I can't (really) test it anyway: insufficient hardware. :)

[Fizick]

It was born in MT thread. See around this post:
http://forum.doom9.org/showthread.php?p=1058626#post1058626
(my hardware is not appropriete too)

[Gannjunior]

Didè, tsp, Fizick...Thanks to all.

As you suggest, splitting the work in two part is the only way to get the 100% of 4 core:

http://img145.imageshack.us/img145/646/dustfk3.th.jpg (http://img145.imageshack.us/my.php?image=dustfk3.jpg)

:-)

The only problem is that i can't automatize it completely because of bitrate distribution. I have to run each first pass together and see wich is the respective saturation level. So i can assign the "enough" bitrate for each part in the second pass. (off course i can previously run also a CT for each part and decide ex ante the reasonable bitrate).

I also tried RemoveNoiseMc and it's pretty filter, but dust is dust and i'm ready to "complicate" my encoding to continue to use it.

PS here it is my script :




#SetWorkingDir("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\")

SetMemoryMax(256)

#LoadPlugin("C:\PROGRA~2\GORDIA~1\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~2\GORDIA~1\AviSynthPlugins\decomb.dll")
LoadPlugin("C:\PROGRA~2\GORDIA~1\AviSynthPlugins\UnDot.dll")
#LoadPlugin("C:\PROGRA~2\GORDIA~1\AviSynthPlugins\SimpleResize.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\peachsmoother.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\unfilter.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\LoadPluginEx.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\DustV5.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\mpasource.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\masktools.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\warpsharp.dll")
import("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\seesaw.avsi")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\removegrain.dll")
Loadplugin("C:\Program Files (x86)\GordianKnot\AviSynthPlugins\repair.dll")

raw_video=avisource("E:\reflex_vhs.avi").Tweak(hue=1,bright=-3,cont=1,sat=1).FieldDeinterlace()

audio_raw=wavsource("e:\reflex_audition_denoised.wav").DelayAudio(-0.12)

mix=audiodub(raw_video,audio_raw)

########

#KernelDeInt(order=1,sharp=true)


#######

trim_ep_20=trim(mix,9909,58355)+trim(mix,66328,78200)

ep20=trim_ep_20.crop(8,8,464,560).BicubicResize(480,400,0,0.5)

ep20_filtering=ep20.converttoyuy2().PeachSmoother(NoiseReduction=80,Stability=25,Spatial=1).GoldDust(3).Undot().Converttoyv12()

video_finale=SeeSaw(ep20,ep20_filtering, NRlimit=3, NRlimit2=4, Sstr=1.5, Slimit=5, Spower=5, Sdamplo=6, Szp=16)

#.

return video_finale

#####





ciao!! ;)

[Boulder]

One option is to use a lossless intermediate file (for example ffdshow's HuffYUV in YV12 colorspace) for each part which you then splice and encode to your final format.

[IanB]

Perhaps someone can track down Steady and plead for the source code. Then someone might make it A) a 2.5 filter, B) multi-instance safe and C) thread safe.