Multithreading an own function -> Avisynth crashes

[LeKouz]

Hi.

I want to deinterlace only a few frames of a video. So, i set the video in 3 zones and call for every zone a function.
At first, this is the script:

##############################################################
#Main
##############################################################

#Set MTMode
#-------------------------------------------------------------
setMTMode(2,4)


#Color Settings
#-------------------------------------------------------------
ColorMatrix()


#Set zones
#-------------------------------------------------------------
part1 = last.Trim(1,3785).noninterlaced()
part2 = last.Trim(3786,3916).interlaced()
part3 = last.Trim(3917,180000).noninterlaced() #"180000" -> max. 2h with 25fps

Distributor()


#Combine zones
#-------------------------------------------------------------
together = part1 ++ part2 ++ part3


#Return video
#-------------------------------------------------------------
return(together)


##############################################################
#Functions
##############################################################

function noninterlaced(clip) {
clip

#Denoising
#----------------------------------------------------------------
super = MSuper(pel=2)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)

#Sharpening
#----------------------------------------------------------------
FFT3DFilter(bt=-1, sharpen=0.2)
}


function interlaced(clip) {
clip

#Deinterlacing
#----------------------------------------------------------------
QTGMC(Preset="Slow", Sharpness=1.0, FPSDivisor=2, ShutterBlur=3, ShutterAngleSrc=45, ShutterAngleOut=180)
}

Without "setMTMode" the script works fine.
But with, VirtualDub/avisynth crashed after a few frames.

Problem Event Name: APPCRASH
Application Name: VirtualDub.exe
Application Version: 1.9.11.0
Application Timestamp: 4d150e35
Fault Module Name: KERNELBASE.dll
Fault Module Version: 6.1.7601.17651
Fault Module Timestamp: 4e211319
Exception Code: e06d7363
Exception Offset: 0000b9bc
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 2055
Additional Information 1: 6574
Additional Information 2: 65749d29014f76f7de3a735bc1061484
Additional Information 3: fca2
Additional Information 4: fca25117ee1bcc1a6f8b81e4f1b5e967



So, how can I use the script multithreaded? The rendering with only 1 core takes to much time. :(

[Chikuzen]

Remove Distributor().
It has only bad influence on the recent applications except avs2pipe(mod).

[LeKouz]

Remove Distributor().
It has only bad influence on the recent applications except avs2pipe(mod).

Without Distributor(), after 42 frames VirtualDub crashed again :(
Problem Event Name: APPCRASH
Application Name: VirtualDub.exe
Application Version: 1.9.11.0
Application Timestamp: 4d150e35
Fault Module Name: KERNELBASE.dll
Fault Module Version: 6.1.7601.17651
Fault Module Timestamp: 4e211319
Exception Code: e06d7363
Exception Offset: 0000b9bc
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 2055
Additional Information 1: 9013
Additional Information 2: 90137f3a2a4aee2a862ee5b22fa401ce
Additional Information 3: 28c6
Additional Information 4: 28c6675dd9b0639677c904cc69e8b633


But isn't Distributor() needed to collect the "multithreaded" frames in the same order as they came in?

[SSH4]

Trims + QTGMC, mdegrain
are not stable on MT Avisynth :( but QTGMC is the first reason to fail on MT. Because it is so huge and complex script :(

If i use QTGMC, i made it in one avisinth script and out to huffyuv lusless avi.

[Groucho2004]

But isn't Distributor() needed to collect the "multithreaded" frames in the same order as they came in?

Yes, but VDub does that for you. If you add the call manually, you're unnecessarily increasing the number of threads.

In order to track down the problem:

1. Which version of Avisynth are you using?
2. What source filter are you using and which SetMTMode() statement did you place before it? (Post the entire script!)

3. Read this (http://forum.doom9.org/showthread.php?p=1312666#post1312666) and this (http://forum.doom9.org/showthread.php?p=1423459#post1423459).

[Taurus]

Trims + QTGMC, mdegrain
are not stable on MT Avisynth :( but QTGMC is the first reason to fail on MT. Because it is so huge and complex script :(

If i use QTGMC, i made it in one avisinth script and out to huffyuv lusless avi.
No,
I'm using trims, QTGMC, degrainers, sharpeners, colorizers, etc.
in one script all the time.
No crashes, almost 100% on all cores.
It's all about setting up your script in the right manner, using the right plugins, the right avisynth version and some good luck:p.
And a rock stable machine.
Thanks to all the developers, scripters for this wonderful tools.

Merry Christmas to all.

[LeKouz]

Yes, but VDub does that for you. If you add the call manually, you're unnecessarily increasing the number of threads.

In order to track down the problem:

1. Which version of Avisynth are you using?
2. What source filter are you using and which SetMTMode() statement did you place before it? (Post the entire script!)

3. Read this (http://forum.doom9.org/showthread.php?p=1312666#post1312666) and this (http://forum.doom9.org/showthread.php?p=1423459#post1423459).

I'm using now Avisynth-Version: 2.5.8.5 (16.04.2010)
It's the MT Version, so i don't need to load the MT.dll-Plugin.
I'll try it again with Avisynth 2.6 MT. The last time i had some errors, but can't remember the reason.

Here is the full script, with loaded plugins.
#Max Memory
#-------------------------------------------------------------
#SetMemoryMax(2*512)


#Aktivate Multithreading
#-------------------------------------------------------------
setMTMode(2,4)


#Plugins
#-------------------------------------------------------------
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\decomb.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ColorMatrix.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools_26.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrain.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\nnedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\FFT3DFilter.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\QTGMC_3.32.avsi")


#Source
#-------------------------------------------------------------
mpeg2source("episode01.d2v")



##############################################################
#Main
##############################################################


#Color Settings
#-------------------------------------------------------------
ColorMatrix()


#Set zones
#-------------------------------------------------------------
part1 = last.Trim(1,3785).noninterlaced()
part2 = last.Trim(3786,3916).interlaced()
part3 = last.Trim(3917,180000).noninterlaced() #"180000" -> max. 2h with 25fps


#Combine zones
#-------------------------------------------------------------
together = part1 ++ part2 ++ part3


#Return video
#-------------------------------------------------------------
return(together)


##############################################################
#Functions
##############################################################

function noninterlaced(clip) {
clip

#Denoising
#----------------------------------------------------------------
super = MSuper(pel=2)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)

#Sharpening
#----------------------------------------------------------------
FFT3DFilter(bt=-1, sharpen=0.2)
}


function interlaced(clip) {
clip

#Deinterlacing
#----------------------------------------------------------------
QTGMC(Preset="Slow", Sharpness=1.0, FPSDivisor=2, ShutterBlur=3, ShutterAngleSrc=45, ShutterAngleOut=180)
}

I need to test other settings. At first i'll update Avisynth to 2.6 MT and check the plugins...
THX @ all for the feedbacks :)

[Taurus]

mt_masktools_26.dll = wrong version for avisynth 2.58. Use mt_masktools_25.dll
Try SetMTMode 3 or 5 before your source filter (mpeg2source)
SetMTMode 2 for your heavy filters.
Good luck :p

[LeKouz]

Now it works.
Changed AviSynth to 2.6 MT.
I use now "SetMemoryMax(2*512)". The script needed more memory.
For QTGMC i use "SetMTMode(3)" because with "setMTMode(2)" it crashed after a few frames. It's slower than with Mode 2, but the 130 frames, which needed to deinterlace, are not much, so it's worth to ignorate the speed :).

So this is the final script:
##############################################################
#Initialize
##############################################################


#Define max. memory
#-------------------------------------------------------------
SetMemoryMax(2*512)


#Activate multithreading
#-------------------------------------------------------------
setMTMode(2,4)


#Load Plugins
#-------------------------------------------------------------
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\decomb.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ColorMatrix.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools_26.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrain.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\nnedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\FFT3DFilter.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\QTGMC_3.32.avsi")


#Source
#-------------------------------------------------------------
setMTMode(5)
mpeg2source("episode01.d2v")


##############################################################
#Main
##############################################################


#Set zones
#-------------------------------------------------------------
setMTMode(2)
part1 = last.Trim(1,3785).noninterlaced().ColorMatrix()
setMTMode(3)
part2 = last.Trim(3786,3916).interlaced().ColorMatrix()
setMTMode(2)
part3 = last.Trim(3917,180000).noninterlaced().ColorMatrix()


#Combine zones
#-------------------------------------------------------------
together = part1 ++ part2 ++ part3


#Return video
#-------------------------------------------------------------
return(together)


##############################################################
#Functions
##############################################################

function noninterlaced(clip) {
clip

#Denoising
#----------------------------------------------------------------
super = MSuper(pel=2)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)


#Sharpening
#----------------------------------------------------------------
#FFT3DFilter(bt=-1, sharpen=0.2)
}



function interlaced(clip) {
clip

#Deinterlacing
#----------------------------------------------------------------
QTGMC(Preset="Slow", Sharpness=1.0, FPSDivisor=2, ShutterBlur=3, ShutterAngleSrc=45, ShutterAngleOut=180)
}

[kypec]

#Load Plugins
#-------------------------------------------------------------
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\decomb.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ColorMatrix.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools_26.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrain.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\nnedi3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\FFT3DFilter.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\QTGMC_3.32.avsi")


You don't need to load plugins that are stored in Avisynth's autoload directory manually -> either move those DLLs to another directory or remove all LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\???.dll lines from your script.