VDUB crash with avisynth

[gwendolien]

I'm recently having problems with VDUB/Avisynth usage.
VDUB is used to write the result video of the avisynth script.
After a restart of the PC, most of the times the avs script I'm using loads OK. But after a variable numer of frames treated by VDUB, VDUB crashes.

Then, VDUB crashes every time I try to load the avs script in a new instance of VDUB.
Restarting the PC the avs loads fine again, but, VDUB crashes after another number(random) of frames. etc. etc.

Going through the video in avisynth itself, produces NO unexpected failures/flaws...

The crash of VDUB always states an "access error/violation".
What could possibly cause this behaviour?

[Guest]

Please post the script that causes crashing.

[gwendolien]

Thanks for the swift response neuron2.

# BECK_RECONSTRUCTIE_C.AVS
LoadPlugin("C:\AV_Stuff\Video\Avisynth\Plugins\RT_Stats26.dll")
LoadPlugin("C:\AV_Stuff\Video\Avisynth\Plugins\GrunT.dll")
LoadPlugin("C:\AV_Stuff\Video\Avisynth\Plugins\GScript.dll")
SetMemoryMax(448)
SetMTMode(2,5)

dvd_t=AviSource("G:\DVD_Deinterlaced_Cropped.avi")#.ConvertToYV12#
SetMTMode(2)
Global dvd=dvd_t.ConvertToRGB24().Crop(1, 0, 700, 494).ConvertToYV12
SetMTMode(2,5)

vhs_t=AviSource("G:\VHS_Deinterlaced_Cropped.avi").ConvertToYV12#.ColorYUV(Levels="PC->TV")#SOURCE is YUY2
SetMTMode(2)
vhs_t=vhs_t.Lanczos4Resize([<"Wf", 760, 790, 776>],[<"Hf", 480, 510, 494>])
bg=BlankClip(vhs_t, Length=vhs_t.FrameCount, Width=870, Height=508, color=$000000)
vhs_u=bg.Overlay(vhs_t, opacity=1.0, x=[<"Xc", 0, 28, 17>], y=[<"Yc", -10, 10, 7>])#.ConvertToYV12(matrix="PC.709")# OVERLAY OUTPUT RGB32
vhs_v=vhs_u.Crop(14, 4, 776, 494)#.ColorYUV(off_y=6, gain_y= 50, cont_y=110, off_u=-0, cont_u=100, off_v=-1, cont_v=6)

dvdL=dvd.Crop(0, 0, 34, 0)
dvdR=dvd.Crop(dvd.Width-42, 0, 0, 0)
dvdvhs=StackHorizontal(dvdL, dvd, dvdR)
vhs_w=vhs_v.HistogramAdjustRGB(dvdvhs)
Global vhsfull=vhs_w.Overlay(dvd.Crop(100, 394, 500, 70), opacity=1.0, x=132, y= 394)
Global vhsdvd=vhsfull.Crop(32, 0, 700, 494)

vhs_replace=vhsfull # vhsfull / vhsfull0 / vhsfull1 / vhsfull2 / vhsfull3
vhs_replace_a=vhs_replace.Tweak(Bright=-0, Hue=-0, Cont=1.0, Coring=false)#.RGBAdjust_gwe(6, -9, 0) # LEFT side version
vhs_replace_b=vhs_replace.Tweak(Bright= 5, Hue=0, Cont=1.0, Coring=false)#.RGBAdjust_gwe(6, -9, 0) # RIGHT side version
vhsfullmaskright=GraMaMa(vhsfull, mode=4, a=0, rad=vhsfull.Width).ConvertToRGB32
Global vhs_corr=Overlay(vhs_replace_a, vhs_replace_b, 0, 0, Mask=vhsfullmaskright) # GRADUALLY CHANGE FORM VHS_REPLACE_A @ LEFT TO VHS_REPLACE_B AT RIGHT

Global vhs_aligned=ScriptClip(vhs_corr, """LumaCorrelation(vhsdvd, dvd, 7, 11, 32)""")# for aligning the VHS to the DVD

# CREËER DE BENODIGDE MASKERS VOOR HET VLOEIEND LATEN OVERLOPEN VAN HET VHS MATERIAAL NAAR HET DVD MATERIAAL

leftwidth=Ceil(vhsfull.Width/4)+Int((vhsfull.Width+1)/4)
rightwidth=vhsfull.Width-leftwidth
br=4*[<"Breedte", 1, 26, 10>]
maskleft1=GraMaMa(vhsfull.Crop(0, 0, 400+br, 0, align=true), mode=4, a=30+int(0.80*(br)), b=494, rad=br).Levels(16, 1, 235, 0, 255, coring=false)
left=BlankClip(maskleft1, Width=600, color=$000000)
maskleft=maskleft1.Overlay(left, opacity=1.0, x=30+int(0.80*br)).Crop(0, 0, leftwidth, 0, align=true)
maskright1=GraMaMa(vhsfull.Crop(378-br, 0, 392+br, 0, align=true), mode=4, a=342-int(0.85*br), b=494, rad=br).Levels(16, 1, 235, 0, 255, coring=false)
right=BlankClip(maskright1, Width=342-2*int(0.425*br), color=$000000)
maskright=maskright1.Overlay(right, opacity=1.0, x=0).Crop(0, 0, vhsfull.Width-leftwidth, 0, align=true)
maskfull=StackHorizontal(maskleft,maskright).Levels(16, 1, 235, 0, 255, coring=false)#.IsFormat
vhs_left=vhs_aligned.Crop(0, 0, 34, 0, align=true)# VHS aligned to DVD
vhs_right=vhs_aligned.Crop(734, 0, 0, 0, align=true)# VHS aligned to DVD

# GENERATE FINALLY THE OUTPUT.....

rslt0=vhs_aligned.Overlay(dvd, Mask=MaskFull.Invert.Crop(32, 0, -44, 0), x=37)# for VHS2DVD alignment
#rslt0=vhs_aligned.Overlay(vhs_corr, Mask=MaskFull, x=0)# for VHS2DVD alignment
return rslt0

#=================================================================================================================================================

Function IsFormat(clip c)
{
text=IsYV12(c)?"YV12":ISYUY2(c)?"YUY2":IsRGB24(c)?"RGB24":IsRGB32(c)?"RGB32":"other"
return Subtitle(c, text+"\n"+String(c.Width)+"x"+String(c.Height), size=13, y=c.height-26, lsp=0)
}


#==============================================================================================================================================

Function LumaCorrelation(clip clip0, clip clip1, clip clip2, int radX, int radY, int off)
{
Width1=clip0.Width()
Height1=clip0.Height()
clipz=clip0.Lanczos4Resize(2*Width1, 2*Height1)
X=2*radX
Y=2*radY
W=-2*radX
H=-2*radY
BestX=255
BestY=255
diff=0
n=current_frame
S=""
clipa=clip1.Lanczos4resize(2*clip1.Width, 2*clip1.Height)
clipb=clip2.Lanczos4resize(2*clip2.Width, 2*clip2.Height)
start=RT_Timer()
GScript("""
for (i=-radX, radX, 1){
for (j=-radY, radY, 1){
tdiff=RT_LumaCorrelation(clipa, clipb, n=n, x=X, y=Y, w=W, h=H, n2=n, x2=X+i, y2=Y+j, interlaced=false)
if(tdiff>diff){
diff=tdiff
BestX=i
BestY=j
}
# WriteFileStart(clip0, "LumaDiff.txt", String(i)+" / "+String(j)+" | "+String(diff), append=true)
}#j=j+1 loop
S=S+"\n"
}#i=i+1 loop
""")
end=RT_Timer()
duration=String(end-start,"%.2f")
bg=clipz.BlankClip(Length=clipz.FrameCount, color=Color_Black)
result=bg.Overlay(clipz, x=BestX, y=BestY, opacity=1.0).Bicubicresize(Width1, height1)# for VHS2DVD alignment
#result=clipz.Overlay(clipb, x=2*off-BestX, y=-BestY, opacity=1.0).BicubicResize(Width1, height1)# for DVD2VHS alignemnt
#WriteFileStart(clipz, "Beck_analysis.txt", String(n), """ ";" """, String(BestX), """ ";" """, String(BestY), """ ";" """, String(diff), append=true)
return result # FOR REAL OUTPUT
return result.Subtitle("X="+String(BestX)+"\nY="+String(BestY)+"\n"+String(diff)+"\n"+duration, x=30, size=15, lsp=0)#FOR DEBUGGING
}

#===========================================================================================================================================================

Function HistogramAdjustRGB(clip clip1, clip clip2)
{
clipared=clip1.ConvertToRGB24.ShowRed("YV12")
clipagreen=clip1.ConvertToRGB24.ShowGreen("YV12")
clipablue=clip1.ConvertToRGB24.ShowBlue("YV12")
Global clipbred=clip2.ConvertToRGB24.ShowRed("YV12")
Global clipbgreen=clip2.ConvertToRGB24.ShowGreen("YV12")
Global clipbblue=clip2.ConvertToRGB24.ShowBlue("YV12")
cliprsltred=clipared.ScriptClip("""HistogramAdjust(type="match", mclip=clipbred, mf=current_frame)""")
cliprsltgreen=clipagreen.ScriptClip("""HistogramAdjust(type="match", mclip=clipbgreen, mf=current_frame)""")
cliprsltblue=clipablue.ScriptClip("""HistogramAdjust(type="match", mclip=clipbblue, mf=current_frame)""")
return MergeRGB(cliprsltred, cliprsltgreen, cliprsltblue).ConvertToYV12#(Matrix="PC.709")
}


I tested also with VFAPI and just "video output", not the VFAPI codec.
The script is loaded properly, but the output size seems to be limited ther to 4GB file size. The processing continues for a while, with the filesize staying constant at 4GB....
Don't know what the final outcome will be, the whole clip takes about 13 hours(estimated)

[StainlessS]

What is this ? (something to do with AVSPMod ?)


vhs_t=vhs_t.Lanczos4Resize([<"Wf", 760, 790, 776>],[<"Hf", 480, 510, 494>])

[gwendolien]

Yes, Stainless, I'm using AvsPMod here to develop/test scripts.

[StainlessS]

SEE two posts later.

EDITED:

[gwendolien]

Thanks StainlessS.
I'm using avisynth 2.6.0.3(MT version)
Should the MergeRGB be replaced with MergeRGBA?
I've terminated the VFAPI-process because the thing simply does not produce antything bigger than 4GB while VFAPI states a steady increasing % of file handled...
I'm now trying to use the installed VFAPI codec and then via the fake avi file produced, load the clip in VDub and write an uncompressed version of the avisynth filtered output.(while going to bed myself now...)

[StainlessS]

There is a lot of conversion to rgb24 in that func, there, do it once for each input clip and then use those instead.

EDIT: NO Problem in v2.6a4, problem originated in MergeRGB() in RGBManipulate() plugin.

EDITED:

[gwendolien]

The second VFAPI attempt produces also a VDUB crash, this time after around 39000 frames. Going through the output avi, I noticed a random occurrance of the following message as in the frame attached. This behaviour is ABSENT when viewing manually through the clip in AVSPMod. What is this message about? Could this have any bearing to the VDUB crashes?

The line referred to was:
clipz=clip0.Lanczos4Resize(2*Width1, 2*Height1)
in function LumaCorrelation

[IanB]

Unfortunately the total error text message is long and the ScriptClip error text painter does not wrap long text.

Best guess is you are tripping line 140 in avisynth2/avisynth/src/filters/resample.cpp...
catch (SoftWire::Error err) {
env->ThrowError("Resize: SoftWire exception : %s", err.getString());
}
...This normally only happens with incorrect code during development and usually complains about an incorrect instruction element not the SoftWire::Assembler::x86 class itself.

Now this code is very well tested and you ran it successfully 39000 times, so something else is in play and Occam's razor would have to assume that a malloc failed due to insufficient or fragmented memory.


What is the total process memory usage just before the crash?


If you use an official Avisynth build without MT does the script run to completion?


You could possibly restructure your script with the mindset "ScriptClip only the decision, never the code". This avoids continuously building and tearing down script graphs which tends to expose any memory leaks in plugins. ConditionalSelect (http://avisynth.nl/index.php/ConditionalSelect) and friends can be helpful here.

[Mole]

Often SetMTMode(2) is simply not stable with certain script.

Try to use SetMTMode(3) or SetMTMode(5) and see if it's still crash the same way. If not, then you know it's the MTMode.

Also, why are there several places with
SetMTMode(2,5)
SetMTMode(2)

Aren't we supposed to use SetMTMode(2,5) only once at the beginning, then SetMTMode(2) later.
Also, I've heard it's recommended to use SetMTMode(3,5) for video input, then change to SetMTMode(2).

When you use SetMTMode(2,5) at the beginning, another SetMTMode(2) is redundant, isn't it?

[gwendolien]

Thanks IAnB
"What is the total process memory usage just before the crash?"
answer: approimately 700 MB as I recall it.

If you use an official Avisynth build without MT does the script run to completion?
Answer: don't know because the 3 fps then drops to below 1 fps....

You could possibly restructure your script with the mindset "ScriptClip only the decision, never the code". This avoids continuously building and tearing down script graphs which tends to expose any memory leaks in plugins. ConditionalSelect and friends can be helpful here.
Answer: I'm afraid I can not follow your reasoning/advice here: my knowledge of Avisynth is simply too rudimentary, I'm afraid.
Could you be somewhat more specific in what you are aiming at?

[gwendolien]

Trying to get errorfree results I got the following observations:

When loading the VFAPI avi into VDub sometimes Vdub crashes;I restart of the PC then almost always gives a succesfull loading into VDub.

When I start with "save as" In VDub right after the loading of the VFAPI avi, VDub always freezes, while the CPU-percentage stays around 13%, but nothing happens at least for minutes.

When I wait say a minute after the successfull loading of the VFAPI avi, VDub responds indeed to the "save as" command and produces a valid avi file as result.

Because I seldom/never saw any strange behaviuor as noted in a couple of my posts earlier BEFORE say 14000 frames, I now am dealing with the clip in chunks of 10 000 frames a time and that process seems to result always in flawless output....

But of course one wonders what is causing this erratic behaviour....

[gwendolien]

Thanks Mole for your response.
I guess my (lack of)knowledge/understanding the MT stuff is debet to these scriptlines.
Your reasoning seems logical to me: I will give it a try tonight with the whole 145 000 frames clip and see what comes out tomorrow morning...

[gwendolien]

After 45 hours the resultant avi appeared to be OK!!
Remains the question of the response of VDub when loading the VFAPI script with the SetMTMode(2,5) statements in it(after ca. 40 000 frames or so...)

Thanks to all who responded!