On processing a file in the same script that saved it

[tin3tin]

This is the workflow: an automatically generated avisynth script saves a ssa subtitle file timed to the various edits in the script. In the end of the avisynth script the saved ssa subtitle file is loaded and added to the videoclip with vsfilter.

So in other words a script saves a file(with multiple appends) and in the end of the script the file is loaded and used by the same script.

This process unfortunately goes wrong on longer scripts with a lot of writefile-appends, because I think the script starts to open the saved file before all the writefiles are finished saving to disk, and I get an writefile error.

A simple solution would be if vsfilter would be able to open a string variable instead of a file or longer strings would be allowed in writefile: http://forum.doom9.org/showthread.php?t=147536
But until that might happen does anyone have an idear on how to force vsfilter to wait loading the file until all writefile functions are finished saving to disk?

[Gavino]

Are you talking about WriteFile or WriteFileStart here?

WriteFileStart closes the file on each write so it should not cause problems with later reads. With WriteFile, you can use flush=true to force this behaviour. However, WriteFile does its work 'per-frame' during frame serving so I don't think you can use it to write the entire file before starting to read it.

Could you post an outline of your script to show the relationship between the writing and the reading?

[tin3tin]

Yes, I'm using writefilestart.

Here's an example(the writefilestart error shows itself with slideshows consisting of 200+ slides on my computer(hhd speed influence?)):

# Script gererated by DVD slideshow GUI
loadplugin("C:\Program Files\DVD slideshow GUI\bin\vsfilter.dll" )

# Definitions:
SafeH = 380 
SafeW = 576
SafeAviH = multifour(380) # resizing avis within width and height 
SafeAviW = multifour(576)

# Import of background image - the proportions of this will be the proportions of the final film. 
bname = "C:\Program Files\DVD slideshow GUI\background_golden.jpg" 
SafebH = 480 # resizing background within width and height 
SafebW = 720 
rate = 29.970
bname = LanczosResize(imagesource(bname,1,1,rate),SafeBW,safeBH)
a = bname
at = bname
b = bname
bt = bname

# Import of audiotrack to be faded out with last still 
loadplugin("C:\Program Files\DVD slideshow GUI\bin\NicAudio.dll" )
audiotrack = NicMPG123Source("C:\Users\Tin2tin\Music\sample.mp3").eqaudio() 
am = bname
am = loop(am,int(29.970*(audiotrack.AudioLengthF/audiotrack.AudioRate)))
music = AudioDub(am,audiotrack) 
music = Loop(music, int(375/music.Framecount())+1)
Try {
imgout=ImageSource("C:\Users\Tin2tin\.DVDslideshowGUI\_temp_files\%06d.ebmp", 0,3, rate)
imgout=imgout.Levels(32, 1, 235, 16, 235, coring=false)
}
Catch(err_msg) {
imgout=blankclip(1,720,480, fps=29.970).converttoyv12()
}
SafeH = 480 
SafeW = 720

# The filename and location of the exported ssa file with subtitle values for each text. 
ssa = ""
writefilestart(blankclip(),"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = false)
ssafile = "C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp_subtitles.ssa" 
ssa = ssa + "[Script Info]"+ chr(10)
ssa = ssa + "ScriptType: v4.00"+ chr(10)
ssa = ssa + "Collisions: Normal"+ chr(10)
ssa = ssa + "PlayResX: 720"+ chr(10)
ssa = ssa + "PlayResY: 480"+ chr(10)
ssa = ssa + "Timer: 100,0000"+ chr(10)
ssa = ssa + chr(10)
ssa = ssa + "[V4 Styles]"
writefilestart(BlankClip(),"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""
ssa = ssa + "Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, TertiaryColour, BackColour, Bold, Italic, "
ssa = ssa + "BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, AlphaLevel,Encoding"
writefilestart(BlankClip(),"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""
ssa = ssa + "Style: Default,Arial,30,16777215,65535,0,0,0,0,1,2,0,2,20,20,70,0,1"
writefilestart(BlankClip(),"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""+ chr(10)
ssa = ssa + "[Events]"+ chr(10)
ssa = ssa + "Format: Layer, Start, End, Style, Actor, MarginL, MarginR, MarginV, Effect, Text"+ chr(10)
writefilestart(BlankClip(),"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""
ssa = ssa + "Dialogue: 0,"+ convert2hhmmscff(0, rate)+ ","+ convert2hhmmscff(100, rate)+",Default,NTP,0000,0000,0000,!Effect,{\pos(360,420)}subtitle" + chr(10)
writefilestart(BlankClip(),"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""

# PROGRAM 
# Slide no:1
inpoint = 1
a = (trim(imgout,000001,000001)) 
a = loop(a,100)
at = (trim(imgout,000001,000001)) 
at = TransPrep(at,25)
musicbite = trim(music,1,a.framecount) 
a = audiodub(a,musicbite)
musicbite = trim(music,a.framecount,framecount(a+audiodub(at,BlankClip(length=at.framecount, fps = rate, audio_rate=48000, stereo = true)))) 
at = audiodub(at,musicbite).eqaudio()
outpoint = a.framecount+at.framecount

# Transition no:1
bt = (trim(imgout,000002,000002)) 
bt = loop(bt,25)
musicbite = trim(music,framecount(a),framecount(a+at)) 
bt = audiodub(bt,musicbite).eqaudio()
at=at.SSRC(48000,fast=true) 
bt=bt.SSRC(48000,fast=true) 
ft=25
c = (import("C:\Program Files\DVD slideshow GUI\transitions\Dissolve.avs"))
c = audiodub(c,BlankClip(length=c.framecount, fps = rate, audio_rate=48000, stereo = true))
d = a + c
ssa = ssa + "Dialogue: 0,"+ convert2hhmmscff(d.framecount+1, rate)+ ","+ convert2hhmmscff(d.framecount+100+1, rate)+",Default,NTP,0000,0000,0000,!Effect,{\pos(360,420)}subtitle"+ chr(10)

writefilestart(d,"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""

# Slide no:2
a = (trim(imgout,000002,000002)) 
a = loop(a,100)
at = (trim(imgout,000002,000002)) 
at = loop(at,25)
musicbite = trim(music,framecount(d),framecount(d+audiodub(a,BlankClip(length=a.framecount, fps = rate, audio_rate=48000, stereo = true)))) 
a = audiodub(a,musicbite).eqaudio()
musicbite = trim(music,framecount(d+a),framecount(d+a+audiodub(at,BlankClip(length=at.framecount, fps = rate, audio_rate=48000, stereo = true)))) 
at = audiodub(at,musicbite).eqaudio()
outpoint = d.framecount+a.framecount+at.framecount

# Transition no:2 - Stillimage
bt = (trim(imgout,000003,000003)) 
bt = loop(bt,25)
musicbite = trim(music,framecount(d+a),framecount(d+a+audiodub(at,BlankClip(length=at.framecount, fps = rate, audio_rate=48000, stereo = true)))) 
bt = audiodub(bt,musicbite).eqaudio()
at=at.SSRC(48000,fast=true) 
bt=bt.SSRC(48000,fast=true) 
ft=25
c = (import("C:\Program Files\DVD slideshow GUI\transitions\Dissolve.avs"))
c = audiodub(c,bt).eqaudio()
d = d + a + c
ssa = ssa + "Dialogue: 0,"+ convert2hhmmscff(d.framecount+1+25, rate)+ ","+ convert2hhmmscff(d.framecount+100, rate)+",Default,NTP,0000,0000,0000,!Effect,{\pos(360,420)} subtitle"+ chr(10)
writefilestart(d,"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""

# Slide no:3 - last

b = (trim(imgout,000003,000003)) 
b = loop(b,100)
outpoint = d.framecount+b.framecount
musicbite = trim(music,framecount(d),framecount(d)+100)
b = audiodub(b,musicbite).eqaudio()
d = d + b 
d = pointchk(inpoint, outpoint) ? d : d.trim(0,inpoint)+audiodub(d.trim(inpoint,outpoint),fadeout2(audiodub(d.trim(inpoint,outpoint),music.trim(inpoint,outpoint).fadein2(25)),25))
writefilestart(d,"C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp"+".ssa","ssa",append = true)
ssa = ""
last=0
d.trim(2,d.framecount) 

#Functions:
function TransPrep( clip TpClip, int TPLength)
{
TpFrames = TpClip.framecount()
TpClip= (TpFrames-abs(TpLength)<0) ? TpClip+loop(trim(TpClip, TpFrames, TpFrames),(abs(TpLength+TpFrames))) : trim (TpClip, TpFrames-abs(TpLength), TpFrames)
Tpclip= (TpLength>0) ? loop(trim(TpClip, TpFrames, TpFrames),abs(TPLength)) : TpClip
Return TpClip
}


function TransPrepBack( clip TpClip, int TPLength)
{
TpFrames = TpClip.framecount()
Tpclip= (TpLength>0) ? loop(trim(TpClip, TpFrames, TpFrames),abs(TPLength)) : TpClip
TpClip= (TpFrames-abs(TpLength)<0) ? loop(trim(TpClip, 0, -1),(abs(TpLength+TpFrames)))+TpClip : trim (TpClip, 0, TpFrames)
Return TpClip
}


#code by tsp - cf=current_frame,fr=framerate 
function convert2smpte(int cf,float fr) 
 {
  hh=int(cf/3600/fr)
  mm=int((cf-hh*3600*fr)/60/fr)
  ss=int((cf-hh*3600*fr-mm*60*fr)/fr)
  ff=int((cf-hh*3600*fr-mm*60*fr-ss*fr))
  smpte=string(hh,"%02.0f")+":"+string(mm,"%02.0f")+":"+string(ss,"%02.0f")+"."+string(ff,"%02.0f")  return smpte }
function convert2hhmmscff(int cf,float fr) 
 {
  hh=int(cf/3600/fr)
  mm=int((cf-hh*3600*fr)/60/fr)
  ss=int((cf-hh*3600*fr-mm*60*fr)/fr)
  ff=int((cf-hh*3600*fr-mm*60*fr-ss*fr))
  smpte=string(hh,"%02.0f")+":"+string(mm,"%02.0f")+":"+string(ss,"%02.0f")+"."+string(ff,"%02.0f")  return smpte }

function convert2srt(int cf,float fr) 
 {
  hh=int(cf/3600/fr)
  mm=int((cf-hh*3600*fr)/60/fr)
  ss=int((cf-hh*3600*fr-mm*60*fr)/fr)
  fftemp=int(cf-hh*3600*fr-mm*60*fr-ss*fr)
  ff=((fftemp))
  smpte=string(hh,"%02.0f")+":"+string(mm,"%02.0f")+":"+string(ss,"%02.0f")+","+string(ff,"%03.0f")
  return smpte
 }

function Trim2(clip c, int start, int "end", int "length")
{
    Assert(!(Defined(end) && Defined(length)),
    \      "Trim2: <end> and <length> parameters cannot be used together")
    Assert(!Defined(length) || length >= 0,
    \      "Trim2: invalid length: " + String(length))

    end = Defined(end)
    \     ? ((end < 0) ? (c.FrameCount() + end) : end)
    \     : Max(start, c.FrameCount())
    end = Defined(length) ? (start + length) : end

    Assert(start >= 0, "Trim2: start frame out of bounds: " + String(start))
    Assert(end >= start, "Trim2: end frame out of bounds: " + String(end))

    start = Min(start, c.FrameCount())
    end = Min(end, c.FrameCount())

    # we can't use Trim(start, end - 1) in case end == 1
    return (start == end)
    \      ? c.NullClip()
    \      : c.Trim(start, -(end - start))
}

function Trim3(clip c, int start, int "end", int "length")
{
    Assert(!(Defined(end) && Defined(length)),
    \      "Trim3: <end> and <length> parameters cannot be used together")
    Assert(!Defined(length) || length > 0,
    \      "Trim3: invalid length: " + String(length))

    end = Defined(length) ? (start + length) : Default(end, c.FrameCount())
    end = (end < 0) ? (c.FrameCount() + end) : end

    Assert(start >= 0 && start < c.FrameCount(),
    \      "Trim3: start frame out of bounds: " + String(start))
    Assert(end > start && end <= c.FrameCount(),
    \      "Trim3: end frame out of bounds: " + String(end))

    return c.Trim2(start, end)
}

function Min(val a, val b) { return (a < b) ? a : b }

function Max(val a, val b) { return (a > b) ? a : b }

function Clamp(val n, val low, val high) { return Max(low, Min(n, high)) }

# MaskTransition by stickboy
function MaskTransition(clip c1, clip c2, imgpath , int dur) 
{
ramp = imagesource(imgpath, 0, 0, 1, false).BilinearResize(c2.Width(),c2.Height()).ConvertToRGB32()
ramp = ramp.Lanczos4Resize(720,480)
ramp = ramp.AssumeFPS(c1.FrameRate()).Loop(dur)
transitionMask = Animate(ramp, 0, dur, "Levels",
\                  0, 1.0, 00, 0, 255,
\               255, 1.0,  255, 0, 255)

    overlap = transitionMask.FrameCount()
    trans1 = c1.Trim3(c1.FrameCount() - overlap)
    trans2 = c2.Trim3(0, length=overlap)
    trans = Overlay(trans2, trans1, mask=transitionMask)

    video = trans 
	return video
}

function Min(val a, val b) { return (a < b) ? a : b }

function Max(val a, val b) { return (a > b) ? a : b }

function AddBack(clip imagef, clip imageb, int dur) 
{ 
  b = imageB.converttorgb32() 
  f = imageF.Levels(0, 1, 255, 21, 255, coring=false).converttorgb32() 
  fa = imageF.converttorgb32() 
  x = (width(b)/2)-(width(f)/2) 
  y = (height(b)/2)-(height(f)/2) 
  c= overlay(b,f,x,y,mask=showalpha(fa), mode="blend") 
return c.converttoyv12() 
} 

function SafeSize(image, int SafeW, int SafeH, float fps) 
{ 
  a = image 
  w= Width(a)
  h= Height(a)
  h = (w > SafeW) ? multifour(h * SafeW/float(w)) : h 
   w = (w > SafeW) ? SafeW : w 
   w = (h > SafeH) ? multifour(w * SafeH/float(h)) : w
    h = (h > SafeH) ? SafeH : h  
 return ((Width(a) != w) || (Height(a) != h)) ? a.LanczosResize(multifour(w), multifour(h)) : a 
} 

function Has169(clip rescheck)
{
return rescheck
}

function HasAudio(clip chkaud)
{
    return chkaud.AudioRate() != 0
}

function HasMono (clip mon)
{
    return audiochannels(mon)!=2
}

function pointchk (inval, outval)
{
    return inval>outval
}

function multifour (val mfour)
{
    return (int(mfour/4))*4
}

function eqaudio(clip eqaud)
{
eqaud = eqaud.HasAudio() ? eqaud : audiodub(eqaud,BlankClip(eqaud.framecount, fps = 29.970, audio_rate=48000))
eqaud = HasMono(eqaud) ? MergeChannels(eqaud.GetChannel(1), eqaud.GetChannel(1)) : eqaud
return eqaud.ResampleAudio(48000)
}

function AddAlphaBack(clip f, clip b)
 {
  f = f.Levels(0, 1, 255, 20, 255, coring=false)
  x = (width(b)/2)-(width(f)/2)
  y = (height(b)/2)-(height(f)/2)
 return Layer(b,f,"add",255,x,y)#,0,use_chroma=true)
 }

function AddAlphaBackNoLev(clip f, clip b)
 {
  x = (width(b)/2)-(width(f)/2)
  y = (height(b)/2)-(height(f)/2)
 return Layer(b,f,"add",255,x,y)
 }


 function SafeSizeKB(image, int SafeW, int SafeH) 
{ 
  a = image 
  w= Width(a)
  h= Height(a)
  h = (w > SafeW) ? multifour(h * SafeW/float(w)) : h 
   w = (w > SafeW) ? SafeW : w 
   w = (h > SafeH) ? multifour(w * SafeH/float(h)) : w
    h = (h > SafeH) ? SafeH : h  
 return ((Width(a) != w) || (Height(a) != h)) ? a.LanczosResize(multifour(w), multifour(h)) : a 
} 
 
function importavs(string impavs)
{
import(impavs).AssumeFPS(29.970, true).eqaudio()
}

ResampleAudio(48000)  
ChangeFPS(29.970)
textsub("C:\Users\Tin2tin\.DVDslideshowGUI\DVDsGUItemp.ssa")
converttoyv12()

[jmartinr]

You could also do this in 2 stages.

The 1st run you create the ssa-file. The second run you check for the existence of the ssa-file. If it's there you use that.

But this solution always needs a 1st run of course.

[Gavino]

Another approach you could consider is generating the ssa file separately inside your program instead of in the Avisynth script. I can see that this would mean recoding some of the calculations currently done in the script into your program's source language, so maybe you would only want to do it as a last resort.

What is the actual error you are getting?

[tin3tin]

Doing it from within the avisynth script is the most exact way (for me) to deal with the subtitles.

The error is something like:

Write: File 'DVDsGUItemp.ssa' cannot be opened.
((Heres the path of the script), line 3191)

The actual error line is changing a lot - and sometimes it runs without any errors at all. That why I suspect the speed of the hhd might have an influence.

[Gavino]

Could it simply be a resources problem rather than a timing problem? The error message suggests to me that it never gets as far as the textsub - to test that theory, what happens if you comment it out?

Originally Posted by tin3tin

A simple solution would be if vsfilter would be able to open a string variable instead of a file or longer strings would be allowed in writefile: http://forum.doom9.org/showthread.php?t=147536

You could try using the workaround function I posted in that thread, ie build up the complete ssa contents in a single string and call my function once only. That will still result in multiple instances of WriteFileStart, but a lot fewer of them.

[tin3tin]

I tried to comment out the textsub - you're right the error is still there. It doesn't have anything to do with vsfilter.

One of my supections is that the ssa file isn't released from previous use before a new file is saved with the same name. But the error is typically related to the 100+ no. of writefilestart instance. If It wasn't released from memory it wouldn't hvae been possible to run the first many instances. I guess that it comes down to some kind of limitation of the maximum no. of instances...?

(When using ssa tags you'll easily use up the max number of letters pr. writefilescript - so eventhough it's a brilliant script example - I don't think it will solve this speciffic problem)

[Gavino]

Originally Posted by tin3tin

When using ssa tags you'll easily use up the max number of letters pr. writefilescript - so eventhough it's a brilliant script example - I don't think it will solve this speciffic problem

From the script you posted, it seems you are writing about 50-60 characters per slide, each in its own call to WriteFileStart. Since my WriteString function calls WriteFileStart once for every 254 characters, you can cut down the number of calls by a factor of 4 or 5.

OK, it probably doesn't solve the problem, but could allow you to handle many more slides before you hit the limit (if indeed that's what the problem is).

[DonQ]

Most probably not related, but what happens if you turn your antivirus off?

[Gavino]

@tin3tin

Have you found any solution to this problem?
I don't think you are hitting a limit on the number of WriteFileStart instances per se. I created a script with 3000 instances (dead easy with GScript ) and it worked just fine. Perhaps you are just running out of memory with the combination of things you have in your script when there are a large number of slides involved.

[tin3tin]

Im not near my computer this week so i can't test my latest teory on this problem. Now i think it's the media player locking the avs and the ssa and when running it in another player at the same time the error happens because the ssa is locked and can't be overwritten.