trouble encoding file with many clips

[tacman1123]

i have a series of small clips that i´m combining together, and something is failing along the way. i´m hoping some of the fine folks here can help to me to how to solve the problem.

first, the file runs fine in vlc and wmp, which makes me think that is´t not the problem of opening too many files. however, when i try to encode it using hcencoder, about halfway through it fails without any message. same problem with both hcenc .22 and .23 (the most recent version)

i ran it the script through something else (alas, i forget at the moment which encoder) and did get a windows error message. when i looked at the debug, it said it failed on subtitleex(), so i thought that maybe it was something to do with the spanish language characters i´m using.

however, i don´t really think it's that, because i took all the smaller scripts (the ones that are included by the main file) and ran each on through hcenc, no problems. so i'm back to thinking it has something to do with the larger grouping.

the main file is pretty much just appending one file after another

f = blank()
f = f + include(scene1.avs)
repeated many time

what i´m hoping to find out is a way to turn on debugging in avisynth or hcenc or somewhere, so that when it fails i have somewhere to look.

if it is a file limit (63, i think i recall reading somewhere) how would it manifest itself? again, the fact that it works fine in wmp makes me think that it is something else.

muchas gracias

tac (in puebla, mexico at the moment, trying to burn a dvd of video clips i shot earlier for some friends...)

[Ranguvar]

Please post your entire script :)

[tacman1123]

I've traced the problem down to repeated calls to a routine that fades in a caption (the person's name and title):

function add_caption(clip a, string "name", string "title", int "startframe", int "framecount") {

firstcaptionedframe = Default(startframe, 30)
framecount = Default(framecount, 80)
cc=a.Trim(firstcaptionedframe, firstcaptionedframe+framecount-1)

fontsize=22
firstframe=0
effects="f(30,15)"

bg=blankclip(length=10, width=round(a.width/2), height=40, color=$0FF000).Greyscale()
cc=Overlay(cc, bg, x=a.width-bg.width, y=a.height-(bg.height+fontsize), opacity=0.2)
ytop = -(bg.height)

textcolor=$00FFFFFF
cc=cc.SubtitleEx(name, x=-60, y=ytop, font="arial", effects="b" + effects,size=fontsize,textcolor=textcolor)


# effects="f(40,10)"
fontsize=16
cc=cc.SubtitleEx(title, x=-60, y=ytop+fontsize, font="arial", effects=effects,size=fontsize)
Replace(a, cc, firstcaptionedframe, framecount)

return last
}

function Replace (clip old, clip inclip, int frame, int numframes)
{

numframes = ((numframes == 0) || (numframes > inclip.framecount)) ? inclip.framecount : numframes

return old.Trim(0, frame - 1) + inclip.Trim(0, numframes) + old.Trim(frame + numframes, 0)

}

This routine seems to work fine in small scripts, but when called too often (not sure what the number is yet), the hcenc just fails. Perhaps it's a memory problem with SubtitleEx(). Of maybe the way I'm doing the Replace()?

Because I'm using Spanish-lanugage characters, I can't use Subtitle().

Tx,

Tac

[tacman1123]

Whatever it is, it seems to be hard to narrow down. Now I tried a script without any calls to my add_caption() routine or to subtitleEx(), and still got the failure. I need to figure out if it's related to the number of DirectShowSource() calls.

Feedback on add_caption is welcome, though, it took me a while to write, as I'm still getting used to avisynth, but the result is a documentary-style name and title that fades in for a few seconds during the clip.

[Gavino]

Certainly sounds like a resource problem of some kind. These things can be hell to track down - buena suerte. :)

One idea to reduce resource usage if you're calling add_caption many times would be to do the background Overlay just once on the whole clip, ie take the Overlay outside and pass in both the original and the overlayed clip as parameters.

A further approach would be to use ConditionalReader (http://avisynth.org/mediawiki/ConditionalReader), specifying the captions (names and titles) in a text file, with the appropriate frame number ranges.

But then again, maybe the problem lies elsewhere anyway.

A couple of minor comments on the code:
- The name and title parameters should not be optional, as you provide no defaults. (Remove the quotes around their names.)
- Beware that your Replace function won't work if the bit you're replacing includes the first or last frame (ie frame is 0 or frame+numframes >= old.framecount).

BTW when you said f = f + include(scene1.avs), did you mean Import? Maybe we need to see exactly how your sources are brought in.

[gzarkadas]

You use many trims in your function and it may be the case if you are nesting too many calls of it that you run out of stack space. This has been pointed out at this forum but I do not have handy at the moment a suitable thread reference and feel too lazy to use search :p. Your best bet is to modify your code and use runtime filters, as Gavino suggested. They are generally a little bit slower but they do succeed when other solutions fail due to their better utilisation of resources.

[tacman1123]

Any idea why the final video runs in WMP but won't encode with HEnc? I'm guessing it has some sort of resources conflict, but one would like if it was just too many trims or too deep a stack that this would show up in both renderings.

I remember seeing a post about handling lots of clips that used a combination of scripts to create a VirtualDub script, and for some reason that worked. I'll look for that and see if it works.

Tac

PS Yes, I meant import(). At the time, I was typing on another computer because I couldn't get my laptop connected. I'll post a full script that fails if I can reduce it. Thanks for the help and suggestions!

[IanB]

1. Using Import to join all your scripts is the right way.

2. How many DirectShowSource() instances are we talking about?

3. DirectShowSource() has a LogFile option.

4. It helps a lot if you quote the exact full error message text, and include your complete scripts (nuke the source filenames if they are sensitive).

[tacman1123]

Alas, no error message of any sort. It (HcEnc) simply dies after about a few thousand frames. Rebders fine in WMP.

I'm reworking the program to include a count of the files. Thx for the confirmation on the import. Another approach I've seen, which I can use, is to grab all the clips into separate variables, then add them all one at a time, e.g.


f1 = DirectShowSource("a.avi");
f2 = DirectShowSource("b.avi");
f3 = DirectShowSource("c.avi");
f1 + f2 + f3


No idea which is better.

Tac

[mikeytown2]

I sometimes encounter an error, when the thread tries to get more then 1.5GB or so. Add this to the top of your script:
SetMemoryMax(1024)

[tacman1123]

The file encodes using QuEnc, but not HcEnc, but I'm not sure that all the options are set the same.

I tried the SetMemoryMax(1024), no difference. But I'm now wondering if it has something to do with a specific clip, as I just encoded a larger file, with more DirectShowSource calls, without a problem. One difference is that the script that fails is calling some older avi files that were created with a C# program I wrote a while ago, based on an older version of the DirectShow library from Microsoft. So even though the file plays in WMP, something's not working when HcEnc encodes it.

I have to see if I get the same errors with the newer files.

Thanks for all the suggestions.

Tac

[Gavino]

So even though the file plays in WMP, something's not working when HcEnc encodes it.
I assume by "the file" here, you mean your script rather than just playing the AVI itself directly in WMP.

That being so, WMP should see exactly the same data as HcEnc does, as it all comes via Avisynth. That suggests to me the problem is in the encoder, although it could just be that the extra memory requirements of encoding (over simply playing) take you over some limit.