Photo scripting: crash @ 2GB

[gamez]

Hi there,

since weeks I am struggling with the following, thank you already for reading this post and any advise you may have.

I want to create a video from a set of, say, 200 digicam photos (jpeg), each has a resolution of 2048x1536 pixels and about 500K. In my script, I create a clip c per photo using ImageSource() and add some zoom in/out motion using Animate with BilinearResize. I then append the clips, either using each time v=v+c (no fade) or v=Dissolve(v, c) (classic fade) or v=MaskTransition(v, c) (masked fade). Of course, I can give you all the details if you wish.

The resulting script plays correctly in MediaPlayer, however not in real-time, so it must be syntactically and functionally correct. But when I feed it into any encoder (Dr. DivX, TMPGEnc, QuEnc, NuEnc, HCEnc...) I notice (taskmgr) that during the encoding process more and more memory is allocated (some 20 - 30 MB per photo) and when we reach 2GB ... BOOOOM... the encoder application just plainly disappears without a trace.

I tried various versions of Avisynth 2.5.x without success

My questions:
- Is there anything wrong with the script, am I doing something not allowed?
- How would YOUR script look like?
- Is there a way to reduce memory usage by changing the script logic?
- Is there a way to explicitely free memory?
- Is there a known memory leak in AviSynth?
- Is there a known limitation at 2GB?

Again, thanks for reading and your advise.

gamez.

[unskinnyboy]

:script:

[gamez]

I have added 2 attachments, hope it works.
Please remove ".txt" and put your own photos...
The masks are borrowed from the program DVDslideshowGUI.

See you,
gamez

[unskinnyboy]

Attachments are still awaiting approval, but I guess what you probably need here is to use the SetMemoryMax(int) function in your script and reduce the memory allocated to AVISynth.

where, int = value in MB that defines the amount of memory that will be allocated to a script when it is loaded.

[Soulhunter]

Is there a known limitation at 2GB?
Yup, afaik windoze cant allocate more than 2GB per process!


Bye

[gzarkadas]

- Is there a way to reduce memory usage by changing the script logic?Since your script is a linear series of commands, you can split it in as many pieces (separate .avs scripts) are needed to get each step's required memory below 2GB. Each step must return a video that the next step will use as input. That way you only have to include an extra AVISource command for each step after the first.

If you want to automate the execution of the chain, you may try DubMan (http://sourceforge.net/projects/dubman/).

- Is there a way to explicitely free memory?I have been trying it for some time but did not find a way to do this by scripting; I guess there is no way.
This is a feature request for next versions of Avisynth :cool:

[WarpEnterprises]

Maybe you consider using Zoom?

www.avisynth.org\warpenterprises\#zoom

[gamez]

@WarpEnterprises

Thanks a lot for your hint, I installed your plugin right away.

Sorry, somehow I can't manage to get the Zoom parms right; could you be so kind to help me translate the syntax, it will be piece of cake for you to understand how I use Animate today in my ZoomIn/ZoomOut functions (see attached scripts).

Thank you!!!


@gzarkadas

Will try your advice as well, thanks!


gamez

[WarpEnterprises]

I will try to translate...

Meanwhile still another option which may suit exactly your needs:
vcmohan made some nice filters which are much easier to use (for many effects and transitions):
www.avisynth.org\warpenterprises\#EffectsMany
www.avisynth.org\warpenterprises\#HollywoodSQ
www.avisynth.org\warpenterprises\#TransAll

[gamez]

Those effects are indeed breathtaking, really stunning.
I surely will study them as soon as I have sorted out my memory consumption problems and as it seems today they are more related to the zooming of one picture than to the fade-over transitions between the pictures.

[gamez]

@WarpEnterprises

Hello again,

I've spend some time to find the required maths that enable my scripts to use your zoom plugin instead of Animate/BilinearResize; for my function ZoomIn it goes like this:

function ZoomIn(clip v, int x, int y, int left, int top, int right, int bottom)
{
# compute zoom factor for x
w1 = float(v.width)
w2 = float(right-left)
fx = string(x/w1) + "+n*" + string(x*(w1-w2)/(framecount(v)*w1*w2))

# compute zoom factor for y
h1 = float(v.height)
h2 = float(bottom-top)
fy = string(y/h1) + "+n*" + string(y*(h1-h2)/(framecount(v)*h1*h2))

# compute source center for x
w2 = float(right+left)
sx = string(w1/2) + (w2 > w1 ?
\ "+n*" + string((w2-w1)/(2*framecount(v))) :
\ "-n*" + string((w1-w2)/(2*framecount(v))))

# compute source center for y
h2 = float(bottom+top)
sy = string(h1/2) + (h2 > h1 ?
\ "+n*" + string((h2-h1)/(2*framecount(v))) :
\ "-n*" + string((h1-h2)/(2*framecount(v))))

return v.ConvertToRGB32.zoom(
\ factorx=fx, factory=fy, srcx=sx, srcy=sy,
\ width=x, height=y).ConvertToYV12
}


For ZoomOut, it's just the other way round, no big deal. Unfortunately, I have to tell you that I've found the following (sorry, I don't want to criticize, it's just the things I have noticed...)

- when the zoom focus is different from the picture's center, the resulting zoom motion looks quite strange in my eyes, it does not zoom linearily to the target area but in a kind of curved or contorted motion - this is probably due to the way how the x/y-factors interact with the source coordinates
- the version using zoom is three times slower than before
- the quality of the resulting resized photos using BilinearResize is significantly superior than the ones resized with zoom - as you mention yourself in the accompanying textfile
- although the memory allocation is indeed different than with Animate/BilinearResize, my "main" script with the 200 photos still does not succeed to complete the encoding process, it even aborts earlier than before

I am very sorry that I was not able to use your function to my advantage, I'm sure it has nice features useful under other circumstances.

If you have any other idea, please let me know!

gamez

[WarpEnterprises]

To your memory problem: did you try the script with VirtualDub? It has better error messaging and you can check when the memory climbs (e.g. if you step frame-by-frame or only in preview mode or if the memory gets freed again if you do nothing).

[Prettz]

You know, no PC application was ever supposed to hit the 2GB address space limit while Win32 was still around. Yet you've just managed to do it. Props.

[WarpEnterprises]

But there is no obvious reason why this amount of memory is actually needed.
It looks like there is a missing memory deallocation or the like.

[gzarkadas]

@ gamez,

I have made some further investigation on this and I believe I have found the "suspect" (which I must admit I was suspecting some time ago, when I was crashing-at-approx-2GB regularly trying to move 12 polygons on spiral paths in a single script until I gave up and made a script chain to accomplish it; see here (http://avslib.sourceforge.net/examples/test_anim_006.html)).

The "suspect" is Overlay(); just one call inside the transition function but this is enough (since it is repeated about 200 times) to blow the script out of memory.

Incidentally, this also means there is currently no way to encode and store the script as is in a full video file in one single step. It must be broken in pieces of say 25-40 photos at the most. I used 25, but this is because the test-case photos I have used are actually 2,25 MB each; with yours (0,5MB) I believe you could go up to 40 per piece.

If you are still working on this, I can post an almost-ready to run set of scripts, which I have used as test-case (I will need a couple of days though to document it properly). :D

[Pookie]

I had a "similar" (sorta) problem on some workstations at work that were trying to render some complex scenes. Even though they had plenty (4GB) of RAM, they kept running out of memory. There's a Boot.ini function /3GB that can help if you have SP2 installed (yuck). I was able to get the machines to render by installing an app called FSAutoStart, which will shut down all the un-needed Windows Services and applications, reclaim the RAM, run the APP, then restart all the programs. It works quite well, and it's easy to install and uninstall if need be. I use it at home as well to give my machine that "freshly booted feeling".

http://www.kensalter.com/forum/forum.asp?FORUM_ID=14

There's also a free command line app called Memdefrag which you can run at timed intervals to recover the RAM. http://web.telia.com/~u32102551/

Finally, you can try O&O Software Clevercache which will optimize your memory dynamically and (supposedly) speed up your system. They have a 30 day eval.

[gzarkadas]

Ok, here is my test case.
The attached scripts implement the test case; the initial "builtin.avs" attached by gamez is also needed to run the normal chain (see below).
At the end are the results that IMHO point to a leak in the implementation of the Overlay() filter.
By the way the scripts are also a nice example of using arrays:
to make your life better (and the code listings much shorter) in general,
to create a for...next loop in Avisynth.

What the script-set does in brief is to code all of the images and intermediate bitmaps as array data and them to execute a for...next loop to combine them to the final result.
The breakdown to pieces of 25 images per step (8 steps in total) is hard-coded inside the mv.avs script's body. The global variable chain_part informs the script about what part of the chain to execute. It has to be set for each step from the command line (see section 1.2 below).

1. Prerequisites

1.1. Required software:

Avisynth, version 2.5.6
AVSLib (http://sourceforge.net/projects/avslib/), version 1.0.0.
The 1.0.1 patch version of AVSlib (from the above location). The patch must be installed after the installation of AVSLib 1.0.0. The patch offers (for Avisynth 2.5.6 and later only) arrays of practically unlimited length; it is also needed for the scripts to execute.
Configuration of your system so that at least 2.2 GB of total (ram+swap) memory are available (the "Limit" field at Task Manager's dialog box should report > 2200000)
VirtualDub, version 1.5.10 or later
XviD 1.0.3 Codec

The later two are needed for exactly reproducing my setup and comparing the results; I assume any selected video compression method will work for encoding the results.

1.2. Script edits / creation of environment


The path on the first line of the mv.avs (main) script must change to the location where you have installed the AVSLib loader (avslib-h.avsi).
From a command prompt you must cd to the scripts' directory and issue the following two commands:

echo return 0 > mv_chain.ini
echo return 0 > mv_built.ini

This will setup the main script for normal operation at step 0 of the script chain.
For later steps (1,2,...,7) you must issue a

echo return X > mv_chain.ini

command (X = 1,2,...,7) before invoking the main script again.
For each successive step you must save the output video as: mvout0.avi, mvout1.avi, mvout2.avi, ... and so on up to the last step.
If you are gamez:

i. Edit the line (no #6 inside mv.avs):
global __debug__ = true to
global __debug__ = false
ii. Check the "bmps" and "bmp_ids" arrays in mvinc.avsi; they may need some adjustment because I think I have missed three items during copying and pasting from your sources. This is the only thing that may make the output to deviate from the intensions of your initial script.

If you are not gamez:

i. Create a folder and put inside:
a. 402 JPEG photos with dimensions at least those reported at the top message of this forum thread, named {anything}001.jpg, ..., {anything}402.jpg
b. one JPEG photo with same dimensions and name "z.jpg" (for the transition effect).

Edit lines #19 and #75 in mvinc.avsi to point at the above folder (don't forget to include {anything} in a1., above, at the end of the path).
Edit the "chain_last" function in mv.avs (lines #48 - 58) to insert an audio of your choice (this will only be called at step 7 of the script chain).


2. Execution

i. As said above, before each step, issue this command (at the command prompt, with active dir the script's dir):
echo return X > mv_chain.ini
Substitute 0,1,2,3,4,5,6,7 for X. Save the output video as mvoutX.avi.

ii. Executing the normal (with effects and the call to Overlay()) chain is the default, if all steps above have been made. If after playing with debug chains you want to revert to normal execution, issue this command (at the command prompt):
echo return 0 > mv_built.ini

iii.To execute a debug chain, issue one of the three commands below:
echo return 1 > mv_built.ini (uses Layer instead of Overlay)
echo return 2 > mv_built.ini (in addition to the case above, does not use Animate)
echo return 3 > mv_built.ini (in addition to the cases above, does not use the transition mask effect)

iii.For inspecting memory consumption, record the total memory before and after opening and encoding (saving as AVI) the script in VirtualDub.

3. Results

With my setup:
1. images of ~2.2 MB each
2. VirtualDub 1.5.10 as renderer
3. XviD 1.0.3 Codec for video compression, with settings:
a. profile / level: AS @ L5
b. target quantizer: 1.00
c. quantization type: MPEG
d. quarter pixel: true
e. global motion compensation: true
f. Motion search precision: 6
g. Use chroma motion: true
h. Trellis quantization: true
4. PC: Pentium-M 1.7GHz / 1GB RAM / ATI Mob. Radeon 9600 AGP, 350MHz, 64MB / XP-Pro, SP2 / avisynth.dll 2.5.6.0

The following results are obtained:

1. Executing the normal chain, required ~ 1.34 GB memory to fully encode the first part (step 0).

2. Executing the first debug chain (Overlay replaced by Layer in the transition function), required ~ 0.52 GB to fully encode the first part (step 0)

3. Executing the other debug chains (without Overlay + Animate / without Overlay + Animate + Transition) required similar amount of memory with the first debug chain.

4. This means ~ 0.82 extra GB are due to Overlay() alone (for about 4000 frames, ie ~ 200 KB per frame).

5. The memory consumption between opening the script and the appearance of the first frame at VirtualDub's input window was about the same for all cases (approx. 0.3 GB). The build-up of the great difference in consumption between the normal and the debug chains is performed during the serving of frames to the encoder.

6. The extra memory required to encode the normal chain is 160% of the debug chains; this is too much and thus it is probably a sign of a memory leak somewhere inside the Overlay() code.

[IanB]

Problem is an interaction between Animate and Overlay. Animate env->Invoke's a chain of filters per different parameter frame. It maintains a small cache of these chains. Each instance of Overlay malloc's 3 YV24 full size work frames. The way the scripts are written causes huge numbers of instances of whole filter chains to concurrently exist each with several Overlay instances. Carefull restructuring of the scripts to avoid recursive calls to Animate may help.

[gzarkadas]

Problem is an interaction between Animate and Overlay. Animate env->Invoke's a chain of filters per different parameter frame. It maintains a small cache of these chains. Each instance of Overlay malloc's 3 YV24 full size work frames. The way the scripts are written causes huge numbers of instances of whole filter chains to concurrently exist each with several Overlay instances. Carefull restructuring of the scripts to avoid recursive calls to Animate may help.

Ok, we 'll live with it. After all we can divide and conquer and there are also the VirtualDub's job files to assemble all pieces together. Thus, we are not doomed, we are happy doom-ers instead :) .

However:
1. the Animate + Overlay combination is a damn useful one.
2. the initial script of gamez which also suffered from the same memory problems does not make recursive calls. It is a linear flow of function calls which result in the following linear combination of calls:
Animate (BilinearResize)
Overlay
Animate (BilinearResize)
Overlay
...
etc.


Now, if what you are saying is that what appears as a linear script flow in the source is in fact a huge recursive call list during execution because each filter pull frames from the previous and thus all up to the top, then simply the task under question (open a bunch of images and make an animated slideshow with a simple transition) cannot be accomplished by a linear-flow script for bunch size > 30 - 50 (depending on their size) in Win32.

I will then give up any animation attempts with Animate and stick to ScriptClip which (at least this is my conclusion up to now) is a much less memory-demanding filter even when combined with Overlay.

I must admit though, that writing scripts as a huge F(frame) function in order to be memory-efficient and manage to accomplish conseptually simple-to-medium level tasks such as a ~50 pictures slideshow with a bit of zooming is not a pleasant situation from the user's perspective. Higher-level primitives are always more easy to comprehend and use, especially from non-developers.

Is there a hope for a better (regarding memory use) Animate / Overlay support in future versions of Avisynth ?

[ShawnFumo]

Interestingly enough, I'm also having a problem with a memory leak, but I'm not using animate or overlay! It is related to my earlier post about getting around avisynth's simultaneous avi issue. To recap, I'm using a script to load all the files into virtualdub and frameserving out. I also generate trims in avs and use that to access the original clips from out of the big concatenated "file" coming from VD.

From there, I'm de-interlacing with or without slow motion, adding titles, and basically dissolving everything together.

My problem is that there seems to be a memory leak in that every time it switches to a new clip a chunk of ram gets taken out and never gets released. It wouldn't be a problem if it was smaller, but it totals a half-hour. I can split it in half, at which point it goes from a start of about 200megs (seems reasonable with all that's going on) up to around 750megs! If I try to do the whole thing, virtualdub will choke during compression. This also happens when just playing a file. If you seek to the middle, it starts leaking from there, etc.

If anyone wants to check out my scripts, look here:
http://www.yoyoing.com/shawn/videofiles/avisynth/

It is basically _edit.avs that is the main file with the edits and functions. _trims.avs contains the trim points. _append.syl is the script for virtualdub.

I need to experiment more, but it seems weird that this is happening considering I'm not doing anything particularly crazy for the actual edting itself...

Shawn

[ShawnFumo]

Wow.. this is unexpected.. I did experimenting with turning various stuff on and off and it is actually "subtitle" which is killing it! I commented those lines out and it is totally fine...

I guess I'll have to look and see if there's an alternative to the built-in subtitle, like a plugin of some kind? Or perhaps I can write out a text file like .srt and then burn that into the video or mux it into the mp4 file or something...

Kind of annoying, though, since I was generating the subs on the fly with incrementing and everything (it is an instructional vid, so each trick is numbered with additional info. on a web page for each).

Shawn

[mg262]

I'm sure I've seen a text-rendering filter other than subtitle, which was designed to be much faster. Unfortunately I can't find it despite considerable searching for combinations of text, font, render, subtitle, etc. I think it was over a year old. Does anyone know what I'm talking about?

Edit: not SubtitleEx.

[foxyshadis]

vsfilter? (requires an external file)

[mg262]

I don't think it was a proper subtitle-filter. Maybe I'm imagining this...

Edit: I was misremembering this: http://forum.doom9.org/showthread.php?t=91968

[ShawnFumo]

Hmm.. I wonder if the subtitle command internally uses overlay and that's why it has the issue?

I was able to mux a .srt file into the .mp4 using MeGUI (which uses mp4box), but I'm not sure I like that solution. It is nice that it is a softsub, but it is a pain to have to right-click something and enable subs all the time. Having the subs separate works, but then you have two files and need to have a thing like vsfilter. Mac side could be tricky in either case.

I think I'll take Foxy's advice and try to load vsfilter as a plugin in avisynth. I'll have to explore ASS and/or ttext in order to get positioning correct. I could always use subtitle for just the titles since I don't have many and the leak wouldn't be too bad, but then the two sets of text might look different from each other. I'll have to explore a bit more..

But if anyone finds any text plugins that work similar to the internalsubtitle (aka without external sub files), let me know.

Thanks for ideas guys!
Shawn

[sh0dan]

Hmm.. I wonder if the subtitle command internally uses overlay and that's why it has the issue?

It doesn't, but it does allocate a rather large buffer, which is why the result is the same, when used many times with animate.

If any of the Animate usage in the script could be replaced by conditional filters, your memory leakage should be gone though.

[ShawnFumo]

Shodan, I'm not using animate in my script, though? When it comes down to it, I have a bunch of trims with individual subtitles on them which are being dissolved together which was causing the trouble in my case..

Shawn

[ShawnFumo]

If I get a chance, I'll see if I can do some sort of test case to see what exactly it takes for subtitle to eat up memory.

Anyway, I did find a solution for myself. I played around with Aegisub (cool program!) and found that the ASS (advanced substation) format has a lot of neat features (even some animation). So, I modified my script to output a file with subtitle lines and made a simple batch file which merges it with a header containing the styles used.

This has a cool after-effect that I can output a totally clean video and use subtitles for all my text. Then I can either distribute the two files, or mux the subs into the mp4 (cool, but not the best supported yet), or burn them in using the VSFilter as a plugin as foxy mentioned. I've tried all the methods and it is nice to have flexibility.

At first I was thinking the tricks in the video would be only marked numerically and have descriptions on a website. That'd let me update names and maybe translate to another language. But using subs, I don't mind putting the name of each trick in the video too, since I can change it easily without having to recompress. :)

Of course I just now saw Basilik's SubtitleEx plugin, which maybe would have worked better for my original issue, but oh well..haha

Shawn