Memory Leak

[leoenc]

I posted this before in the alternative codecs forum, because I thought it is related more to Expression Encoder, but now I am starting to think it might be a general AVIsynth issue.

The problem I am facing is that AVIsynth seems to have memory leaks, even loading the simple script.

When opening scripts in Expression, RAM gets filled up with each new script that is opened (even though the previous one is closed). I am talking about the simplest file loader AVS, no filters.
See my discussion here also in the Expression forum:
http://social.expression.microsoft.com/Forums/en-US/encoder/thread/b70bf740-3cc1-41ff-ad38-6bc4623bfaa8

Does anyone know about this?

I'm using 2.5.8 but I did also try the latest 2.6 alpha builds with no success.

[jpsdr]

Have you try to use SetMemoryMax ? It affects only the cache, and your problem may simply come from this.

[leoenc]

I did, with various settings. There was no change.

[StainlessS]

From the linked thread in first post:
To recreate you can simply load an AVS script into Expression and watch the memory in Task Manager for the Encoder process. Then make a new project and notice that the memory hasn't cleaned up properly. Only closing Encoder frees up all the accumulated RAM.

The last sentence seems to indicate that it is the Encoder that is hanging on to RAM and maybe not 'letting go'
of the Avisynth AVS, perhaps I'm being simplistic.

EDIT: Is there an option to 'Close' a project and not just create a new one, if so, what does that do?

[Didée]

Hmh. I can't tell much about those low-level things in Avisynth ... however,


Hello Leo,

I have been looking into this issue and have come to the conclusion that this is an AVISynth memory leak problem (ie. below the point where encoder can control).

Making the same exercise with VirtualDub: Make AVS's with just return(AviSource(...)), open/seek/close, open/seek/close, open/seek/close, ... repeat 20 times in total, using official Avisynth 2.5.8, 32bit, no MT:

Opening 1st avs: Vdub: 10xxx kB memory usage
opening 2nd avs: Vdub: 11xxx kB memory usage
opening 3rd avs: Vdub: 11xxx kB memory usage
...
opening 20th avs: Vdub: 11xxx kB memory usage


I don't see any memory leak there.


If-I-remember-correctly, years in the past, older builds of VirtualDub had problems with not correctly releasing the memory when unloading Avisynth scripts. This has been fixed long since, though.

Maybe that MS Expression thingumbob has a similar problem?

[leoenc]

@StainlessS,
There is no Close option, but when you make a new project, it's the same thing. I've tested loading an AVI and then choosing a new project - memory is released. Not so with AVS.

@Didée,
You are right. I will mention this to MS. Thanks!

[StainlessS]

when you make a new project, it's the same thing

Should be the same thing but seems it isn't. As previously stated, the encoder is not releasing a 'Handle'
or something else to prevent memory release. When contacting MS, emphasize what you have already
said in that thread, ie:

Only closing Encoder frees up all the accumulated RAM


EDIT: If it were Avisynth at fault, the RAM would be lost till reboot (Can anybody contradict that,
taking into consideration the previous quote?).

EDIT: The encoder may be detecting the AVS specifically (it knows that it is, not a real AVI) and
maybe scanning the AVS file, perhaps to ascertain the original AVI file and obtain MediaInfo
that is unavailable from the AVS alone. This would explain the difference as the AVS is
being handled differently to a standard AVI. It is whatever is being done differently that
is not being cleaned up & released properly. (perhaps???)

[IanB]

This feels like the IScriptEnvironment is not being released. When using the AVIFile interface it is imperative to close the AVIFile stream handles when finished with the script. There is much code out there that leak handles due to slack and careless coding. Not closing ordinary AVI files leaks 1 file handle and a very small amount of memory.

If using the Avisynth C or C++ API interfaces you must release the IScriptEnvironment when you are finished with it. With older versions of the C API this was not possible, recent versions (2.6) have a DeleteScriptEnvironment() function added for this purpose. With the C++ API doing a delete env; in your code may not always work correctly when using disparate MSVCRT environments, there is now a IScriptEnvironment::DeleteScriptEnvironment() (2.6) method available to release the IScriptEnvironment from within the same MSVCRT environment that originally created it.

Each IScriptEnvironment maintains a full independent set of Avisynth resources, i.e. cache, vartable, graph, locals, etc, ...


Avisynth is implemented as a Process class library, exiting the process is always enough to recover any memory, there are no global system resources used by Avisynth that would require a reboot. However some recalcitrant processes refuse to die without prejudiced help to euthanase them.

[leoenc]

@IanB,

Thanks a lot for the explanation.
Can I quote you on this in my discussion with MS in the Expression Encoder forum? It should help the Expression developers to fix the leak.

[javlak]

This feels like the IScriptEnvironment is not being released. When using the AVIFile interface it is imperative to close the AVIFile stream handles when finished with the script. There is much code out there that leak handles due to slack and careless coding. Not closing ordinary AVI files leaks 1 file handle and a very small amount of memory.

If using the Avisynth C or C++ API interfaces you must release the IScriptEnvironment when you are finished with it. With older versions of the C API this was not possible, recent versions have a DeleteIScriptEnvironment() function added for this purpose. With the C++ API doing a delete env; in your code may not always work correctly when using disparate MSVCRT environments, there is now a IScriptEnvironment::DeleteIScriptEnvironment() method available to release the IScriptEnvironment from within the same MSVCRT environment that originally created it.

Each IScriptEnvironment maintains a full independent set of Avisynth resources, i.e. cache, vartable, graph, locals, etc, ...


Avisynth is implemented as a Process class library, exiting the process is always enough to recover any memory, there are no global system resources used by Avisynth that would require a reboot. However some recalcitrant processes refuse to die without prejudiced help to euthanise them.

Hi. Pardon my ignorance but a) I can't seem to find a DeleteIScriptEnvironment method in 2.5.8 avisynth.h and what I was able to find in the 2.6.0 110525 one was a DeleteScriptEnvironment. Is this a typo and what, in your opinion, is the best way to emulate this in 2.5.8 since delete env can miss? env=0? Maybe just add this in avisynth.h v2.5.8 like it is in v2.60 ( virtual void __stdcall DeleteScriptEnvironment() = 0; ) and call it whenever we want?

Thanks.

[IanB]

@leoenc,

Sure. :D

@javlak,

Yes it's DeleteScriptEnvironment (bad memory/typo) and yes it's a 2.6 feature.

For usage with a 2.5 avisynth.h you could update the IScriptEnvironment vtable declaration to include the extra entries. In your code you should check for version 5 (or latter) of the API to avoid running off the end of the vtable.

Something a bit like this should cope :-...
bool API26;
IScriptEnvironment* env = CreateScriptEnvironment(5); // Ask for 2.6 API

if (env)
API26 = true; // Bonus!
else {
API26 = false;
env = CreateScriptEnvironment(); // default API
}

...

if (API26)
env->DeleteScriptEnvironment();
else
delete env; // pray for a non-disparate MSVCRT
env = 0;

[javlak]

OK, thanks a lot!