Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th September 2012, 09:04   #1  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
Memory leak when using env->NewVideoFrame(vi)

Hi all

I'm trying to develop an interface to use some avisynth filters in an NLE in MS Visual Studio 2005. I got it to work apart from a memory leak which I'm trying to fix for a while now.

Code:
void myNLEclass::myNLEplugin()
{
  IScriptEnvironment* env = CreateScriptEnvironment(AVISYNTH_INTERFACE_VERSION);
  env->AddFunction("myNLEsource", "i", Create_myNLEsource, myNLEclassInstance);
  PClip myclip = env->Invoke("myNLEsource",0).AsClip();
  PVideoFrame myframe = myclip->GetFrame(_myNLEtime, env);
  // Here I plan to apply some avisynth filters and hand over the result to my NLE later
  delete myframe;
  delete myclip;
  delete env;
}
I also implemented the Avisynth plugin myNLEsource which sets the vi properties, creates a new frame using env->NewVideoFrame(vi) and gets the video frame from myNLEclassInstance.
Everything works fine if I leave away the three delete statements, but the used memory is growing with every processed frame until it crashes. If I put the three delete statements, it breaks at delete myframe. I get "Windows has triggered a breakpoint in..." and it breaks in new.cpp at the line "while ((p = malloc(size)) == 0)".

In the beginning I linked to avisynth dynamically using:
Code:
avsDLL = LoadLibrary("avisynth.dll");	
IScriptEnvironment* (__stdcall * CreateScriptEnvironment)(int version) =
                              (IScriptEnvironment*(__stdcall *)(int)) GetProcAddress(_avsDLL, "CreateScriptEnvironment");
I read that it could be a problem if the memory for the frame is allocated in a library which was compiled in release version and trying to use delete in debug version. So compiled everything in release version, but it didn't help.
Then I read that it might be a problem allocating the memory in a dll which was compiled with a different version of VS. So I compiled avisynth myself and linked statically adding avisynth.lib to my project. But still it doesn't work.

Any ideas what I'm doing wrong? Any suggestions how to free the memory?

Thanks a lot for any help!

Stefan

Last edited by Guest; 11th September 2012 at 14:18. Reason: fix wide post
sscheidegger is offline   Reply With Quote
Old 11th September 2012, 10:39   #2  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
Since myframe and myclip are local variables of function myNLEplugin, you shouldn't need to 'delete' them explicitly - in fact, it is wrong to do so as they are not direct pointers. Instead, you should set them to 0.

There is a potential problem with 'delete env', which is why DeleteScriptEnvironment() has been added to Avisynth 2.6. See this thread.

Perhaps there is something in your myNLEsource class which is retaining a reference to the video frames.
What does the code of your GetFrame() function look like?
__________________
GScript and GRunT - complex Avisynth scripting made easier

Last edited by Gavino; 11th September 2012 at 11:59. Reason: Add info on DeleteScriptEnvironment()
Gavino is offline   Reply With Quote
Old 11th September 2012, 13:11   #3  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
I thought I need to delete myframe because env->NewVideoFrame(vi) allocates memory using malloc or similar. Is this not the case in the current avisynth version (2.5.8)?

GetFrame looks like this:
Code:
PVideoFrame __stdcall myNLEsource::GetFrame(int n, IScriptEnvironment* env) {
  myNLEimage* src(myNLEclassInstance->getSrcClip()->fetchImage(n));
  PVideoFrame dst = env->NewVideoFrame(vi);
  unsigned char *pdst = dst->GetWritePtr();
  for(int y = y1; y < y2; y++) {
    for(int x = x1; x < x2; x++) {
      unsigned char *srcPix = (unsigned char *)  (src ? src->getPixelAddress(x, y) : 0);
      if(srcPix) {
        for(int c = 0; c < nComponents; c++) {
          pdst[x*(nComponents) + c] = srcPix[mapping[c]];
        }
      }
    }
  pdst = pdst + dst->GetPitch();
  }
  return dst;
}
So could it be that the memory leak is actually somewhere here? But I don't think I need to delete something or set to 0 explicitly here. Do I?

What's the best thing to do with env in my case using v2.5.8? Can I also do nothing or set it to 0?

Thanks a lot for your help!
sscheidegger is offline   Reply With Quote
Old 11th September 2012, 14:06   #4  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,380
On 2.5.8, I think the best you can do is try replacing your code:
Quote:
Originally Posted by sscheidegger View Post
Code:
  delete myframe;
  delete myclip;
  delete env;
with:
Code:
  myframe = 0;
  myclip = 0;
  delete env;
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 11th September 2012, 16:06   #5  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
When I do this, it runs without breaking. But a memory leak persists!
sscheidegger is offline   Reply With Quote
Old 12th September 2012, 04:16   #6  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
How are you determining there is a leak?

env->NewVideoFrame(vi) allocates new VideoFrameBuffer's while the total pool is less than the SetMemoryMax value. When the pool is full it reuses a previous instance, unless all the ones greater than or equal to the required size are in use, in which case it allocates a new one.
IanB is offline   Reply With Quote
Old 12th September 2012, 16:30   #7  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
I realize there is a leak because the process of my NLE grows with every processed frame until it crashes (if the avisynth interface plugin is running).

My NLE creates a new instance of myNLEclass for every frame it processes. So for every frame "env = CreateScriptEnvironment()" was called. I wondered if therefore Avisynth allocates one VideoFrameBuffer every time. So I made env a meber variable of the base calss. Like that CreateScriptEnvironment() is called only once and Avisynth should be able to limit the maximum buffer size. My new code looks like that:

Code:
void myNLEclass::myNLEplugin()
{
  myNLEbaseClassInstance->env_->AddFunction("myNLEsource", "i", Create_myNLEsource, myNLEclassInstance);
  PClip myclip = myNLEbaseClassInstance->env_->Invoke("myNLEsource",0).AsClip();
  PVideoFrame myframe = myclip->GetFrame(_myNLEtime, myNLEbaseClassInstance->env_);
  // Here I plan to apply some avisynth filters and hand over the result to my NLE later
  myframe = 0;
  myclip = 0;
}
Unfortunately the memory leak is still there! Even if I add "env_->SetMemoryMax(16);" the process of my NLE grows up to 1.5 GB where it crashes! I think I'll try to install a memory leak debug library to identify where exactly the leak is. (I tried before but gave up soon when I didn't get it to run!)
sscheidegger is offline   Reply With Quote
Old 12th September 2012, 22:53   #8  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
Enough with the keyholing, If you want help show us the whole thing. You are apparently doing something unexpected and deeper analysis in needed.

Did you take full note of the this :-
Quote:
Originally Posted by Gavino View Post
...
There is a potential problem with 'delete env', which is why DeleteScriptEnvironment() has been added to Avisynth 2.6. See this thread.
...
If you "delete env;" in a different RT the memory is not actually released! This applies even with different static instance of the same revisions of MSVCRT.
IanB is offline   Reply With Quote
Old 18th September 2012, 10:15   #9  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
Hi all

I went on analyzing the memory leak. Attached is the whole code that I have so far. It's a combination of the OFX Support basic example and the avisynth simplesample in a source and an include file.

I used _CrtDumpMemoryLeaks() to track the memory leaks and found that with every processed frame the memory is growing by two additional blocks. I also found out that, if I remove line 48 in the source file (GetFrame), the memory is growing by only one block per processed frame. If I also remove line 46 (Invoke("SimpleSample",0)) the memory is not growing anymore and my OFX plug-in is running fine.

So I think now that env_ = CreateScriptEnvironment(AVISYNTH_INTERFACE_VERSION); is being called only once, we can be sure the problem is not there.

Any suggestions where to check next?

Thanks again!
Attached Files
File Type: txt ofx2avisynth.cpp.txt (20.6 KB, 16 views)
File Type: txt ofx2avisynth.h.txt (5.4 KB, 11 views)
sscheidegger is offline   Reply With Quote
Old 18th September 2012, 13:53   #10  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,041
I didn't look very hard, but to me it looks like every time you call multiThreadProcessImages(), you not only re-export the SimpleSample() function with env->AddFunction() (don't do that), you also create a new SimpleSample filter instance which you never remove. Also, while it's possible to have an Avisynth plugin in an arbitrary DLL as long as you supply the appropriate entry point, I strongly suggest you factor out your Avisynth plugin code to a separate project that compiles into a separate DLL. If you do insist on keeping it in the same project, you don't need to implement the AvisynthPluginInit2() entry point because you're never importing yourself as a plugin.
TheFluff is offline   Reply With Quote
Old 18th September 2012, 22:30   #11  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
Yes as TheFluff says, load avisynth.dll once, create IScriptEnvironment once, add your functions once, Invoke your graph once. Then use that graph lots, i.e only call the GetFrame as required.

As this is not a plugin you do not need AvisynthPluginInit2(), but might be useful to test your function standalone in an external environment.
IanB is offline   Reply With Quote
Old 21st September 2012, 12:33   #12  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
Thanks a lot for the advice! I had tried to add the functions and invoke the graph in the constructor of BasicPlugin before. This approach failed because at that point the information about the OFX src clip is not available yet.
After your hint I realized that I can also do those things once in BasicPlugin but outside the constructor and it works very well. The memory leak is gone!

I was also wondering if I need AvisynthPluginInit2() in my case. I think I'll leave it away.

Unfortunately I already have the next memory leak. I'm trying to use the mvtools plugin. I am invoking the mvtools functions only once, but when rendering the memory of the process is again growing a lot. If anyone is familiar with mvtools and has an idea what could be the reason, I'd be happy to learn.

Thank you very much for all the help!!!
sscheidegger is offline   Reply With Quote
Old 21st September 2012, 16:38   #13  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
Maybe I was blaming mvtools too early. I tested the problem the whole afternoon and I get the impression that the memory requirement increases a lot with every filter that I invoke in C++.

Finally I tried to move this part to an avs script. So in C++ I only invoke
Code:
dclip_ = env_->Invoke("Import","C:\\path\\test.avs").AsClip();
I was testing like that and it seems to run without problems. Also when I use mvtools in the script it works fine!

Very nice! I like to go to weekend having solved a problem!
Thanks a again for the support!
sscheidegger is offline   Reply With Quote
Old 21st September 2012, 18:48   #14  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,041
AvisynthPluginInit2() does exactly one thing: it's the entry point used by Avisynth when loading a DLL as a plugin. If you don't compile your project to a DLL, or if you don't intend for your DLL to be loaded as a plugin, you don't need it. In your case, you do the reverse of plugin loading: instead of letting Avisynth calling your code, you load avisynth.dll directly and call Avisynth code instead.
TheFluff is offline   Reply With Quote
Old 24th September 2012, 14:27   #15  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
Okay, now its very clear. Thanks a lot!

One more question: Is there a way to clear the graph and start invoking it from zero? I have the situation that the user can set the path to an avs script and the NLE plugin calls once:
Code:
dclip_ = env_->Invoke("Import", avsPath2_.c_str()).AsClip();
However, if the user changes the path to a different avs script, the NLE needs to call the above line again and invoke the new script. I think I'm running again into memory problems if I just call this line repeatedly for different avs scripts without somehow tyding up before. Or should that be fine?
sscheidegger is offline   Reply With Quote
Old 24th September 2012, 22:41   #16  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
Code:
PClip dclip_;
...
dclip_ = 0;
IanB is offline   Reply With Quote
Old 25th September 2012, 10:29   #17  |  Link
sscheidegger
Registered User
 
Join Date: Sep 2012
Posts: 14
So should I do something like that?
Code:
PClip dclip_;
while(avsFileChanged)
{
    dclip_ = 0;
    dclip_ = env_->Invoke("Import", avsPath2_.c_str()).AsClip();
}
Does it actually make a difference if I leave away dclip_ = 0; ?
sscheidegger is offline   Reply With Quote
Old 25th September 2012, 11:08   #18  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
Setting the PClip to zero first releases all the graph resources before building a new graph. In general it does not matter much, but if any component uses a single use resource then the second Invoke may fail.
IanB is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:43.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.