PVideoFrame lifetime?

[Guest]

I need a definitive answer to the following question about the Avisynth core, possibly from Richard Berg, biodoche, or dividee.

Ben's tutorial states that a frame won't be recycled until all references to it go away. If I declare a PVideoFrame in a filter's class data, one would think the frame would be stable as long as the filter is instantiated, i.e., between GetFrame() calls. But sh0dan asserts that the frame will get recycled.

There are several filters that will require extensive redesign if this is true.

Can someone that really knows address this issue? I am not dissing sh0dan because he has stated that he finds the code very cryptic and doesn't fully understand it.

Finally, it seems a serious architectural limitation to not have a way to declare a stable PVideoFrame.

Thank you.

[Richard Berg]

I won't pretend to understand the videobuffer life-cycle from start to finish -- frankly, that's half the reason I want to rewrite it -- but looking at the code again I think sh0dan is wrong. (It may be a communication thing though; in one sense /all/ framebuffers you come across may or may not have been recycled previously, and you never hold onto one directly.) If nothing else, the complexity makes me pretty certain nobody's messed with it since Ben.

Are you able to compile Avisynth? The most instructive thing for me is just tracing through with a source debugger. In fact I added some null-transform routines yesterday for the very purpose of hunting down the last few bugs in the cache, BitBlt, etc.

[sh0dan]

My main question is:

How can AviSynth know when a filter is done processing a videoframe, and there are no references to it? If a videoframe is requested by a filter, and places in a local variable - does AviSynth get a call somewhere telling it that the local variable has been invalidated (when the function has finished).
In my head, a frame->free() function would be needed for AviSynth to know when it can recycle frames, if it was allowed for filters to get a stable videoframe.

I see no internal filters preserving frames across GetFrame calls (by storing them in member variables for instance). Because of that, and because of what I think I can put together from Ben's code, this is my guess on how it works.

So - create your own framebuffers (new or malloc), and blit the information over there - it's the only safe solution I see.

[trbarry]

I see no internal filters preserving frames across GetFrame calls (by storing them in member variables for instance). Because of that, and becauseof what I think I can put together from Ben's code, my

I was under the impression from Ben's write up that Avisynth did some fancy ref counting on PVideoFrame's every time you returned from a filter. But maybe I mis-remember that.

It would be nice if someone could really understand and explain that process. For instance, does it make a difference if a PVideoFrame is in a member vs a local variable? And what, if anything, happens to ref counts when a filter returns to its caller?

So I have no idea how it's done internally but I hope the quote above ("across GetFrame calls") is referring to calls TO the filter and not FROM the filter. For instance, any IVTC or deinterlace may reference multiple frames from a clip to compare and process them. So I would hate to think that referencing, say, a previous frame somehow affected my access to the current frame, all in the same invocation of my filter. But I'm assuming that's probably not the case here.

- Tom

[Richard Berg]

does AviSynth get a call somewhere telling it that the local variable has been invalidated (when the function has finished).
Yes. When a local variable goes out of scope, its destructor is called:

~PVideoFrame() { if (p) p->Release(); }

For instance, does it make a difference if a PVideoFrame is in a member vs a local variable?
For class members, the same destructor is called when the YourFilter class is deleted, namely here:
~FunctionTable() {
while (local_functions) {
LocalFunction* prev = local_functions->prev;
delete local_functions;
local_functions = prev;
}
I think...

[sh0dan]

When saying "across GetFrame calls" I mean from one call to GetFrame (to get frame x) to the next GetFrame call (to get frame y).

What you say about Ben's thoughs does make much sense.
A filter can request or create several frames, but only one can be returned, thus only the returned frame will be said to have a reference, since all internal frames are considered "temporary frames" only needed to create the retured frame. Thus all frames not returned by the filter is said to have no references, and can therefore be recycled.

[Bidoche]

What's so complicated to understand with refcounted VideoFrame ?

it's simply count the number of PVideoFrame pointing to itself
PvideoFrame operations are done so this refcount is correctly updated.

And so when there is no longer a PVideoFrame pointing to it, the VideoFrame puts oneself in the recycle bin to be recycled

[Guest]

Originally posted by Bidoche
What's so complicated to understand with refcounted VideoFrame ?

it's simply count the number of PVideoFrame pointing to itself
PvideoFrame operations are done so this refcount is correctly updated.

And so when there is no longer a PVideoFrame pointing to it, the VideoFrame puts oneself in the recycle bin to be recycled So that means I have to do both the declaration of a PVideoFrame and the NewVideoFrame() call and assignment to my PVideoFrame in my filter constructor? My goal is to allocate a VideoFrame that will remain stable across GetFrame() calls to my filter. Thank you.

[Bidoche]

As the framework is now, you can't safely assume a VideoFrame will stay stable.
Of course it should, but a missing MakeWritable somewhere and your frame is messed up.
That's why I was saying earlier that MakeWritable should be performed somehow when calling GetWritePtr. (then you are safe)

But for now it's impossible (can't modify the VideoFrame * when calling a method of the VideoFrame...)

It can be done if you distinguish VideoFrame * and const VideoFrame *
(I am doing it in my changes)

For now, the only solution is to not share your frame with others, you build it, you keep it somewhere and you never share it with anybody, you make copies for that.

[Guest]

Thank you, biodoche. Currently, I do only keep it and don't share it out, copying it when required by others. But what I am doing is declaring the PVideoFrame in the filter class but doing the NewVideoFrame() call and assignment to my PVideoFrame in the GetFrame(). Is that an error?

[sh0dan]

@Biodochzeck: Very nice to hear! You should have in mind, that you are dealing with a bunch of people very good at image processing, and not that much into "system" programming - at least that's true for me :)

First time for a long time I've been glad to be wrong. Now let's get this bug hunted down! :)

Edit: Find it :)

[Bidoche]

@sh0dan

I would say it goes both sides, when you are talking about smart image processing, sometimes it goes completely over my head :p

@neuron2

you keep the PVideoFrame as a filter member and make the copy in GetFrame.
What's wrong with that ? and anyway how would you do otherwise ?

Edit: And please note that my nick is Bidoche, not biodoche.
At first I thought it was a typo, but you seem to be consistent with it...

[Bidoche]

For correcting the bug, an easy workaround :

you insert _ASSERTE(IsWritable()); inside GetWritePtr() in VideoFrame
and it will forbid changes who should not happen
Or maybe even a ThrowError...

That would be rather a hack than a real fix, but it should work

[Guest]

Originally posted by Bidoche
you keep the PVideoFrame as a filter member and make the copy in GetFrame. What's wrong with that ?That's what I was wondering. :-)

and anyway how would you do otherwise ?You could do this in your filter constructor:

framep = NewVideoFrame(vi);

...where framep was declared in the filter class. And as opposed to doing the NewVideoFrame() call in GetFrame().

"Bidoche". Sure no problem. A foolish consistency is the hobgoblin of little minds. Isn't it?

[Bidoche]

@neuron2

I thought you were already doing that for the saved PVideoFrame.
Can you post the relative code, I will be easier to understand each other that way.

[sh0dan]

I'll get it plugged in ASAP, so we can locate the source of this problem.

BTW, it funny how wrong names stick. I guess if search for "convultion 3d" you probably get 20 posts from me, until I finally figured out it was convolution3d. ;)

[Guest]

Sorry to keep harping guys. You know, I wouldn't want to be seen as harrassing anyone. :)

My crash is not happening on a GetWritePtr() call but on the GetFrame() to get a previous frame.

I am going to follow Richard's suggestion to trace through Avisynth, because sh0dan has not been able to duplicate the problem. And besides, maybe I'll learn something. :)

[Guest]

Originally posted by Bidoche
@neuron2

I thought you were already doing that for the saved PVideoFrame.
Can you post the relative code, I will be easier to understand each other that way. Actually, there are two cases. In Dup, yes, I do it that way, i.e., declare PVideoFrame in filter member and do NewVideoFrame in filter constructor, and finally copy data into it in GetFrame(). But in MarcFD's Cnr2(), he puts the NewVideoFrame() call in GetFrame and arranges it to be called only once. I will post code when I get home from work.

Also, I will post some details about the crash to see if you have any ideas.

EDIT: Thank you everyone for your help. Your assistance is greatly appreciated.

[Bidoche]

Yes, but it's GetWritePtr() who allows modification of frames who should not (ie those who are shared)

[Guest]

OK, then we are facing two different problems it appears.

Let's squash them both. :)

[Guest]

OK, I built Avisynth and traced the problem down. I found that I was in fact sharing the private copy! That caused the refcount to go to 2 and subsequent GetWritePtr() calls returned 0, causing the crashes.

So, there is just this problem and not two problems.

Thank you all for your assistance. I have released fixed versions of Dup() and Cnr2() that don't suffer from this problem.

I hope the core developers will consider doing something to improve this situation.

EDIT: I felt a rush of adrenaline as I traced through Avisynth. Thanks for motivating me to do it, Richard. I'm going to learn a lot now. :)

[sh0dan]

neuron2: Glad to hear! I'll update the plugin package with the new Dup as soon as you release it :)

Seems like we also have a dll requiering a debug dll, so we'll have to update anyway.

[Bidoche]

Since we talked about VideoFrame recycling and such.

I am wondering if someone knew if recycling them is really worth the hassle.
I understand that recycling buffers (who are huge) is really important, but VideoFrame are small objects (32 maybe 64 bytes).
So is it really a gain to use specific recycling rather than having new do the job...

[sh0dan]

IMO there is no reason at all to recycle them, unless it serves a structural purpose.

[Bidoche]

As far as I know, there is no such one.
The only reason is can see for such recycling are :
- if it's faster... but it can be a big gain
- if it's help limiting memory fragmentation... but I think new already does some tricks about that (when malloc don't)

[Guest]

Now I know why there are multiple refcounts. :)

[Bidoche]

multiple refcounts ?
Where do you see that, there are only one per object.

[Guest]

I mean one for the buffer and one for the pointer.

[Bidoche]

you mean: one for the VideoFrame and one for the VideoFrameBuffer

[Guest]

If that is a question, then yes. If a statement, then you're right. :)

[Richard Berg]

What is your answer, neuron? I know when I was first looking over the code I was confused why it was necessary to ref-count the pointers when the buffers are already ref-counted; as Bidoche said, it only saves a few dozen bytes.

[Guest]

Oh, I just meant originally I was confused to see two refcounts and wondered why it was done. You gents are just echoing my intuition. When I said now I see why there are two, I then understood that it was an attempt to recycle the PVideoFrames as well as the buffers. I agree with Bidoche.

[Bidoche]

In fact, if VideoFrame were not refcounted and handled with PVideoFrame, but copied at each time.
It would solve some problems we have now, then we could built-in MakeWritable in VideoFrame and we'll be assured that buffers can't be modified.

In the other hand, it would probably arise some problems too.

NB: I'm not saying we should do that.