? A script to make Avisynth's cache grow >2GB ?

[Didée]

In order to test something (regarding 64bit Avisynth), I wanted to make a script that lets Avisynth's frame cache grow bigger than 2GB, using only internal core filters. But hmmh, I don't see an easy way to do so! Something very convoluted like e.g.

interleave( selectevery(n,0).temporalsoften(7,...).temporalsoften(7,...),
\ selectevery(n,1).temporalsoften(7,...).temporalsoften(7,...),
\ ..etc..etc..
\ selectevery(n,n-1).temporalsoften(7,...).temporalsoften(7,...),
\ ).temporalsoften(7,...).temporalsoften(7,...)

surely can make the cache grow pretty big, but hey, somehow that isn't the right thing.

Any nice & clean way to let the cache grow VERY big, with only core filters? And preferably without conditional environment?

[cretindesalpes]

Maybe by resizing pictures to insane sizes ? A single 10000x10000 picture in RGB32 would occupy 0.4 GB.

[IanB]

Apart from bugs that leak PVideoFrame instances ideally it should not normally be possible.

From 2.5.8 the memory manager actively defends against exceeding the SetMemoryMax value. And the value is internally limited to the amount for free physical ram available less a small amount. From 2.5.8 SetMemoryMax(0) will return the actual limit being used. The various MT builds floating around still use the old 2.5.6 memory manager/cache model which has many pathological problems. That combined with the shotgun multi-threading approach leaves plenty of room to squander PVideoFrames in those versions.

Things that allocate static PVideoFrames like BlankClip can be used to lock up chunks of the managed memory pool. Things that allocate private internal buffers like Overlay can be used to lock up chunks of the address space that is not managed. A 16000x16000 RGB32 image would use about 1GB per instance.

As I have pointed out in the past in order for the cache to be useful some filter within a script must require the same frame to be provided more than once. Trivial examples are temporal filters that want previous, current and next input frames for each output frame.

[Didée]

I see ... in the middle of a forest of bugs, hacks, fixes and workarounds, it's not easy for a poor little script writer to tell the road from the moat.

Point is, during testing Avisynth64 with ressource hungry scripts, I found scripts that work as long as memory usage (per taskmanager) is below 2GB, but die away when going above 2GB. Not surprisingly, it's scripts using MVTools ... and I basically wanted to isolate if it's Avisynth64 itself which causes the bug, or if it's MVTools.

After all, not having the 2GB barrier is one major point of Avisynth64 ... but at least up to now, it seems that this very barrier still does exist. Which is a pity.

[um3k]

This is kind of off topic, but I just had a realization. If Didée created the likes of TempGaussMC on a stone age computer, does that mean his scripts with the new computer are going to give impossibly amazing results...on a supercomputer (and too slow to use on anything else)?

[Mystery Keeper]

This is kind of off topic, but I just had a realization. If Didée created the likes of TempGaussMC on a stone age computer, does that mean his scripts with the new computer are going to give impossibly amazing results...on a supercomputer (and too slow to use on anything else)?

They are going to run faster, that's all. You know, 0.5 FPS vs 2 FPS IS a big difference.

[Selur]

only thing you should fear now is Didée creating even slower scripts ;)

[Groucho2004]

Not surprisingly, it's scripts using MVTools ... and I basically wanted to isolate if it's Avisynth64 itself which causes the bug, or if it's MVTools.

Are you using a 64 bit version of MVTools? Is there even one?

[um3k]

only thing you should fear now is Didée creating even slower scripts ;)

That's what I was talking about. :devil:

[Didée]

Are you using a 64 bit version of MVTools? Is there even one?
Sure, there is. Have a look here (http://forum.doom9.org/showthread.php?t=152800) ...


They are going to run faster, that's all. You know, 0.5 FPS vs 2 FPS IS a big difference.
Basically correct, but the numbers don't fit. You think 4 times faster would be an event?

When coming from an ancient singlecore, it's really impressive what modern multicores can crunch. With computationally intensive tasks like x264 encoding or heavy Avisynth scripts, my actual numbers are more like 1.9 fps vs. 62 fps. Yes, really. 30 to 35 times faster. Feels like Futureworld.


only thing you should fear now is Didée creating even slower scripts ;)
For the moment, rather not. The actual task is "wood working". The new rig doesn't fit in the old desk (which I once handmade from solid wood) anymore ... and worse, the massive table board has warped over the years. Needs a complete overhaul, quite some work to do, a few weeks to finish. For the time being, the new PC is residing in a place too uncomfortable to do much work with it.

[Motenai Yoda]

mmm
a ripetition of
a=last
overlay(a,a.blur(1))
a=last
overlay(a,a.TemporalSoften(7,0,0,0,2))
?