Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
16th April 2024, 23:06 | #2821 | Link |
Acid fr0g
Join Date: May 2002
Location: Italy
Posts: 2,577
|
Do we still need to declare SetMemoryMax?
I mean, almost 70% of the computers are now 64-bit with gazillions of GBs of RAM. Is it of any use? Can't we simply let modern Windows to manage memory?
__________________
@turment on Telegram |
Yesterday, 04:47 | #2823 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,731
|
The default is quite conservative and is sometimes not enough if you work on 4K sources with heavy filtering and use multithreading. There's no real harm setting the max higher if you have the memory available, Avisynth will use what it needs and not all of it automatically.
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
Today, 00:09 | #2826 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,904
|
Quote:
There's no need to set it, Avisynth will manage that automatically. By default, in modern x64 systems, it will use 4GB of RAM which is plenty. Remember that this is the memory of the cache, in other words the memory used to store frames without having to fetch them again, NOT the RAM used by filters in general. This means that even if you were to set SetMemoryMax(512) but you were heavily filtering a UHD content, Avisynth would still use way more than 512MB of RAM. Remember that the cache is there only so that if you have temporal filters and encoders that need to access frames in a non linear fashion, Avisynth won't have to compute them every single time it moves forward and backward and forward and backward etc. The bigger the cache the more frames are gonna be stored, but storing too many frames can be detrimental as it's gonna use RAM that could otherwise be used by - let's say - the encoder, which is why the cache size is calculated on the fly on the basis of the available RAM and it never exceeds 4GB. Assuming RGB48 (which is overkill anyway), we have: 3840×2160×48 = 398131200 bits = 0.0498 gigabyte That's just for one frame, so at the current default Avisynth can cache more than 80 frames. Obviously working with RGB48 UHD is a bit unrealistic, but if we work in YUV that number increases and it even goes much further if we work in 8bit and at lower resolutions. My point is that it's already implemented correctly and I don't see a reason to change that. If anything, having the SetMemoryMax() is useful if you wanna reduce the cache. |
|
Today, 04:20 | #2827 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,067
|
It looks you not understand how frame-based AVS+ MT is working.
To make things as fast as possible (to minimize CPU stall on threads sync) it looks AVS+ simply put several frames cache around _each_ filter in the filtergraph and in _each_ logical thread. So any time system have some free logical CPU cores it can load it with some useful computing. But it cause awful RAM consuming in any 'complex' scripts of a several filters in a chain. Total RAM in a cache is about NumFrames_in_Prefetch_x_NumFilters_x_NumThreads_x_FrameSize. |
Today, 04:59 | #2828 | Link |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,731
|
Regarding the 4GB, which is unfortunately not enough in many cases: https://forum.doom9.org/showthread.p...75#post1913375
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
Today, 10:57 | #2830 | Link | ||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,904
|
Quote:
Quote:
For instance, my scripts are like: Code:
video=LWLibavVideoSource("video.mxf") audio=LWLibavAudioSource("audio.mxf") AudioDub(video, audio) ConvertBits(16) ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false) ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.5, contrast_X=0.9) ConvertXYZtoYUV(pColor=0) without prefetch. In the example above, the filters would create their own threadpool which is how I think it's intended to be. Call me traditionalist but I see Prefetch() as an evolution of the old MT Modes from 2009 and in my head they should only be used if the filter you're calling is old and single-threaded only and of course at your own risk as it might misbehave, especially if it filters temporally. This however brings me to a different question, then. Is there an advantage in raising the SetMemoryMax() value from 4GB to something higher IF I don't use Prefetch()? Last edited by FranceBB; Today at 11:00. |
||
Today, 12:08 | #2831 | Link |
Registered User
Join Date: Jul 2018
Posts: 1,067
|
If you not use Prefetch() it mean AVS+ will run all filtergraph with single threaded cache mode ? So the equation for total cache size is about
NumFrames_in_Prefetch_x_NumFilters_x_FrameSize. Default num frames in AVS+ cache (for each filter) is at least several ? (2..3 to 10 ? or num of physical cores ?). "Is there an advantage in raising the SetMemoryMax() value from 4GB to something higher IF I don't use Prefetch()?" It may depends on workflow. And easy tested between 4 GB and all RAM avaialble. The idea of SetMemoryMax() may be complex hint to AVS memory manager like to save from swapping of too much allocated cache pages attempt to lower number of cached frames if total process RAM allocation approaches the current RAM limit. Because with increasing number of frames in the cache user may got some performance boost (depending of filters used and many more effects) setting too low MemoryMax may cause decreasing of performance in some use cases. It is better to check with exact end user workflow. Last edited by DTL; Today at 12:15. |
Thread Tools | Search this Thread |
Display Modes | |
|
|