Doom9's Forum - View Single Post

-Vit- · 2nd June 2011, 13:26

Avisynth is sadly rather buggy with multi-threading and limited to 2GB memory use. There are an unfortunate few whose systems just won't play nice and can never get multi-threading working effectively. However, for many, multi-threading can be made stable and it is worth it for any moderately complex script. I have made a collection of tweaked plugins that give me perfect stability for my QTGMC related processing, they might help you too. This collection includes some of the key plugins you need, including the correct versions of RemoveGrain and its friends; picking the working versions of those plugins is a bit tricky [make sure you delete all old versions from you autoload plugins folder]

Going to x64 is the only solution to the memory issue, but many plugins don't exist in 64-bit land and there are further stability issues. However 2Gb is enough unless you're doing very complex processing on SD or moderately complex processing on HD.

This is the multi-threading information I provide for QTGMC, simplified to the general case:
____

For best script performance try going multi-threaded. You will need an "MT" version of avisynth and there are several versions. I recommend one of SEt's versions but if you get stability problems (crashes) then try the others. These are the options: Standard 2.57MT, SEt's 2.58MT or SEt's 2.6MT. Or you can convert to a completely 64-bit workflow (must also get 64-bit versions of all plugins): x64-2.58MT.

You need to tweak multi-threaded scripts to suit your system. Here is a basic template - read and follow the comments carefully and provide the values noted (you cannot use this code just as it is):

Code:

SetMemoryMax(M)  # Optional line. Leave it out at first. See below for value M
SetMTMode(5, X)  # See below for value X
YourSource("yourfile")  # DGDecode_mpeg2source, FFVideoSource, AviSource, whatever your source requires
SetMTMode(2)

# Your filters / processing goes here

Distributor() # This line may or may not be necessary, try removing it and see if that fixes crashes / gives more speed

Setting X
- Start at the number of logical cores in your machine (note: with HyperThreading enabled, logical cores = 2x physical cores)
- If it crashes, decrease 1 at a time
- Otherwise increase 1 at a time until CPU usage just reaches 100%, don't go too far or it will actually slow things down.

Setting M
- First try without the SetMemoryMax line
- However, selecting a good SetMemoryMax value might better utilise memory and increase stability. Particularly important for very complex scripts or HD
- Try values 400,600,800,1000 etc. Sometimes reducing this value can allow you to increase X for a speedup

2nd June 2011, 13:26	#4 \| Link
-Vit- Registered User Join Date: Jul 2010 Posts: 448	Avisynth is sadly rather buggy with multi-threading and limited to 2GB memory use. There are an unfortunate few whose systems just won't play nice and can never get multi-threading working effectively. However, for many, multi-threading can be made stable and it is worth it for any moderately complex script. I have made a collection of tweaked plugins that give me perfect stability for my QTGMC related processing, they might help you too. This collection includes some of the key plugins you need, including the correct versions of RemoveGrain and its friends; picking the working versions of those plugins is a bit tricky [make sure you delete all old versions from you autoload plugins folder] Going to x64 is the only solution to the memory issue, but many plugins don't exist in 64-bit land and there are further stability issues. However 2Gb is enough unless you're doing very complex processing on SD or moderately complex processing on HD. This is the multi-threading information I provide for QTGMC, simplified to the general case: ____ For best script performance try going multi-threaded. You will need an "MT" version of avisynth and there are several versions. I recommend one of SEt's versions but if you get stability problems (crashes) then try the others. These are the options: Standard 2.57MT, SEt's 2.58MT or SEt's 2.6MT. Or you can convert to a completely 64-bit workflow (must also get 64-bit versions of all plugins): x64-2.58MT. You need to tweak multi-threaded scripts to suit your system. Here is a basic template - read and follow the comments carefully and provide the values noted (you cannot use this code just as it is): Code: SetMemoryMax(M) # Optional line. Leave it out at first. See below for value M SetMTMode(5, X) # See below for value X YourSource("yourfile") # DGDecode_mpeg2source, FFVideoSource, AviSource, whatever your source requires SetMTMode(2) # Your filters / processing goes here Distributor() # This line may or may not be necessary, try removing it and see if that fixes crashes / gives more speed Setting X - Start at the number of logical cores in your machine (note: with HyperThreading enabled, logical cores = 2x physical cores) - If it crashes, decrease 1 at a time - Otherwise increase 1 at a time until CPU usage just reaches 100%, don't go too far or it will actually slow things down. Setting M - First try without the SetMemoryMax line - However, selecting a good SetMemoryMax value might better utilise memory and increase stability. Particularly important for very complex scripts or HD - Try values 400,600,800,1000 etc. Sometimes reducing this value can allow you to increase X for a speedup Last edited by -Vit-; 2nd June 2011 at 13:45.