Avisynth 2.6 MT

[Gavino]

I see what you mean - the documentation is misleading.

[Wilbert, can this be improved? Perhaps remove 'taps' from Basic usage for Blackman, Lanczos and Sinc?]

[Boulder]

Hi SEt,

have you seen this thread yet: http://forum.doom9.org/showthread.php?t=151248? There seems to be something in the MT Avisynth build that a recent x264 doesn't like.

[SEt]

Seems like it now requires manual call to distributor() as ffdshow. Try adding at the end of script the same
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

[Boulder]

Seems like it now requires manual call to distributor() as ffdshow. Try adding at the end of script the same
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : lastSeems to work, thanks :)

[Dark Shikari]

Do note that when the x264-side fix is committed, such a call will likely break things horribly.

[kemuri-_9]

we implemented the same call that is used in the vfw interface, script wise this comes out to

cur_mt_mode = GetMTMode(false)
(cur_mt_mode > 0 && cur_mt_mode < 5) ? Distributor() : last

[Adub]

So is it recommended to be using SET's build of Avisynth 2.6 Alpha when I want to use MT, or should I use the newer version of the official Avisynth 2.6 Alpha (090927) release?

[cweb]

So is it recommended to be using SET's build of Avisynth 2.6 Alpha when I want to use MT, or should I use the newer version of the official Avisynth 2.6 Alpha (090927) release?
The latter doesn't support MT while SET's build does...

[Adub]

Ah, I thought that 2.6 was going to incorporate MT anyways. Thanks for clearing everything up.

[kemuri-_9]

afaik the official alpha build release is just a partial merge of the 2.6 branch into the main branch (formerly 2.5.x).
not everything of 2.6 was merged in...

where as set's build is from the 2.6 branch itself so it contains all the features of 2.6

[cweb]

afaik the official alpha build release is just a partial merge of the 2.6 branch into the main branch (formerly 2.5.x).
not everything of 2.6 was merged in...

where as set's build is from the 2.6 branch itself so it contains all the features of 2.6
Yes I think that's why the official alpha doesn't support MT...

[Boulder]

SEt, I get unhandled win32 exception crashes with a recent ICL build of x264 (r1416) and your Avisynth 2.6 MT build. Your multithreaded Avisynth v2.5.8 seems to work fine.

This is the script I used:
SetMTMode(5)
MPEG2Source("1.d2v")
SetMTMode(2)
clip1=last.Trim(0,74268)
clip2=last.Trim(74269,0)
clip1 ++ clip2.YADIFMod(edeint=NNEDI2(clip2))
Crop(2,2,-6,-2,true)
MCTemporalDenoise(chroma=true,enhance=true,sigma=7)

MCTemporalDenoise (http://forum.doom9.org/showthread.php?t=139766)
http://kemuri9.net/dev/x264/x264_r1416_set.zip

[SEt]

Script itself works fine for me, but eats 2.7GB of memory (with 720x480 resolution and 8 threads) what your build of x264 can't provide.
Change in that x264_icl.exe (32-bit of course) byte at offset 0x106 from 0x03 to 0x23 and run it on 32-bit system with /3GB or any 64-bit system.

[Boulder]

Script itself works fine for me, but eats 2.7GB of memory (with 720x480 resolution and 8 threads) what your build of x264 can't provide.
Change in that x264_icl.exe (32-bit of course) byte at offset 0x106 from 0x03 to 0x23 and run it on 32-bit system with /3GB or any 64-bit system.

Could this be fixed by using a lower SetMemoryMax()? I think I have it automatically set at 384MB (I have 2GB of memory). When looking at Task Manager, the x264 process consumes 700-800MB of memory.

[SEt]

There is no good way to limit memory usage of MT cache in avisynth now other than using less threads.

[Chainmax]

So, usage still consists of putting MT v0.7's dll in the plugin folder and the replace windows\system32's avisynth.dll with this one, right? Also, is this:

LoadPlugin("X:\wherever\MT.dll")
**load other filters**

SetMTMode(5)
**load source**
SetMTMode(2)
**rest of the filterchain**

still a good way to use multithreading or are there more nuances to it in this newer version?

[levi]

You dont need to load the mt.dll unless you mt a specific function

otherwise just load your favorite flavor of the patched version of avisynth will enable setmtmode()

http://avisynth.org/mediawiki/MT

Only nuance I found is disabling hyperthreading gives better performance.

[boondoggle]

I get an error when i press any arrow key in AvsPmod i.e just look through movie for about 10 seconds then it gives this:

Traceback (most recent call last):
File "AvsP.pyo", line 5662, in OnMenuVideoNextFrame
File "AvsP.pyo", line 9191, in ShowVideoOffset
File "AvsP.pyo", line 8950, in ShowVideoFrame
File "AvsP.pyo", line 9499, in PaintAVIFrame
File "pyavs.pyo", line 322, in DrawFrame
File "pyavs.pyo", line 301, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception code 0xe06d7363

No filters just AVISource(clip). Any ideas?

[boondoggle]

Oh, and it appears only with your version of avisynth of course. Its quite annoying having to choose between about 40% speed increase or being able to edit scripts meanwhile processing :/

[Bob Wya]

Hi

Stupid question... But please, please can we have access to the .avs script name in the Avisynth 2.6 API (so it is available to externally linked filters)!! It's been in reasonably high demand (as a feature) and surely it's just a 5 line function/public class method... I spent hours looking through the main Avisynth 2.58 source for a way to access the avs filename for a filter I am currently working on... :mad:

:thanks:
Bob

[kemuri-_9]

there is no guarantee that an avs script was even used, so how are you expecting to handle that situation?

[Bob Wya]

there is no guarantee that an avs script was even used, so how are you expecting to handle that situation?

It is only a problem if you expose the API to Avisynth users!! I am talking about allowing filter developers access to this new function in their own filter classes (via the Avisynth.h header). Obviously if no Avisynth script is being used just do something sensible and return a null string "" or similar (pretty obvious I would of thought).

Bob

[Gavino]

Obviously if no Avisynth script is being used just do something sensible and return a null string "" or similar
As well as in the case where no script is being used (eg a program using the API via the library interface), it would also have to return a null value if called at run-time (GetFrame), since the script filename is a compile-time concept and the information is not present in the compiled filter graph.

[Bob Wya]

As well as in the case where no script is being used (eg a program using the API via the library interface), it would also have to return a null value if called at run-time (GetFrame), since the script filename is a compile-time concept and the information is not present in the compiled filter graph.

In my filter I need the name of the .avs script at the filter constructor stage - when it should be available (- right?)

The script name can be saved in a class variable in the filter constructor to use later on (e.g. in GetFrame()) anyway I guess.

Bob

[IanB]

From the current CVS :- global_var_table->Set("$ScriptName$", AVSValue());
global_var_table->Set("$ScriptFile$", AVSValue());
global_var_table->Set("$ScriptDir$", AVSValue());
{ "ScriptName", "", ScriptName },
{ "ScriptFile", "", ScriptFile },
{ "ScriptDir", "", ScriptDir },
...
HANDLE h = ::CreateFile(full_path, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
if (h == INVALID_HANDLE_VALUE)
env->ThrowError("Import: couldn't open \"%s\"", full_path);

env->SetGlobalVar("$ScriptName$", env->SaveString(script_name));
env->SetGlobalVar("$ScriptFile$", env->SaveString(file_part));

*file_part = 0;
CWDChanger change_cwd(full_path);

env->SetGlobalVar("$ScriptDir$", env->SaveString(full_path));

DWORD size = GetFileSize(h, NULL);
DynamicCharBuffer buf(size+1);
BOOL status = ReadFile(h, buf, size, &size, NULL);
CloseHandle(h);
if (!status)
env->ThrowError("Import: unable to read \"%s\"", script_name);
...
AVSValue ScriptName(AVSValue args, void*, IScriptEnvironment* env) { return GetVar(env, "$ScriptName$"); }
AVSValue ScriptFile(AVSValue args, void*, IScriptEnvironment* env) { return GetVar(env, "$ScriptFile$"); }
AVSValue ScriptDir (AVSValue args, void*, IScriptEnvironment* env) { return GetVar(env, "$ScriptDir$" ); }
ScriptName is the raw input string to Import.
ScriptFile is the filename part of the parsed full pathname of the script file.
ScriptDir is the directory part of the parsed full pathname of the script file.

[Gavino]

From the current CVS :-
...
env->SetGlobalVar("$ScriptName$", env->SaveString(script_name));
env->SetGlobalVar("$ScriptFile$", env->SaveString(file_part));

*file_part = 0;
CWDChanger change_cwd(full_path);

env->SetGlobalVar("$ScriptDir$", env->SaveString(full_path));
...

Is that the only place the variables get set?
To allow for the nested nature of Imports, I would expect to see them being saved on entry to function Import and restored on exit.
(This would also reset them to null for the run-time phase.)

[Bob Wya]

ScriptName is the raw input string to Import.
ScriptFile is the filename part of the parsed full pathname of the script file.
ScriptDir is the directory part of the parsed full pathname of the script file.

Ah thanks for getting back to me! I thought I had read it was going to get added in to a new build of Avisynth. It might seem lazy that I haven't downloaded the 2.6 source code to check this... I'm just suffering brain burnout from trying to understand how the 2.58 code hangs together!!

:thanks:
Bob

[IanB]

Is that the only place the variables get set?
To allow for the nested nature of Imports, I would expect to see them being saved on entry to function Import and restored on exit.
(This would also reset them to null for the run-time phase.SourceForge.net > Find Software > avisynth2 > SCM Repositories > avisynth2 > avisynth > src > core > parser > script.cpp (http://avisynth2.cvs.sourceforge.net/viewvc/avisynth2/avisynth/src/core/parser/script.cpp?r1=1.27&r2=1.28&pathrev=MAIN)

[dansrfe]

I'm getting a "Addborders must add by mod2" error which I never got with the 2.5.8 MT build and have been adding odd values for addborders for a while with 2.5.8 MT.

[kemuri-_9]

I'm getting a "Addborders must add by mod2" error which I never got with the 2.5.8 MT build and have been adding odd values for addborders for a while with 2.5.8 MT.

make sure your video's colorspace isn't something that has mod2 requirements.

[dansrfe]

The resulting resolution after doing addborders is definitely like mod8 with it going to 1920 x 1080. Colorspace is YV12.

EDIT: Do we need to install AviSynth 2.6.0 Alpha 2 from soureforge on top of the existing 2.5.8 installation (or not) and then replace the 2.6 dll by SEt?

[kemuri-_9]

The resulting resolution after doing addborders is definitely like mod8 with it going to 1920 x 1080. Colorspace is YV12.

Oh, it seems that in 2.58 the AddBorder values rounded to requirements of the colorspace.
so when you specified odd values, it either rounded down or up to the nearest compliant value.
this behavior was changed for 2.6 to throw the error.


EDIT: Do we need to install AviSynth 2.6.0 Alpha 2 from soureforge on top of the existing 2.5.8 installation (or not) and then replace the 2.6 dll by SEt?

no, you can replace SEt's 2.6 dll over the 2.5.8 installed dll, but you won't have documentation matching your installed version in that case.

[dansrfe]

It's strange because i suddenly started getting this error even when I replace the dll back to 2.5.8.5. I always used to use Addborders with odd values, most of the time, for adding dvd borders to make it 720 x 480 and never had any problems. Also i want to point out that I can see the preview in AvsP without the error, it's only when I preview through MPC-HC that I get the error. I hope there is some fix because I'm not sure why I've suddenly started seeing the error in MPC. I've used mod2 for a long time in on-the-fly post=processing and no errors have popped up until today :(

[boondoggle]

Could someone please just check if my error in AvsP (holding arrow key to play through a movie for 10 sec and then hangs, even without any filters) only happends for me or if its a known bug?

[Gavino]

SourceForge.net > Find Software > avisynth2 > SCM Repositories > avisynth2 > avisynth > src > core > parser > script.cpp (http://avisynth2.cvs.sourceforge.net/viewvc/avisynth2/avisynth/src/core/parser/script.cpp?r1=1.27&r2=1.28&pathrev=MAIN)
Thanks, Ian - I see you already had my point covered. Sorry I was too lazy to check it out before posting.

Thinking about how these new functions work, I believe that if called from a function they will return the file currently being compiled rather than the file containing the function declaration (unlike mf's and stickboy's hack with try/catch from this thread (http://forum.doom9.org/showthread.php?t=66627)). Personally I think that is prefererable (and it would be difficult to do it any other way), but it's something users need to be aware of.

[kemuri-_9]

It's strange because i suddenly started getting this error even when I replace the dll back to 2.5.8.5. I always used to use Addborders with odd values, most of the time, for adding dvd borders to make it 720 x 480 and never had any problems. Also i want to point out that I can see the preview in AvsP without the error, it's only when I preview through MPC-HC that I get the error. I hope there is some fix because I'm not sure why I've suddenly started seeing the error in MPC. I've used mod2 for a long time in on-the-fly post=processing and no errors have popped up until today :(

As I've not seen what AvsP does nor have I ever used it, I don't know why you don't get the error there but do get it with mpc-hc.
It could be converting to RGB which would allow for non mod2 values...

Truthfully i prefer the error over 'just working' in this situation:

when i had a YV12 video and did AddBorders(0,1,0,1), the 1 pixel top was rounded down to 0 and the 1 pixel bottom was rounded up to 2.
This also has it to where AddBorders(0,1,0,0) does absolutely nothing and AddBorders(0,0,0,1) adds 2 pixels to the bottom.

This 'just working' situation is irregular and it should've thrown an error to begin rather than working in the above cases.

I would suggest that you conform to the change and stop using values in AddBorders that is not compliant with the colorspace you are using.

[IanB]

Given this thread is about SEt's build of the old CVS Avisynth_26 code parking branch, I am quite surprised at how few problems people actually have had. None of the code in that branch was ever tested past the ability to clean compile. I do remember fixing the code for AddBorders() and Letterbox() to handle the new planar formats, so it's quite likely that there are bugs in this build. If there is a problem with the current AddBorders() in AviSynth 2.6.0 Alpha 2 then they should be reported in its thread

[dansrfe]

Can someone please fix this build or make a new MT build for 2.6 that doesn't throw mod2 Addborders errors.

[kemuri-_9]

Can someone please fix this build or make a new MT build for 2.6 that doesn't throw mod2 Addborders errors.

what? we have already discussed this.
using values in addborders that do not comply with the underlying csp is now a user error, fix your script.

[dansrfe]

Why is it "wrong" to use odd values in Addborders to result in a mod2 or mod4 resolution. AviSynth 2.5.8 MT doesn't cause me any problems or "green"borders or anything of the like and the result after using Addborders with odd and even values gives the exact resolution I desire without issues. I understand resizing to mod2, using functions/plugins/scripts in mod2 or mod4 or mod8 but adding black pixel rows in an odd amount doesn't seem like it should be causing "trouble".

[leeperry]

I've run a few test w/ a big script using LSF/ColorMatrix/SmoothLevels/GrainFactory3(based on AddGrainC) on XPSP3 using 4 cores off a 3.5Ghz Q9450:
Avisynth 2.57 MT: 70fps
Avisynth 2.58 MT: 53fps
Avisynth 2.6 MT: 77fps

I get 200fps w/ SmoothLevels(preset="tv2pc",multithread=2,Smode=0,useopt=true) using Avisynth 2.57, and 275fps using Avisynth 2.6....talk about being late to the party :rolleyes:

I'm not sure whether I should use MT.dll from 2.57 or 2.58? They both seem to work and I get slightly more fps from 2.57. I hope it'll be stable, thanks for the fps boost! :)

[IanB]

Why is it "wrong" to use odd values in Addborders to result in a mod2 or mod4 resolution. AviSynth 2.5.8 MT doesn't cause me any problems or "green"borders or anything of the like and the result after using Addborders with odd and even values gives the exact resolution I desire without issues. I understand resizing to mod2, using functions/plugins/scripts in mod2 or mod4 or mod8 but adding black pixel rows in an odd amount doesn't seem like it should be causing "trouble".
Well I've wasted some of my valuable time and done some archaeology to check the code SEt actually used and it is correct.

For versions 2.5.8 and prior AddBorders() silently chopped odd values to force them even :- if (vi.IsYUV()) {
// YUY2 can only add even amounts
left = left & -2;
right = (right+1) & -2;
if (vi.IsYV12()) {
xsub=1;
ysub=1;
top=top& -2;
bot=(bot+1)& -2;
}
}Thus scripts with errors were silently allow to give wrong results.

2.6 now throws a script error when a value incompatible with the current pixel format is given :- if (vi.IsYUV()) {
if (!vi.IsY8()) {
xsub=vi.GetPlaneWidthSubsampling(PLANAR_U);
ysub=vi.GetPlaneHeightSubsampling(PLANAR_U);
}

const int xmask = (1 << xsub) - 1;
const int ymask = (1 << ysub) - 1;

// YUY2, etc, ... can only add even amounts
if (_left & xmask)
env->ThrowError("AddBorders: YUV image can only add by Mod %d (left side).", xmask+1);
if (_right & xmask)
env->ThrowError("AddBorders: YUV image can only add by Mod %d (right side).", xmask+1);

if (_top & ymask)
env->ThrowError("AddBorders: YUV image can only add by Mod %d (top).", ymask+1);
if (_bot & ymask)
env->ThrowError("AddBorders: YUV image can only add by Mod %d (bottom).", ymask+1);
}Only Y8 and YV24 can have arbitrary odd values for AddBorders()

[dansrfe]

Then why do I still get the desired output resolution after addborders is used with calculated values using addition of the borders? For example if I have a video that has the correct A/R at 720 x 358 and I want to add borders to make it 720 x 480 (disregard all A/R flags atm); why does this work? Why do I get exactly 720 x 480 while using addborders(0,61,0,61)? If it does indeed round the addborder parameters to mod2 in 2.5.8 then why don't I get something like 720 x 478?

[Gavino]

If it does indeed round the addborder parameters to mod2 in 2.5.8 then why don't I get something like 720 x 478?
Because it rounds the top border down to mod2 (giving 60) and the bottom border up to mod2 (giving 62).

[dansrfe]

ah^. That clears up the confusion for me :). Ok I guess I'll have to revise my scripts a bit then :D

[Boulder]

Does anyone have a recent MT build? Avisynth 2.6 has had quite a few fixes since September 2009.

[leeperry]

Does anyone have a recent MT build? Avisynth 2.6 has had quite a few fixes since September 2009.
http://forum-images.hardware.fr/images/perso/rastaman40.gif

[DVDBob]

Can i to this:
MPEG2Source("My movie.d2v",cpu=0)
Crop(14,2,-18,-10)
mt("""MCTemporalDenoise(settings="low")""")

Or should i do this:
SetMTMode(5,0)
MPEG2Source("My movie.d2v",cpu=0)
Crop(14,2,-18,-10)
MCTemporalDenoise(settings="low")
SetMTMode(2,0)

I have a 2 core cpu.

[Fullmetal Encoder]

Has anyone tried FFMS2 with Avisynth 2.6? I have been able to set up the x64 FFMS2 to work with JoshyD's 64-bit 2.5.8 Avisynth but after hearing such great performance out of SEt's 2.6 I decided to run some benchmarks with it to see if I could do better than I have been but I can't get a single version of FFMS2 to work with it. I've tried the vanilla ffmpegsource-2.14 and ffmpegsource-2.1.4-mt both with SetMTMode and without and all I get is a VirtualDub that just hangs or throws a KERNALBASE error or an Avisynth read error (unknown exception).

I've been using SEt's 2.6 version released 09/19/2009 with the following script (the most basic script I've attempted):

LoadPlugin("X")
FFVideoSource("Y")
QTGMC(Preset="Slow")
SelectEven()

Where X is the full path to FFMS2.dll and Y is the full path to the MKV I'm working with. What I'm doing is dragging and dropping this avs script onto the 32-bit VirtualDub, selecting "Direct stream copy" in VirtualDub and then saving the file as an AVI. The process seems to work for a few seconds as VirtualDub's progress window pops up but I inevitably get "Error reading source frame 0: Avisynth read error: Avisynth: unknown exception." I've tried using SetMemoryMax() to no avail as well as various SetMTMode values without any success. I've also double checked to make sure my paths are correct and that I'm using the 2.6 version of Avisynth from the version.avs script.

I'm using a Core i7 860 with 8GB of RAM running Windows 7 Professional 64-bit. This problem is starting to drive me nuts as I have a pretty powerful system but can't use more than a fraction of it for my encoding and despite the searching I've done in the forums and on the internet I can't figure out what could be causing this.

I would very much appreciate it if someone could shed some light on this for me.

Thanks in advance!

[Didée]

I've been using SEt's 2.6 version released 09/19/2009 with the following script (the most basic script I've attempted):

LoadPlugin("X")
FFVideoSource("Y")
QTGMC(Preset="Slow")
SelectEven()
The most basic script you have tried is - a script with QTGMC. Now, look at that.


drive me nuts as I have a pretty powerful system but can't use more than a fraction of it for my encoding
I've pretty much the same HW as you, and have no problems to use 90~99% CPU for filtering+encoding. With 2.5.8.5.MT, that is.