I made a mt 2.5.8 avisynth.dll and mt.dll

[Jeremy Duncan]

I updated the avisynth.dll and the src. Links are in the first post.
I also updated the instructions to build the dll at the bottom of the first post.

What changed? I fixed the bug Gavino reported using the code he provided to fix the bug, and I made the name change atak snapjera requested.

[Gavino]

I fixed the bug Gavino reported using the code he provided to fix the bug
(I take it this refers to the audio cache bug.)

Thinking about the MT implications of this led me to have a look at the source code of CacheMT::GetAudio.
I see that it is the same as Avisynth 2.57 and does not include the changes made (to Cache) in Avisynth 2.58. Ironically, this also means that it does not suffer from the bug (when SetMTMode is used), but really CacheMT::GetAudio should be the same as Cache::GetAudio.

[Jeremy Duncan]

Interesting. I wonder if it would be faster if it has the 2.5.8 code in the cachemt?

I updated the dll and src links in the first post.

[Gavino]

I wonder if it would be faster if it has the 2.5.8 code in the cachemt?
Most scripts would show no difference, since the audio cache only comes into play if the same section of audio is accessed more than once (eg when mixing channels).

The main reason for making them the same is for consistency and ease of future maintenance. In the longer term (Avisynth 2.6?), it might be better to derive Cache and CacheMT from a common parent class and put the audio code in that parent.

[thewebchat]

This is probably a dumb question, but how does AviSynth MT behave when no instances of MT or SetMTMode have been called? Is it functionally equivalent to normal AviSynth?

[cweb]

I'm getting odd behaviour when I use SetMTMode before directshowsource

SetMtMode(2,2)
video=directshowsource("00003.ts").KillAudio()


My source is converted from my camera's hdvideo (50p).

If I put the SetMtMode after the directshowsource all is fine.
Otherwise I get some strange jerkiness or repeats of previous parts of the video throughout..
I will try to prepare a sample for you as soon as I can.

Mediainfo reports the following on my source ts file (not sure why it's saying NTSC, I think it should say PAL!).

Maximum Overall bit rate : 35.5 Mbps

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.0
Format settings, CABAC : No
Format settings, ReFrames : 1 frame
Duration : 6mn 25s
Bit rate : 13.6 Mbps
Width : 1 280 pixels
Height : 720 pixels
Display aspect ratio : 16/9
Frame rate : 50.000 fps
Standard : NTSC
Resolution : 24 bits
Colorimetry : 4:2:0
Scan type : Progressive
Bits/(Pixel*Frame) : 0.296
Stream size : 626 MiB (92%)

[saint-francis]

I'm getting odd behaviour when I use SetMTMode before directshowsource

SetMtMode(2,2)
video=directshowsource("00003.ts").KillAudio()


My source is converted from my camera's hdvideo (50p).

If I put the SetMtMode after the directshowsource all is fine.
Otherwise I get some strange jerkiness or repeats of previous parts of the video throughout..
I will try to prepare a sample for you as soon as I can.
)

SetMTMode (2) doesn't work with DirectShowSource. That is the classic example of SetMTMode (2) and DSS. You're lucky it completed encoding. The reason you don't have the issues when you put it after DSS is because it isn't getting called. If you use SetMTMode you have to begint he script with it. You can change between modes during the script but you need to begin with it.
What you need to do is begin the script with SetMTMode (3) and after DSS is called use mode 2. I have found that mode 3 causes havoc with some other filters I usually use.

[BigDid]

... but how does AviSynth MT behave when no instances of MT or SetMTMode have been called? Is it functionally equivalent to normal AviSynth?
Hi,

I had already verified that (with a 2.58 avisynth-mt.dll) there was nearly no difference scripting no MT or scripting with setmtmode(2).
Here is a more complete test that confirm there is no or very little speed difference between the different scripting exception being the MT() mode.
I use last SET avisynth MT 2.58 dll:
http://forum.doom9.org/showthread.php?p=1301410#post1301410
A slow dual-core with avisynth 2.58 mono or MT dll
Script is mpeg2source+ crop+lanczosresize+
#( the Setmtmode or MT scripting is inserted here when needed)
fft3dfilter(plane=4, sigma=1, bt=4).lsfmod()

1/ 2.58 Mono-dll => 7,25 fps
2/ 2.58 MT-dll, no MT scripting => 7,19fps
3/ 2.58 MT-dll, setmtmode(2) => 7,15fps
4/ 2.58 MT-dll, MT("...") => 10,19 fps

Conclusion being: for a marginal gain, if you own a single core, stick to avisynth 2.58 mono dll. If you own a multiple core, use the 2.58 MT dll and try to get the MT() mode to work ;)

Nota: anybody with a quad-core wanting to redo these simple tests is welcome cause mileage may vary.

Did

Edit: the conclusion applied to the question in the above post gives the following answer: don't worry with the setmtmode(2) it should not be quicker than normal scripting IF you have the avisynth 2.58 MT dll.

[aegisofrime]

MT() Breaks some filters, like TGMC for example. Tried and tested on my side. For TGMC SetMTMode(2,4) on my Phenom II X4 955 along with a Distributor() at the end of the script gives me a very nice boost.

[Gavino]

... along with a Distributor() at the end of the script gives me a very nice boost.
Doesn't Distributor() get added automatically? (see here (http://avisynth.org/mediawiki/MT_modes_explained))
What effect does adding another one have?

[Bluedeep]

Thank's for this version ;)

Do you accept donations?

[Jeremy Duncan]

Thank's for this version ;)
Your welcome.


Do you accept donations?

No.

[BigDid]

... For TGMC SetMTMode(2,4) on my Phenom II X4 955 along with a Distributor() at the end of the script gives me a very nice boost.
Hi,

Glad for you but that doesn't give any numbers for comparison!?

Did

[thewebchat]

To do parallel temporal processing, would it be reasonable to do something like this?
StackVertical(SelectEven,SelectOdd)
MT("filter",2,0)
Interleave(Crop(last.width,0,0,0-last.height/2),Crop(last.width,last.height/2,0,0))

Or maybe something like this?
AssumeFieldBased
Weave
MTi("filter")
SeparateFields
AssumeFrameBased

[Jeremy Duncan]

thewebchat,

I couldn't tell you since I don't use that code.

Everybody,

I have updated the src and avisynth.dll and linked to them in the first post.

[aegisofrime]

Hi,

Glad for you but that doesn't give any numbers for comparison!?

Did

My bad, wasn't keeping track of this thread.

As TGMC does take a while for the FPS numbers to more or less "settle down", I waited until 1 minute of processing to do a measurement.

Without Distributor()

loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrainSSE3.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RepairSSE3.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\nnedi.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools-25.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mvtools2.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\MT.dll")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\TGMCAlpha_3.avsi")

SetMTMode(2,4)
DGDecode_mpeg2source("F:\Raws\Test.d2v",info=3,idct=5)
TempGaussMC_Alpha3(2, 1, 1, EdiMode="NNEDI2")


FPS: 5.89

With Distributor at the end,

loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RemoveGrainSSE3.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\RepairSSE3.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\nnedi.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools-25.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\mvtools2.dll")
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins\MT.dll")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\TGMCAlpha_3.avsi")

SetMTMode(2,4)
DGDecode_mpeg2source("F:\Raws\Test.d2v",info=3,idct=5)
TempGaussMC_Alpha3(2, 1, 1, EdiMode="NNEDI2")

Distributor()


FPS: 14.1

Edit: I would like to clarify that the above results are with the pre-rendering job on MeGUI. Interestingly, if encoded with x264 @ CRF21 (that is, pre-rendering mode turned off), the speed difference shrinks to around 7 FPS for non-distributor and 7.5 FPS for Distributor O.o

I believe this is due to x264 actually being a bottleneck... Before anyone asks, the reason why I use pre-rendering is that I have noticed some filters require it. FFT3DGPU gives me crap when I do a TGMC 1 pass encode...

[Gavino]

October 6, 2009
I updated some files in the src, core folder. I got the new ones from SEt's 2.6 src. I read the bug fix he made in that 2.6 thread of his today and so decided to make a new dll with the fix and thought it would be cool to include other mt files from his src so I did.
I forget which ones though.
Do you realise how lame that sounds?
Don't you understand the importance of software configuration management (http://en.wikipedia.org/wiki/Software_configuration_management) to a project like this?

[Guest]

Don't you understand the importance of software configuration management (http://en.wikipedia.org/wiki/Software_configuration_management) to a project like this? If you followed this thread from the beginning you would know the answer to that question.

[Jeremy Duncan]

Do you realise how lame that sounds?
Don't you understand the importance of software configuration management (http://en.wikipedia.org/wiki/Software_configuration_management) to a project like this?

I know which files but I didn't want to be specific and say certain ones.

Here is the list I made in this link (http://forum.doom9.org/showthread.php?p=1306028#post1306028):
core folder

avisynth_c.cpp
cache.cpp
cachemt.cpp
main.cpp
mt.cpp
mt.h
softwire_helpers.cpp

I know it's not softwirehelpers. I think I tried main but that didn't work. One of the mt, I think mt.cpp didn't work. And I think avisynth_c didn't work either.
That leaves cache, cachemt and one of the mt files I ported from the 2.6 src to my 2.5.8 build.

The cachemt didn't work so I had to splice the new code into my old code and so it's a mix.

[Arshad07]

I'm getting an error while using MT although i've got c++ installed;

Error :
this application has requested the Runtime to terminate it in an unusual way

Please contact the applications support team for more information

http://i38.tinypic.com/iei2ow.jpg

[Jeremy Duncan]

I'm getting an error while using MT although i've got c++ installed;

Error :
this application has requested the Runtime to terminate it in an unusual way

Please contact the applications support team for more information

When you installed the new dll's, did you reboot twice?

[Groucho2004]

I know it's not softwirehelpers. I think I tried main but that didn't work. One of the mt, I think mt.cpp didn't work. And I think avisynth_c didn't work either.
That leaves cache, cachemt and one of the mt files I ported from the 2.6 src to my 2.5.8 build.

The cachemt didn't work so I had to splice the new code into my old code and so it's a mix.

Priceless.

You should really put a warning in front of your download links saying that you have no clue about software development and source control and that you mix code from various versions randomly.

You seem to be under the impression that downloading and installing a Visual Studio trial version instantly gives you the knowledge required to understand the code you're attempting to compile.

[Jeremy Duncan]

Priceless.

You should really put a warning in front of your download links saying that you have no clue about software development and source control and that you mix code from various versions randomly.

You seem to be under the impression that downloading and installing a Visual Studio trial version instantly gives you the knowledge required to understand the code you're attempting to compile.

I won't do as you ask. If people want to try it then they can.

I posted this build not so it can be all things to all people but that it can work for me in ffdshow, using the avisynth code I put in ffdshow avisynth tab.
I was very fortunate to have SEt's help in the process.

The gnu asks that my personal build be posted for a few years so I have to post it and here in this forum is as good as any place.

If this build doesn't work for you then go try somebody elses build, or use tsp's official builds. :thanks:

[Groucho2004]

I won't do as you ask. If people want to try it then they can.

Fair enough, I wasn't expecting you to do that anyway.

[aegisofrime]

For what it's worth, Jeremy's builds work fine for me. For some strange reason, Set's builds makes my encoding processes drop to 0 FPS after a while :/

[SEt]

For some strange reason, Set's builds makes my encoding processes drop to 0 FPS after a while :/
And of course you used my latest 2.6 build and not some random old one?

[aegisofrime]

And of course you used my latest 2.6 build and not some random old one?

Yup. I'm using the 2009.09.19 version. :/

[Arshad07]

Hey jeremy, how should i multithread this script

AVCSource("C:\BD\VZ.dga")
crop( 0, 130, 0, -130)
Spline64Resize(1920,816) # Spline64 (Sharp)
YLevels(1, 1, 250, 0, 255)
Tweak(hue=0, sat=1, bright=0, cont=1.02, coring=true, sse=false)
HDRAGC(min_sat=0.1,max_sat=5.9,coef_sat=2.5)
DeGrainMedian(limitY=4, limitUV=6, mode=3, interlaced=false, norow=false)
LSFmod(Smode=3,Lmode=1,ss_x=1.0,ss_y=1.0,strength=80)

[Atak_Snajpera]

did you try SetMTMode(2,0) above AVCSource?

[Arshad07]

^ Didnt work unfortunately :(

[Atak_Snajpera]

http://avisynth.org/mediawiki/MT_support_page#SetMTMode.28.29_Mode

[actarusfleed]

Hi Jeremy,
my question is very simple: Can we use LSF with your avisynth.dll ?

SetMTMode(2,8)
multinum=2
multiden=1
mode=2
spar=0
pel=1
blkh=16
blkv=16
ffdShow_source()
super=MSuper(pel=pel,hpad=blkh, vpad=blkv, levels=4)
backward_vec1=MAnalyse(super, isb=true, blksize=blkh, blksizev=blkv, searchparam=spar, plevel=2, levels=4)
forward_vec1=MAnalyse(super, isb=false, blksize=blkh, blksizev=blkv, searchparam=spar, plevel=2, levels=4)
MBlockFps(super, backward_vec1, forward_vec1, num=FramerateNumerator(last)*multinum, den=FramerateDenominator(last)*multiden, mode=mode)
distributor()

When I install the DLLs that this script needs I install your avisynth.dll library in the folder "system32" in windows.

http://www.mediafire.com/?o1dccmm1l1t

The same thing happens when I've to install LSF.

So my question is: Is there a avisynth.dll that goes right for LSF and Motionflow script together?

Thank you,
actarus.

[Jeremy Duncan]

dlls. Updated january 10, 2010. These were compiled using the windows 7 sdk so it may be more compatible with windows 7. (http://www.mediafire.com/?zy2xm2ejvzg)

[hydra3333]

I'm not sure what to do. I want to try out a 2.5.8 MT ... however
i) I read on various threads that Jeremy Duncan has issues understanding what he's doing with code (noted in a few places) and therefore I can't really trust his versions
ii) SET's version here http://forum.doom9.org/showthread.php?t=148117 has a range of issues as mentioned in the 1st post and subsequent posts in that thread

Anyone care to comment on experiences with reliabilty of each version as at Feb 2010 ?

[cweb]

I'm not sure what to do. I want to try out a 2.5.8 MT ... however
i) I read on various threads that Jeremy Duncan has issues understanding what he's doing with code (noted in a few places) and therefore I can't really trust his versions
ii) SET's version here http://forum.doom9.org/showthread.php?t=148117 has a range of issues as mentioned in the 1st post and subsequent posts in that thread

Anyone care to comment on experiences with reliabilty of each version as at Feb 2010 ?
SET's 2.6 version works best for me personally. No problems with reliability.

[crasus]

I would love to hear if somebody tried any of these two version with the latest MCTemporalDenoise and if there are any advices regarding a proper usage.

As far as I understand untill know, fft3dGPU should be disabled in the script and fft3d used instead.

The script I'd like to use :

TDeint(order=1, mtnmode=3)
Crop(16,58,-24,-86)
BicubicResize(640,384,0,0.5)
MCTemporalDenoise(settings="very high", twopass=false, useTTmpSm=true, sharp=false, adapt=false, deblock=true)
greyscale()

All sugestions welcomed. Cheers!

[n0an]

Awesome work! :)

[Jeremy Duncan]

Awesome work! :)

:thanks:

[burfadel]

Is there any relevance between the work done with Avisynth-MT x64 and the 32 bit avisynth?
http://forum.doom9.org/showthread.php?p=1374605#post1374605

[osgZach]

Relvance? In what way? They are independent projects. In particular JoshyD's is focused on the move to an x64 encoding pipe, and once that is stable probably some improvements if possible.

Jeremy's seems focused on providing multi-threaded support on the x32 side of things.. Although most feel he has no idea what he is doing (I have no opinion and have not tried his builds). But this particular project will likely be rendered obsolete once Avisynth 2.6 comes out.

[Gannjunior]

Hi,

i'm trying to use MT in this script:


SetWorkingDir("d:\avs_plugins\AviSynthPlugins\")
LoadPlugin("MT.dll")
Loadplugin("d:\avs_plugins\warpsharp.dll")

LoadPlugin("SangNom.dll")
LoadPlugin("average.dll")
LoadPlugin("Degrainmedian.dll")
Loadplugin("removegrain.dll")
LoadPlugin("Repair.dll")
LoadPlugin("gradfun2db.dll")

LoadPlugin("DGAVCDecode.dll")
LoadPlugin("DGDecode.dll")

loadplugin("fft3dfilter.dll")
loadplugin("hqdn3d.dll")
LoadPlugin("convolution3d.dll")
LoadPlugin("undot.dll")
LoadPlugin("mvtools2.dll")
LoadPlugin("mt_masktools-26.dll")
LoadPlugin("TTempSmooth.dll")

import("d:\avs_plugins\denoisers\MCTemporalDenoise.v1.4.08.avsi")
import("d:\avs_plugins\denoisers\GradFun2DBmod.v1.5.avsi")
import("LSFmod.v1.9.avsi")
LoadPlugin("GradFun2DB")

LoadPlugin("D:\Program Files (x86)\StaxRip\Applications\AviSynth plugins\DGAVCDecode.dll")

AVCSource("L:\test_5d2\MVI_0135 temp files\MVI_0135.dga")
MCTemporalDenoise(settings="high")

I tried to use SetMTmode() in different way but with no success.

How could I set MT in the correct way to use all my i-920 cores in the script above?

Thanks in advance

ciao!

[osgZach]

All I see is a LoadPlugin() for MT.dll

Where did you attempt to SetMTMode and how did you call it?

[Gannjunior]

before MCTtemporal...

could you help me modify my script adding it in the correct place of the script?

thanks!

[osgZach]

Honestly I'm not that experienced with StMTMode.. Most people know a lot more than me..

Something real basic you could try is this..

SetMTMode(5,0) # put this BEFORE AVCSource - 5 = mode 5 (recommended for source filters). 0 = auto detect threads I believe. You define threads once and only ever change the MODE from then on.

Then after your AVCSource but Before MCTTemporal - SetMTMode(2)

That is one very basic way to try it. I have seen some other examples with a bit more code than that though, so hopefully someone better than me will come along and help.. I am used to using the MT filter on a single line instead of threading the whole script.

[prokhozhijj]

OS: Windows 7 64-bit.
CPU: Intel Core 2 Quad 9505.
RAM: 4Gb
Coder: x264, v.1592 32-bit.
Avisynth: 2.5.8 with dlls from this theme.
Source video resolution: 720x576
MVTools: v.2.5.10 (also tried 2.5.9)

I tried to use script listed below with different numbers of threads and different volume of memory.
With number of threads less than 5 - CPU load is no more than 50%.
With SetMemoryMax=2048Mb - CPU load is no more than 50%.
With number of threads more than 4 and SetMemoryMax=1200Mb - scripts hangs with module name error mvtools2.dll.

Could anybody help me with this problem?

Script looks like this:

SetMemoryMax(1200)
SetMTmode(5,8)

DGDecode_mpeg2source("G:\video\_new\ub_s\s 3\d3\encode\d3_1.d2v", idct=3, info=3)
AssumeTFF()

SetMTmode(2)

crop(10,68,-18,-64)

#######################################
# TempGaussMC
#######################################
Import("D:\arh\media\edit\video\_Filters\TempGaussMC\TempGaussMC_beta2u.avsi")
clip=TempGaussMC_beta2u(EdiMode="nnedi2",qual=3,tr2=2,lossless=-1)


clip=clip.SRestore(frate=25)


return clip

[Didée]

With number of threads less than 5 - CPU load is no more than 50%.

That's normal. You have an HT CPU - 4 physical cores, 8 "logical" cores. When 4 computational intensive threads are running, they will be assigned to the 4 physical cores. Taskmanager will show 50% load because only 4 of the 8 present cores are used. However, the real load in fact is much higher: when the 4 physical cores are already "maxed out", then the additional 4 virtual cores would not give you another 50% of performance. In practice, they could give you some 5%-20% of additional performance (depending on many things). A realistic number is +10%.


With number of threads more than 4 and SetMemoryMax=1200Mb - scripts hangs with module name error mvtools2.dll.

You are in 32bit application land, and your whole processing chain is hitting the "not more than 2GB memory per application" limit.
TGMC needs quite some memory even single-threaded. Now, with some uber-simplification just to make understanding more easy, using SetMTMode(mode,#threads) is quite similar to running TGMC #threads times in parallel. Say that singlethreaded TGMC requires 400MB RAM. Now multiplicate that by 8 threads, and you have a problem. (Yes, I know that SetMTMode in reality uses less ressources.)

Add to that that you're encoding directly to x264, where x264 by itself can easily require anything from 500MB to 1 GB or 1.5GB of RAM, depending on settings. (RC-lookahead plays a major role.)


Alltogether, that's lots of memory that is required .... and since for 32bit applications only 2GB are allowed, it's just too much.

The possibilities are

- stick with SetMTMode(2,4)

- or: make a lossles encoding first, then feed that into x264 (get x264 memory requirement out of the game, so Avisynth has more memory available)

- or: switch to 64bit Avisynth & x264_x64.


In any case, reduce that overly huge SetMemoryMax. It only controls the frame cache of Avisynth, and you really don't need that much. The more frame cache you specify, the less memory is available for the actual script.


My "best guess" is: if you have it running with SetMTMode(2,4), then stick with that. That's about as much as you'll get from TGMC on a CPU with 4 physical cores. You might even consider to specify *less* threads for x264. When the input contains the TGMC turtle, powered by 4 threads already, then it should be safe to reduce x264 to e.g. threads=4. It's default is 12 threads (on a 4-Core with HT), and that won't do much good in this case. 4 threads should be enough to slip into the little free space of Avisynth threads. Anything more will only add unnecessary overhead, for no actual benefit.

[prokhozhijj]

Didée, thank you very much for explanation. I knew nothing about memory allocation within avisynth. So your explanation very useful for me. I'll try to reduce memory size for inner buffer cache.

But let me comment a little some sentences.

You have an HT CPU - 4 physical cores, 8 "logical" cores. When 4 computational intensive threads are running, they will be assigned to the 4 physical cores. Taskmanager will show 50% load because only 4 of the 8 present cores are used. However, the real load in fact is much higher: when the 4 physical cores are already "maxed out", then the additional 4 virtual cores would not give you another 50% of performance. In practice, they could give you some 5%-20% of additional performance (depending on many things). A realistic number is +10%.

When I am running my script with 5 or more threads processing speed equals to around 6 fps. But with 4 or less threads speed is much more modest - it is about 3 fps. So difference is much more than 10%.

[supernater]

In SEt's 2.5.8 MT package he has modified versions of some plugins such as directshowsource and tcpdeliver. Does your version require modified versions of these plugins as well?

[Jeremy Duncan]

I forget, but if I remember correctly the direct show source dll is different from the avisynth dll. Same for tcp deliver dll. I only made the avisynth dll. It's a regular avisynth dll, no special requirements besides using mt dll.

[Mrsash]

OK I reinstalled everything. I remember I had an option other than disabled and libavcodec in ffdshow, but it seems to have disappeared. Am I missing anything? I have just come back after a break to encoding and need some assistance. Basically ffmpeg_mt is missing in the list.
thanks


Forgot to add. Installed latest ffdshow(ffdshow_rev3914_20110629_clsid.exe)