Avisynth 2.5.8 MT [Archive]

SEt

30th June 2009, 18:16

Note: Not maintained anymore, use Avisynth 2.6 (http://forum.doom9.org/showthread.php?t=148782) builds instead.

Version 2009.07.12
Compiled dlls: http://www.mediafire.com/file/nnbngfccnj2/avisynth258MT.7z
Sources: http://pavelsx.googlepages.com/avisynth258MT_src.7z

Build is statically linked with runtime library, so you don't need to install any VisualStudio runtime for it to work.

Changes:

threading fixes
code cleanup (for loop scope conformance,...)
added hack to allow C plugin to identify itself as cache
fixed minor memory leak

Known problems:
Multithreaded caching is quite bad what results in eating all memory and crashing after that. Workarounds:

split script into several parts and process them one by one
reduce number of used threads (in SetMTMode or MT)
try different SetMemoryMax values, the more isn't always the better, i often get good results at 512 or 1024

There should be no crashes due to threading, but if you still experience not 'not enough memory' crashes with MT or correct SetMTMode - rebuild offending plugin with avisynth.h from these sources.

interaction with ffdshow:

patched ffdshow.ax revision 2975: http://www.mediafire.com/file/jz2jyluzzny/ffdshow.7z
patched ffdshow.ax revision 3173: http://www.mediafire.com/file/ntnnnz0hyxy/ffdshow.7z
patch: http://pavelsx.googlepages.com/ffds_avs_patch.diff

Patch fixes incorrect timestamps, redice crashes on stop, reduce memory usage a bit.

For using SetMTMode in ffdshow you must uncheck 'Add ffdshow video source', start your script with something like
SetMemoryMax(512)
SetMTMode(3,4)
ffdshow_source()
SetMTMode(2)
and end with
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

'Buffer ahead' value should be at least 'source_frames_ahead_for_result_frame' + 'number_of_threads', so for example for MDegrain2 with 8 threads it should be 10+.

juGGaKNot

30th June 2009, 18:52

Compiled dlls: http://www.mediafire.com/file/ybbztyywdzn/avisynth258MT.7z

A small tutorial one how to use ? simple script

avisource
converttoyv12(pc.601)

leeperry

30th June 2009, 19:14

hi SEt, does it work w/ the sample I sent you? coz this one gives a DrWatson error on XP SP3 related to the "avisynth" component of fffdshow.

there's also some crazy memory leaks on 4 threads w/ AddGrainC 1.4 in Avisynth MT 2.57, if you could fix them that'd be really awesome! but 2.57 works fine w/ AddGrainC 1.1, so I dunno who's at fault tbh.

can you please submit your ffdshow changes to the SVN?

Adub

30th June 2009, 20:04

AddGrain has had memory leak issues in the past, most of which were resolved but I think that there were one or two left over. Didee and neuron2 know more about them.

Edit: Also, SEt, is this a different (more up-to-date) version than what you posted in the earlier thread?

SEt

30th June 2009, 20:16

juGGaKNot, i don't understand you.

leeperry, it does work with your sample and filters with 4 threads and SetMemoryMax(512) - i get memory usage ~1.8Gb. Since you haven't posted your script - you have to figure out yourself correct SetMemoryMax and max number of threads to fit in 2Gb.

ffdshow changes are not required and far from beautiful, so i'd leave them as is for anyone who want to apply them. Only recompilation of ffdshow.ax is required - you can use other dlls from other builds of the same revision.

Adub, it's updated version - changelog is relative to my previous one. What looks like 'huge memory leak' is actually the way multithreaded cache works in 2.5.8 - it can stop somewhere below 2Gb and everything will be fine or it won't and something will crash due to end of address space.

Adub

30th June 2009, 20:18

Great, thanks for the info. Can I ask what you did with Directshowsource.dll and TCPDeliver.dll?

SEt

30th June 2009, 20:25

Just rebuilt with new header so they are more thread-safe.

Leak

30th June 2009, 20:48

[...]
interaction with ffdshow:

patched ffdshow.ax revision 2975: http://www.mediafire.com/file/ie1yetdxj0m/ffdshow.7z
patch: http://pavelsx.googlepages.com/ffds_avs_patch.diff

Patch fixes incorrect timestamps, redice crashes on stop, reduce memory usage a bit.
It's not that I don't believe you, but what incorrect timestamps are you talking about here? Your patch doesn't even go near timestamp calculation, unless I'm missing something... :)

np: Yage - 1st Calling (Future Sound Of London - By Any Other Name)

SEt

30th June 2009, 21:04

leeperry

30th June 2009, 22:22

leeperry, it does work with your sample and filters with 4 threads and SetMemoryMax(512) - i get memory usage ~1.8Gb. Since you haven't posted your script - you have to figure out yourself correct SetMemoryMax and max number of threads to fit in 2Gb.
well I set it to 64mb ;)
but you can try to use AddGrainC 1.4 on 8 threads and it will leak memory like crazy using 2.57 and won't obey SetMemoryMax() :(

AddGrainC 1.1 works fine, no leak.

I'll try your new build :thanks:

leeperry

30th June 2009, 23:16

juGGaKNot

1st July 2009, 09:24

juGGaKNot, i don't understand you.

Its avisynth 2.5.8 so i can use it to feed x264 frames no ?

Or is it for other use ?

Leak

1st July 2009, 11:09

Incorrect timestamps are fixed by disabling passFirstThrough associated code that breaks them. Try yourself: add '#DEBUG' as first line in ffdshow avisynth script, enable debug messages and see debug output by DbgView. In "[?] TimgFilterAvisynth: Looked up frame X, using frame Y" X should be the same as Y for correct timestamps.
Hmmm... it's been a while since I last worked on the timestamp calculation, but I'll take a look at it this weekend - but I'd rather fix it completely instead of taking out passFirstThrough... :)

Adub

1st July 2009, 16:35

Its avisynth 2.5.8 so i can use it to feed x264 frames no ?

Or is it for other use ?

Uh, yes. Although you have always been able to feed x264 frames. With any Avisynth version. I don't really know why you are asking a question with such an obvious answer.

juGGaKNot

1st July 2009, 17:47

Uh, yes. Although you have always been able to feed x264 frames. With any Avisynth version. I don't really know why you are asking a question with such an obvious answer.

Well do i have to change anything in the script to use mt ? or just copy the dll ?

A small tutorial one how to use ? simple script

avisource
converttoyv12(pc.601)

onesloth

1st July 2009, 18:55

Well do i have to change anything in the script to use mt ?
http://avisynth.org/mediawiki/MT

Jeremy Duncan

1st July 2009, 23:32

SEt,

What did you do to the softwire, soundtouch, pfc code in your src?

SEt

2nd July 2009, 17:41

Just some code cleanup as said tn the first post.

leeperry

7th July 2009, 17:42

Jeremy Duncan

7th July 2009, 21:23

I'm using the first build you posted, and sometimes KMP crashes while opening and windows tells me that it doesn't have enough virtual memory...this never happened w/ 2.57 :confused:

how can I help to pinpoint the hypothetical memory leak exactly?

Try my build. If it works it's just SEt's project settings not the code.

Then SEt can fine tune his debugging.

I know the poject settings can affect the stability of the dll's.

lych_necross

8th July 2009, 07:12

:rolleyes:...oh boy...
@leeperry
reread the first post. It explains the memory leaks. Another build won't change anything.

leeperry

8th July 2009, 10:27

ouh, rebuilding plugins...I'm using LSF/GrainFactory3 and rgb3dlut()

GrainF3 is based on AddGrainC, which is pretty buggy...especially in the memory leak department. I think I'll stick to 2.57 then..

Fizick

8th July 2009, 17:17

SEt,
The question from plugins writer:
is plugins compiled with your new modified 2.5.8MT header avisynth.h 100% compatible with older versions of Avisynth (2.5.6-2.5.7), including non-MT ?

(my previous versions of MVTools and other plugins used old avisynth.h from Avisynth v2.5.5 for wide compatibility)

SEt

8th July 2009, 19:28

It's as compatible as the header from official Avisynth 2.5.8.

The only additional problem you can get with my header is with compiling InterlockedIncrement -> _InterlockedIncrement mapping (depending on included headers and their order this can be redefinition). This is pure minor performance optimization for VS and that #if section can be safely removed without affecting the correctness.

If you want to use older header - just apply the difference of my header with official 2.5.8 without the end of IScriptEnvironment and "friend class CacheMT;" as these are MT-specific additions (won't harm even if present in header as long as you don't call them).

Fizick

9th July 2009, 18:34

OK. Thanks. I will do it in next release :)

Well, i changed avisynth.h not out of nowhere:

void Release() { if (refcount==1) InterlockedDecrement(&vfb->refcount); InterlockedDecrement((long *)&refcount); }is definitely not thread safe and

So, the current code is fully thread-safe? :

void Release() { VideoFrameBuffer* vfb_local = vfb; if (!InterlockedDecrement(&refcount)) InterlockedDecrement(&vfb_local->refcount); }

Next question from dilettante: is any not-thread-safe code in new avisynth.h ?
For example, is IsWritable and GetWritePtr code thread-safe?

SEt

9th July 2009, 23:47

void Release() { VideoFrameBuffer* vfb_local = vfb; if (!InterlockedDecrement(&refcount)) InterlockedDecrement(&vfb_local->refcount); }Is thread-safe for how it's used in Avisynth and plugins. Note that similar following code is not:
void Release() { if (!InterlockedDecrement(&refcount)) InterlockedDecrement(&vfb->refcount); }

Next question from dilettante: is any not-thread-safe code in new avisynth.h ?
For example, is IsWritable and GetWritePtr code thread-safe?
Well, it's not like i'm threading guru myself. Probably there is some not thread-safe code. And yes, IsWritable and GetWritePtr are very suspicious - i've used to crash a lot because of them earlier (NewVideoFrame, GetWritePtr with refcount 2 at that moment, NULL pointer dereference) but now it works far more stable (test run successfully run 10 hours while earlier it would crash after ~half an hour).

In general i don't like at all how this Read-Write state of frames work in current Avisynth - such things should be completely inside Avisynth itself, not in every plugin. Only than you would be able to say that they are thread-safe.

I have several suggestions for radical changes in Avisynth plugin API and now is good time for them - i think version 2.6 can drop backward compatibility for easier to use, safer for threading and internally flexible (like caching implementation) without every plugin recompilation API.

Fizick

10th July 2009, 17:45

1. In my personal opinion (from user and plugin developer's point of view), we may broke plugins 2.5 compatibility in nearest Avisynth version only as a follows:
by providing new command LoadNewPlugin for new API plugins, and preserve LoadPlugin for old 2.5 API plugins. (So, do not break nothing, but do not provide much support of new features to old API)

2. probably we can return to discussion about roadmap of Avisynth, while IanB live in real life (may be hunting for crocodiles in Australian rivers :)
If you have not only suggestions, but also some time to implement your suggestions (with some discussion), you are welcome to contribute to some new version branch, for example 2.7. (Ask Wilbert for sourceforge access as a team member.)

3. May be I am complitely wrong both in 1 and in 2 points above (it was hot day). :)

Fizick

11th July 2009, 06:49

SEt, in meantime, can you provide (for me) some scenarios, showing thread-safety-notsafety of code fragments above?

SEt

11th July 2009, 13:45

void Release() { if (refcount==1) InterlockedDecrement(&vfb->refcount); InterlockedDecrement((long *)&refcount); }Consider the following situation:
Two threads are holding this object so refcount=2.
They at the same time call Release().
Condition (refcount==1) is false for both, after that both decrement refcount and it becomes 0, but no one decrements vfb->refcount.
void Release() { if (!InterlockedDecrement(&refcount)) InterlockedDecrement(&vfb->refcount); }Avisynth keeps a list of VideoFrame objects and reuses them instead of allocating/deallocating each time. After refcount is decremented to 0 object is considered free and can be reused immediately, so before we read vfb->refcount it can be overwritten by vfb of newly created frame (and i indeed seen this happen in practice). This results in decrementing refcount of wrong vfb.
void Release() { VideoFrameBuffer* vfb_local = vfb; if (!InterlockedDecrement(&refcount)) InterlockedDecrement(&vfb_local->refcount); }Is free from both problems.

buletti

11th July 2009, 16:50

Hmmm... it's been a while since I last worked on the timestamp calculation, but I'll take a look at it this weekend - but I'd rather fix it completely instead of taking out passFirstThrough... :)
Is there any news regarding the ffdshow patch? Do we need another solution or may Set's patch go into the ffdshow trunk? It would be good, if the MT-enabled Avisynth wasn't bound to ffdshow revision 2975 only.

Fizick

11th July 2009, 21:12

SEt, thanks. :)

Next question. One user reported a problem with your build of Avisynth2.5.8MT with the crazy script, including AnimeIVTC:
http://forum.ixbt.com/topic.cgi?id=29:9331-153#4577
Version 2.5.7MT and official 2.5.8 works without this problem.

SEt

11th July 2009, 22:33

AnimeIVTC is so monstrous that it's unusable i'd say. There are also several problems.

One of them is Cdeblend - it doesn't (always) work in official 2.5.8 too. Here is the patch:
--- Cdeblend_v1.1b.avsi 2007-05-18 16:40:16.000000000 +0400
+++ Cdeblend_v1.1b.avsi 2009-07-12 06:34:31.155704600 +0400
@@ -66,8 +66,6 @@
btemp = default(dclip,input)
global blendclip= (fnr==false ? btemp : clense(btemp)).bilinearresize(int(width(btemp)*xr)*8,int(height(btemp)*yr)*8)

-rate = framerate(input)
-
###### masking ######
diff = mt_makediff(blendclip, blendclip.trim(1,0))
global mask = mt_lutxy(diff, diff.trim(1,0), yexpr="x 128 - abs y 128 - abs < x 128 - abs x y + 256 - abs < & x 128 - abs x 128 - abs 0.5 ^ - 2 ^ y 128 - abs x y + 256 - abs < y 128 - abs y 128 - abs 0.5 ^ - 2 ^ x y + 256 - abs x y + 256 - abs 0.5 ^ - 2 ^ ? ? x 128 - y 128 - * 0 > -1 1 ? * 128 +", uexpr="x", vexpr="x")
@@ -129,6 +127,6 @@
mode > 1 ? scriptclip(output, "evaluate(LumaDifference(blendclip.trim(1,0),blendclip.trim(3,0)), AverageLuma(mask.trim(1,0)))") :
\ scriptclip(output, "evaluate(LumaDifference(blendclip,blendclip.trim(2,0)), AverageLuma(mask))")

-recl = last.changefps(rate*2).changefps(rate,linear=true)
-return(recl.addborders(0,0,8,0).crop(0,0,-8,-0))
+recl = changefps(FrameRateNumerator(), FrameRateDenominator(), linear=true)
+return recl
}

The last two lines probably try to force frame requests in linear order but script still breaks with SetMTMode.

The other one i remember is EEDI2 distributed with it is built with very buggy OpenMP runtime. Just rebuilding solves the problem of random crashing.

Fizick

12th July 2009, 01:14

here is the simplest broken script with 2.5.8MT:

blankclip
return changefps(23.976)

SEt

12th July 2009, 05:04

Problem fixed, see the first post for updated version.

Notes:
lol, of course it will break if you change things like
union { float f; unsigned i; } value;
mantissa = (value.i & 0x7FFFFF) + 0x800000; // add implicit bit on the left

into
union { double f; unsigned i; } value;
mantissa = (value.i & 0x7FFFFF) + 0x800000; // add implicit bit on the left

Seems like Jeremy was removing warnings this way >_>. I've just rolled back affected files.

Fizick

12th July 2009, 06:34

OK, thanks! It works now. But I do not see updated source code.

EDIT - now I see it (fps.cpp) (firstly i missed your point about rolling back, not modifying)

(Oh, those warnings... :) )

Jeremy Duncan

12th July 2009, 09:53

Interesting thread. No real points to make but i would like to thank SEt for his fine efforts in this stuff.

I have done some tests with the tsp and set and my versions here: link (http://forum.doom9.org/showthread.php?p=1304695#post1304695). I just wanted you to see for something interesting to eyeball. I know I like to read such things. Anyway, keep up the interesting good work Mr. SEt. :)

Gavino

12th July 2009, 10:08

Seems like Jeremy was removing warnings this way >_>. I've just rolled back affected files.
Have you reviewed all changes introduced by Jeremy? It would be reassuring for us all (him included) to know that there are no similar problems elsewhere in the code.

On a different note (prompted by seeing the Cdeblend code above): do ScriptClip and friends work reliably under MT Avisynth? Can different threads interfere with each other when setting 'current_frame'?

Leak

13th July 2009, 17:25

Is there any news regarding the ffdshow patch? Do we need another solution or may Set's patch go into the ffdshow trunk? It would be good, if the MT-enabled Avisynth wasn't bound to ffdshow revision 2975 only.
Sorry, I was being swamped with stuff at work so I couldn't really get myself to do anything code-related at home... :(

I do have 2 weeks of vacation come next week, so I'll be taking a look at it then. :)

np: Tortoise - Charteroak Foundation (Beacons Of Ancestorship)

Jeremy Duncan

16th July 2009, 21:04

Have you reviewed all changes introduced by Jeremy? It would be reassuring for us all (him included) to know that there are no similar problems elsewhere in the code.

I compared the files in the src. The two src I compared were the original japan src and SEt july 12 src.

I found these files were not the same in the src.

stdafx.h
avisynth.cpp
avisynth.h
cache.cpp
cachemt.cpp
main.cpp
mt.cpp
mt.h
softwire_helpers.cpp
expression.cpp
color.cpp
convolution.cpp
edit.cpp
focus.cpp
greyscale.cpp
histogram.cpp
layer.cpp
layer.h
levels.cpp
merge.cpp
planeswap.cpp
resample.cpp
resize.cpp
transform.cpp
source.cpp
____________
pfc folder
string.cpp
utf8.cpp

Softwire folder
assembler.cpp
instructionset.cpp
operand.cpp

japan src (http://www.mediafire.com/?wky0vnztzqy)
SEt july 12 src (http://www.mediafire.com/?qnnnmzjrynz)
ultraedit. sw I used to compare (http://www.ultraedit.com/products/ultraedit.html)

click the ultraedit link, go to product tour and highlight the "Save time by comparing your files to track editing changes.
show/hide". IT will show you how to use the compare feature in this sw.
You open two files, then click the blue "uc" globe icon. When the compare screen opens then you press the arrow icon to go where the different code is.

I have to compare the distribute folder now and then go to my src and then the official 2.5.8 src so I will be updating this post later.
I did not compare the directshowsource or tcpdeliver files as I did not build these so didn't fiddle with their files.

Edit,

I have now finished comparing the japan and SEt src and distrib folder and the changes are in the list above. These are the files with different code.
________________________________________________________________
Edit,

I have redone the test with my src. My src is really SEt july 12 src but with a few files changed. Which files you ask? It's at the bottom of the edit below.

I tried to use ultraedit to compare the official avisynth 2.5.8 to both the japan and SEt src and mine too but it was all garbage and I couldn't get it done.
So I redid the work above but used my src instead of the one used before, cause I wanted too. :)
You probably see more files in the list below. I was more careful this time cause I knew how to do it right after the first try in the list above.

Anyway, below is the files changed in the SEt july 12 avisynth src from the original Japan avisynth mt src from way back when:

Audio folder
audio.cpp: lines 1400, 1450
avs-soundtouch.cpp: line 86
supereq.cpp: line 165

Convert folder

core folder

avisynth_c.cpp: lines 123
cache.cpp: lines 39, 514, 523, 533, 553, 824
cachemt.cpp: lines 295
main.cpp
mt.cpp: lines 102, 200, 284
mt.h: line 82
softwire_helpers.cpp: line 68

core parser folder
expression.cpp: lines 87, 93, 691

filters folder
color.cpp: lines 129, 169, 176, 209
convolution.cpp: line 216
edit.cpp: lines 404, 474, 596, 859, 878, 887, 896, 905, 914, 923, 933, 946, 959
focus.cpp: lines 339, 481, 486, 725, 753, 1088, 1104, 1132, 1148
greyscale.cpp: line 95
histogram.cpp: lines 495, 507, 548, 582, 604, 667, 674, 694, 698, 709, 713, 775, 782, 792, 810, 818, 824, 829, 837, 843, 847, 864, 870, 876, 883, 890
layer.cpp: lines 514, 1005, 2300
layer.h: line 160
levels.cpp: lines 136, 143, 277, 299, 952, 1057
merge.cpp: lines 94
planeswap.cpp: line 348
resample.cpp: lines 331, 379, 410, 437, 470, 1705
resize.cpp: lines 260, 272
transform.cpp: lines 149, 159, 382, 385, 390, 395, 398, 403, 435, 441, 452, 456, 511

filters overlay, conditional folders

sources folder
source.cpp: lines 90, 96

sources, avi folder
AVIReadHandler.cpp: line 1287

Note: Changes in the src tested from the SEt july 12 avisynth src.

change 1.) The main src folder, not the src subfolders, is from my src.
These are mainly settings for when I use visual studio to compile the dll, vcproj and other such files.

Change 2.) I also put the avisynth.cpp and avisynth.h files from my src into the SEt july 12 src.

Change 3.) Also I used the distrib, include: softwire, soundtouch, pfc files from my src.

other than that, the rest of the files were from the SEt july 12 avisynth src.

I'm sorry I changed so many files. :)

SEt

18th July 2009, 16:45

Jeremy Duncan

19th July 2009, 03:23

I've reviewed only changes between the version where this casting was added and previous one. Other files changes seems harmless.
The real problem is: why AVSValue stores floating point values as float, function named AsFloat, but returns double? I think it creates confusion for plugin writers too.

From the list below some files I have written double instead of float.

I know the folder and file names but not the specific files.
Just compare the files from the japan src to the one in your july 12 with ultraedit.
The japan version of these files is the same as the official src.

Here is the list, one or two of them have the double error I told you about. The numbers are lines different from the official and japan src.:

filters folder
color.cpp: lines 129, 169, 176, 209
convolution.cpp: line 216
edit.cpp: lines 404, 474, 596, 859, 878, 887, 896, 905, 914, 923, 933, 946, 959
focus.cpp: lines 339, 481, 486, 725, 753, 1088, 1104, 1132, 1148
greyscale.cpp: line 95
histogram.cpp: lines 495, 507, 548, 582, 604, 667, 674, 694, 698, 709, 713, 775, 782, 792, 810, 818, 824, 829, 837, 843, 847, 864, 870, 876, 883, 890
layer.cpp: lines 514, 1005, 2300
layer.h: line 160
levels.cpp: lines 136, 143, 277, 299, 952, 1057
merge.cpp: lines 94
planeswap.cpp: line 348
resample.cpp: lines 331, 379, 410, 437, 470, 1705
resize.cpp: lines 260, 272
transform.cpp: lines 149, 159, 382, 385, 390, 395, 398, 403, 435, 441, 452, 456, 511

levi

3rd October 2009, 21:14

SetMTMode(2,0)
mpeg2source("D:\my.d2v")
Load_Stdcall_Plugin("C:\Program Files (x86)\AviSynth 2.5\plugins\yadif.dll")
Yadif(order=-1)
crop(248,136,1428,816)
LanczosResize(624,352)
Undot()
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

Crashes half way through 2nd xvid pass with vdub 1.8.8. Any ideas?

BigDid

3rd October 2009, 22:12

...
Crashes half way through 2nd xvid pass with vdub 1.8.8. Any ideas?
Hi,

What if:
1/ you begin the 1st line of the script with setmemorymax(xxx) ?
xxx being 256K for 1Gb, 512k or more for 2Gb.

2/ You do not use Setmtmode(); see
http://forum.doom9.org/showthread.php?p=1330112#post1330112
or no Setmtmode() but MT("...") when possible ?

Did

SEt

4th October 2009, 01:22

levi, posted 2.5.8 build has several obvious bugs that are fixed in 2.6, but 2.6 has its own bugs. I suggest waiting for stable 2.6. If you need something MT right now - try my 2.6 build.
And you should always put source filter in MTMode 3, not 2.

BigDid, if you can't use MTMode it doesn't mean others should not.

BigDid

4th October 2009, 03:43

...
BigDid, if you can't use MTMode it doesn't mean others should not.
Hi,

It is the other way around, I can use either setmtmode(1 or 2 up to 6) or MT(). I just don't have speed gain with setmtmode(), I do with MT() and I posted in Jeremy's thread in answer for a question related to speed:
http://forum.doom9.org/showthread.php?p=1330112#post1330112

Unless my english is that bad :eek: , it was the meaning of my last post: use MT 2.58 dll but without setmtmode() to see if same speed (and no bugs) and use MT() whenever possible.
Did

levi

4th October 2009, 04:15

Thanks for advices guys. Moving to SEt's 2.6 thread. :)

Jeremy Duncan

31st December 2009, 04:55

interaction with ffdshow:

patched ffdshow.ax revision 2975: http://www.mediafire.com/file/jz2jyluzzny/ffdshow.7z
patch: http://pavelsx.googlepages.com/ffds_avs_patch.diff

Patch fixes incorrect timestamps, redice crashes on stop, reduce memory usage a bit.

For using SetMTMode in ffdshow you must uncheck 'Add ffdshow video source', start your script with something like
SetMemoryMax(512)
SetMTMode(3,4)
ffdshow_source()
SetMTMode(2)
and end with
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

'Buffer ahead' value should be at least 'source_frames_ahead_for_result_frame' + 'number_of_threads', so for example for MDegrain2 with 8 threads it should be 10+.

Can you please make a new ffdshow.ax patch and post it here using ffdshow generic version 3164 ? :)

SEt

31st December 2009, 16:25

Updated first post with build of ffdshow.ax revision 3173. But seriously, if you want patched version – use posted patch file and build it yourself for any revision you need.

Jeremy Duncan

31st December 2009, 18:15

Updated first post with build of ffdshow.ax revision 3173. But seriously, if you want patched version – use posted patch file and build it yourself for any revision you need.

Thank you.

leeperry

4th January 2010, 14:18

is the patched ffdshow.ax any useful w/ 2.57MT(my plugins were not recompiled for 2.58)? Leak didn't think these issues needed to be fixed in ffdshow?

SEt

5th January 2010, 23:02

It's useful with any Avisynth if you want frame-accurate seeking.

leeperry

6th January 2010, 21:59

otherwise it would add delay? I wonder why Leak didn't implement it if it's needed, I'll try it! :thanks:

PS: oops, I tried some modified ffdshow a while ago, and it was going completely wonky from time to time IIRC...half speed and so :o

Jeremy Duncan

20th January 2010, 08:43

SEt, your patch works fine on 32 bit xp and vista, but on windows 7 64 bit, the picture is very distorted. It looks like the resizing it all wrong.
I thought it was the 32 bit build being incompatible with 64 bit, so I made a 64 bit ffdshow.ax using your patch and clsid's 64 bit ffdshow, both ffdshow and patch were build 3200 but the artifact of bad resizing was still there.
I writing this to you in the hope you can revise your patch so it works on windows 7 64 bit. If there's any tests I can run to help you I will do so, I can build ffdshow.ax 64 bit in minutes. :)

Edit, I fixed the problem here is the new diff file: diff file i made (http://www.mediafire.com/?dnljddfjzlm)

SEt

22nd January 2010, 00:49

Jeremy Duncan

22nd January 2010, 03:21

Jeremy, you were told many times – you have absolutely no idea of what you are doing. Of course you are free to continue it for yourself, but please don't confuse less experienced users here. Image distortion you see likely has no relation to my patch and you'd see it without patch too. Not only contents of your 'patch' make no sense at all, it's not even a valid .diff file so you can't apply it.

I patched the two files from the src, imgfilters, avisynth folder using your patch, but I didn't make a new diff file.
May I ask you to look at the two files I patched and post a diff file.
I will post the original files then the patched files so you can see them both. I can read the diff file and copy and paste them into the file but on how to build a diff file I'm not too sure yet. :)

patched files (http://www.mediafire.com/?cdwzfhzzhlr)
original files (http://www.mediafire.com/?zhnzxmony0q)

I said could you make a new diff file since they have updated the files in the svn so I'm not entirely sure your diff file in the first post is still correct.

22nd January 2010, 03:38

Jeremy, it's time to take your meds now.

Jeremy Duncan

22nd January 2010, 06:21

cy, rule #4. I won't report you though, but just so you know, I can if I wanted too. :)

tedkunich

22nd January 2010, 07:24

I wish this forum had the ability to use a killfile... threads of late have become useless drivel, wasting bandwidth...

Jeremy, when you finally graduate high school (I assume you are teenager), I would recommend that you elect computer science as a major in college - then in a few years, please take a look back at your posts and you will (hopefully) finally understand what everyone is trying to explain to you. Until then, please spare us your naive, youthful over exuberance on this matter, it is really wearing thin.

Peace,

T

Jeremy Duncan

22nd January 2010, 10:08

read this reply in a different thread. I'm sorry for my error. (http://forum.doom9.org/showthread.php?p=1366545#post1366545)

mikey1984

16th February 2010, 13:26

Hi, ive just installed modded version of avisynth and mt.dlls from this thread.

I experienced grt speed improvements already.

I also noticed that encodes without resizing are ~ 20-30fps faster than the ones with resizing, and i have question, which resize filter is compatibile the best with those mods and/or multithread-friendly? So far i only used lanczos.

Any ideas, suggestions?

trackerz

16th March 2010, 17:41

.....

Hiritsuki

18th March 2010, 19:37

i got this error

I don't know MT("""""") can use multi filter in once?

osgZach

18th March 2010, 22:05

You can make multiple filter calls like that, but I don't know if there is a limit, or if certain ones just don't work. However I think the triple quotes were supposed to be to help make sure it went through properly..

in my case I just do this for my specific need

MT("""TempGaussMC_beta2().SelectEven()""",overlap=x,threads=x)
# x = w/e your values are

Works great for me

Gavino

19th March 2010, 12:37

You can make multiple filter calls like that, but I don't know if there is a limit, or if certain ones just don't work. However I think the triple quotes were supposed to be to help make sure it went through properly.
Yes, the MT filter argument can include multiple calls, it is just a string passed to the Eval function. The triple quotes are only strictly needed if the string itself includes further quotes.

trackerz' problem might be in applying hqdn3d to the MSuper clip - I don't know if this is allowed or makes sense.

lych_necross

20th March 2010, 07:15

What are you trying to do trackerz?

trackerz

20th March 2010, 10:57

What are you trying to do trackerz?

encoding a 720p mkv to 480p avi

User X

5th May 2010, 06:36

Hi to all.

I have installed Avisynth 2.5.8 with the plugins "MVTools 2.5.10" and this "MT 2.5.8", with the Avisynth configuration activated in FFSshow. Everything on updated XP SP3.
Work fine, but I have noted than this plugin provokes that the Media Player Classic crash passed a determined quantity of video files (some ~10 videos) played or skipped with [Skipe Foward] from list of reproduction (*.MPCPL).
This happens when I activate the Avisynth configuration in FFSshow only when I use this MT plugin. I feel than is a plugin bug.

I wanted to know if to somebody else happens to it or only to me.

Greetings and thanks.

foxyshadis

8th May 2010, 03:36

ok, indeed my sample works now w/ your new MT.dll+avisynth.dll :cool:

now, using this simple script w/ the latest icl10 ffdshoow build from xvidvideo.ru in KMPlayer in XP SP3 :

SetMemoryMax(64)
MT("AddGrainC()",8)

that's AddGrainC 1.1 :

http://img72.imageshack.us/img72/2866/62435091.png

first value is CPU load, second is RAM usage, and third is pagefile usage.

and now AddGrainC 1.4 :

http://img229.imageshack.us/img229/2024/36428209.png

I've put all the builds(+source) from 1.0 to 1.4 here....in case you have any idea what's going wrong : http://www.megaupload.com/?d=BOKPFX19

leaks start at 1.2, so between 1.1 and 1.2 something went wrong w/ MT :(

I already mentioned it to foxyshadis, but never got a reply :o

I'm not sure whether there's any point that I try your ffdshow mod, because SetMTMode() doesn't work too well w/ HR+Reclock...it seems to be asynchronous or something..

I finally reinstalled a build environment and built a fix (http://www.megaupload.com/?d=9ZWS9HDF). (Once I wrapped my head back around the algorithm.) It loops through 256 * threads possible permutations of noise, instead of frames * threads, which got pretty ugly with MT 8-way. Sorry about that, I didn't test on any really long sequences before.

Let me know if it crashes.

Thanks for the download btw, I didn't even have my own sources. (They're on my laptop's pre-reinstall backup.)

fabje

12th June 2010, 16:32

Didée

12th June 2010, 17:00

I just tested this and it works fantastic a result off an 25min recording, including cropping, deinterlace and trimming to a xvid avi file.
Before: Pass 1: 171fps | Pass 2: 165fps
After: Pass 1: 300fps | Pass 2: 223fps
With all due respect, those numbers are suspiciously high. What was the script, the Xvid settings, and the resolution?

With Xvid basically being singlethreaded (except for some exotic, seldomly-used solutions), I wonder what the actual chain was. Perhaps like AviSource().bilinearresize(320,172)?

(On my i7-860, a plain decoding of mpeg2source("PAL_720x576_25fps.d2v") to NULL is in range 550~600 fps. There's not much room at all for filtering or exhaustive encoding settings, in order to get Xvid run at 300fps on 1st pass, or 200+ fps on 2nd pass.)

fabje

12th June 2010, 17:08

With all due respect, those numbers are suspiciously high. What was the script, the Xvid settings, and the resolution?

With Xvid basically being singlethreaded (except for some exotic, seldomly-used solutions), I wonder what the actual chain was. Perhaps like AviSource().bilinearresize(320,172)?

(On my i7-860, a plain decoding of mpeg2source("PAL_720x576_25fps.d2v") to NULL is in range 550~600 fps. There's not much room at all for filtering or exhaustive encoding settings, in order to get Xvid run at 300fps on 1st pass, or 200+ fps on 2nd pass.)
Thanks :)

The source is an IPTV recording with an overall bitrate of 8500 Kbps and resolution of 720 x 576.

This is the avs script:
SetMTMode(5,2)
MPEG2Source("D:\test.d2v")
SetMTMode(2)
LeakKernelDeint(order=1,threshold=8)
Crop(2,2,-2,-2)
BicubicResize(672,368,0,0.5)
Trim(17518,56467)

The specs of the system:
i7 930 @ 3520Mhz
3GB DDR3 RAM
2x 250GB WD RAID0
Windows XP SP3.

If have also an i7 980x @ 3750Mhz, wonder what that will do :p

Didée

12th June 2010, 18:07

Wow, so it seems Xvid can really go *that* fast, even though it's single-threaded? (I haven't yet installed Xvid, hence I didn't test)

fabje

12th June 2010, 19:33

Wow, so it seems Xvid can really go *that* fast, even though it's single-threaded? (I haven't yet installed Xvid, hence I didn't test)
I don't know if xvid is single-threaded I know that you can say in the settings how many threads your pc has.

Didée

12th June 2010, 20:37

Errrrmh ... yes, of course, you're right. Xvid can use MT since ... quite a few years.... :o

(Lame excuse: I've ridden the one-legged horse for so [too] long, that Xvid SMP escaped my notice.)

Atak_Snajpera

26th June 2010, 18:46

lych_necross

27th June 2010, 06:59

Thanks Atak_Snajpera!

supernater

18th July 2010, 15:40

I wanted to say THANK YOU for such a wonderful plugin. It is AWESOME, I get speed boosts by more than 100% sometimes! I wish I'd stumbled onto this thread a long time ago. I have a few questions though...

I ran several scripts on a piece of poorly encoded anime that I was trying to fix up. One of the scripts crashes when going from the first pass to the second pass while the other three don't. Here are my scripts...

Script#1: works (about 4.5 fps)
directshowSource("badanimesource.avi", fps=25.0, convertfps=true, audio=false)
comptest(5)
Deen("a2d",3,7,10,0.5)
dehalo_alpha(ss=1.2)
warpsharp(depth=50, blur=2, bump=50)
limitedsharpenfaster()
DCTFilter(1,1,1,1,1,0.75,0.25,0)

Script#2: works (about 8.2 fps)
SetMemoryMax(512)
SetMTmode(3,0)
directshowSource("badanimesource.avi", fps=25.0, convertfps=true, audio=false)
SetMTmode(2)
comptest(5)
Deen("a2d",3,7,10,0.5)
dehalo_alpha(ss=1.2)
warpsharp(depth=50, blur=2, bump=50)
limitedsharpenfaster()
DCTFilter(1,1,1,1,1,0.75,0.25,0)

Script#3: works (about 8.45 fps)
SetMemoryMax(512)
SetMTmode(2,0)
directshowSource("badanimesource.avi", fps=25.0, convertfps=true, audio=false)
comptest(5)
Deen("a2d",3,7,10,0.5)
dehalo_alpha(ss=1.2)
warpsharp(depth=50, blur=2, bump=50)
limitedsharpenfaster()
DCTFilter(1,1,1,1,1,0.75,0.25,0)

Script#4: first pass works, second pass crashes
SetMemoryMax(512)
SetMTmode(3,0)
directshowSource("badanimesource.avi", fps=25.0, convertfps=true, audio=false)
SetMTmode(2)
comptest(5)
Deen("a2d",3,7,10,0.5)
dehalo_alpha(ss=1.2)
warpsharp(depth=50, blur=2, bump=50)
limitedsharpenfaster()
DCTFilter(1,1,1,1,1,0.75,0.25,0)
SetMTMode(1)
GetMTMode(false) > 0 ? distributor() : last

I've reproduced this result several times on different anime sources. I've installed everything properly, I'm running avisynth 2.5.8 and ffdshow r2975 with the ffdshow.ax file that SEt posted, I repladed the appropriate DLL's with what SEt had in the zip file. My question is.. Why does the 4th script crash? I am running it according to SEt's recommendations, right? Can anyone see any reason why this isn't working? Is there any reason why I shouldn't use the second or third script even though I omitted the lines SetMTMode(1) and GetMTMode(false) > 0 ? distributor() : last ? Those two scripts still gave me a speed boost and seemed to work fine.

doc_dvxm

22nd July 2010, 14:24

I've created a AviSynth MT 2.5.8 installer based on SEt's sources
Installer: http://www.mediafire.com/file/kjdzqyz4dyj/AviSynthMT_258.exe
Sources: http://www.mediafire.com/file/zt4gzeoo3nk/AviSynthMT_258_INSTALLER.7z

Thanx atak :)

I am running 32 bit MeGUI + 32 Bit Avisynth on my Win 7 Ultimate 64 Bit - my processor is i7930 btw. Why I am running 32 bit softwares - because I am not able to resolve problem with 64 bit codecs - especially with VC-1 codec based m2ts files. ffdshow don't opens the VC-1 codec based m2ts files. I can't use DGIndexNV, because my graphic card isn't Nvidia. So I decide to use the 32 bit softwares. When I am encoding HD movies from BluRay sources, now everything is fine. But sometimes we encode a Bluray sources to 480p avi files. And then, I am using a given Here (http://forum.doom9.org/showthread.php?t=144852) MT.dll & Avisynth.dll. On my previous machine, when I need a SD encoding, ı have switched to given here (http://forum.doom9.org/showthread.php?t=144852)MT.dll in avisynth/plugins folder and avisynth.dll in system32 folder and then I was able to encode 480p with multithreaded dll's. After finishing 480p encoding, I have switched to original MT.fll & Avisynth.dll's. Then I was able to do HD encodes.

Today, I have tried to do same, but avisynth stops work.
The produced error is:

Evaluate: System exception - Acces violation. When selected value for MT "".......,4". Same thing was happened when MT "".......,6" is selected.

Can I use the installed avisynthMT for both purpose - HD & SD encoding - on my 64 bit system which works with 32 bit softwares?

And how many cores I must select from MeGUI? 6? 8?

Of'coz I'll try it, but maybe any suggestion will make my job more easy?

Thanx in advance....

doc

asarian

24th July 2010, 00:35

Is there a SetMTmode() command to disable multithreading altogether? (as in: running as stable and single-threaded as non-MT AviSynth). Because you can't install the two AviSynth packages side-by-side.

Atak_Snajpera

24th July 2010, 00:41

asarian

24th July 2010, 02:13

If you don't specify SetMTMode() then you have single threaded mode. Don't worry about stability in MT version. I've been using it for long time and no problems so far. No need to go back. Avisynth 2.6 will also by based on this version.
Hehe, you read my mind: I was indeed worried about stability. :)

Well, thanks again, Atak. You've been most helpful of late!

fabje

9th October 2010, 00:07

Ok guys I have a weird problem.
A few month ago I posted here also that it was working fine with these results:
Before: Pass 1: 171fps | Pass 2: 165fps
After: Pass 1: 300fps | Pass 2: 223fps

And that was with this system:
i7 930 @ 3520Mhz
3GB DDR3 RAM
2x 250GB WD RAID0
Windows XP SP3.

Now I changed the OS to Windows 7 32Bit and this specs:
i7 930 @ 3740Mhz
3GB DDR3 RAM
2x 500GB WD RAID0

But now I'm getting these rules:
Pass1: 219fps | Pass2: 165fps

Anyone here knows whats going on?
This are the settings that I use:
SetMTMode(5,2)
MPEG2Source("D:\test.d2v")
SetMTMode(2)
LeakKernelDeint(order=1,threshold=8)
Crop(2,2,-2,-2)
BicubicResize(672,368,0,0.5)

Usedocne

9th October 2010, 03:28

fabje

9th October 2010, 06:04

@fabje

The speed difference could be caused by a number of different things. Like maybe you used different XviD settings than last time? Or maybe you changed a BIOS setting (HT off)?

BTW congrats on that OC. :devil:
The only thing that is changed in the bios is that the MP is set on 21 instead of 22 to get a higher BLCK. The rest is the same for what I know.

AlekseiV

9th October 2010, 16:11

fabje

9th October 2010, 18:23

You should really install Windows 7 64-bit. There are no downsides anymore (and Avisynth64 and x264_64 are both ~10-20% faster than their 32-bit versions).
Also try using SetMTMode(3) with MPEG2Source; it works for me (but crashes colormatrix)

Being more relevant: Have you tried going back to your original overclocking settings to see if video encoding goes back to that speed?
I have test is with my original overclock settings and that wasn't the trick :(

And if i'm going to use 64bit how about the filters Leakkerndeint, Yadif and LancozResize?
And I have only 3GB Ram is 64bit still faster?

Usedocne

11th October 2010, 02:24

And if i'm going to use 64bit how about the filters Leakkerndeint, Yadif and LancozResize?
64-bit vers. of Leakkerndeint (http://code.google.com/p/avisynth64/wiki/PluginLinks) and Yadif (http://yo4kazu.110mb.com/). I think LancozResize is already built into avisynth64.
And I have only 3GB Ram is 64bit still faster?
Win7(x64) is not as resource hungry as ppl might think.

AlekseiV

11th October 2010, 16:40

And I have only 3GB Ram is 64bit still faster?Yes.

The only reason Windows 7 even has a 32-bit version is for netbooks and other low-end systems which do not have x64-capable processors.

cweb

12th October 2010, 14:15

You should really install Windows 7 64-bit. There are no downsides anymore (and Avisynth64 and x264_64 are both ~10-20% faster than their 32-bit versions).
Also try using SetMTMode(3) with MPEG2Source; it works for me (but crashes colormatrix)

Being more relevant: Have you tried going back to your original overclocking settings to see if video encoding goes back to that speed?

Do you know of any good detailed guides to installing win7 64-bit when you have win7 32-bit installed?

lych_necross

13th October 2010, 07:39

Do you know of any good detailed guides to installing win7 64-bit when you have win7 32-bit installed?
Check out these sites for more info:

http://www.winsupersite.com/win7/upgrade_02.asp
http://windows.microsoft.com/en-US/windows7/32-bit-and-64-bit-Windows-frequently-asked-questions

cweb

13th October 2010, 08:34

Check out these sites for more info:

http://www.winsupersite.com/win7/upgrade_02.asp
http://windows.microsoft.com/en-US/windows7/32-bit-and-64-bit-Windows-frequently-asked-questions

It seems that this is what is involved for win7 32 to 64 bit migration:

If you want to move from a 32-bit version of Windows to a 64-bit version of Windows 7 or vice versa, you'll need to back up your files and choose the Custom option during Windows 7 installation. Then, you'll need to restore your files and reinstall your programs

cweb

15th October 2010, 08:51

It seems that this is what is involved for win7 32 to 64 bit migration:

I went ahead and tried it. In the process I increased my windows partition size as I know that it was a teeny bit too small.
It worked essentially. I used Windows Easy Transfer and managed to transfer most of my stuff.
I reformatted the windows partition and did a clean 64 bit win7 install there.
I had an external backup of all my files too. I did lose some program settings including putty but nothing too important. You do have to reinstall software but it's doable for a technical user. It may not be straightforward for non-techies I should add.

Now to reinstall itunes (sans bonjour) - the 64 bit version now - and avisynth 64 bit and stuff..

bayex

1st November 2010, 18:23

X6 AMD Phenom processor usage does not exceed 30% when I use a filter with a kind of encoding.
I use Mt avisyth SetMTMode topic (2, 6) and after 5 minutes this time fails to x264 x264 error 6 kernel works but gave about 80% off.

What could be the reason.

My system is:

Main card: asus crosshair formula 4 iv

Processor: Phenom 1090 x6 t
http://img830.imageshack.us/img830/2126/adszxi.jpg

SetMTMode(2, 6)
import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpen.avs")
import("C:\Program Files\AviSynth 2.5\plugins\Soothe.avs")

AVCSource("C:\Users\bayex\Desktop\hade.dga")
Crop(0, 140, -0, -140)

Lanczos4Resize(1920, 800)

dull = last
sharp = dull.LimitedSharpen(ss_x=1,4, ss_y=1,4, Smode=3, strength=75, radius=0, Lmode=3, overshoot=1)
Soothe(sharp, dull, 65)

Blu-ray would encode

32-bit operating system for win7

athlonx2 amd 5000 + processor with the same scripts can use a comfortable lead in t does not exceed 20% of the other cores with Phenom ii2 x6 why active filters do not support multi-core processor amd lullaby why the problem is creating a new problem of how I can make it. It hurt too much aid they will be happy masters in this case squeezed

kemuri-_9

2nd November 2010, 00:27

Zep

3rd November 2010, 13:26

X6 AMD Phenom processor usage does not exceed 30% when I use a filter with a kind of encoding.
I use Mt avisyth SetMTMode topic (2, 6) and after 5 minutes this time fails to x264 x264 error 6 kernel works but gave about 80% off.

What could be the reason.

SetMTMode(2, 6)
import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpen.avs")
import("C:\Program Files\AviSynth 2.5\plugins\Soothe.avs")

AVCSource("C:\Users\bayex\Desktop\hade.dga")
Crop(0, 140, -0, -140)

Lanczos4Resize(1920, 800)

dull = last
sharp = dull.LimitedSharpen(ss_x=1,4, ss_y=1,4, Smode=3, strength=75, radius=0, Lmode=3, overshoot=1)
Soothe(sharp, dull, 65)

Blu-ray would encode

32-bit operating system for win7

i already told you what the reason is and how to fix in the other thread. I ran the same code and it works fine for me when you do what I said in the other thread. Did you even try what I suggested to see if it works for you like it does for me?

Zep

3rd November 2010, 13:33

it would help if you actually posted whatever error you're getting with x264 (is what I interpreted from that non english gibber), as letting us guess isn't going to help things too much.

either way, I suspect that you're hitting the 2GB memory limit for 32bit processes.
known possible fixes & workarounds
A) get a 64bit operating system and use 64bit processes on it.
B) split avisynth and x264 into separate processes (e.g. with avs2yuv)
C) set avisynth's MemoryMax to a low amount
D) Don't Multithread

I ran the avs on my XP 32 bit 2 gigs and it ran fine. my guess was the setMTmode 2,6 on AVCsource chokes and sure enough when i set it to 5,6 it ran fine. Not saying that is 100% why he is getting trouble but it does work for me ok.

kemuri-_9

3rd November 2010, 13:43

I ran the avs on my XP 32 bit 2 gigs and it ran fine. my guess was the setMTmode 2,6 on AVCsource chokes and sure enough when i set it to 5,6 it ran fine. Not saying that is 100% why he is getting trouble but it does work for me ok.

AVCSource is outdated, unsupported, and frequently broken.
he should use something that's not in this kind of shape...

but yes, the MTmode 2 is known to break source filters, it is widely recommended/required to use 5 instead.

SilaSurfer

9th November 2010, 17:49

Hi. I'm testing this version of avisynth on my Pentium Dual Core processor and this is my script.

SetMemoryMax(512)

LoadPlugin("C:\Program Files\GordianKnot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("E:\R_O_T_K\ColorMatrix\ColorMatrix.dll")
LoadPlugin("E:\R_O_T_K\Mask\mt_masktools-25.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RemoveGrainSSE2.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RepairSSE2.dll")
Import("E:\R_O_T_K\Lsfmod.avsi")
SetMTMode(2)
MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v", cpu=0)
Colormatrix()
crop(2,80,712,416)
LsfMod(defaults="slow", preblur="on", kernel=19,dest_x=720, dest_y=288)

and I'm getting ~11fps compared to 8fps with standard Avisynth 2.5.8. This is my first usage of Avisynth MT version so it kind of new to me. Please correct the script if I'm doing anything wrong. Thanks in advance!

Zep

10th November 2010, 18:16

What is your CPU usage? Try this...

LoadPlugin("C:\Program Files\GordianKnot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("E:\R_O_T_K\ColorMatrix\ColorMatrix.dll")
LoadPlugin("E:\R_O_T_K\Mask\mt_masktools-25.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RemoveGrainSSE2.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RepairSSE2.dll")
Import("E:\R_O_T_K\Lsfmod.avsi")

SetMTMode(5)
SetMemoryMax(512)

MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v", cpu=0)
ChangeFPS(last,last,true)

SetMTMode(2)

Colormatrix()
crop(2,80,712,416)
LsfMod(defaults="slow", preblur="on", kernel=19,dest_x=720, dest_y=288)

SilaSurfer

10th November 2010, 18:53

I'm getting 100% of cpu usage with the script that I posted so above. With this script I'm getting maximum results

SetMemoryMax(512)

LoadPlugin("C:\Program Files\GordianKnot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("E:\R_O_T_K\ColorMatrix\ColorMatrix.dll")
LoadPlugin("E:\R_O_T_K\Mask\mt_masktools-25.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RemoveGrainSSE2.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RepairSSE2.dll")
Import("E:\R_O_T_K\Lsfmod.avsi")

SetMTMode(2)

MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v")

Colormatrix()

crop(2,80,712,416)

LsfMod(defaults="slow", preblur="on", kernel=19,dest_x=720, dest_y=288)

gave 12fps!

With MT()call:

SetMemoryMax(512)

LoadPlugin("C:\Program Files\GordianKnot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("E:\R_O_T_K\ColorMatrix\ColorMatrix.dll")
LoadPlugin("E:\R_O_T_K\Mask\mt_masktools-25.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RemoveGrainSSE2.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RepairSSE2.dll")
Import("E:\R_O_T_K\Lsfmod.avsi")

MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v")

Colormatrix()

crop(2,80,712,416)

mt("spline36resize(720,last.height)")
mt("spline36resize(last.width,288)",splitvertical=true)

MT("""LsfMod(defaults="slow", preblur="on", kernel=19)""",2)

16 fps. It seems that MT() call gives faster results. Could you help me enter command for MT() so I can use LSFMOD for resizing also so I can ditch Spline36Resize step. Thanks for your response!

Zep

12th November 2010, 01:39

SetMTMode(2)

MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v")

is bad. MPEG2Source can really screw up and hang and slow down with it. you should move it above SetMTMode(2)
also adding ChangeFPS(last,last,true) right after MPEG2Source can many times increase speed since it sorta forces
it to cache/read ahead.

anyway what happened when you tried my version?

Zep

12th November 2010, 01:56

MT("""LsfMod(defaults="slow", preblur="on", kernel=19)""",2)

16 fps. It seems that MT() call gives faster results. Could you help me enter command for MT() so I can use LSFMOD for resizing also so I can ditch Spline36Resize step. Thanks for your response!

Odds are you will need to call LSFMOD twice and do what you did for spline. Me i would leave it how it is and not try to resize in LSFmod. I never use MT() anyway cause it can screw things up unless you have a lot of over lap (depends on the filters) which then slows it down so....

zambelli

12th November 2010, 04:12

SetMTMode(2)

MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v")

is bad. MPEG2Source can really screw up and hang and slow down with it. you should move it above SetMTMode(2)

Have you been able to find any MT mode that works with DGMPGDec? I think I tried all of them in both your x86 and JoshD's x64 mod and it'd always either crash right away or hang after several hundred frames.

BTW, it's interesting that http://avisynth.org/mediawiki/MT_support_page lists MPEG2Source() as being compatible with mode 1. Should that be updated? Or did something change between MT 2.5.7 and MT 2.5.8 that's breaking MPEG2Source?

also adding ChangeFPS(last,last,true) right after MPEG2Source can many times increase speed since it sorta forces
it to cache/read ahead.
Wow, I'll have to try that. So, you're suggesting:

MPEG2Source("foo.d2v")
ChangeFPS(last,last, true)
SetMTMode(2)
#other functions and filters
Is that right?

Didée

12th November 2010, 14:29

Zep

12th November 2010, 23:08

Have you been able to find any MT mode that works with DGMPGDec? I think I tried all of them in both your x86 and JoshD's x64 mod and it'd always either crash right away or hang after several hundred frames.

BTW, it's interesting that http://avisynth.org/mediawiki/MT_support_page lists MPEG2Source() as being compatible with mode 1. Should that be updated? Or did something change between MT 2.5.7 and MT 2.5.8 that's breaking MPEG2Source?

Wow, I'll have to try that. So, you're suggesting:

MPEG2Source("foo.d2v")
ChangeFPS(last,last, true)
SetMTMode(2)
#other functions and filters
Is that right?

DGMPGDec. no idea have not used it in ages :)

AS for MPEG2Source mode 3 works ok 80% of the time but the slight speed up IMHO is not worth the risk. 4 is slower than no MT. Lets face it any decent pc should be able to read frames in at 100FPS+ with a single non MT MPEG2Source so I just do not see the need/risk. I have never been able to get mode 1 to run more than a few seconds before it crashes

yeah that code is what i am suggesting. note though that you should always test and experiment. So many times moving a filter around in the chain has either sped up (or slowed it down lol) so much I always try a few versions and never just go with the first one.

and like Didee said add a setMTmode(5,whatever) at the start

Zep

12th November 2010, 23:20

Almost. Dont forget to put SetMTmode(5) before the mpeg2source() call. (The first MTmode call defines the number of threads, and must appear before any kind of video frames are created. In case of doubt, put SetMTmode(5,<threads>) as the very 1st line in the script.)

Linguistical correction: "adding ChangeFPS ... can many times increase speed" sounds like it would give an X-fold speed turbo. Better let's formulate it like so: "adding ChangeFPS(last,last,true) right after XYZ-source() can increase speed in some cases. (YMMV)."

i get good speed up on simple scripts doing that but none on more complex ones. What speed up it does do helps mostly just on 50+ FPS when random reads and decoding partial GOPS is happening quickly. The more complex stuff is so slow mpeg2source() has no problem keeping up without ChangeFPS(last,last,true)

zambelli

12th November 2010, 23:35

DGMPGDec. no idea have not used it in ages :)

AS for MPEG2Source mode 3 works ok 80% of the time but the slight speed up IMHO is not worth the risk.
But wait... DGMPGDec is MPEG2Source. DGMPGDec is the plugin and MPEG2Source is one of its functions.

Are we talking about 2 different versions of MPEG2Source?

and like Didee said add a setMTmode(5,whatever) at the start
That's the problem - it doesn't work for me. It hangs my app several hundred frames into it.

The only way I could ever get MPEG2Source() to work in a multithreaded script was to move any calls to SetMTMode() down below MPEG2Source.

Gavino

13th November 2010, 00:43

The only way I could ever get MPEG2Source() to work in a multithreaded script was to move any calls to SetMTMode() down below MPEG2Source.
As I understand it (not entirely sure), unless a SetMTMode appears before the first video-processing function in the script, you will get no multithreading at all.

Zep

15th November 2010, 06:51

But wait... DGMPGDec is MPEG2Source. DGMPGDec is the plugin and MPEG2Source is one of its functions.

Are we talking about 2 different versions of MPEG2Source?

That's the problem - it doesn't work for me. It hangs my app several hundred frames into it.

The only way I could ever get MPEG2Source() to work in a multithreaded script was to move any calls to SetMTMode() down below MPEG2Source.

I use the MPEG2Source that is in the DGDecode.dll which is part of DGIndex/DGDecode. IIRC that one via DGMPGDec is old old stuff. Do a search and refresh my/our memory :D

zambelli

17th November 2010, 21:31

I use the MPEG2Source that is in the DGDecode.dll which is part of DGIndex/DGDecode. IIRC that one via DGMPGDec is old old stuff. Do a search and refresh my/our memory :D
Its author Donald Graft still refers to the whole package as DGMPGDec. See here: http://neuron2.net/dgmpgdec/dgmpgdec.html

Zep

17th November 2010, 22:26

Its author Donald Graft still refers to the whole package as DGMPGDec. See here: http://neuron2.net/dgmpgdec/dgmpgdec.html

interesting he removed the bins. bummer

so you are using 1.5.8? Doesn't change much since mode 1 and 2 fails with it. (for me anyway) Mode 3 is the only mode i found worth trying but not enough speed up to risk using it in the long run IMHO.

zambelli

17th November 2010, 22:59

interesting he removed the bins. bummer
I believe it was in order to avoid potential legal trouble with MPEG-LA.

so you are using 1.5.8? Doesn't change much since mode 1 and 2 fails with it. (for me anyway) Mode 3 is the only mode i found worth trying but not enough speed up to risk using it in the long run IMHO.
Yeah, I'm using 1.5.8. I can't get any mode of it to work without crashing the process. The only way I can get past the crashing is by not calling SetMTMode() at all before MPEG2Source(), though some have said that just disables multithreading entirely so it might be just a red herring.

Didée

17th November 2010, 23:41

@Zep - What is that with mode 1, 2, 3 for mpeg2source? Are you trying to multithread the source filter itself? Usually the source filter is fast enough to just not care about it - for Mpeg-2, anyway! So mode 5 for the source filter, the other modes for video filters that come later.
If you're really going to run so ultrafast that the source filters matters, then I'd rather use ffms2 with its internal multithreading.

@zambelli - something seems not right on your side. I've not noticed any problems related to mpeg2source (DGDecode 1.5.8) when using SetMTmode. The occasional problem with other (processing) filters, but not mpeg2source.

IanB

18th November 2010, 07:21

Mode 5 is always recommended for source filters for formats that contain intra (B) frame encodings like mpeg2. With intra frame encoding the frames need to be decoded out of order, e.g With an IBBPBBPBBPB... mpeg 2 stream the decoder may decode frame 0 directly, but needs to decode frames 0 and 3 before it can decode frames 1 and 2. Similarly it cannot decode frame 3 without frame 0 or frame 6 without frame 3. And needs frames 3 and 6 to decode frames 4 and 5.

WIth modes 1, 2 and 3 graph splitting occurs and multiple instances of each filter are created. Each instance only processes a portion of the frames. This is an effective way to process frames in parallel on multiple cores, provided the filter has no context from the calculation of adjacent frames.

Intra frame source filters have a pant load of context from adjacent frames, so much so that in mode 2 all the instance end up needing to do all the work anyway, so you get no speed improvement, just hotter CPU cores.

With formats that are all I frames like huffyuv and DV there is no context between frames and mode 2 with a source filter in that case may well be beneficial.

Hagbard23

18th November 2010, 09:47

Thanks a lot IANB for your clarification...its something i wanted to know for a while...good explanation...

Nephilis

18th November 2010, 19:49

Stick with MT(".........",threads=?). A lot better than SetMTMode, which is problematic and incompatible with many filters

SilaSurfer

18th November 2010, 20:33

~Zep~

LoadPlugin("C:\Program Files\GordianKnot\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT.dll")
LoadPlugin("E:\R_O_T_K\ColorMatrix\ColorMatrix.dll")
LoadPlugin("E:\R_O_T_K\Mask\mt_masktools-25.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RemoveGrainSSE2.dll")
LoadPlugin("E:\R_O_T_K\Removegrain\RepairSSE2.dll")
Import("E:\R_O_T_K\Lsfmod.avsi")

SetMTMode(5)
SetMemoryMax(512)

MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v")
ChangeFPS(last,last,true)

SetMTMode(2)

Colormatrix()
crop(2,80,712,416)
LsfMod(defaults="slow", preblur="on", kernel=19,dest_x=720, dest_y=288)

Well tried it but it seems I'm getting "Thread Deadlock" in VirtualDub, when encoding it just hangs no matter where I put SetMTMode(5,0), before Source call or as the first line in the script. Probably LSFMOD not fully compatible. Why is MT() mode bad, it works faster on my system and the above mentioned problems don't occur.

Zep

18th November 2010, 23:24

@Zep - What is that with mode 1, 2, 3 for mpeg2source? Are you trying to multithread the source filter itself? Usually the source filter is fast enough to just not care about it - for Mpeg-2, anyway! So mode 5 for the source filter, the other modes for video filters that come later.
If you're really going to run so ultrafast that the source filters matters, then I'd rather use ffms2 with its internal multithreading.
.

not me zambelli is or was. Me I NEVER use anything other than mode 5 on mpeg2source. If you read a few posts back i mentioned that IMHO MT on it is not needed since any decent PC can do 100+ FPS with Mpeg2source.

ffms2 does not keep sync so be careful. Many times frame count does not match that of DGIndex etc... (for me anyway)

Zep

18th November 2010, 23:37

~Zep~

Well tried it but it seems I'm getting "Thread Deadlock" in VirtualDub, when encoding it just hangs no matter where I put SetMTMode(5,0), before Source call or as the first line in the script. Probably LSFMOD not fully compatible. Why is MT() mode bad, it works faster on my system and the above mentioned problems don't occur.

===========================================
I have never ever had a problem with this

SetMTMode(5)
SetMemoryMax(512)

MPEG2Source("E:\R_O_T_K\VIDEO_TS\VTS_03_1.d2v")
ChangeFPS(last,last,true)
===========================================

so test the below filters in mode 5 to see which one is causing the problem. then mode 3 it or leave it in mode 5

SetMTMode(2)

Colormatrix()
crop(2,80,712,416)

SetMTMode(5)
LsfMod(defaults="slow", preblur="on", kernel=19,dest_x=720, dest_y=288)
SetMTMode(2)

yes end with mode 2 cause for some reason it speeds everything up. In fact sometimes I need to add something after it just to trigger the speed up like

fadeout0(10)

Zach

22nd November 2010, 18:11

Ignorant question, but that's why we ask questions: to remove ignorance. :)

Is this build fully backwards compatible with the official 2.5.8 build, i.e., will all my old scripts (and plug-ins) work "as is?"

Note: I understand that not everything is multi-thread friendly or capable, or, at the very least, will need a custom build plug-in with MT() functionality in mind. I am just asking if all my old single thread based plug-in and scripts will work with this "2.5.8 MT" build.

lych_necross

23rd November 2010, 08:52

Zach

23rd November 2010, 16:17

If you do come across any problems, please post them in this thread so we can help. :)
Will do!

:thanks:

XhmikosR

12th December 2010, 14:17

That patch has been already applied in ffdshow's svn for quite some time now (r3358). Just FYI, the patches apply to source files and not binaries.

rodgar

3rd June 2011, 11:36

Recently I noticed a recurring VirtualDub crash in connection with MT. I use FastDegrain() wich in my case only uses MVTools.
The Input Video is a segmentedAVI Capture via VD - ffdshow dv.

Here are 2 ErrorLogs from VD (Seems to be always the same problem):
VirtualDub crash report -- build 32842 (release)
--------------------------------------

Disassembly:
04dfa2c0: 0401 add al, 01h
04dfa2c2: 0f6e1411 movd mm2, [ecx+edx]
04dfa2c6: 0f60c7 punpcklbw mm0, mm7
04dfa2c9: 0f60d7 punpcklbw mm2, mm7
04dfa2cc: 0ff5c1 pmaddwd mm0, mm1
04dfa2cf: 0ff5da pmaddwd mm3, mm2
04dfa2d2: 0ffeeb paddd mm5, mm3
04dfa2d5: 0ffee0 paddd mm4, mm0
04dfa2d8: 83c608 add esi, 08h
04dfa2db: 83c104 add ecx, 04h
04dfa2de: 3b4c240c cmp ecx, [esp+0ch]
04dfa2e2: 72d3 jc 04dfa2b7
04dfa2e4: 01fe add esi, edi
04dfa2e6: 0f6fc4 movq mm0, mm4
04dfa2e9: 0f62e5 punpckldq mm4, mm5
04dfa2ec: 0f6ac5 punpckhdq mm0, mm5
04dfa2ef: 0ffee0 paddd mm4, mm0
04dfa2f2: 0f72e407 psrad mm4, 07h
04dfa2f6: 0f6be4 packssdw mm4, mm4
04dfa2f9: 8b442420 mov eax, [esp+20h]
04dfa2fd: 0f7e2418 movd [eax+ebx], mm4
04dfa301: 83c304 add ebx, 04h
04dfa304: 739a jnc 04dfa2a0
04dfa306: 8b5c2410 mov ebx, [esp+10h]
04dfa30a: 8b742414 mov esi, [esp+14h]
04dfa30e: 8b7c2418 mov edi, [esp+18h]
04dfa312: 83c41c add esp, 1ch
04dfa315: c3 ret
04dfa316: f7db neg ebx
04dfa318: 8b542438 mov edx, [esp+38h]
04dfa31c: 89f1 mov ecx, esi
04dfa31e: 8d041b lea eax, [ebx+ebx]
04dfa321: 8b742428 mov esi, [esp+28h]
04dfa325: 89c7 mov edi, eax
04dfa327: f7df neg edi
04dfa329: c1e303 shl ebx, 03h
04dfa32c: 01fa add edx, edi
04dfa32e: 29d9 sub ecx, ebx
04dfa330: 037c2420 add edi, [esp+20h]
04dfa334: 0fefff pxor mm7, mm7
04dfa337: 55 push ebp
04dfa338: 89c5 mov ebp, eax
04dfa33a: 8db600000000 lea esi, [esi+00]
04dfa340: 0fb7042a movzx eax, word ptr [edx+ebp]
04dfa344: 0fb75c2a02 movzx ebx, word ptr [edx+ebp+02h]
04dfa349: 0f6f0ca9 movq mm1, [ecx+ebp*4]
04dfa34d: 0f6f5ca908 movq mm3, [ecx+ebp*4+08h]
04dfa352: 0f6e0406 movd mm0, [esi+eax] <-- FAULT
04dfa356: 0f6e141e movd mm2, [esi+ebx]
04dfa35a: 0f60c7 punpcklbw mm0, mm7
04dfa35d: 0f60d7 punpcklbw mm2, mm7
04dfa360: 0ff5c1 pmaddwd mm0, mm1
04dfa363: 0ff5da pmaddwd mm3, mm2
04dfa366: 0f6fe0 movq mm4, mm0
04dfa369: 0f62c3 punpckldq mm0, mm3
04dfa36c: 0f6ae3 punpckhdq mm4, mm3
04dfa36f: 0ffec4 paddd mm0, mm4
04dfa372: 0f72e007 psrad mm0, 07h
04dfa376: 0f6bc0 packssdw mm0, mm0
04dfa379: 0f7e042f movd [edi+ebp], mm0
04dfa37d: 83c504 add ebp, 04h
04dfa380: 73be jnc 04dfa340
04dfa382: 5d pop ebp
04dfa383: 8b5c2410 mov ebx, [esp+10h]
04dfa387: 8b742414 mov esi, [esp+14h]
04dfa38b: 8b7c2418 mov edi, [esp+18h]
04dfa38f: 83c41c add esp, 1ch
04dfa392: c3 ret
04dfa393: f7db neg ebx
04dfa395: 8b542438 mov edx, [esp+38h]
04dfa399: 89f1 mov ecx, esi
04dfa39b: 8d041b lea eax, [ebx+ebx]
04dfa39e: 8b742428 mov esi, [esp+28h]
04dfa3a2: 89c7 mov edi, eax
04dfa3a4: f7df neg edi
04dfa3a6: c1e304 shl ebx, 04h
04dfa3a9: 01fa add edx, edi
04dfa3ab: 29d9 sub ecx, ebx
04dfa3ad: 037c2420 add edi, [esp+20h]
04dfa3b1: 0fefff pxor mm7, mm7
04dfa3b4: 55 push ebp
04dfa3b5: 89c5 mov ebp, eax
04dfa3b7: 89f6 mov esi, esi
04dfa3b9: 8dbc2700000000 lea edi, [edi+00]

Built on Aegis on Fri Dec 24 13:18:44 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7601) [Service Pack 1]

EAX = 00000000
EBX = 00000000
ECX = 001efa20
EDX = 0219e1d0
EBP = fffffd30
ESI = 021a0048
EDI = 021e04f0
ESP = 3825fd2c
EIP = 04dfa352
EFLAGS = 00010206
FPUCW = 027f
FPUTW = 96aa

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'ffmpeg'...

...reading address 021A0048.

Pointer dumps:

ECX 001efa20: 00000000 40000000 00000000 00000000 33797430 00000000 331bf236 0c001658
EDX 0219e1d0: 001e00b0 00000000 00000000 00000000 af13f2a2 08041756 02252060 0219ed18
EDI 021e04f0: 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040
ESP 3825fd28: 00000000 0000023b 000005a0 021e01a0 021e04f0 021e0ae0 022116a0 0000023b
3825fd48: 00000001 04e03d7c 021e0220 00000168 021a0048 000000b4 00008000 001eeee0
3825fd68: 0219df00 00000004 00000000 000002d0 00000168 00000000 00000000 00000000
3825fd88: 00000000 00000000 02206f28 00000000 021ec2e8 021a0048 0217ffc4 021e0810

Thread call stack:
04dfa352: ffmpeg!yadif_filter [04bd0000+219c50+10702]
7771e003: ntdll!RtlFreeHeap [776f0000+2df85+7e]
04e047e4: ffmpeg!yadif_filter [04bd0000+219c50+1ab94]
76651148: kernel32!WaitForSingleObject [76640000+11136+12]
04e12037: ffmpeg!sws_getContext [04bd0000+23a1e0+7e57]
7525a53a: msvcrt!_lock [75250000+a449+f1]
75261287: msvcrt!_itow_s [75250000+1123b+4c]
75261328: msvcrt!_endthreadex [75250000+112bc+6c]
766533ca: kernel32!BaseThreadInitThunk [76640000+133b8+12]
77729ed2: ntdll!RtlInitializeExceptionChain [776f0000+39e6f+63]
77729ea5: ntdll!RtlInitializeExceptionChain [776f0000+39e6f+36]

-- End of report

Second one:
VirtualDub crash report -- build 32842 (release)
--------------------------------------

Disassembly:
6d22a2c0: 0401 add al, 01h
6d22a2c2: 0f6e1411 movd mm2, [ecx+edx]
6d22a2c6: 0f60c7 punpcklbw mm0, mm7
6d22a2c9: 0f60d7 punpcklbw mm2, mm7
6d22a2cc: 0ff5c1 pmaddwd mm0, mm1
6d22a2cf: 0ff5da pmaddwd mm3, mm2
6d22a2d2: 0ffeeb paddd mm5, mm3
6d22a2d5: 0ffee0 paddd mm4, mm0
6d22a2d8: 83c608 add esi, 08h
6d22a2db: 83c104 add ecx, 04h
6d22a2de: 3b4c240c cmp ecx, [esp+0ch]
6d22a2e2: 72d3 jc 6d22a2b7
6d22a2e4: 01fe add esi, edi
6d22a2e6: 0f6fc4 movq mm0, mm4
6d22a2e9: 0f62e5 punpckldq mm4, mm5
6d22a2ec: 0f6ac5 punpckhdq mm0, mm5
6d22a2ef: 0ffee0 paddd mm4, mm0
6d22a2f2: 0f72e407 psrad mm4, 07h
6d22a2f6: 0f6be4 packssdw mm4, mm4
6d22a2f9: 8b442420 mov eax, [esp+20h]
6d22a2fd: 0f7e2418 movd [eax+ebx], mm4
6d22a301: 83c304 add ebx, 04h
6d22a304: 739a jnc 6d22a2a0
6d22a306: 8b5c2410 mov ebx, [esp+10h]
6d22a30a: 8b742414 mov esi, [esp+14h]
6d22a30e: 8b7c2418 mov edi, [esp+18h]
6d22a312: 83c41c add esp, 1ch
6d22a315: c3 ret
6d22a316: f7db neg ebx
6d22a318: 8b542438 mov edx, [esp+38h]
6d22a31c: 89f1 mov ecx, esi
6d22a31e: 8d041b lea eax, [ebx+ebx]
6d22a321: 8b742428 mov esi, [esp+28h]
6d22a325: 89c7 mov edi, eax
6d22a327: f7df neg edi
6d22a329: c1e303 shl ebx, 03h
6d22a32c: 01fa add edx, edi
6d22a32e: 29d9 sub ecx, ebx
6d22a330: 037c2420 add edi, [esp+20h]
6d22a334: 0fefff pxor mm7, mm7
6d22a337: 55 push ebp
6d22a338: 89c5 mov ebp, eax
6d22a33a: 8db600000000 lea esi, [esi+00]
6d22a340: 0fb7042a movzx eax, word ptr [edx+ebp]
6d22a344: 0fb75c2a02 movzx ebx, word ptr [edx+ebp+02h]
6d22a349: 0f6f0ca9 movq mm1, [ecx+ebp*4]
6d22a34d: 0f6f5ca908 movq mm3, [ecx+ebp*4+08h]
6d22a352: 0f6e0406 movd mm0, [esi+eax] <-- FAULT
6d22a356: 0f6e141e movd mm2, [esi+ebx]
6d22a35a: 0f60c7 punpcklbw mm0, mm7
6d22a35d: 0f60d7 punpcklbw mm2, mm7
6d22a360: 0ff5c1 pmaddwd mm0, mm1
6d22a363: 0ff5da pmaddwd mm3, mm2
6d22a366: 0f6fe0 movq mm4, mm0
6d22a369: 0f62c3 punpckldq mm0, mm3
6d22a36c: 0f6ae3 punpckhdq mm4, mm3
6d22a36f: 0ffec4 paddd mm0, mm4
6d22a372: 0f72e007 psrad mm0, 07h
6d22a376: 0f6bc0 packssdw mm0, mm0
6d22a379: 0f7e042f movd [edi+ebp], mm0
6d22a37d: 83c504 add ebp, 04h
6d22a380: 73be jnc 6d22a340
6d22a382: 5d pop ebp
6d22a383: 8b5c2410 mov ebx, [esp+10h]
6d22a387: 8b742414 mov esi, [esp+14h]
6d22a38b: 8b7c2418 mov edi, [esp+18h]
6d22a38f: 83c41c add esp, 1ch
6d22a392: c3 ret
6d22a393: f7db neg ebx
6d22a395: 8b542438 mov edx, [esp+38h]
6d22a399: 89f1 mov ecx, esi
6d22a39b: 8d041b lea eax, [ebx+ebx]
6d22a39e: 8b742428 mov esi, [esp+28h]
6d22a3a2: 89c7 mov edi, eax
6d22a3a4: f7df neg edi
6d22a3a6: c1e304 shl ebx, 04h
6d22a3a9: 01fa add edx, edi
6d22a3ab: 29d9 sub ecx, ebx
6d22a3ad: 037c2420 add edi, [esp+20h]
6d22a3b1: 0fefff pxor mm7, mm7
6d22a3b4: 55 push ebp
6d22a3b5: 89c5 mov ebp, eax
6d22a3b7: 89f6 mov esi, esi
6d22a3b9: 8dbc2700000000 lea edi, [edi+00]

Built on Aegis on Fri Dec 24 13:18:44 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7601) [Service Pack 1]

EAX = 00000000
EBX = 00000000
ECX = 05070440
EDX = 007e9170
EBP = fffffd30
ESI = 04fc9068
EDI = 033741b0
ESP = 3e8efd2c
EIP = 6d22a352
EFLAGS = 00010202
FPUCW = 027f
FPUTW = 96aa

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'ffmpeg'...

...reading address 04FC9068.

Pointer dumps:

ECX 05070440: 00000000 40000000 80808080 80808080 80808080 80808080 029a216f 0c1d125c
EDX 007e9170: 7a7d00b0 4bd7b630 cc37ff95 c975a951 809a21ed 0a001353 04fc6f10 025eea48
EDI 033741b0: 40404040 40404040 40404040 40404040 40404040 40404040 40404040 40404040
ESP 3e8efd28: 00000000 0000023b 000005a0 03373e60 033741b0 033747a0 03369fa0 0000023b
3e8efd48: 00000001 6d233d7c 03373ee0 00000168 04fc9068 000000b4 00008000 0506f900
3e8efd68: 007e8ea0 00000004 00000000 000002d0 00000168 00000000 00000000 00000000
3e8efd88: 00000000 00000000 0506f868 00000000 05054c28 04fc9068 04fab0a4 033744d0

Thread call stack:
6d22a352: ffmpeg!yadif_filter [6d000000+219c50+10702]
7746e003: ntdll!RtlFreeHeap [77440000+2df85+7e]
6d2347e4: ffmpeg!yadif_filter [6d000000+219c50+1ab94]
74ee1148: kernel32!WaitForSingleObject [74ed0000+11136+12]
6d242037: ffmpeg!sws_getContext [6d000000+23a1e0+7e57]
74b9a53a: msvcrt!_lock [74b90000+a449+f1]
74ba1287: msvcrt!_itow_s [74b90000+1123b+4c]
74ba1328: msvcrt!_endthreadex [74b90000+112bc+6c]
74ee33ca: kernel32!BaseThreadInitThunk [74ed0000+133b8+12]
77479ed2: ntdll!RtlInitializeExceptionChain [77440000+39e6f+63]
77479ea5: ntdll!RtlInitializeExceptionChain [77440000+39e6f+36]

-- End of report

And that's my AVS-Script (If I only delete fastdegrain(), everything goes well)
Import(".\Scripte\LimitedSharpenFaster.avs")
Import(".\Scripte\FastDegrain-ripped.avs")
Import(".\Scripte\FixChromaBleeding.avs")
SetMtMode(2,4)
SegmentedAVISource("video.avi")

ConverttoRGB()
trim(329, 75241)
ReverseFieldDominance()
ConvertToYV12()
FastDegrain()
separateFields()
crop(24,6,-24,-12)
LanczosResize(720,288)
FixChromaBleeding()
LimitedSharpenFaster(strength=145)
weave()

To complete this, here's FastDegrain() (it's only cutted down to what I need)
function FastDegrain(clip src)
{

super = src.MSuper(pel=1)
bvec2 = super.MAnalyse(isb=true, delta=2, blksize=8, overlap=4)
bvec1 = super.MAnalyse(isb=true, delta=1, blksize=8, overlap=4)
fvec1 = super.MAnalyse(isb=false, delta=1, blksize=8, overlap=4)
fvec2 = super.MAnalyse(isb=false, delta=2, blksize=8, overlap=4)

src = src.MDegrain1(super, bvec1, fvec1, limit=255)
src = src.MDegrain2(super, bvec1, fvec1, bvec2, fvec2, limit=255)

return src
}

Gavino

3rd June 2011, 14:09

SetMtMode(2,4)
SegmentedAVISource("video.avi")

ConverttoRGB()

It's more reliable to use mode 5 for source filters.
Also, if your source is interlaced you need to tell ConvertToRGB:
SetMtMode(5,4)
SegmentedAVISource("video.avi")
SetMtMode(2)
ConverttoRGB(interlaced=true)
...

Didée

3rd June 2011, 14:38

Though not related to the crashes, that script is full of issues.

- If the source is interlaced, FastDegrain shouldn't be applied like that. (Mocomp denoising is a temporal-spatial filter, and you know spatial is no good idea on interlaced footage).

- Also, cropping+resizing the fields individually is likely to cause some spatial field misalignment.

- In the stripped fastdegrain function, it seems like an additional msuper() clip should be created after the 1st, and be used in the 2nd mdegrain call. (It can be used like it is now, but the resulting effect probably is not the intended effect.)

rodgar

4th June 2011, 15:02

Good to know. Thanks a lot for the advice, both of you. :)

If the source is interlaced, FastDegrain shouldn't be applied like that. (Mocomp denoising is a temporal-spatial filter, and you know spatial is no good idea on interlaced footage).

I don't know much about all that, but before I used it on the Separated fields like suggested but this way it looks way better. Maybe because of the double of height during processing.

- In the stripped fastdegrain function, it seems like an additional msuper() clip should be created after the 1st, and be used in the 2nd mdegrain call. (It can be used like it is now, but the resulting effect probably is not the intended effect.)

That's right. It's if you set Degrain=3. But I use it as if Degrain would be 2, so I don't need this additional lines.

kenpachi

18th October 2011, 22:28

Guys, I need to ask You something. Is it hard to understand and get on with MultiThreading in AviSynth? I find it very difficult see difference in speed while using unofficial MT285 and this is why I'm puzzled a bit. Even when trying a simple example from here (http://avisynth.org/mediawiki/MT) not mentioning more complex scripts.

Let's take a script as below and encode it with x264 CQ20 High@4.1
SetMTMode(2,x)
src=AviSource("MT_test.avi") # 640x368
SetMTMode(5)
src=src.converttoyuy2().fft3dgpu
SetMTMode(2)
src.LimitedSharpenFaster()
Subtitle("Number of threads used: " + string(GetMTMode(true)) + \
" Current MT Mode: " + string(GetMTMode()))
Trim(62038,65395) # 3357frame sample
ConvertToYV12()

Results:
MT_test_noMT (*) - 6.67fps
MT_test_SetMTMode(2,0) - 6.54fps // 0 - default. Creates 2 threads
MT_test_SetMTMode(2,1) - 6.76fps
MT_test_SetMTMode(2,4) - 6.52fps

* noMT means I used official avisynth 2.5.8.5 x86 and commented all MT-related syntax.

Hardware:
Intel Core 2 Duo CPU T7250 @ 2.00GHz (with Hyper-Threading)
Software:
meGUI with up to date x264

1 thread = 2 threads? Does it make sense? Task Manager would give 100% to all the above choices.

Didée

18th October 2011, 23:12

Rule of thumb: when system load is 100%, then it is 100%. :)

Multithreading makes sense only when the Avisynth script is so slow that it becomes the bottleneck (i.e. when it can't produce frames fast enough for x264, so x264 has to wait for the next frame, and the next, and the next...)

In your case, the script is fast enough even single-threaded. Therefore, nothing to gain from multithreading.

That's probably because of the stamp-sized video. (Joke.) Seriously, 640x368 is so small a resolution, that "even" LSF can run fast enough.

Anyway, you should use SetMTmode(5, [threads] ) in the 1st line. Mode 2 is not generally safe for source filters (sometimes works, most often works not). Mode 5 should always be safe for the source filter.

kenpachi

19th October 2011, 02:38

Argh, I haven't even previewed the script. I picked it from the mentioned link as an example. Didn't even think about how fast the script was. But I got problems achieving speed ups with more complex ones. An example below.

I encoded it gaining only 0.06fps using MT285 vs. 0.23fps using official 285. I can't explain this. I don't remember what Task Manager said, will check it later. I usually encode with let's say 1.4fps. Now it's even more like 1.1 average but I don't want to get too low with speed on long sources. Sure thing, when I have a 2 minute trailer I can let myself get to 0.4, 0.2, but when it's 2hrs it would be insane to accept it.

SetMTMode(5, 4) # Choosed 4 threads as for HyperThreading 2x2=4

LoadCplugin(...)

MPEG2Source(...)
a = NicAC3Source(...)
BlankClip(...).KillAudio() + Trim(...) + BlankClip(...).KillAudio()
AudioDub(last, a)
Trim(0,0)

SetMTMode(2)
last.QTGMC(Preset="slow", FPSDivisor=2, EdiThreads=2)

Deblock_QED(...)

Crop(4, 2, -8, -2)

vectors = last.MVAnalyseMulti(...)
last.MVDegrainMulti(...) # JoseyWells mvtools

GradFunkMirror(...)
Lanczos4Resize(672, 544)

LimitedSharpenFaster(...)

SetMTMode(5)
GrainFactory3(...)

Trim(100, 200) # 101-frame sample

What's more I have a funny feeling it might be faster if I did it doing 2 separated scripts: the first would contain everything to Crop and the second one would perform degraining etc. to GrainFactory3. I've noticed my script needs about 1.5s for a frame to load when QTGMC and mvtools aren't enabled at the same time and it takes about 8s when they are. Even 1.5+1.5 what'd be too high isn't 8.

Didée

19th October 2011, 08:03

Your 2nd example is simply too complex, aka "system overload".

If you give me a 100kg cement bag that I must carry 100 meters, I'll do it in one minute.

If you give me one 200kg cement bag that I must carry 100 meters, I'll NOT do it in 2 minutes. Because I will break down on the way.

SEt

20th October 2011, 23:55

Source filters should be put in mode 3, not 5. It has less overhead and effect is the same for source filters.

If you want to combine some single-threaded initial processing with the rest of the script multithreaded - try putting first part in separate non-MTMode script that second part opens as AviSource("first_part.avs"). Avoid using mode 5.

Always put even empty brackets after function calls with MTMode Avisynth.

And the most important: do not use this Avisynth version - it has many threading bugs. Use builds of Avisynth 2.6 instead.

sutha25

28th March 2014, 12:59

can some one please share the Avisynth.DLL MT 2.5.8 i searched Google and couldn't find 1....Mediafire Link file removed

turbojet

29th March 2014, 02:26

http://www.mediafire.com/download/5n1iujf48l8dotg/avisynth-2.58mt.dll

2.6 MT and avs+ is much more stable for me.

sutha25

29th March 2014, 02:53

http://www.mediafire.com/download/5n1iujf48l8dotg/avisynth-2.58mt.dll

2.6 MT and avs+ is much more stable for me.

i tired that bro but still same is not like it use to be i use to get 100-125FPS and now iam getting 9-15FPS i haven't changed anything then the DLL

turbojet

29th March 2014, 06:54

You formatted, that's about the largest change you could do. The installer and that dll is the official build, no need to ask in 2 other places. What's your script?

sutha25

29th March 2014, 06:58

You formatted, that's about the largest change you could do. The installer and that dll is the official build, no need to ask in 2 other places. What's your script?

HI thanks for the reply where can i get the MT dll then?

SetMTMode(2)
LoadPlugin("C:\Users\Administrator\Documents\megui\tools\ffms\ffms2.dll")
FFVideoSource("D:\Downloads\Playground.mp4")
Crop(2, 88, -2, -88)
Spline36Resize(720,304)
fft3dfilter(sigma=1.0, bt=5, bw=32, bh=32, ow=16, oh=16, sharpen=0.4)
LSFmod( strength=75, smode=3, soothe=true, keep=20, edgemode=0, edgemaskHQ=false)
Tweak(Sat=1.15,Bright=5,cont=1.15)

please let me know where i can get the DLL mt edition so i can gain my speed agian

turbojet

29th March 2014, 07:03

sutha25

29th March 2014, 07:08

http://www.mediafire.com/download/97w3y45d7atp3af/MT.dll but your script doesn't use it.

Are you opening the script with something other than x264?
You might need to add distributor() at the end.

It should be a pretty fast script. What cpu are using?

i need the MT AVISYNTH.dll iam opening the script with AVSpmod and then using megui to encode to AVI format from .MP4