Avisynth 2.6 MT [Archive] - Page 21

ryrynz

4th August 2015, 10:10

Is this working for everyone on Windows 10? I upgraded and now am getting a crash when ffdshow tries to do anything with Avisynth ticked even if it's blank.
I've reinstalled MT via the installer but with no joy. At this point I'm expecting ffdshow is playing up, hopefully an easy fix. Can anyone confirm?

luquinhas0021

4th August 2015, 13:39

Reel.Deel

4th August 2015, 14:01

ryrynz, Avisynth 2.6.0 official is working on Windows 10. By the way, the last release of Avisynth 2.6 MT was in february, time when the Avisynth 2.6 official was in RC2 or RC3. Then Avisynth 2.6 MT is more stable (Less bugs) than Avisynth 2.6 official? The resources are the same? or Avisynth 2.6 MT or Avisynth 2.6 official has more resources than the other?

What are you talking about?

AviSynth MT is basically AviSynth 2.6 with multi-threading. AviSynth MT without MT enabled should work just like AviSynth 2.6. Read here (http://forum.doom9.org/showpost.php?p=1725579&postcount=1018) for peace of mind. You should also try reading up before asking such questions. Especially when most of those are already answered in the OP (http://forum.doom9.org/showthread.php?t=148782).

Groucho2004

4th August 2015, 14:14

Is this working for everyone on Windows 10? I upgraded and now am getting a crash when ffdshow tries to do anything with Avisynth ticked even if it's blank.
Since Avisynth seems to be working on W10 I'd venture a guess that your problem is related to ffdshow.

Edit: Just saw that you also suspect ffdshow to be the culprit.

luquinhas0021

4th August 2015, 14:28

I've had no problem with Avisynth 2.6 itself on Windows 10. ffdshow sucks, my opinion. I preffer LAV Filters.

Groucho2004

4th August 2015, 14:32

ffdshow sucks, my opinion. I preffer LAV Filters.
You have to learn to read and understand the posts you reply to. ryrynz is using a feature in ffdshow that lav filters do not have.

ryrynz

4th August 2015, 14:41

Since Avisynth seems to be working on W10 I'd venture a guess that your problem is related to ffdshow.

Edit: Just saw that you also suspect ffdshow to be the culprit.

Mm.. I hope it's something clsid can fix. Guess I'll ask him and see.
If he can't I'll just bug some devs to go hunting through the code and patch it. Can't have this happening after all, not without anyone else having implemented a replacement for the raw filter yet.. still waiting...

sausuke

10th August 2015, 09:53

can anyone guide me on how to use all the CPU cores of my PC. I have 5960x (8 cores) but only 45% utilization on my CPU. This is the script I'm using though? what can I change? thank you in advance

Cores=8
SetMemoryMax(512)
SetMTMode(3,8)
<input>.ConvertToYV12()
<deinterlace>
<crop>
<denoise>
<resize>

btw I frameserving from Sony Vegas to Megui

LigH

10th August 2015, 10:12

sausuke

10th August 2015, 10:45

This is no valid AviSynth script. May be a template, but who knows...

And apart from that, even with optimal tuning for multi-threading, a CPU utilization >80% is hard to guarantee, there are always limits of parallelizability (means, usually there is an element in a sequence which must wait for intermediate results, a.k.a. "bottleneck"). You omitted all interesting details from your script, so we will never know which filter may be the rather sequential one...

sorry I'm really noob on script :( I just copy this on this site

http://www.spirton.com/convert-videos-to-60fps/

Cores=4
SetMemoryMax(512)
SetMTMode(3,Cores)
PluginPath = "C:\Program Files (x86)\MeGUI\tools\avisynth_plugin\"
LoadPlugin(PluginPath+"svpflow1.dll")
LoadPlugin(PluginPath+"svpflow2.dll")
Import(PluginPath+"InterFrame2.avsi")
<input>.ConvertToYV12()
SetMTMode(2)
<deinterlace>
<crop>
<denoise>
<resize>
InterFrame(Cores=Cores)

I just change the cores to 8.. it manage to use all of my 8 cores to 100% though but with normal encoding example 60fps to 60fps it only uses half of my CPU.. unlike that double frame 30 to 60fps script :( again sorry I'm really new at this

Groucho2004

10th August 2015, 11:02

I just change the cores to 8.. it manage to use all of my 8 cores to 100% though but with normal encoding example 60fps to 60fps it only uses half of my CPU.. unlike that double frame 30 to 60fps script :( again sorry I'm really new at this
What's this irrational obsession with CPU usage lately? People appear to be determined to max out their cores but efficiency and therefore the speed of the frame serving and/or encoding chain seems irrelevant.

Try different numbers for "cores", check the speed and, very important, memory usage with AVSMeter. If you are using internally multi-threaded plugins, set their "threads" parameters to "1" (in some rare cases cores/2 or cores/4 is better). Not sure about the "cores" parameter for Interframe. Try different values.

sausuke

10th August 2015, 11:52

What's this irrational obsession with CPU usage lately? People appear to be determined to max out their cores but efficiency and therefore the speed of the frame serving and/or encoding chain seems irrelevant.

Try different numbers for "cores", check the speed and, very important, memory usage with AVSMeter. If you are using internally multi-threaded plugins, set their "threads" parameters to "1" (in some rare cases cores/2 or cores/4 is better). Not sure about the "cores" parameter for Interframe. Try different values.

Sorry.. thank you for those info.. I seen my past Queue info on my Megui and seen info about FPS and I thought I don't have increased performance but I was wrong.. seen 28 fps on my 60fps encodes on 4790k there unlike now that is 73fps on my 5960x.. again :thanks: you're right it doesn't have to be full utilization of cores :o

Groucho2004

10th August 2015, 12:09

Sorry
Don't be. You're new here and I should probably have posted the rant elsewhere.
Anyway, I'm glad it's working for you.

ryrynz

10th August 2015, 13:26

What's this irrational obsession with CPU usage lately? People appear to be determined to max out their cores but efficiency and therefore the speed of the frame serving and/or encoding chain seems irrelevant.

People just think the two are linked without looking at the results.. if you're not maxxing it out then you must not be utilizing it fully right?
That's the thinking, it's just one of those things you'll have to deal with because to the uninitiated it makes some sense.. as you'd likely expect.

LigH

10th August 2015, 13:51

Well, if people are already "spoiled" by the rather good scalability of x264, they will be sobered by x265 which is more limited regarding parallelizability and will most probably not utilize a CPU with many cores to the max, even though the busy cores will be utilized well with efficient SIMD instructions. Some algorithms just have their bottlenecks you can't avoid easily. But you can circumvent it (e.g. by running several jobs in parallel).

~ VEGETA ~

12th December 2015, 06:53

Groucho2004

12th December 2015, 09:30

I had an issue with this MT version, the issue is that i produces white frame every group of frames when using such a script:

setmemorymax(2048(
setmtmode(2)
mpeg2source("d2v")
colormatrix("from d2v")
setmtmode(3)
deband
dehalo+sharpen+line darken+aa

when I removed the mt commands, it worked nice. The problem shows an artifact like ffms2's seeking problems too.
You're using the MT statements wrong. Try something like this:
setmemorymax(1200)
setmtmode(5)
mpeg2source("d2v")
colormatrix("from d2v")

setmtmode(2)
deband
dehalo+sharpen+line darken+aa

~ VEGETA ~

12th December 2015, 16:35

what is the difference between setmtmodes? I am sure I used 3 for source and 2 for others just like I see here all the time. what are the other modes?

Groucho2004

12th December 2015, 16:41

what is the difference between setmtmodes? I am sure I used 3 for source and 2 for others just like I see here all the time. what are the other modes?
You could have a look at the first post in this thread, it explains the modes.

LigH

12th December 2015, 16:56

Some people reported mode 5 useful for DGDecNV due to its hardware decoder chip access. But differences to mode 3 are small.

Selur

13th December 2015, 16:40

I even use mode 6 for DGDecNV. :)

speedyrazor

18th December 2015, 21:58

I will publish my MT mod later with this and another changes. Currently I prepare it to use in my pro postproduction environment to be compatible with 4K sources.
Also I changed some programs, so it is simpler now to export a 16 bit video to FFmpeg directly (to encode for ProRes 10/16 bit and use in pro software).

This sounds excellent rean, I would love to give it a try.
When do you think you may have a version for us to try please?

~ VEGETA ~

19th December 2015, 16:40

it happened again with this:

setmemorymax(2048)
setmtmode(2)
source=lwlibavvideosource("C:\prereleases\natsume\ep1.mkv")

setmtmode(3)
#dark areas
dark_mask=Levels(source,30, 1,40, 255,0, coring=false)
dark_deband=gradfun3(source,thr=0.3,radius=10).f3kdb(grainc=80,dynamic_grain=false,range=20)
dark_clip=mt_merge(source,dark_deband,dark_mask)

+other filters here (masks)
+insertsign here (using lwlibavvideosource not avisource)

again, in avsp preview no problem was shown even when putting setmtmode stuff.... but final encode showed them.

Groucho2004

19th December 2015, 16:51

~ VEGETA ~

19th December 2015, 17:43

That script won't even work
You're still using setmtmode ass backwards
A correct preview in AVSP doesn't mean that the entire encode will be OK

- it worked, it was faster than without mt. However it had issues like I said.

- you are right! I used 2 instead of 3 and vice versa... it is a typo! I assure you it is! (will it be the solution?)

- well, it should be ok because it is what is going to be encoded... but I am not 100% sure about it.

Groucho2004

19th December 2015, 19:54

- it worked, it was faster than without mt.No, it does not. The script as you posted it has invalid syntax and does not return a clip.

well, it should be ok because it is what is going to be encoded... but I am not 100% sure about it.Even if previewing the entire script in AVSP works fine doesn't mean that it won't crash when you include the encoder in the process.

real.finder

20th December 2015, 09:07

AVSP didn't call distributor() if you use setmtmode, without distributor() it will not be real MT (it will be faster than non MT though)

you can put distributor() in AVSP in the bottom of the script and it will be like what you got after encoding and you will get the real speed of MT, but because of avsp stability probably you will get a lot of crashes that not happen in the encoding

Groucho2004

20th December 2015, 10:16

AVSP didn't call distributor() if you use setmtmode, without distributor() it will not be real MT (it will be faster than non MT though)
You're right about AVSP not calling Distributor() but the rest is wrong.
You don't have to understand the inner workings of SEt's MT mod to know that, you can simply test it with AVSMeter (set "InvokeDistributor" to "0" in AVSMeter's ini file).

you can put distributor() in AVSP in the bottom of the script and it will be like what you got after encoding and you will get the real speed of MT, but because of avsp stability probably you will get a lot of crashes that not happen in the encoding
What?

LigH

20th December 2015, 10:34

The main differences between AvsPmod and an encoder will probably be: You will preview most often only one frame, rarely play a scene for a brief time, but hardly run the whole movie from the start to the end (which may accumulate RAM requirements over some time); and an encoder will require much more own RAM to buffer several frames out of a GOP to get encoded with all probable reference candidates (leaving less for AviSynth), while a preview only displays the currently played frame.

An explicit Distributor() call is only required for environments not handling multithreading on their own. It is not required when serving an encoder like x264, it knows how to handle this situation correctly. It is even counterproductive in this case, multiplying the number of threads. I am not sure if AvsPmod requires it, but I doubt, it has been used for several years, also in MT workflows, so I would assume the developers already know to distribute threads without explicit call in the script. Correct me if I'm wrong (that happens regularly).

~ VEGETA ~

20th December 2015, 11:34

No, it does not. The script as you posted it has invalid syntax and does not return a clip.

You misunderstood me again! I know the thing I posted won't work because it has syntax problems... except that I said the script would be like it. Look here:

+other filters here (masks)
+insertsign here (using lwlibavvideosource not avisource)

I know that

dark_clip=mt_merge(source,dark_deband,dark_mask)

won't return a clip and that is why i told you there are other lines following it.. plus, I already stated that the thing worked! and the encode started... I just didn't want to spam the thread with my full script.

Even if previewing the entire script in AVSP works fine doesn't mean that it won't crash when you include the encoder in the process.

this is what I wanted to understand. In the official AVS it is always working right.

Groucho2004

20th December 2015, 13:18

I am not sure if AvsPmod requires it, but I doubt, it has been used for several years, also in MT workflows, so I would assume the developers already know to distribute threads without explicit call in the script. Correct me if I'm wrong (that happens regularly).
It does not call Distributor(), I checked the source and also tested it.
So, in order to run/preview a script containing setmtmode() statements as "designed" (i.e. multi-threaded) in AVSPMod, one must place the call in the script (and not forget to take it out before encoding).

StainlessS

20th December 2015, 15:44

one must place the call in the script (and not forget to take it out before encoding).

If using some kind of avs auto creating thing (eg Avisynthesizer_Mod), to negate need to add/remove Distributor, can use RT_GetProcessName,
[EDIT: from RT_Stats]

RT_GetProcessName(bool "parent"=false,bool "debug"=false)
Returns string, name of current process eg "VirtualDubMod.exe" or "AvsPMod.exe" or name of parent process
(the one that started our process eg "Explorer.exe"). Debug outputs some info to DebugView.

Something like

(RT_GetProcessName()=="AvsPMod.exe") ? Distributor() : NOP

EDIT: If you want to use above but dont need full RT_Stats library, then get GetProcessName plugin from here:- http://forum.doom9.org/showthread.php?p=1607929#post1607929

Groucho2004

20th December 2015, 17:20

If using some kind of avs auto creating thing (eg Avisynthesizer_Mod), to negate need to add/remove setmtmode, can use RT_GetProcessName,
[EDIT: from RT_Stats]

RT_GetProcessName(bool "parent"=false,bool "debug"=false)
Returns string, name of current process eg "VirtualDubMod.exe" or "AvsPMod.exe" or name of parent process
(the one that started our process eg "Explorer.exe"). Debug outputs some info to DebugView.

Something like

(RT_GetProcessName()=="AvsPMod.exe") ? setmtmode() : NOP

Very nice but I think the function should be Distributor(), not SetMTMode().

Groucho2004

20th December 2015, 17:29

You misunderstood me again! I know the thing I posted won't work because it has syntax problems... except that I said the script would be like it.
If posting a script then post the exact script, not "something like it".
You may think it won't make a difference but even a tiny detail can break it. This applies particularly to people who are not well versed with Avisynth syntax.

StainlessS

20th December 2015, 17:36

Thanx Grouchy, Fixed. (what do I know about MT, nuttin :) )

~ VEGETA ~

20th December 2015, 20:15

If posting a script then post the exact script, not "something like it".
You may think it won't make a difference but even a tiny detail can break it. This applies particularly to people who are not well versed with Avisynth syntax.

I am well versed with avs. And I already pointed out that the script actually worked. worked = 100% good syntax.

the problem was on the lines that i pointed out... the rest of the script was just more filters....

bxyhxyh

25th December 2015, 17:17

I am well versed with avs. And I already pointed out that the script actually worked. worked = 100% good syntax.

the problem was on the lines that i pointed out... the rest of the script was just more filters....
Even if your script works perfectly ok with AvsPmod or non MT Avisynths. It unexpectedly crashes when you do real encoding with Avisynth MT. It depends on plugins you use or other minor things. That's why he's keep saying post your full script.

speedyrazor

27th December 2015, 22:36

Hi, I have this spec machine to work with:

Windows Server 2012
Dual Xeon E5-2650 v2 (16 cores, 32 threads)
128 GB Ram

I want to run 4 separate instances of the below script:

SetMemoryMax(400)
SetMTMode(3,4)
QTInput("test.mov", quality=100, audio=2)
SetMTMode(2)
ColorMatrix(mode="Rec.709->Rec.601", clamp=0)
SmoothDeinterlace()
Spline36Resize(720, 576)
TimeStretchPlugin(tempo = 25.0/24.0*100.0)
AssumeFPS(25, 1)
Distributor()

My question is.....
Is this the 'best' way to be running this script, SetMTMode(3,4) to open the file, then SetMTMode(2) to run the rest? As I want to run 4 separate instances of this script at the same time, all running different things (but pretty much the same as above), should I just be using SetMTMode(1) for everything? I have read-up on SetMTMode but couldn't quite understand what would work best for me in this situation.

Kind regards.

LigH

27th December 2015, 22:44

MT mode 1 is only available for very few functions, which are programmed very cooperatively, regarding multi-threading. In general, your combination of
SetMTMode(3,threads)
SourceFilter()
SetMTMode(2)
OtherProcessingFilters()
is the recommendable way.

speedyrazor

8th January 2016, 21:22

Hi I am using the below script, which I added a few bits to recently, but now some bits are not working, AssumeFPS, etc. What am I doing wrong? Also, have I got the SetMTMode(2) in the correct place?

SetMemoryMax(512)
SetMTMode(3,4)
vid1 = QTInput("//192.168.0.100/media/temp/sample.mov", quality=100, audio=2)
blackVideo1 = blankclip(vid1, length=10)
blackVideo2 = blankclip(vid1, length=10)
vid = Trim(vid1,25,312)
return blackVideo1 + vid + blackVideo2
SetMTMode(2)
TimeStretchPlugin(tempo = (1.0*24000.0/1001.0)/24.0*100.0)
AssumeFPS(24000, 1001)
Distributor()

Kind regards.

LigH

8th January 2016, 21:26

Quite ovbious mistake:

SetMemoryMax(512)
SetMTMode(3,4)
vid1 = QTInput("//192.168.0.100/media/temp/sample.mov", quality=100, audio=2)
blackVideo1 = blankclip(vid1, length=10)
blackVideo2 = blankclip(vid1, length=10)
vid = Trim(vid1,25,312)
return blackVideo1 + vid + blackVideo2
# You returned the video here. Nothing below doesn't happen anymore.

SetMTMode(2)
TimeStretchPlugin(tempo = (1.0*24000.0/1001.0)/24.0*100.0)
AssumeFPS(24000, 1001)
Distributor()

When you use the "return" statement, this will be the final filter step.

Furthermore, please be certain whether to use Distributor() or not, when you are going to remove the return statement.

speedyrazor

8th January 2016, 21:43

When you use the "return" statement, this will be the final filter step.
Furthermore, please be certain whether to use Distributor() or not, when you are going to remove the return statement.

So moving the return to the bottom, I now get Invalid arguments to function "TimeStretchPlugin"

SetMemoryMax(512)
SetMTMode(3,4)
vid1 = QTInput("//192.168.0.100/media/temp/sample.mov", quality=100, audio=2)
blackVideo1 = blankclip(vid1, length=10)
blackVideo2 = blankclip(vid1, length=10)
vid = Trim(vid1,25,312)
SetMTMode(2)
TimeStretchPlugin(tempo = (1.0*24000.0/1001.0)/24.0*100.0)
AssumeFPS(24000, 1001)
return blackVideo1 + vid + blackVideo2
Distributor()

Boulder

8th January 2016, 21:52

Isn't it TimeStretch and not TimeStretchPlugin?

http://avisynth.nl/index.php/TimeStretch

speedyrazor

8th January 2016, 21:54

Isn't it TimeStretch and not TimeStretchPlugin?

http://avisynth.nl/index.php/TimeStretch

No, It's a plugin here

https://www.doom9.org/showthread.php?p=1723420

And it worked before adding the other stuff in.

LigH

8th January 2016, 22:03

You assigned all the intermediary clips to named clip variables. Therefore the implicit variable "last" (which is always used when there is no explicit variable name) has no content.

Instead of returning the concatenation of blackVideo1 + vid + blackVideo2, you should have used its result as another intermediary step.

Your script looks like you tried to copy&paste from different locations without understanding what's going on; having understood the meaning of "last" is one of the most important things in AviSynth scripting.

SetMemoryMax(512)
SetMTMode(3,4)
vid1 = QTInput("//192.168.0.100/media/temp/sample.mov", quality=100, audio=2)
black10 = blankclip(vid1, length=10)
black10 + vid1.Trim(25,312) + black10
# ^ means verbosely:
# last = UnalignedSplice(black10, vid1.Trim(25,312), black10)
SetMTMode(2)
TimeStretchPlugin(tempo = (1.0*24000.0/1001.0)/24.0*100.0)
AssumeFPS(24000, 1001)

# Distributor()
# ^ use only if you know where, when, and why!
# else:
return last
# ^ is in fact the default, could even be omitted, in theory...

speedyrazor

8th January 2016, 23:05

Your script looks like you tried to copy&paste from different locations without understanding what's going on; having understood the meaning of "last" is one of the most important things in AviSynth scripting.

Im sorry to say you are right! Thanks for pointing me in the right direction.
Using my script, I am feeding it straight into ffmpeg, I was using Distributor() as it more than doubled the ffmpeg encoding speed.

Groucho2004

8th January 2016, 23:35

Using my script, I am feeding it straight into ffmpeg, I was using Distributor() as it more than doubled the ffmpeg encoding speed.
In that case it seems obvious that ffmpeg does not invoke "distributor()" in its Avisynth input code so you are right to place the call in the script.

Why are you using Avisynth MT for this script? If I recall correctly, the "TimeStretchPlugin" function is extremely fast, I don't see why you would increase complexity and the number of threads for no apparent reason.

I suggest you comment the SetMTMode() and Distibutor calls and compare the speed.

speedyrazor

9th January 2016, 09:28

In that case it seems obvious that ffmpeg does not invoke "distributor()" in its Avisynth input code so you are right to place the call in the script.

Why are you using Avisynth MT for this script? If I recall correctly, the "TimeStretchPlugin" function is extremely fast, I don't see why you would increase complexity and the number of threads for no apparent reason.

I suggest you comment the SetMTMode() and Distibutor calls and compare the speed.

Ill test and report back, thanks for the advice.