Avisynth 2.6 MT

[LigH]

Yes, and regarding the first line, depending on your CPU:

SetMTMode (5, threads)

Setting the number of threads is only valid in the very first use of SetMTMode. It is only recommendable if you have a high number of CPU cores, to avoid too many copies all using their own RAM in only 2 GB of a 32-bit process memory (only 2 GB if the calling program is not "Large Address Aware"; if it is LAA, up to 4 GB).

You don't need to reduce the number of threads if you only have a DualCore CPU, or up to QuadCore without HyperThreading. But with a HexaCore, or a QuadCore with HyperThreading (which virtually doubles the number of cores to 8), multithreading with 6 or even 8 threads (= copies of your script plugins) may take too much RAM and cause crashes.

[GMJCZP]

Yes, and regarding the first line, depending on your CPU:

SetMTMode (5, threads)

Setting the number of threads is only valid in the very first use of SetMTMode. It is only recommendable if you have a high number of CPU cores, to avoid too many copies all using their own RAM in only 2 GB of a 32-bit process memory (only 2 GB if the calling program is not "Large Address Aware"; if it is LAA, up to 4 GB).

You don't need to reduce the number of threads if you only have a DualCore CPU, or up to QuadCore without HyperThreading. But with a HexaCore, or a QuadCore with HyperThreading (which virtually doubles the number of cores to 8), multithreading with 6 or even 8 threads (= copies of your script plugins) may take too much RAM and cause crashes.

Naturally, include threads, in my case my PC is Core 2 Duo:
SetMTMode (5,2)...
Is valid your note about threads in other cases too.

[GMJCZP]

I just did a test using Preroll(video = 24) on my first script (fps VideoSource = 23.976) and not get hung up, this time I did not use frfun3d, which is very heavy (I added 1 hour processing).
Looks promising, as I had a crash before removing frfun3d.

We need more tests, using as a starting point and empirically Preroll(video = round(fps video source))

[Boulder]

There is something funny here, I just noticed some weird artifacts when encoding with x264. I was able to reproduce it by using this script and the sample clip. Sometimes there are no artifacts right after the scene change, but usually they are there. If I uncomment the RequestLinear line, I haven't been able to get the artifacts. You can use any settings in x264, they don't seem to have any effect. I use piping to feed to a 64-bit x264 executable, hence the Distributor call.

SetMTMode(5)
DGSource("convictiontest.dgi")
#RequestLinear(clim=100)
SetMTMode(2)
Bicubicresize(1280,688,b=-0.6,c=0.3)
Distributor()

Can anyone replicate this one?

Here's the sample: https://drive.google.com/file/d/0BzeF_1syecQwOVRLd09KQWxQZnM/view?usp=sharing

[Groucho2004]

Can anyone replicate this one?
Tried 3 times, can't reproduce it with your clip and script. Are these subtle artifacts or obvious ones?

[Boulder]

They are very obvious ones.

http://i.imgur.com/1s7exoql.jpg (http://imgur.com/1s7exoq)

It's possible that it's just my computer that shows the issue since MT is a tricky beast.. I did notice that it doesn't matter which MTMode is in use, if there's a Distributor call in the script, the artifacts appear (without RequestLinear).

The difficult thing is that I don't know if Avisynth or x264 is the one to blame:confused:

[LigH]

Don't use Distributor() except you are certain that the calling process needs it. One of the few places where it may be required is in the ffdshow postprocessing, but x264 does not need it.

[Groucho2004]

The difficult thing is that I don't know if Avisynth or x264 is the one to blame:confused:
If you get the artifacts by changing the script I suppose it's safe to assume that the problem is in your AVS -> pipe chain.

Are you using the latest AVS MT? Also, try this (https://www.dropbox.com/s/2w7ea8j1ppvwak2/avs2yuv.zip?dl=0) version of avs2yuv, it does not require the Distributor call in any case.

The RequestLinear() function uses a lot of memory, try increasing the default cache size from 512MB to 1024MB (SetMemoryMax).

[Boulder]

I've been using avs2pipemod for piping for years and that requires Distributor to make MT work, and yes, the latest build of AVS MT is in use. I'll test avs2yuv shortly..

EDIT: avs2yuv is more stable but not faultless :( Uncommenting RequestLinear still fixes the issue, the same goes for Preroll if the amount of frames is large enough (25 frames seems to suffice). SetMemoryMax is at 768MB at the moment.

[Groucho2004]

@Boulder
I assume that this is a purely academic exercise since your script certainly does not need speeding up with Avisynth MT, right?

[Boulder]

The script is simple as I worked my way down to the simplest form which would show the artifacts. My normal script has a custom made function for denoising etc. which is rather heavy duty. I just happened to notice those errors by accident when previewing stuff before moving it to my networked media player.

Actually I can reproduce it even with a simple

DGSource("convictiontest.dgi")
SetMTMode(2)
ConverttoYV12()

so it doesn't even need the resizing part. Changing the cropping in the index file also doesn't change the outcome.

[Groucho2004]

DGSource("convictiontest.dgi")
SetMTMode(2)
ConverttoYV12()
That script wouldn't even use multi-threading since it has no SetMTMode before the source filter.

[Boulder]

Sorry, I do have that one set as a default with an avsi file in my plugins folder along with the SetMemoryMax call. Saves time when writing the script ;)

[Sparktank]

@Boudler

Have you tried remuxing to MKV and using FFMS2 ?
(not sure if FFMS2 still has issues with transport streams, can't remember the exact issues it had but haven't really followed for over a year on FFMS2 and transport streams.)

EDIT: That IS DGdecNV.
Old DGdec uses MPEG2Source(). :p
Have you tried updating DGdecNV and/or your graphics card drivers?
NVidia just updated in the past week.

Also, taro_06 (aka Astrataro) has updated avs4x264 with bugfixes and new support for x265 and x262.
avs4x26x - Modified avs4x264
http://tmod.nmm-hd.org/avs4x26x/
"avs4x26x.exe" --x26x-binary "64_8.exe" -o "output.mkv" "script.avs"
Document says default binary is "x264_64", but I rename mine.

It's recently updated and pretty much stable.

[foxyshadis]

Boulder, what are your system specs? Maybe it's a specific issue with an instruction set, or the video card if you use DGDecodeNV.

[Boulder]

I'm running an i5-4670K with a GeForce 520-based videocard.

With that simple script, I was able to get the artifacts also when inputting the script directly in a 32-bit x264 so the next step must be eliminating DGDecodeNV out of the equation.

[Boulder]

I remuxed the transport stream to mkv and tried FFVideoSource. It seems that the artifacts are gone, however the output from x264 is not the exact same every time (based on the final bitrate). I don't know if this is due to the nature of FFVideoSource or x264 threading. The next thing I'm going to do is to experiment with various nVidia driver versions.

[huhn]

x264 with multi threading doesn't have a bit identical output so this should be your issue.

[Boulder]

Yes, I recall reading that sometime. The difference in bitrate was extremely small, less than one kbps. When the artifacts appeared, the difference was quite a bit larger ;)

The display driver versions don't affect the output so it looks like I'm going to have to live with it and use RequestLinear if anyone else is not able to reproduce it.

[Seedmanc]

Just my 2c here, I remember experiencing artifacting with DG-related stuff as well, especially with its NV-enabled part, it never works right, so I would vote for it being the cause.

As for identical output, I thought it's only non-identical if you're using --non-deterministic in arguments for x264, isn't it so? Maybe the gui/other program you're using to launch x264 adds it, just an assumption.

[GMJCZP]

I just did another test, this time using frfun3d and hung.
I had to code as suggested Boulder.
I do not know whether to use RequestLinear (), but do not know exactly how it is used, and if prevent more crashes. :confused:

@Boulder
Regarding your problem, I do not know about piping, but you know exactly which version of x264.exe are you using? This way someone could look into a repository and try to recreate your case (in CLI mode, maybe).
It is normal that the bitrate increases when appears such artifacts.

[l33tmeatwad]

Created a standalone installer that fully sets up AviSynth with the MT version included.

<LINK REMOVE DUE TO BEING OUTDATED>

Edit: Figured I would also point out I changed the "Play" option to open in MPC-HC and I fixed the problem where in windows 8 it won't properly display the icon in Add/Remove programs on x64 systems because of how it writes to the registry. It includes AVSTP in the installer as well.

Edit: Link removed, now outdated.

[Xor]

How to verify if MT is enabled?

On Os WinServer2012 X64 i have intalled dll on "syswow64" with last Megui_2513, on cpu 6/12 core (E5-1650) but rendering/encoding time appears the same as single core cpu.

[LigH]

Simply exchanging the DLL does not magically use more threads. You will have to set up multithreading manually by using SetMTMode in the script (MeGUI won't do that for you) because multithreading filters in AviSynth is not really trivial, different filters require different modes or may corrupt the video, even crash; this function will not be available if you did not exchange the DLL correctly.

You can check if an MT mode was set using GetMTMode. It will return different integer values depending on its bool parameter 'threads':

current_mode = GetMTMode(false)
num_threads = GetMTMode(true)
debug_text = "Current MT mode = "+String(current_mode)+" using "+String(num_threads)+" threads"
BlankClip().Subtitle(debug_text)


When no MT mode was initialized using SetMTMode(), I get a current mode -1 returned. If I did not have an MT version of AviSynth installed, I would instead get an error that this function is not defined.

[LigH]

Dear SEt.

Now that there is AVS 2.6.0 RC 1 [150114] (https://sourceforge.net/projects/avisynth2/files/AviSynth_Alpha_Releases/AVS%202.6.0%20RC%201%20%5B150114%5D/), I hope there will also be an MT DLL available again...

[SEt]

It will be, but a bit later.

[LigH]

:thanks: in advance...

[Ajvar]

Yes, will keep refreshing this page once a day;)

[SEt]

Updated to current CVS. Also updated build environment to VS2013.

Note: It would be much easier to build these if at least non-MT changes would be merged upstream but no one cares, sigh...

[Boulder]

Thanks as always!

[Groucho2004]

Thanks for the new build.

Also updated build environment to VS2013.
However, that makes it incompatible with WinXP. I think there is a compatibility option for XP.

Or is the drop of XP intended?

[LigH]

http://cosgan.de/images/smilie/froehlich/e035.gif (http://www.cosgan.de/smilie.php) Hooray!

[SEt]

Have I said anywhere that XP is dropped? Then it's still supported.

[Groucho2004]

Have I said anywhere that XP is dropped? Then it's still supported.
See here (http://stackoverflow.com/questions/19724029/windows-xp-and-windows-server-2003-support-in-visual-studio-2013). Your new DLL does not work on XP.

[Taurus]

Thanks SEt for the build :thanks:

I can confirm the MT version is not working on WinXP 32bit.
The "normal" version 2.6.0 RC1 is doing fine.
The 2013 redist is installed (verified)
I dont care too much caus I'm normally working on Win7 Pro 64bit or Win 8.1.
But for compatibility reasons we should have a look on this.

[SEt]

Redownload – I've tweaked compilation a bit, should be ok now.

My builds never require any redists, so you don't need to worry about them.

[Groucho2004]

Redownload – I've tweaked compilation a bit, should be ok now.
Works now on XP, thank you.

[LigH]

Is there any additional change to log, apart from the AviSynth development?

[l33tmeatwad]

Standalone installer for the latest MT build.

Download (https://www.mediafire.com/?kofmx4apcw0qccz)
(NSIS Script (https://mega.co.nz/#!aRlwRYJa!DWb0U6b72lhQm5Uq1g1Gp0nFXSHiBhJ0VH6wE2RauTE))

Note: I changed the "Play" option to open in MPC-HC and I fixed the problem where in windows 8 it won't properly display the icon in Add/Remove programs on x64 systems because of how it writes to the registry. It includes AVSTP in the installer as well.

[SEt]

Is there any additional change to log, apart from the AviSynth development?
Just current CVS + MT + the same old my minor patches.

Standalone installer for the latest MT build.
You can safely drop msvcp60.dll from there – it shouldn't be used by anything.

[Overdrive80]

Thanks, Sir.

[l33tmeatwad]

You can safely drop msvcp60.dll from there – it should not used by anything.Thanks for the tip, updated the installer.

[Sparktank]

:thanks: Liking 2015 already.

Thanks for all the hard work guys. :)

[GMJCZP]

Thank you SEt and l33tmeatwad, working in WXP!

[Ajvar]

Thank you SEt! Will gonna use it now!;)

[GillesH]

Hello Set
And a great thank you for this great work.
Just a question :
What are the changes from the previous version.
Is it the same as version 2.6 RC1 with MT more ?
thank you

[LigH]

@ GillesH:

Already asked:

Just current CVS + MT + the same old my minor patches.

More or less all changes are in the base project of AviSynth.

[bilditup1]

Hmm, avs development not dead then? Good news...
(and thanks SeT!)

[Pat357]

Since I updated my Avisynth.dll from an older 2.6.0.3 MT version to the newest one (2015-01-25), I get an error when I use Seesaw :
AVSMeter 1.9.4.0 (x86) by Groucho2004
AviSynth 2.60, build:Jan 25 2015 [17:03:53] (2.6.0.5) (MT)

Script error: Invalid arguments to function "sharpen2"
(k:\programs\AviSynth 2.5\special filters\SeeSaw_2006.01.02\SeeSaw_.avs, line 74)
(K:\film\Safe-House-2012\Safe_10a.avs, line 43)

Line 43 in my script is the actual call for "SeeSaw (a,b, NRlimit=3, NRlimit2=4, Sstr=1.5, Slimit=5, Spower=4, Sdamplo=5, Szp=16, bias=40)"
Line 74 in the SeeSaw_.avsi points the arguments of a function sharpen2(). See attachment for this SeeSaw_.avsi (txt format) or here http://avisynth.nl/images/SeeSaw.avs

Output from FFmpeg :
[avisynth @ 02920280] Script error: Invalid arguments to function "sharpen2"
(k:\programs\AviSynth 2.5\special filters\SeeSaw_2006.01.02\SeeSaw_.avs, line 74)
(\Safe_10a.avs, line 43)
\Safe_10a.avs: Unknown error occurred

About SeeSaw and download the Seesaw.avsi script : http://avisynth.nl/index.php/SeeSaw
Download Seesaw.avsi : http://avisynth.nl/images/SeeSaw.avs

My script looks like this :

......
LoadPlugin("...\plugins\RemoveGrainSSE2.dll")
LoadPlugin("...\plugins\RepairSSE2.dll")
LoadPlugin(...\plugins\RSharpenSSE2.dll")
LoadPlugin("...\Vaguedenoiser0351\VagueDenoiser.dll")
Loadplugin("..\degrainmedian_20061008\degrainmedian.dll")
LoadPlugin("path...\mt_masktools-26.dll")
Import("path .....\SeeSaw_2006.01.02\SeeSaw_.avsi")

FFmpegSource2("Safe.mkv",vtrack=-1,atrack=-1, fpsnum=24000, fpsden=1001, width=720, height=304, colorspace="YV12", threads=1)
a = last
b=a.degrainmedian(mode=2).VagueDenoiser(threshold=0.9, method=1, nsteps=6, chromaT=0.9)
SeeSaw(a,b, NRlimit=3, NRlimit2=4, Sstr=1.5, Slimit=5, Spower=4, Sdamplo=5, Szp=16, bias=40)
## SeeSaw(a,b, NRlimit=3, NRlimit2=4, Sstr=1.5, Slimit=5)

The error seems to point to a function ""sharpen2" inside the "Seesaw.avsi" .

If I use less parameters for SeeSaw by replacing the line :
SeeSaw(a,b, NRlimit=3, NRlimit2=4, Sstr=1.5, Slimit=5, Spower=4, Sdamplo=5, Szp=16, bias=40)
by this
SeeSaw(a,b, NRlimit=3, NRlimit2=4, Sstr=1.5, Slimit=5)
the script works and produces nice output.

Attached is the "SeeSaw.avsi" SeeSaw v0.3e (02 Jan 2006) as txt file.

Any idea why the newer Avisynth MT versions give an error, while the previous older 2.6.0.3 that I 've used did not ?
Is this a problem with the parameters itself or maybe with the Seesaw script ?

[SEt]

Does official version http://forum.doom9.org/showthread.php?t=171668 work or produce the same error? If error is still present - it's non-MT issue and should be reported to the official thread.