Avisynth+ plugin modernization efforts

[tormento]

Still, an important question: what are the most common uses of the filters this package?
Get a look at SMDegrain on Doom10, however AFAIK the quantile part.

[cretindesalpes]

You can replace Quantile used in the modified MinBlur function by any 5x5 and 7x7 spatial median filter.

[tormento]

You can replace Quantile used in the modified MinBlur function by any 5x5 and 7x7 spatial median filter.

Such as?

You know I am an evergreen newbie ;)

[cretindesalpes]

medianblur for example.

[tormento]

\ : (r==2) ? mt_makediff(clp,clp.MedianBlur(2,2*medf,2*medf), U=uv2,V=uv2)
\ : mt_makediff(clp,clp.MedianBlur(3,3*medf,3*medf), U=uv2,V=uv2)
Tried: medf is not defined.

[tormento]

That's bacause you only copy-pasted these two lines, and did not compare the entire function to see there is any other difference.
You are right. :o

[mawen1250]

A small problem in MedianBlur2 0.94:
When setting -255 for radius, the plane is set to 0 instead of 255.

[TurboPascal7]

Okay, due to popular demand of exactly zero people, new test version of SangNom2 (https://dl.dropboxusercontent.com/u/54253260/avs/mt/SangNom2%2B.dll).
Unlike the previous one, this version will use internal threading when frame-level threading in avs+ is disabled (no prefetch(x) call). In this aspect it should be same or a bit more efficient than the stable version. As usual, avs+ is required.
If nothing bad happens, this will become the next "stable" version.

A small problem in MedianBlur2 0.94:
When setting -255 for radius, the plane is set to 0 instead of 255.
Will be fixed in the next build (whenever that happens). Thanks for the report.

[tormento]

Okay, due to popular demand of exactly zero people, new test version of SangNom2 (https://dl.dropboxusercontent.com/u/54253260/avs/mt/SangNom2%2B.dll).
Quantile is still waiting you ;)

[TurboPascal7]

Quantile is still waiting you ;)
As multiple people already told you, it's possible to replace it with MedianBlur in most cases. I won't touch it.

[jpsdr]

Okay, due to popular demand of exactly zero people, new test version of SangNom2 (https://dl.dropboxusercontent.com/u/54253260/avs/mt/SangNom2%2B.dll).
Unlike the previous one, this version will use internal threading when frame-level threading in avs+ is disabled (no prefetch(x) call).
Is it possible to have a link to the source ?
I only want to see how it's done to check if frame-level threading is enabled or not.

Thanks.

[TurboPascal7]

Is it possible to have a link to the source ?
I only want to see how it's done to check if frame-level threading is enabled or not.

Thanks.
Source code is always linked on the first page. Here's a link (https://github.com/AviSynth/SangNom2/tree/avs-mt) to the branch.

[ultim]

Source code is always linked on the first page. Here's a link (https://github.com/AviSynth/SangNom2/tree/avs-mt) to the branch.

The API will only be frozen when AviSynth+MT is released as stable. Publishing plugins that use anything from IScriptEnvironment2 earlier than that is at your user's risk if it breaks.

[mark0077]

What has to happen for plugins like SVP to work with avisynth+. Do the SVP developers explicitly add support? Do you think it would increase performance for a plugin like svp

[TurboPascal7]

Okay, here's something a bit more fun for you to play.

CLExpr (https://github.com/tp7/CLExpr) filter - avs expr filter, implemented in OpenCL.
Provided interface is almost identical to that of masktools2. Functions are called cl_expr, cl_exprxy and cl_exprxyz (try to guess which masktools functions they map to). In theory, you can use all expressions and functions that worked with masktools. Offx/offy/w/h parameters aren't implemented - if people are interested in those, I could add them of course.

16-bit support is provided with the common lsb parameter. Set it to true to process stacked 16-bit clips.

This plugin is highly experimental and could easily kill your GPU driver like it's the most natural thing to do. Drivers usually can restart themselves but I warned you.

There's very little point in threading this plugin.

Why expr?
For most common uses on 8-bit clips this filter will be slower than masktools (especially the b branch in this thread). Unless you have a very good video card/motherboard and a very bad CPU. On-board GPUs might probably provide a bit better performance, I never tried. Dunno about performance on CPU either, right now it searches only for GPU devices.
But you can process 16-bit clips with it, which you obviously can't with mt_lutxy and mt_lutxyz.

Performance is mostly limited by memory transfer and hardly depends on complexity of the expression you pass to it. For example on my system cl_expr("x log log log log log",u=3,v=3) has the same performance as cl_expr("x 1 +",u=3,v=3). No dup/swap functions provided.

Why OpenCL?
Two reasons:
1. It was very fun.
2. It's a lot easier to implement with OpenCL than with regular C/C++ (vsynth). Basically it works by calling mt_infix() on the expression and feeding that to OpenCL compilation engine. Done.

That said, my experience with OpenCL is limited to ~10 hours, so I'm probably doing something very dumb in there. Please feel free to tell me how to improve memory transfer rates, device detection etc.

Binary
Here (https://dl.dropboxusercontent.com/u/54253260/avs/test/CLExpr.dll). Only x86 version for now, but it works on x64 without problems.

EDIT: reuploaded the binary because NVIDIA apparently doesn't like long/ulong types and e.g. (ulong)(round(3.0f)) is suddenly equal to zero, while (uint)(round(3.0f)) is 3.

[Mystery Keeper]

Drivers usually can restart themselves but I warned you.Only after Windows 7. On XP overloaded (caught in eternal loop, for example) video driver will cause major hanging.

[Rumbah]

EDIT: reuploaded the binary because NVIDIA apparently doesn't like long/ulong types and e.g. (ulong)(round(3.0f)) is suddenly equal to zero, while (uint)(round(3.0f)) is 3.

Yeah, if I remember correctly rounding in a shader does not have to mean rounding to the next integer. It's something that the hardware can decide. I remember problems because a nvidia card always rounded to the next even integer.

[SEt]

TurboPascal7, I haven't looked at code or filter at all, but maybe instead of (uint)(round(3.0f)) you actually wanted something like convert_uchar_sat_rte(3.0f) ?

[TurboPascal7]

TurboPascal7, I haven't looked at code or filter at all, but maybe instead of (uint)(round(3.0f)) you actually wanted something like convert_uchar_sat_rte(3.0f) ?
Yeah that would work (except uint instead of uchar). I ended up using size_t and ptrdiff_t though and they work but don't provide the manual convert functions.

[SEt]

It should work with uint, ulong or any other standard OpenCL type, even vector ones.
As for converting floating point value to size_t... is a very strange operation. Also, I think they "work" because they are actually "uint" for 32-bit code and the bug you encountered was converting to ulong.

[TurboPascal7]

It should work with uint, ulong or any other standard OpenCL type, even vector ones.
As for converting floating point value to size_t... is a very strange operation. Also, I think they "work" because they are actually "uint" for 32-bit code and the bug you encountered was converting to ulong.
Yeah of course it should work, it just doesn't. I blame NVIDIA for that. Never tried it on AMD cards.

And yes, size_t most likely works because it's uint, I never actually checked the behaviour. I will probably change it to uint later because getting values outside of uint range on any stage of expression calculation is almost impossible in real world.

Converting is required for doing stuff like bitshifts and other binary operations supported by masktools.

[SEt]

In terms of correctness I recommend testing also against CPU implementations, especially Intel one as something totally different. Not just to get peace of the mind that it is NV bug, but because different implementations forgive some deviations from the standard that you might be used to after some time coding for specific implementation and don't even notice (like mixing pointers to different memory types on NV or unaligned shared memory access on AMD).

[DeathAngelBR]

I'm getting this error with x86 masktools2.dll and avs+ installed.

http://i.imgur.com/7qpU021.png

LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\removegrain\RemoveGrainSSE2.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\removegrain\RepairSSE2.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\removegrain\RSharpenSSE2.dll")
#LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\mt_masktools-26.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\masktools2.dll")
#LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\mvtools2.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\AddGrainC.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\dither.dll")
Import("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\dither.avs")

LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\LSMASHSource.dll")

src = "I:\BDMV Symphogear G Vol. 1\BDMV\STREAM\00003.m2ts"
src2 = "I:\BDMV Symphogear G Vol. 1\BDMV\STREAM\00009.m2ts"

video_ep01 = LWLibavVideoSource(src,cache=true)
video_ep02preview = LWLibavVideoSource(src2,cache=true)

video_ep01 ++ video_ep02preview

#super = MSuper(pel=2, sharp=1)
#backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
#forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
#MDegrain1(super, backward_vec1,forward_vec1,thSAD=64)
#dither_convert_8_to_16().dither_resize16(1280,720,kernel="spline64").smoothgrad().gradfun3(lsb_in=true,lsb=true,smode=2,mask=3).dither_out()

[TurboPascal7]

Try loading it with some other program maybe? It's a bit hard to extract much useful info from that error message.

[DeathAngelBR]

Try loading it with some other program maybe? It's a bit hard to extract much useful info from that error message.

Same error (messed up symbols) with megui.

avs4x264mod CLI also crashes.

"avs4x264mod.exe" --seek-mode safe --x264-binary "x264tMod10bit.exe" --preset veryslow --tune animation --crf 13.0 --deblock -1:-1
--keyint 240 --qcomp 0.8 --fade-compensate 0.8 --aq-mode 3 --aq-strength 0.85 --merange 16 --psy-rd 0.60:0.00 --no-dct-decimate
--no-fast-pskip --colorprim bt709 --transfer bt709 --colormatrix bt709 --fps 24000/1001 --force-cfr --input-res 1280x720 --frames 35030
--input-depth 16 --opts 3 --output "E:\SYMPHOGEAR\Symphogear G EP01 video tmod.264" "SymphogearG01.avs"

[TurboPascal7]

Well, I'm no wizard. If you have the appropriate runtime installed (if you installed avs+ using the installer) there's no reason why it shouldn't work for you.

[DeathAngelBR]

Well, I'm no wizard. If you have the appropriate runtime installed (if you installed avs+ using the installer) there's no reason why it shouldn't work for you.

I *might* have ruled out what's causing it.

Commenting the removegrain dll lines makes the error go away.

Does avs+ have built-in multithreading? avstp.dll also causes that error.

VirtualDub 1.10.4 is also crashing with avs+:

An out-of-bounds memory access (access violation) occurred in module 'kernel32'...
...writing address 0AB749B4.

RgTools.dll = same error as with removegrain dlls.

[cretindesalpes]

avstp.dll also causes that error.
avstp.dll is not a plug-in and must not be loaded with LoadPlugin().

[Lenchik]

Put it in the usual AviSynth 2.5\plugins\ directory, or load it manually with LoadPlugin("path\avstp.dll") in your script.
Put it in the usual AviSynth 2.5\plugins\ directory, or load it manually with LoadPlugin("path\avstp.dll") in your script.
Update manuals maybe?

[cretindesalpes]

Uh no just discard my previous message, I was completely absent-minded… Yes, you can load it with LoadPlugin().

[aldix]

Sorry if that's the wrong spot, but how to set the threads in a script with tfm().tdecimate()?

I put just SetFilterMTMode("DGDecode_mpeg2source", 3) at front & Prefetch(4) in the end and it locked in at 3+GB RAM.

What must I do?

[turbojet]

Prefetch(4) defines 4 threads, if you have the avsi that set's filtermode then you are using 4 threads with tfm().tdecimate() otherwise you'll have to setfiltermtmode.

If this is all that's in your script, in my limited tests it's slower and uses more cpu with mt then without unfortunately. TFM is suspected.

[aldix]

Here's the relevant script snippet:


SetFilterMTMode("DGDecode_mpeg2source", 3)
MPEG2Source("c:\users\admin\desktop\stuff.d2v")

tfm().tdecimate().colormatrix()

Crop(0, 6, -0, -0)

# denoise

# sharpen

# resize

Prefetch(4)


What am I doing wrong here? Source is TS made indexed with DGSource.

AvspMod (the version allowed for Avisynth+ MT) hangs at ~4GB memory and program shuts down with "run out of memory" error.

x264 encoding (with Mulder's GUI) stops with "potential deadlock". Doesn't even start.

[Evil_Burrito]

This could be a bug or I could be doing something wrong.

When I use the test test version of masktools2+ with the maa2 script (v0.41), the encoded output (tried x264, utvideo, and magicyuv) has white over the anti-aliased areas. The white doesn't appear when using the test version with avspmod or when I encode using the pre-release version from the first post.

[bxyhxyh]

When I use Film Restore package by videoFred, there appears "Clense doesn't have a named argument cache" error.
Can you implement it for compatibility?

[TurboPascal7]

When I use Film Restore package by videoFred, there appears "Clense doesn't have a named argument cache" error.
Can you implement it for compatibility?

You can try creating a wrapper function that would have all parameters like clense plus the "cache" parameter, which would be simply ignored. I'm not interested in adding a useless parameter to the plugin.

[GMJCZP]

I thank you deeply for your crusade, TurboPascal7.

I don't know if this is the correct thread to ask other plugins updates. Otherwise, if you could tell me how to do. :)

[DARKAN]

Help! I checkmate plugin fails to boot. Gives an error:
http://i58.fastpic.ru/big/2015/0326/7d/16ce8fe6310a7d1b4429fc06f3b15c7d.png
In the script prescribed this command:
checkmate(100).DFMDeRainbow(maskthresh=3,mask=false, interlaced=false)
Resetting the system did not help. Using Windows 7 Enterprise x64. Probably the thing in a processor that I use. I have a processor AMD, in others, who say that they have the plug-worth processor Intel.
This plugin checkmate x86 from this topic does not work, but the plugin works checkmate here (http://forum.doom9.org/showpost.php?p=1571520&postcount=20).

[TurboPascal7]

It appears I accidentally used SSE4.1 there while checking only for SSE2. The offending intrinsic is _mm_min_epu16. If anybody is willing to verify that it's save to replace it with _mm_min_epi16 or keep SSE2 compatibility in some other way it'd be nice.

Until then - yeah, some older AMD CPUs are out of luck.

EDIT: innocenat just built a version with replaced intrinsics (https://dl.dropboxusercontent.com/u/54253260/avs/checkmate.dll), if it works properly for you I'll add it to github releases.

[DARKAN]

innocenat just built a version with replaced intrinsics (https://dl.dropboxusercontent.com/u/54253260/avs/checkmate.dll), if it works properly for you I'll add it to github releases.
Yes, it works. I have a processor AMD A6-3650 APU with Radeon (tm) HD Graphics (~ 2593). Thank you! :)

[DeathAngelBR]

Well, I'm no wizard. If you have the appropriate runtime installed (if you installed avs+ using the installer) there's no reason why it shouldn't work for you.

Ok this is odd...

I gave rgtools another try - uninstalled/reinstalled visual studio and avisynth -, but it still refused to work... until I moved it to the autoload plugin (default) folder. Any reason it won't load from, let's say, avisynth\plugins\extrasorwhatever\ folder?

[TurboPascal7]

No it should load (manually with LoadPlugin, of course) just fine from any folder whatsoever - in fact for debugging purposes I usually loaded libraries from some obscure debug folders. Does it show an error or anything?

[DeathAngelBR]

No it should load (manually with LoadPlugin, of course) just fine from any folder whatsoever - in fact for debugging purposes I usually loaded libraries from some obscure debug folders. Does it show an error or anything?

Yea, it's that one >>>> http://forum.doom9.org/showthread.php?p=1673689#post1673689

I guess I should format considering I can't encode anything 720p/1080p without random crashes and reboots.

[Wilbert]

I'm trying the updated MaskTools (link: http://github.com/tp7/masktools/releases/download/b1/masktools2-25-x86.zip). The script


(...)
mt_xxpand() # or mt_expand()


results in 'there is no function named mt_xxpand' (or mt_expand). Is it still available or is it called differently?

[Groucho2004]

I'm trying the updated MaskTools (link: http://github.com/tp7/masktools/releases/download/b1/masktools2-25-x86.zip). The script


(...)
mt_xxpand() # or mt_expand()


results in 'there is no function named mt_xxpand' (or mt_expand). Is it still available or is it called differently?
This may not be related to your problem but why are you using the 2.5.8 version? There is a version for 2.6.

Do you have the correct MS runtimes installed (msv*110.dll)?

[Reel.Deel]

mt_expand() / mt_inpand() work for me, mt_xxpand does not exist. I tried the 2.5 version just to make sure but I'm also wondering the same thing as Groucho2004.

[Groucho2004]

mt_expand() / mt_inpand() work for me, mt_xxpand does not exist. I tried the 2.5 version just to make sure but I'm also wondering the same thing as Groucho2004.
mt_expand also works for me, "mt_xxpand_multi.avsi" is part of the Dither package. Not sure if Wilbert was referring to that.

[Wilbert]

I posted the wrong link ;( I'm using this one: http://github.com/tp7/masktools/releases/download/b1/masktools2-x86.zip. But you are right i needed to install the runtimes libs for 2012. It works now thanks.

[StainlessS]

mt_expand(u=3,v=3) works for me using Wilbert 2nd link (avs v2.6 official, v2.58 will not see v2.6 dll).

EDIT: Oh good, you got it working :)

[jpsdr]

Ported version of AutoYUY2, based on neuron2's pluggin, updated to API v2.6 and x64 version, so, not working with avisynth 2.5.8.
Sources are here (https://github.com/jpsdr/AutoYUY2).
Binaries are here (https://github.com/jpsdr/AutoYUY2/releases/download/20150902/AutoYUY2_20150902.7z).

New version, with YV16 ouput support.

EDIT : Corretion of a typo/bug.