AviSynth+ thread Vol.2

[magnetite]

Okay, here's the avsinfo log and the Avisynth+ 3.6.0 setup log.

[qyot27]

Originally Posted by magnetite

Had similar issues when trying out the 3.5.2 install of Avisynth+.

The problem is on MeGUI's side. They hardcode each individual AVISYNTH_INTERFACE_VERSION to specific routines in their homespun AviSynthWrapper.dll thing and any substantive version bumps to the AviSynth API will therefore make MeGUI choke on itself.

This is what gdb tells us about what MeGUI is doing:

(gdb) r
Starting program: /e/Documents/MeGUI-2913-64/MeGUI.exe
[New Thread 10080.0x15f0]
[New Thread 10080.0x1a0c]
[New Thread 10080.0x2e14]
[New Thread 10080.0x1b8c]
[New Thread 10080.0x2884]
[New Thread 10080.0x2944]
[New Thread 10080.0x2b1c]
[New Thread 10080.0x1ca0]
[New Thread 10080.0x6bc]
[New Thread 10080.0x228c]
[Thread 10080.0x228c exited with code 0]
[New Thread 10080.0x93c]
[New Thread 10080.0x1518]
[New Thread 10080.0x720]
[New Thread 10080.0x2964]

Thread 1 received signal SIGSEGV, Segmentation fault.
0x00007ffc08765020 in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
(gdb) bt
#0  0x00007ffc08765020 in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
#1  0x00007ffc0876542e in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
#2  0x00007ffc08765b39 in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
#3  0x00007ffc08a26efe in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
#4  0x00007ffc08765494 in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
#5  0x00007ffc08765b39 in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
#6  0x00007ffc0872ce1b in avs_get_read_ptr_p () from /e/Programs/AviSynth+/AviSynth64.dll
#7  0x0000000180002543 in dimzon_avs_init_2 () from /e/Documents/MeGUI-2913-64/AvisynthWrapper.DLL
#8  0x00007ffbacc8ea90 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(gdb)

The exact part of avisynth_c.cpp it's choking on (at line 314):
https://github.com/AviSynth/AviSynth...nth_c.cpp#L311

[pinterf]

Originally Posted by feisty2

I was randomly browsing avisynth's filter sdk and the invoke API doesn't seem very elegant, it is currently

auto args = std::array{ AVSValue{ arg1 }, AVSValue{ arg2 }, ... };
auto result = env->Invoke("Filter", AVSValue{ args.data(), args.size() });

it could be much prettier using the following syntax

auto result = env["Filter"](arg1, arg2, ...);

or even better with named function call

auto result = env["Filter"]("param1", arg1, "param2", arg2, ...);

function call with arbitrary arguments could be implemented by variadic templates, simple example.
also see its implementation in vsfilterscript

Interesting ideas, they are a bit abstract for our present thinking.
I wish that was the only task to simplify. Instead there are many other opportunities to make the code a bit cleaner. E.g. removing all checks for 16 byte frame aligment in Avisynth (no unaligned frames allowed), and there are still other ancient techniques there which are sometimes workarounds for a VS2005 behaviour.

[pinterf]

Originally Posted by qyot27

AviSynth+ 3.6.0 has been released.

Thanks, it's a milestone.

Known issues
- Due to interface extensions, some plugins relying on old IScriptEnvironment2 no longer work (=crash)
- CPP 2.5 plugins which are using "Invoke" will probably have problems.
- MEGUI (?)

Rebuild list
- KNLMeansCL (fails because using env2->SetFilterMTMode instead of cache hints - see my mods in source)
I hope this link is temporary, I don't want to hijack this project.

- chikuzen's plugins rebuilt by Asd-g
https://github.com/Asd-g?tab=repositories

- GScript
See download link in Groucho2004's topic

[pinterf]

I'd like to say a big thanks to Nekopanda. Some years ago he started an independent fork of Avisynth+ for his needs (you may know it as CUDA fork), which I wasn't even aware of for a long time, how serious changes were incorporated in that version.

He made many fixes and enhancements to the core.

- Multithreading fixes (the infamous ScriptClip issues)
(originally I only wanted to cherry-pick from Nekopanda fork to solve this problem, but the changes were not independent from the core changes he made as well so after spending weeks on resolving conflicts with our existing stuff, finally I had to integrate almost everything from there)
- language extension: function objects and variables. In interface: PFunction
- fine tune cache system, set optional caching strategy
- multiple Prefetch with new parameters
- 'escaped' string literals starting with e prefix e.g. e"Hello world\r\n"
- filter graph export

[Boulder]

Here's an example of a script which starts pounding on the GPU at 100%. I've tried changing the prefetch value, but I've not found a good value which would put the CPU (12c/24t) to work at 90-100%. In Vapoursynth, a very similar script uses 95-100% of the CPU and is much faster. GPU usage stays below 10% almost all the time.

DGSource("potter_stone.dgi", ct=280, cb=280, cl=0, cr=0) # UHD source

c2 = convertbits(bits=16)
c2blur = c2.blur(0.2)
prefilt = convertbits(bits=10)

w = prefilt.width()
h = prefilt.height()
prefilt = prefilt.removegrain(12, 12).gaussresize(w, h, 0, 0, w+0.0001, h+0.0001, p=2).mergeluma(prefilt, 0.1)

sharp_luma = c2.sharpen(0.6)
sharp_chroma = c2.sharpen(0.2)
sharp = sharp_luma.mergechroma(sharp_chroma)

superanalyse = prefilt.msuper(pel=2, hpad=16, vpad=16, sharp=2, rfilter=4)
supermdg = sharp.msuper(pel=2, hpad=16, vpad=16, levels=1, sharp=2, rfilter=4)

fv1 = manalyse(superanalyse, isb=false, delta=1, blksize=64, overlap=32, search=5, searchparam=8, pelsearch=8, truemotion=false, dct=5, mt=false)
bv1 = manalyse(superanalyse, isb=false, delta=1, blksize=64, overlap=32, search=5, searchparam=8, pelsearch=8, truemotion=false, dct=5, mt=false)
fv1 = mrecalculate(superanalyse, fv1, thsad=100, blksize=32, overlap=16, search=5, searchparam=6, truemotion=false, dct=5, mt=false)
bv1 = mrecalculate(superanalyse, bv1, thsad=100, blksize=32, overlap=16, search=5, searchparam=6, truemotion=false, dct=5, mt=false)
fv1 = mrecalculate(superanalyse, fv1, thsad=100, blksize=16, overlap=8, search=5, searchparam=6, truemotion=false, dct=5, mt=false)
bv1 = mrecalculate(superanalyse, bv1, thsad=100, blksize=16, overlap=8, search=5, searchparam=6, truemotion=false, dct=5, mt=false)

fv1scaled = fv1.mscalevect(bits=16)
bv1scaled = bv1.mscalevect(bits=16)

c2blur.mdegrain1(supermdg, bv1scaled, fv1scaled, thsad=200, thsadc=200, plane=4, limit=255, limitc=255, thscd1=200, thscd2=70)

Prefetch(24)

[pinterf]

And a list about changes that affect future plugins.
Both serious and not so serious plugin writers have to keep in mind.

So just some thoughts.

[Frame properties]
AviSynth+ now have frame property support which was imported from VapourSynth. Thanks to Myrsloik.

Note that this is only the framework, a possibility, the 0th step.
We'll write on it later, documentation is lagging here - but it should appear on a refreshed Filter SDK.

Until then, even if you don't use them, you have to know some facts.

Frame properties are passed along and inherited with the frames in the filter graph.

Inheritance is broken if a filter does not behave in a frame-property friendly way. There is not problem when a plugin is using MakeWriteable. But when using NewVideoFrame which accepts only a format input (VideoInfo) the chain is broken. There is no frame property source for this empty frame. Avisynth core cannot guess it. But now we have a new IScriptEnvironment function "NewVideoFrameP" with the option of specifying the property source which will be copied to the newly created frame as well.

[AviSynth+ headers]

https://github.com/AviSynth/AviSynth...s_core/include

cpp: avisynth.h (and the affected items under /avs)
c: avisynth_c.h

Avisynth+ Interface V8
- cpp interface: IScriptEnvironment was extended
- c interface: functions added to IScriptEnvironment are available as well
- former IScriptEnvironment2 items were moved to IScriptEnvironment (memory pool allocation, free, query of internal core properties, etc..)
- frame property support

[OS, compiler and architecture dependency]
This topic will probably require years.

Firstly you have a little help from Avisynth headers

#include "avs/config.h"

will set you some useful defines you can work with and act upon them (compiler flavours: MSVC, clang, GCC; OS flavours, Windows, POSIX, BSD; machine architectures: Intel, ARM)

At the present state most Avisynth plugins were coded with Microsoft Visual C++. Sometimes it's even difficult to port old sources to the actual MSVC syntax (and I'm not talking about C++17, but even having a valid C++11 syntax).
Then it will fail to build with clang, then it fails with gcc. And we are still on Windows.

Moving to POSIX introduces newer problems (case sensitivity, loading external DLLs such as fftw3, file system usage)

And when all this works you'll recognize that there are ARM machines and your code is full with Intel SIMD parts mixed into the C.

So it will require a big change in coding style and thinking.

VapourSynth is a "bit" ahead of us.

[magnetite]

I passed a note along to the MeGUI developers at Sourceforge about the issue, as well as I posted something in the MeGUI bug thread here.

[tebasuna51]

If MeGUI is broken I think also BeHappy, because use also a special AviSynthWrapper.dll

I read than also many plugins can't work with the new version.

When make a new version must be backward compatible, please begin with AviSynth 4.0, or a new fork, because the changes are too big.

If you plan to do that big changes remember my old suggestion of change the audio property NumChannels with MaskChannels (the NumChannels can be obtained from MaskChannels and the audio are now well defined).

[pinterf]

Originally Posted by tebasuna51

I read than also many plugins can't work with the new version.

List them here, we'll try to solve their problems.

Originally Posted by tebasuna51

When make a new version must be backward compatible, please begin with AviSynth 4.0, or a new fork, because the changes are too big.

Some of the (temporarily) broken plugins violated this remark placed in avisynth.h:

"   Note to plugin authors: The interface in IScriptEnvironment2 is
      preliminary / under construction / only for testing / non-final etc.!
      As long as you see this note here, IScriptEnvironment2 might still change,
      in which case your plugin WILL break. This also means that you are welcome
      to test it and give your feedback about any ideas, improvements, or issues
      you might have."

[tebasuna51]

Originally Posted by pinterf

List them here, we'll try to solve their problems.

The problems must be solved before launch a new version.

We need another set of plugins?

EDIT: maybe a new set called Avs* instead Avs+

[pinterf]

In general you'll need no new set of plugins.

But when you find a plugin which fails, post here or report the issue on github or report to the maintainer of the specific plugin.

[qyot27]

Originally Posted by tebasuna51

If MeGUI is broken I think also BeHappy, because use also a special AviSynthWrapper.dll

AFAICT, it just re-uses the same .dll from MeGUI. Which from the SVN history also previously had to be updated to support interface version 6. And for when AviSynth+ added high bit depth (eventually counted as interface version 7 for all of about a month and half).

From a cursory glance at the sources of the wrapper, I can't even tell why it would be throwing errors on avs_get_read_ptr_p specifically - it doesn't even seem to use the C interface at all. It also seems to be trying to use the init function intended for 2.5.7 instead of the one that was extended for interface version 6 and the high bit depth and MT features Plus introduced.

So all that needs to happen is that AviSynthWrapper.dll needs to be updated to be aware that version 8 exists. If there are any particulars in the API it has to compensate for, then it can do so at the same time in the version 8 loading function.

I read than also many plugins can't work with the new version.

Like pinterf pointed out, it should be only plugins that ignored the warning that the new-to-avsplus IScriptEnvironment2 wasn't stable and was subject to change that got hit by that particular issue.

Unless you were referring to the frame property support, which is simply that old plugins that don't support it will probably interfere with new plugins that do support it, not that the plugins break or cause the script to fail (well, unless the script was relying on the frame properties, maybe? But that's still the new feature ceasing to work, not the old stuff failing). To be honest, the new frame property and in-script array stuff is a little over my head.

When make a new version must be backward compatible, please begin with AviSynth 4.0, or a new fork, because the changes are too big.

That's precisely why it was bumped to 3.6 instead of continuing the 3.5.x series. Remember, AviSynth 2.6 was not perfectly backward compatible with 2.5, either. Plugins broke during the development of 2.6 and client programs (read: FFmpeg, x264) that depended on 2.5 having code baked into its headers broke when it was cleaned up in 2.6 and moved out of the headers, forcing them to either awkwardly try to support both, or to drop support of 2.5 outright.

The major.minor versions in AviSynth (in total, not just Plus) have always been mostly just symbolic. They aren't semantic versions, although since we now support more than just Windows, I am trying to ensure that the third version indicates just bugfixes or changes that don't impact any kind of compatibility. The version that really matters for compatibility checking is AVISYNTH_INTERFACE_VERSION, which was bumped when it changed to signal as much.

Note: the essential information that one actually needs to get and use from the AviSynth(+) API actually did not change here, as evidenced by the fact that a build of FFmpeg from December 30th, using headers that predate native Linux support¹ and still referred to themselves as interface version 6, can still load 3.6.0 and operate correctly.

¹predated 3.4.0, actually; the last time the compat/ headers in FFmpeg were updated (before getting removed in the wake of 3.5.0 being available on more than just Windows) was in May of 2019.

[Kurtnoise]

Hi,

I can concur about AvisynthWrapper stuff. It just needs to be recompiled using #8 as interface.

btw, Im trying to compile ffmpeg w/ avisynth support but I'm getting this error :

C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h: In function 'avs_load_library':
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1450:41: error: macro "AVSC_LOAD_FUNC" passed 2 arguments, but takes just 1
 1450 |   AVSC_LOAD_FUNC(avs_is_444, avs_is_yv24);
      |                                         ^
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1325: note: macro "AVSC_LOAD_FUNC" defined here
 1325 | #define AVSC_LOAD_FUNC(name) {\
      |
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1450:3: error: 'AVSC_LOAD_FUNC' undeclared (first use in this function)
 1450 |   AVSC_LOAD_FUNC(avs_is_444, avs_is_yv24);
      |   ^~~~~~~~~~~~~~
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1450:3: note: each undeclared identifier is reported only once for each function it appears in
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1451:41: error: macro "AVSC_LOAD_FUNC" passed 2 arguments, but takes just 1
 1451 |   AVSC_LOAD_FUNC(avs_is_422, avs_is_yv16);
      |                                         ^
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1325: note: macro "AVSC_LOAD_FUNC" defined here
 1325 | #define AVSC_LOAD_FUNC(name) {\
      |
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1452:41: error: macro "AVSC_LOAD_FUNC" passed 2 arguments, but takes just 1
 1452 |   AVSC_LOAD_FUNC(avs_is_420, avs_is_yv12);
      |                                         ^
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1325: note: macro "AVSC_LOAD_FUNC" defined here
 1325 | #define AVSC_LOAD_FUNC(name) {\
      |
C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h:1453:37: error: macro "AVSC_LOAD_FUNC" passed 2 arguments, but takes just 1
 1453 |   AVSC_LOAD_FUNC(avs_is_y, avs_is_y8);

something wrong within headers ?

[Groucho2004]

Originally Posted by Boulder

Here's an example of a script which starts pounding on the GPU at 100%. I've tried changing the prefetch value, but I've not found a good value which would put the CPU (12c/24t) to work at 90-100%. In Vapoursynth, a very similar script uses 95-100% of the CPU and is much faster. GPU usage stays below 10% almost all the time.

No idea why your GPU load is so high, I can't reproduce that. Have you tried L-Smash source filter with GPU support?

[pinterf]

Originally Posted by Kurtnoise

Hi,

I can concur about AvisynthWrapper stuff. It just needs to be recompiled using #8 as interface.

btw, Im trying to compile ffmpeg w/ avisynth support but I'm getting this error :
[code]C:/Users/Lionel/Downloads/mabs-master/local/include/avisynth/avisynth_c.h: In function 'avs_load_library':

Thanks, fixed on git.

[Boulder]

Originally Posted by Groucho2004

No idea why your GPU load is so high, I can't reproduce that. Have you tried L-Smash source filter with GPU support?

Just tried with LWLibavVideoSource, dead slow too. GPU usage is around 40-50% and the whole process less than 1 fps as CPU usage was in the low 20s. This was without cropping so the frame size was 3840x2160.

[manolito]

Not happy with AVS+ 3.6...

(I use the 32-bit version exclusively)

So far all my old plugins from the pre-modernization effort times seem to work, which is a welcome surprise. But the new MT process handling does not play nice with my older 32-bit StaxRip version (last 32-bit stable version 1.1.90).

Whenever an AVS filter in Staxrip is added or modified, the filter will be called to test it, and after a successful test the process will be terminated and released from memory. With AVS+ 3.6.0 this does not work any more. My source filter is DSS2Mod with LAV Filters, and now I get at least 3 instances of the LAV splitter and LAV Video source filter. This slows down the encoding by about 1 fps, and I think that it is unnecessary.

This is the script created by StaxRip:

LoadPlugin("C:\Program Files (x86)\StaxRip\Applications\AviSynth plugins\Decomb\Decomb.dll")
DSS2("D:\Jurassic World.mkv", preroll=15)
FDecimate(25)
Crop(0,0, -Width % 4,-Height % 4)
ColorMatrix(source=0,dest=2)
RequestLinear(rlim=50, clim=50)
ConvertToYV12()
Spline36Resize(704,396)
# Insert Loc parameter here:
Loc="20,14,-588,-330"
InpaintDelogo(mask="d:\logomask.bmp", Loc=Loc, Mode="Inpaint", Turbo=0)
RequestLinear(rlim=50, clim=50)
FineSharp()
avstp_set_threads(1)
Prefetch(4)
Trim(6151,172368)

For my needs this latest version 3.6 has nothing to offer, I am going back to 3.51. And generally I agree with Tebasuna:

If you want to push new versions without much testing compatibility with established and popular encoder GUIs then please create a new fork. You can make more radical changes then like drop 32-bit support and reject older plugins. And maybe some dev will agree to make a few bugfix updates to the older (traditional, maybe retro) 3.5x fork.


Just my 2 cents...
manolito

[pinterf]

Originally Posted by manolito

If you want to push new versions without much testing compatibility with established and popular encoder GUIs then please create a new fork.

??? Are you aware how many developers are working on Avisynth+ and how many remains when they make a new fork?

[real.finder]

since I am not with or against new fork but if it will be new fork in the end, then maybe better call it AvxSynth+ (since there are old AvxSynth) because it work in Linux and so