Avisynth+

[macromizer]

Yes, because a pause of 5 months (two months if you count the pull from June) is in any way comparable to a 5+ year, prolonged development hell. /sarcasm

Think about it for a second: within a time span of 6 months (September 2013-March 2014), there were enough unique commits to AviSynth+ to outpace the entire development cycle of AviSynth 2.6 up to that point, and even a chunk of 2.5's. It's not unreasonable, especially with the volume of different things that were getting done (64-bit support, intrinsics, MT & caching, etc.), to expect some level of burn-out or slow down. And even granted that, the 'level' that avsplus is currently at is not in any way lagging behind 2.6 - which hasn't had any commits since September 2013 when 2.6a5 was released.

If this was towards me, I would definitely agree that the two are not analogous situations and I wasn't meaning to make it seem that. Avisynth+ with what ultim and others have done has been amazing in kickstarting Avisynth activity again and what they have done is pretty awesome. My point is that I just worry that if everyone else ends up dropping out before ultim comes back that it's just going to suffer the same fate as IanB driving Avisynth alone. It's just so easy to burn out if you're a one-man development band.

[innocenat]

Regarding Linux/OSX/GCC/LLVM port, qyot27 is very spot on. Even then, because of how ssse3/sse4.1 code are interleaved with sse2, it is even quite more work than other project. I have been thinking about this for quite a while, bur haven't start yet.

Ultim is alive and does reply to questions on IRC. If he has more free time I am quite sure he will continue developing. I also plan to continue developing, I just don't have enoigh free time and experience to handle the project myself.

[Seedmanc]

Well, at least Avisynth x64 works, the parts that were ported, that is.

Hm, but what else are we supposed to use instead of Avisynth if it's dying then?
I second this question. It can only die if it were to be replaced by something more effective at solving same tasks, but I can't imagine doing stuff without Avisynth, neither VDub nor Vegas-like editors fill its niche.

[qyot27]

If this was towards me, I would definitely agree that the two are not analogous situations and I wasn't meaning to make it seem that. Avisynth+ with what ultim and others have done has been amazing in kickstarting Avisynth activity again and what they have done is pretty awesome. My point is that I just worry that if everyone else ends up dropping out before ultim comes back that it's just going to suffer the same fate as IanB driving Avisynth alone. It's just so easy to burn out if you're a one-man development band.
No, it wasn't directed at you. It was directed at the 'there's not been an update in a month, so is it dead now? I guess I should move back to 2.6' types of responses from general users...because it started happening not *that* long into the slowdown in activity.

Even then, because of how ssse3/sse4.1 code are interleaved with sse2, it is even quite more work than other project.
Do you mean there's SSSE3/SSE4.1 stuff in the functions with _sse2 suffixes in their names? Or just that all the different functions are all packed into the same source file? I assume it's the latter.

The thing I was getting confused about were the functions suffixed with _ssex and _xsse. I wasn't sure where to put those when splitting the files in convert/.

[feisty2]

Hm, but what else are we supposed to use instead of Avisynth if it's dying then?

imho, avisynth itself might not be the best up till now, but there are so many magic filters and scripts, thats what makes it so irreplaceable

[innocenat]

Do you mean there's SSSE3/SSE4.1 stuff in the functions with _sse2 suffixes in their names? Or just that all the different functions are all packed into the same source file? I assume it's the latter.

The thing I was getting confused about were the functions suffixed with _ssex and _xsse. I wasn't sure where to put those when splitting the files in convert/.

The SSE3 lddqu, SSSE3 pshufb etc are injected by templated parameter into sse2 core function, to reduce code duplication.

I think the core and templated function should be put in #ifdef in header file and include into sse2/3/4 file, but I am not sure if this will work or not.

[foxyshadis]

Well, at least Avisynth x64 works, the parts that were ported, that is.


I second this question. It can only die if it were to be replaced by something more effective at solving same tasks, but I can't imagine doing stuff without Avisynth, neither VDub nor Vegas-like editors fill its niche.

ffmpeg's filtering is becoming much more capable, with a lot of ideas taken from Avisynth filters. Meanwhile, real-time filtering with multiple shaders is now easy during playback, even with low-end GPUs. I don't intend to stop using Avisynth anytime soon, but the writing is on the wall.

[TurboPascal7]

Hm, but what else are we supposed to use instead of Avisynth if it's dying then?

Nothing. Hth, bye.

No, seriously, you don't have to use anything instead of avisynth. Why would you do this? It works for what it was designed for and won't suddenly break tomorrow.

I'm just saying that the whole idea of extremely complex and powerful frameservers is kinda useless these days, when most video requires light processing, that can be done in real-time on playback, if any at all.

[Octo-puss]

I thought Avisynth was used for encoding movie rips somehow? Please don't laugh at me, I don't understand this stuff one bit. I just use MeGUI and it depends on the old single threaded Avisynth, which is painfully slow when using QTGMC (waiting 3 days to encode something on i7 is just.... uh).

[Groucho2004]

I just use MeGUI and it depends on the old single threaded Avisynth, which is painfully slow when using QTGMC (waiting 3 days to encode something on i7 is just.... uh).
I don't think megui is restricted to one Avisynth version. I'm sure you can use either SEt's MT or AVS+ MT.

3 days with an i7 is insane, even with single threaded processing. I'd like to see that script.

[Octo-puss]

I don't think megui is restricted to one Avisynth version. I'm sure you can use either SEt's MT or AVS+ MT.

3 days with an i7 is insane, even with single threaded processing. I'd like to see that script.

No, it's not, but the multithreaded version is horribly bugged and the whole process crashes after several hours unpredictably.
I haven't tried Avisynth+ as it's unfinished.

Yes, that's the downside of QTGMC. It takes a LONG time with single thread. That's why I was hoping Avisynth+ would replace Avisynth.

[Boulder]

Multithreading QTGMC scripts is tricky, sometimes it seems impossible to make it through the whole video without crashing.

As an alternative, you could try using the MVTools version by cretindesalpes (inside the Dither package). It is internally multithreaded and runs faster.

[macromizer]

No, it wasn't directed at you. It was directed at the 'there's not been an update in a month, so is it dead now? I guess I should move back to 2.6' types of responses from general users...because it started happening not *that* long into the slowdown in activity.

Okay. :)

Well after I finish up a couple other projects I've been meaning to finish for more than a year now, I'll look back into possibly doing some more OS X porting work. I'll definitely get in contact if I decide to end up working on it again.

[Groucho2004]

As an alternative, you could try using the MVTools version by cretindesalpes (inside the Dither package). It is internally multithreaded and runs faster.
Never tried this before, very nice speed-up!

[foxyshadis]

No, it's not, but the multithreaded version is horribly bugged and the whole process crashes after several hours unpredictably.
I haven't tried Avisynth+ as it's unfinished.

It's not set's MT or avs+ that's bugged (well, not entirely), it's actually mvtools itself. It's fairly buggy, but a lot of the bugs are only exposed during parallel processing, so you get most of these crashes on slow mvtools scripts where you need the speed the most of all.

It'd be a bit painful but quite possible to create a version of QTGMC that used avs threading for all filters but mvtools, where only the internally threaded version works correctly right now, if anyone is interested. It's past due for a new plugin pack with the improved avs+ versions of filters, too.

Myrsloik is wrestling with that one in his copious free time.

[jpsdr]

Yes, that's the downside of QTGMC. It takes a LONG time with single thread.

One trick is to split your file in several parts and process each part in the same time. You'll keep QTGMC single threaded, but take advantage of several CPU. The best would be that each filter used has internal MT, but it seems it's not the case.

[Dion]

Is the name of the plugin a secret ?

Nope its called SupTitle. It's for burning sup subtitles into a video.

[Elegant]

Any idea what can cause MeGUI or AvsPMod to hang when using AviSynth+? I'm using Windows 7 x64 and even loading a video using any method (DGDecode, FFMS2) in a script causes both programs to hang (only occurs with AviSynth+). Encoding is fine though but I'd prefer to actually see a preview >.>

[Groucho2004]

Any idea what can cause MeGUI or AvsPMod to hang when using AviSynth+? I'm using Windows 7 x64 and even loading a video using any method (DGDecode, FFMS2) in a script causes both programs to hang (only occurs with AviSynth+). Encoding is fine though but I'd prefer to actually see a preview >.>

Which version of AVS+ are you using? 64 or 32 Bit?
Be aware that 64 Bit Avisynth cannot load 32 Bit plugins (like DGDecode) and vice versa.
Define "hang". No error message?
Post your script.

[Elegant]

32 bit version; the 64 bit version shouldn't load with MeGUI nor AvsPMod (this is also version listed on the AviSynth+ website) as far as I know. You are right in assuming there is no error, it simply attempts to open and hangs on both programs. Script examples:

LoadPlugin("C:\Program Files (x86)\MeGUI\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("E:\Output\VTS_01_4.d2v", info=3)

LoadPlugin("C:\Program Files (x86)\MeGUI\tools\ffms\ffms2.dll")
FFVideoSource("L:\MOVIE\BDMV\STREAM\00508.m2ts", cachefile="E:\Output\Movie.ffindex", fpsnum=24000, fpsden=1001, threads=1)

Both cause the issue. Both plugins are 32 bit as that's what MeGUI uses.

[qyot27]

I can encode just fine using x264 but I would prefer the ability to see what my work will look like instead of encoding blind.
VirtualDub? mpv?


And for obviousness' sake, you did already run the video through DGIndex or ffmsindex, right? The .d2v/.ffindex files do exist, is what I'm saying.

[Elegant]

VirtualDub? mpv?


And for obviousness' sake, you did already run the video through DGIndex or ffmsindex, right? The .d2v/.ffindex files do exist, is what I'm saying.

I'm talking command line x264 it works just fine. Yes, I did run them through those 2, one is a DVD and the other is a Bluray. The .d2v/.ffindex were made before I installed AviSynth+ though but it should be unrelated. I can open the files just fine for previewing with 2.6.0a5 MT but AviSynth+ just doesn't play nice for some reason. I was hoping to take advantage of some of AviSynth+'s features once I could actually load a video but so far even running those import lines by themselves causes both programs to hang.

[qyot27]

And if you try opening the scripts in VirtualDub and mpv?

[Elegant]

Unsure of what mpv is but using VirtualDub 1.10.4 also causes it to hang on open. Mind you VirtualDub wasn't able to actually load the DGDecode plugin for that test case but for FFMS2 it hangs.

[qyot27]

Mind you VirtualDub wasn't able to actually load the DGDecode plugin for that test case but for FFMS2 it hangs.
VirtualDub doesn't load AviSynth plugins. Was there an error message when you tried opening the .d2v script?


mpv is a media player:
http://mpv.io

Like most/all mplayer variants, it's typically started from the command line:
mpv test.avs

If the script won't play, copy/paste the error messages here.



How are you switching between the 2.6 MT and AviSynth+ dlls?
How did you install AviSynth+? If it was from the .7z file, did you also install the MSVC 2012 Runtime?
Put DGDecode and FFMS2 in AviSynth+'s plugins folder and remove the LoadPlugin lines from the scripts - does that do anything?

[foxyshadis]

Try Process Explorer, as well. You can open up the process and look at the threads, where you might see which plugin is the one that isn't initializing properly. If it's not using any CPU, that probably means you hit a deadlock, though, which would be weird to see consistently. If all the cpu is in avisynth.dll itself, then you might have to get Visual Studio Express and compile it yourself with debugging on to find out why (ugh), since the Avisynth+ installers don't include debugging info.

[Elegant]

Found it, AvsFilterNet.dll is the trouble maker. I don't think I've used any plugins that have relied on it so I guess I don't really need it.

[asarian]

Use installer from the homepage. If classic AviSynth is already installed, you will be offered the choice of either upgrading or replacing it. In the first case you'll be able to switch back to classic AviSynth by simply uninstalling AviSynth+. Your existing plugins will be kept in both cases.

Hmm, any way I can use my official 32-bit 2.6.0 AviSynth *next* to the 64-bit version of this one? I recall from 2.5.8, that you could use both 32 and 64-bit versions next to each other.

[Groucho2004]

Hmm, any way I can use my official 32-bit 2.6.0 AviSynth *next* to the 64-bit version of this one?
Sure. Install both AVS+ versions (32 & 64) using the installer from avs-plus.net.
Replace the avisynth.dll and the devil.dll in SysWow64 with the files from this archive (https://www.dropbox.com/s/qxj7z43m4y05z9g/AVS260_ICL.zip?dl=0).

[asarian]

Sure. Install both AVS+ versions (32 & 64) using the installer from avs-plus.net.
Replace the avisynth.dll and the devil.dll in SysWow64 with the files from this archive (https://www.dropbox.com/s/qxj7z43m4y05z9g/AVS260_ICL.zip?dl=0).

Thx. :) That's a neat trick. It *will* require me to give up on the official AS, though, right? Reason I wanted to install a parallel 64-bit version, is so I can keeping using MT for the 32-bit version.

[Groucho2004]

It *will* require me to give up on the official AS, though, right? Reason I wanted to install a parallel 64-bit version, is so I can keeping using MT for the 32-bit version.
If you want SEt's 32 Bit MT version, just use that DLL instead of the one I put in the zip file.

Edit: The only "official" version is the one on Sourceforge. All other flavours (SEt's MT, my ICL build, AVS+) can be used and are probably just as stable in most cases.

[feisty2]

Thx. :) That's a neat trick. It *will* require me to give up on the official AS, though, right? Reason I wanted to install a parallel 64-bit version, is so I can keeping using MT for the 32-bit version.

both SEt's mt version and avs+ are improved (but not official) versions, the official avs 2.6 is IanB's version

[asarian]

If you want SEt's 32 Bit MT version, just use that DLL instead of the one I put in the zip file.

Thx again. :) Will do so.

[feisty2]

Thx. :) That's a neat trick. It *will* require me to give up on the official AS, though, right? Reason I wanted to install a parallel 64-bit version, is so I can keeping using MT for the 32-bit version.

u can enjoy mt benefits on avs x64 also, search "mp_pipeline" on forum, the tool gets u mt functions for both ia32 and x64 versions of avs

[asarian]

u can enjoy mt benefits on avs x64 also, search "mp_pipeline" on forum, the tool gets u mt functions for both ia32 and x64 versions of avs

That's great news! Thx! :)

[asarian]

If you want SEt's 32 Bit MT version, just use that DLL instead of the one I put in the zip file.

Edit: The only "official" version is the one on Sourceforge. All other flavours (SEt's MT, my ICL build, AVS+) can be used and are probably just as stable in most cases.

Hehe; since I was already using MT dll's, I guess I never really used to the official package to begin with. :o

[qyot27]

both SEt's mt version and avs+ are improved (but not official) versions, the official avs 2.6 is IanB's version
Technically, only SEt's would be a non-'official' version of 2.6, because it's not really diverged much from classic. AviSynth+ is a separate project that was forked from 2.6, but has diverged significantly to the point that it's just not accurate to label it an 'unofficial' version of 2.6. It'd be like saying GNU and Linux are an 'unofficial' version of BSD.

Although for that matter, anyone that takes a look at AviSynth's development history will see very quickly that things have always been a bit scattershot with the main development. The developers involved often change a lot between the different trees that sprouted up - the original BRG version, the patched versions in the 1.0 series after BRG left (various authors), the development on 2.0 and 2.5.0-2.5.5 (sh0dan, et al.), 2.5.6-2.5.8 and 2.6 (increasingly IanB) are all technically distinct dev branches - 2.0-2.6 simply had the benefit of a single main repository, and just changed lead developers. Had AVS3.0 not died back in 2007, it would have exhibited the same properties - the devs in charge of it were different too, and the SVN repo for it was separate from the CVS repo for 2.0-2.6.

2.5/2.6 only coalesced as 'official' because it became widely used and was updated consistently for a few years, not because the devs that worked on it have a steering committee that gets to decide what 'official' is over the entire AviSynth ecosystem, only in the context of their own branches. AviSynth+ is hardly a deviation from that, and has its own 'official' (or more accurately, 'stable') builds - currently 0.1 r1576, with 'unofficial' ('dev') builds from the main/MT branch, which is up to r1699.

[feisty2]

well, thx qyot27 :), got it, I never really dug much into avisynth history

[asarian]

Technically, only SEt's would be a non-'official' version of 2.6, because it's not really diverged much from classic. AviSynth+ is a separate project that was forked from 2.6, but has diverged significantly to the point that it's just not accurate to label it an 'unofficial' version of 2.6. It'd be like saying GNU and Linux are an 'unofficial' version of BSD.

Although for that matter, anyone that takes a look at AviSynth's development history will see very quickly that things have always been a bit scattershot with the main development. The developers involved often change a lot between the different trees that sprouted up - the original BRG version, the patched versions in the 1.0 series after BRG left (various authors), the development on 2.0 and 2.5.0-2.5.5 (sh0dan, et al.), 2.5.6-2.5.8 and 2.6 (increasingly IanB) are all technically distinct dev branches - 2.0-2.6 simply had the benefit of a single main repository, and just changed lead developers. Had AVS3.0 not died back in 2007, it would have exhibited the same properties - the devs in charge of it were different too, and the SVN repo for it was separate from the CVS repo for 2.0-2.6.

2.5/2.6 only coalesced as 'official' because it became widely used and was updated consistently for a few years, not because the devs that worked on it have a steering committee that gets to decide what 'official' is over the entire AviSynth ecosystem, only in the context of their own branches. AviSynth+ is hardly a deviation from that, and has its own 'official' (or more accurately, 'stable') builds - currently 0.1 r1576, with 'unofficial' ('dev') builds from the main/MT branch, which is up to r1699.

Learn something every day. :) Thx.

[chainik_svp]

Is it dead or alive? o_O
In other words, if I think I found a bug - should I try to fix it myself or post here and wait for support?

===
Short version of issue is: x64 MT version of AVS+ doesn't free memory so if I want to run some memory consuming script inside MPC-HC x64 + ffdshow x64 it will eat some amount of memory on each seek.


enter simple script into ffdshow:
SetFilterMTMode("",2)
SetFilterMTMode("ffdShow_source",3)
ffdShow_source()
BilinearResize(3000,1500)
BilinearResize(1920,1080)
Prefetch(10)


and then just check and uncheck "Avisynth" check box while looking at Task Manager memory graph

===
and it's not an issue with 32-bit version

[ryrynz]

Is it dead or alive? o_O
In other words, if I think I found a bug - should I try to fix it myself or post here and wait for support?


Pretty much dead. I wouldn't rely on anyone getting to this in a timely fashion, if you're able to work on it go for it IMO.

[innocenat]

Is it dead or alive? o_O
In other words, if I think I found a bug - should I try to fix it myself or post here and wait for support?

===
Short version of issue is: x64 MT version of AVS+ doesn't free memory so if I want to run some memory consuming script inside MPC-HC x64 + ffdshow x64 it will eat some amount of memory on each seek.


enter simple script into ffdshow:
SetFilterMTMode("",2)
SetFilterMTMode("ffdShow_source",3)
ffdShow_source()
BilinearResize(3000,1500)
BilinearResize(1920,1080)
Prefetch(10)


and then just check and uncheck "Avisynth" check box while looking at Task Manager memory graph

===
and it's not an issue with 32-bit version

Do you have revision number/where did you get the build? If it's resizer issue the I can look into it, but no promise since ultim isn't here to release it.

[chainik_svp]

It's not a resizer issue, BilinearResize() is used just as an example to illustrate it's not a some filter issue but AVS+ in general.
Still it can be not AVS+ x64 issue but ffdshow x64 but I prefer to think that memory allocated by AVS+ should be freed by AVS+.

where did you get the build?

latest sources from MT branch built by me :)

[chainik_svp]

[for every frame]
{
VideoFrame *frame = it->second;
if (frame->vfb->refcount == 0)
delete frame->vfb;

delete frame;
}

why "if (frame->vfb->refcount == 0)" ??
vfb->refcount decremented in VideoFrame destructor so it can be (and it IS) >0
and ~VideoFrame doesn't delete vfb but just releases it

=====
Fixed it, see https://github.com/AviSynth/AviSynthPlus/issues/49

Is there any chance to include this fix to the main branch and release a new official build?

[foxyshadis]

Ouch. That makes sense, now I know why I was getting out-of-memory when testing some script variations last month.

[chainik_svp]

this isn't all the truth :D

if the filter was instantiated - it'll never be deleted...

in other words if you'll write some filter doing something like this:

SomeFilter::SomeFilter()
{ buffer = new BYTE[1000000000]; }

SomeFilter::~SomeFilter()
{ delete[] buffer; }

than you'll end with 1 GB of memory stolen cause it'll never be freed :)

[qyot27]

My question is: why is this specific to 64-bit, though? The snippet doesn't look specialized to me (noting, of course, I have no real knowledge of what you have to do for 64-bit vs. 32-bit in the code itself).

What I'm getting at is, is there any chance that it's caused by being miscompiled? Does the behavior change with different compilers (VC2012 vs. VC2013 vs. ICL), or different optimization levels in the same compiler?

(although, yes, I recognize the fact that if it's being miscompiled there's a chance of it being from a legitimate bug that simply exposes it, and the bug needs fixing)

[chainik_svp]

My question is: why is this specific to 64-bit, though?

asked myself the very same question, still no answer :D
may be there's a difference in procedure of loading/unloading DLLs between x32 and x64 modes oO

What I'm getting at is, is there any chance that it's caused by being miscompiled?

not a chance, I can see from the code that some objects are NOT deleted by destructor

===
in fact these bugs are very uncommon
I mean the only chance to face them is to use AVS+ inside ffdshow - then you can load/unload AVS several times in the same process

[chainik_svp]

Just wasted all the Sunday to find out that ScriptEnvironment:: prefetcher is not deleted in destructor.
Which leads to all MT-enabled filter objects are not freed.

[mapg]

Hi,

Let me explain the issue.

Maximum riff index added by ffmpeg in the avi video container is 256, after that point a warning is shown as: "Invalid riff index 257 > 256", later ... "Invalid riff index 258 > 256" and so on ...

Well, when I encode using your Avisynth+ 64-bit and AVISource, when encoder (i.e: x264) reaches riff index 257, the image stays frozen from there to the end of video, namely, scenes stop until the end of video.

Therefore if you are encoding an uncompressed AVI video which is bigger than 256~260 GB (approximately), and the input format is a AVS script (using Avisynth+ and JUST the command: AVISource), at some point of time (when riff index > 256 is reached), the video is frozen, damaged after all.

This problem doesn't happen if I use Avisynth+ and tag DirectShowSource instead of AVISource.

I think that there are two options:

1) Avisynth+ team adds some fix to AVISource to avoid this issue.
2) ffmpeg team allows adding riff indexes bigger than 256 to AVIs.

What do you think?

Let me suggest that you should fix it at your end too.

Kindest regards,
Mapg

BTW: I didn't test if this is also happening in Avisynth+ 32-bit