Avisynth+

[tormento]

Avisynth.dll compiled for Debug has no corruption.
You mean you will release a fixed version soon? :cool:

[pinterf]

You mean you will release a fixed version soon? :cool:
Asked the Avisynth Plus Customer Service Center (APCSC) and they said: no.
They also said that the problem was a nice one, but they wished the solution process to their enemies.
Finally they said: yesofcoursewhynot.

[tormento]

Asked the Avisynth Plus Customer Service Center (APCSC) and they said: no.
They also said that the problem was a nice one, but they wished the solution process to their enemies.
Finally they said: yesofcoursewhynot.



LOL... It took months from my first post but... Hey... At last...

[pinterf]

Unfortunately I cannot issue a Prefetch(100) command to myself

[MysteryX]

Quick question. What's the difference between having a filter run in MT=1 with a unique_lock in GetFrame vs having a filter run in MT=3?

In both cases, there is a single instance and the work is sequential. With MT=3, it's always the same thread calling instead of each request coming from a different thread. Besides that, what's the difference?

[tormento]

Unfortunately I cannot issue a Prefetch(100) command to myself

It's great to have you here XD

[ultim]

EDIT: There is a newer version available in this post (http://forum.doom9.org/showthread.php?p=1772944#post1772944).

I have compiled a new build of Avisynth+ for testing, you can find it here (http://avs-plus.net/builds).
(To download all files at once, there is a "Download" button at the top right corner of the page)

This is r2003, and is built on top of the most up-to-date code for Avs+ as of now. It includes all the changes from Avisynth 2.6.1, all of pinterf's work, all code-related pull-requests sent to me, and some additional fixes from me. So it is pretty cutting edge ;) Let us know your experiences. Some notes to bear in mind:
- Important: This build breaks some plugins. If you are a regular user, you might not want this build. It it is for testing anyway.
- This was built with a new compiler (MSVS 2015 Update 3). You'll find corresponding runtimes that you must install first under the same link above. These will be included in the next installer.
- XP-support might not work, I cannot test it. Let me know and I'll try to correct it.
- ConvertStackedToNative/ConvertNativeToStacked have been renamed to ConvertHbdFromStacked/ConvertHbdToStacked (HBD = high bit-depth). These are transitional functions and exist to support old plugins which tried to hack around missing HBD support. We expect that these functions will not be needed in a year from now.
- The high bit-depth support is still provisional. Some important and useful functions for dealing with it are still missing.
- While testing, please keep an extra eye on the correctness of Resizers/Resamplers, mergers, and SoundTouch. Let us know if you see any problems.

Cheers.

[chainik_svp]

Quick question. What's the difference between having a filter run in MT=1 with a unique_lock in GetFrame vs having a filter run in MT=3?

order of the requested frames?
the filter working in mt=3 knows that it processes frames one-by-one so it's free to use some info from the previous frame(s) to process current one
a simple motion blur filter can be a good example

[ultim]

order of the requested frames?

When Prefetch() is used, the order of the frames requested in plugins is not guaranteed in any of the MT modes.

[ultim]

order of the requested frames?
the filter working in mt=3 knows that it processes frames one-by-one so it's free to use some info from the previous frame(s) to process current one
a simple motion blur filter can be a good example

mt=3 only guarantees that the code will not be executed in parallel, but frame requests might still arrive non-sequentially. For filters where this is a problem, Prefetch must not be enabled, but such filters are faulty.

Filters should request each frame they need every time they receive a request and must not try to "remember" frames on their own. This way filters do not depend on a specific order of requests. The caches in Avisynth will make sure that processing is still efficient by avoiding the re-calculation of frames that have been already requested in the recent past.

[qyot27]

- XP-support might not work, I cannot test it. Let me know and I'll try to correct it.
If the last time the VS2015 and XP issues cropped up is still true, it'll need /Zc:threadSafeInit- to work on XP. Dunno if that's the 'correct' thing to do, though.

[qyot27]

Also, for anyone wanting to experiment with native 16-bit output, here are FFmpeg and mpv builds that support opening 16-bit scripts:
http://www.mediafire.com/download/k7e2tb1yavu05tr/ffmpeg_r80789_avshbd.7z
http://www.mediafire.com/download/rsfawx6jjpd07tu/mpv_r43454_avshbd.7z

ConvertHbdFromStacked/ConvertHbdToStacked

are bad function names. You ask users to remember very strange word "Hdb", that is unknown in avisynth world and even in video software world. many-many users who use old plugins will have remember problems, and mistakes.

ConverFromStacked/ConvertToStacked is a very clean name.
'HBD' is a known abbreviation of 'High Bit Depth'. Although I suppose we could just tell people it stands for 'Here be dragons'. :P

[Reel.Deel]

I have compiled a new build of Avisynth+ for testing, you can find it here (http://avs-plus.net/builds).

With this simple script I get an error in VDub (64-bit works though):

ColorBarsHD()
Dither_convert_8_to_16()

Avisynth open failure:
System exception - Access Violation
(M:\AviSynth+ r2003 test\16-bit resizers.avs, line 2)


Edit: also the resizers are messed up, if I comment out the resizers then the output is correct, otherwise I get this (https://s33.postimg.org/yzcd6oldp/avisynth_hbd_test.png?dl=0).


ColorBarsHD(1280, 720)
Dither_convert_8_to_16()
ConvertHbdFromStacked()
Spline64Resize(400,300)
Crop(9,7,-9,-7)
FlipHorizontal().FlipVertical()
TurnLeft().TurnRight().Turn180()
PointResize(3200,2400)
GrayScale()
BilinearResize(1600,1200)
ConvertHbdToStacked()

[Groucho2004]

It is called HDR in a computer graphic world.
High Dynamic Range != High Bit Depth.

[qyot27]

With this simple script I get an error in VDub (64-bit works though):

ColorBarsHD()
Dither_convert_8_to_16()
Ditto here (32-bit Windows 10, Silvermont Z3735F). It seems to happen on any function after the first.

Version() by itself works. Version().ConvertToYV12() throws an Access Violation.

[Groucho2004]

Ditto here (32-bit Windows 10, Silvermont Z3735F). It seems to happen on any function after the first.

Version() by itself works. Version().ConvertToYV12() throws an Access Violation.
Ha, I thought it was a problem with WinXP. Getting clip properties or a simple colorbars() worked, anything further gave me an access violation in avisynth.dll with AVSMeter.
The build could still be incompatible with WinXP though...

[ultim]

High Dynamic Range != High Bit Depth.

Well said. High bit Depth is the ability to use more than 8-bits to represent finer color gradations, and (probably much more importantly) to present more correct processing results due to less errors in in-between operations.

HBD is basically a technical prerequisite of HDR. In addition to HBD, HDR allows to process color and brightness values outside of your limited display range. This usually means 1) using HBD to represent values outside of the valid displayable range, and 2) at the end some kind of tone mapping algorithm to bring it back down to valid pixel values. With HBD Avisynth+ can now do the first of part of HDR, for the second some kind of plugin to perform the tone mapping would be required.

About the naming: "Native" and "Stacked" are even less standardized than "HBD". AFAIK, "native" and "stacked" were invented in this community to give method names to an Avisynth-specific hack.

XP and crashes: Looking into it. I could run an MCTemporalDenoise script and some resizers before I uploaded.

[Groucho2004]

XP and crashes: Looking into it.Not just XP, see qyot27's post.

[qyot27]

About the naming: "Native" and "Stacked" are even less standardized than "HBD". AFAIK, "native" and "stacked" were invented in this community to give method names to an Avisynth-specific hack.
While we're on the subject, are there any plans to also have a convert function to handle the Interleaved16 format? That's actually the one I usually use when dithering up with f3kdb.

I mean, it's not like it's difficult to just switch from using output_mode=2 to using output_mode=1, but having the option would be nice.

[ultim]

I cannot reproduce any of the crashes. Tried 32-bits in VDub, AvsPmod, and AVSmeter, none of them crashed with any of the three scripts posted earlier. Did you guys install the new runtime?
I can however reproduce the corrupted output from the resizer using Reel.Deel's script. Also, I am adding the /Zc switch for XP.

[Reel.Deel]

Did you guys install the new runtime?


Yes, 1st thing I did.

Edit: the filters that have been updated to support HBD allow 16/32-bit, is there a way to specify what bit depth to use?

[qyot27]

Yup, I'd had VS2015 installed for a long while too, but switched back to VS2013 once pinterf really got going and I tried testing 64-bit because Wine didn't like VS2015 binaries. But I did go ahead and install the runtime again, just in case.

And I can also reproduce with my own VS2013 builds, so VS2015 miscompiling it isn't a factor here.

[ultim]

Fixes for the resizer corruption (it only affected some HBD formats) and XP support are on GitHub. I will publish a new build tomorrow, assuming I get a handle on the crashes, because I haven't been able to reproduce them yet. Help appreciated in making them reproducible.

[Reel.Deel]

Help appreciated in making them reproducible.

Found something odd, If use BlankClip().TurnLeft() it works, but if I use it like the example below it gives me the access violation error.


BlankClip()
TurnLeft()

Error from AVSMeter:

Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
Module: Unknown module
Address: 0x00000001

[Groucho2004]

Found something odd, If use BlankClip().TurnLeft() it works, but if I use it like the example below it gives me the access violation error.


BlankClip()
TurnLeft()

Error from AVSMeter:
Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
Module: Unknown module
Address: 0x00000001

The "Unknown module" is probably avisynth.dll but access may be restricted. Do you have UAC enabled? Run AVSMeter as admin.

[Reel.Deel]

The "Unknown module" is probably avisynth.dll but access may be restricted. Do you have UAC enabled?

I have UAC disabled, turned it on but it gives the same error. I was using AVSMeter 2.2.8 , update to 2.2.9 and now it gives the "AVSMeter - Avisynth Script Testing has stopped working". Here's the details:

Problem signature:
Problem Event Name: BEX
Application Name: AVSMeter.exe
Application Version: 2.2.9.0
Application Timestamp: 5776dcf1
Fault Module Name: StackHash_0a9e
Fault Module Version: 0.0.0.0
Fault Module Timestamp: 00000000
Exception Offset: 00000001
Exception Code: c0000005
Exception Data: 00000008
OS Version: 6.1.7601.2.1.0.256.48
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789

Run AVSMeter as admin.

Without loading a script, if I right click AVSMeter.exe, run as admin I get the error above, same as without run as admin. 64-bit works fine.

Edit: works fine in Avs+ r1576 and Avs 2.6.1. Any other Avs+ version, same error. Tested all the way down to r1828.

[Groucho2004]

I have UAC disabled, turned it on but it gives the same error. I was using AVSMeter 2.2.8 , update to 2.2.9 and now it gives the "AVSMeter - Avisynth Script Testing has stopped working". Here's the details:
Fault Module Name: StackHash_0a9e
Haven't seen that before. I assume this only happens with this particular Avisynth version?

[Groucho2004]

works fine in Avs+ r1576 and Avs 2.6.1. Any other Avs+ version, same error. Tested all the way down to r1828.So this only happens with the latest version of AVSMeter?

[Reel.Deel]

So this only happens with the latest version of AVSMeter?

Correct.

[Groucho2004]

Without loading a script, if I right click AVSMeter.exe, run as admin I get the error above, same as without run as admin. 64-bit works fine.

Edit: works fine in Avs+ r1576 and Avs 2.6.1. Any other Avs+ version, same error. Tested all the way down to r1828.
Unfortunately, I can't reproduce this. Could you please try this test build (https://www.dropbox.com/s/y1ut2ofuap8yuoa/AVSMeter.7z?dl=0)?

[Reel.Deel]

Unfortunately, I can't reproduce this. Could you please try this test build (https://www.dropbox.com/s/y1ut2ofuap8yuoa/AVSMeter.7z?dl=0)?

No I get the same error as in post #1931, with or without a script.

[Groucho2004]

No I get the same error as in post #1931, with or without a script.
So, the same as 2.2.8, right?

[Reel.Deel]

So, the same as 2.2.8, right?

Yes. I reverted back to pinterf's r1858, run AVSMeter test build and now I get "bad array new length". Could this have something to do with updating to MS redistributable 2015 update 3? I see that update 2 is gone from the programs list.

[Groucho2004]

Yes. I reverted back to pinterf's r1858, run AVSMeter test build and now I get "bad array new length". Could this have something to do with updating to MS redistributable 2015 update 3? I see that update 2 is gone from the programs list.
No idea...

The difference between AVSMeter 2.2.8 and 2.2.9 is that the new version enumerates and checks the auto-load plugins right at the start and not just when using the "-avsinfo" switch. Maybe you have a dodgy plugin. You should try with an empty auto-load directory.

[Reel.Deel]

No idea...

The difference between AVSMeter 2.2.8 and 2.2.9 is that the new version enumerates and checks the auto-load plugins right at the start and not just when using the "-avsinfo" switch. Maybe you have a dodgy plugin. You should try with an empty auto-load directory.

I used r1858 and the plugins included with Avs+ r2003. I downgraded to what I had (core plugins from r1825 and pinterf's recent VDub filter) and now 32-bit works with v2.2.8 again. So it must be something with the distributed plugins in ultim's r2003.

Edit: using the updated plugins from r2003, if I remove DirectShowSource.dll then 32-bit r1858 works again.

[qyot27]

The only-first-function works thing is really weird. It does 'work' if you use the . syntax, but only for the first *two* functions, the third in the . syntax then fails with an Access Violation.

Bisecting it appears to point to the merge commit fcae803fbd. git log fcae803fbd^..fcae803fbd shows that it merges in some stuff from classic's CVS from very early on in the 2.6.1 cycle, which presents some difficulty in trying to figure out which one to try and revert (and moreover, why this merge would even do anything, since the file paths are different and it should have just been the history that was merged...right?). 10e4dc50a5b seems to be the last commit prior to the merge which works like expected.

[MysteryX]

Do you understand the more important problem? You enter hundreds of Avisynth users to use your own non-intuitive term.

I even cannot remember this HBR DRB BDR etc. It just a letter noise. For example "Stacked" is many years well know term here.

Please write code for real users, not robots or assembly programmers. It is the most common computer software problem for API developers. They create over-complicated stuff in cases where it is not required.
When we consider that these 2 functions exist for the sole purpose of supporting Stack16 format, and Stacked implies that we're working with HBD, "ConvertToStacked" and "ConvertFromStacked" work just fine.

Just like "ConvertToShader" and "ConvertFromShader" in AviSynthShader. I switched to different names and then came back to that. What format it converts to isn't part of function name, but is rather the "format" and "lsb" arguments.

[pinterf]

Uh, what a weekend!
While I was chasing Strava KOMs on my bike in the nice tailwind, ultim gave avisynth+ a tailwind, too :) I will look at all these merging things in the evening, but anyway, it was a good job!

[fAy01]

Uh, what a weekend!
While I was chasing Strava KOMs on my bike in the nice tailwind, ultim gave avisynth+ a tailwind, too :) I will look at all these merging things in the evening, but anyway, it was a good job!

Is there a point of having 4-5 different branches of avs+?

[MysteryX]

Is there a point of having 4-5 different branches of avs+?
Pinterf had to create a new branch for his new development.

Ultim has the official branch and is merging all the changes both from Pinterf and from v2.6.1 into it. That's a good thing.

Whether there's a point in having a v2.6.1 branch, an official AVS+ branch and Pinterf's dev branch is irrelevant; we have that. What matters is to merge it into an official build.

[Reel.Deel]

Maybe you have a dodgy plugin. You should try with an empty auto-load directory.

So I kept getting the same error as in post #1932 (http://forum.doom9.org/showthread.php?p=1772699#post1772699) with 32-bit r2003 and AVSMeter v2.2.9. I cleared out my auto-load directory. Added plugins until I got the error message again, it turns out the latest nnedi3 (v0.9.4.22). If I remove it, AVSMeter works as it should. Kinda odd that 64-bit nnedi3 works though.

Edit: removing nnedi3 from the auto-load folder also fixes the problem reported in post #1930 (http://forum.doom9.org/showthread.php?p=1772694#post1772694).

[Groucho2004]

So I kept getting the same error as in post #1932 (http://forum.doom9.org/showthread.php?p=1772699#post1772699) with 32-bit r2003 and AVSMeter v2.2.9. I cleared out my auto-load directory. Added plugins until I got the error message again, it turns out the latest nnedi3 (v0.9.4.22). If I remove it, AVSMeter works as it should. Kinda odd that 64-bit nnedi3 works though.

Edit: removing nnedi3 from the auto-load folder also fixes the problem reported in post #1930 (http://forum.doom9.org/showthread.php?p=1772694#post1772694).Thanks for looking into this. I don't use that version of nnedi3 since I don't need the additional color spaces. I guess that's why I could not reproduce it.

[qyot27]

Well, it resolved itself here once I replaced the core plugins in plugins+ with the ones built with r2005.

[qyot27]

Thinking about this, can the Access Violation error message be made more verbose or checks made more robust so that errant plugins that cause issues like this get reported in the message and can be dealt with?

[LigH]

One may even think of a debug build using Windows Debug Messages (https://msdn.microsoft.com/en-us/library/windows/desktop/aa363362(v=vs.85).aspx) readable with DebugView (https://technet.microsoft.com/en-us/sysinternals/debugview.aspx) or DebugView++ (https://debugviewpp.wordpress.com/2014/01/04/7/).

[MysteryX]

Thinking about this, can the Access Violation error message be made more verbose or checks made more robust so that errant plugins that cause issues like this get reported in the message and can be dealt with?
I don't even know how to locate such errors when debugging my own code. If there's a way, I'd love to hear about it.

[Groucho2004]

Thinking about this, can the Access Violation error message be made more verbose or checks made more robust so that errant plugins that cause issues like this get reported in the message and can be dealt with?
AVSMeter reports the module that caused the exception but that only works if Avisynth doesn't catch the exception first.
It should be simple to add this to the SEH code in Avisynth.

[ultim]

qyot, Reel, you two are kings! I checked the sources of nnedi3 after Reel found it was a possible cause, and realized what the problem is. The merge commit that qyot bisected reinforced my analysis. In short, nnedi3 needs to be recompiled at a minimum, and better, a small fix be made to it to avoid the problem in the future. EDIT: A proper fix needs to come from Avs+. See later posts.

Here is what happened:
1) In commit 2ee19354 of classic Avs (JEEB's repo), GetVarDef() was added to the IScriptEnv interface, and this was merged to Avs+ in the commit found by qyot.
2) The addition of GetVarDef is in general OK, because it maintains ABI for IScriptEnv, and while it breaks ABI for IScriptEnv2 (because it is a derived interface), it was merged anyway because IScriptEnv2 is not (yet) supposed to be used by external plugins. There is a big fat warning block above the interface definition that warns plugin developers, that the interface is not stable and should only be used for testing.
3) Now comes the fun part. nnedi3 ignored this warning, and in its newest version is relying on the unstable interface, which just got changed, and hence the plugin got broken.
4) War, famine, and epidemics, not to mention the wasted hours for the community in debugging this. Warnings are there for a reason. Once that warning gets remove, only then can you rely publicly on the interface.

This also explains why I couldn't reproduce this: I have an older version of nnedi3 that does not use IScriptEnv2.

It seems nnedi3 is using the unstable API to register its MT mode. The proper way to implement this functionality is instead of using IScriptEnv2, respond to the CACHE_GET_MTMODE query in the plugin's SetCacheHints() with the proper constant. Here (https://github.com/AviSynth/AviSynthPlus/blob/MT/avs_core/filters/field.cpp#L618) is an example of doing it right. This is future proof and is compatible with both classic Avs and Avs+.
EDIT: Ah darn, this functionality got disabled in Avs+ to fix related problems. So a "proper fix" is not that easy. See comment below.

A real fix in Avs+ is not trivial, but a new build will still be issued later tonight to at least fix the HBD resizers and XP compatibility.

[Groucho2004]

One may even think of a debug build using Windows Debug Messages (https://msdn.microsoft.com/en-us/library/windows/desktop/aa363362(v=vs.85).aspx) readable with DebugView (https://technet.microsoft.com/en-us/sysinternals/debugview.aspx) or DebugView++ (https://debugviewpp.wordpress.com/2014/01/04/7/).
This won't tell you much if you don't write the code for catching structured exceptions.

[MysteryX]

Quite a few plugin authors, including me, started using IScriptEnv2 to set the MT mode.

Is there a way to use IScriptEnv2 so that it doesn't crash if the interface changes?