Avisynth+

[jpsdr]

I've made some tests, and i'm screwed, as i was afraid. As i suspected, the fact that Prefetch must be at the end of the script made me unable to get the value using GetProperty(AEP_FILTERCHAIN_THREADS) inside AvisynthPluginInit3, or, in worst case, inside Create_xxxx. I'll probably get the information only within GetFrame, but it's too late.
Using the value of the size of the internal threadpool is not interesting because it may lead to create too much threads, so, for now, i'll use the workaround to add a prefetch parameter to my plugins (the purpose is to try to speed-up things with script using prefecth).

[qyot27]

Why do I have to convert my encode from 10bit to 8bit? It worked fine before. Please help :)
You mean it worked fine before high bit depth support was added to FFMS2's AviSynth interface and it downscaled everything to 8bit on its own?

Report the issue in the avspmod or YATTA threads. That's where the problems are, not here.

[real.finder]

if I have 2 mvtools2.dll one that work with 2.5 in non plus folder and one that work with 2.6 in plugins+ I note that the one in the non plus folder is used in encode! is this normal?

[Groucho2004]

if I have 2 mvtools2.dll one that work with 2.5 in non plus folder and one that work with 2.6 in plugins+ I note that the one in the non plus folder is used in encode! is this normal?It should load from the "+" directory first. Run "AVSMeter -avsinfo -log" and post the created log file.

[real.finder]

It should load from the "+" directory first.

Isn't better to have it opposite? first non plus then plus

in AVSMeter -avsinfo show the non plus first then plus one!

whether in avsdeps.log or in avsinfo.log

[jpsdr]

Isn't better to have it opposite? first non plus then plus


I think it's the way avs+ is designed.
The plugins+ directory is "only" for the avs+ system plugins, the plugins directory is for the user plugins. "Theoricaly", you shouldn't put plugins in the plugins+ directory. As it's where are supposed to be the system plugins, it's (from my point of view) normal to have this directory read first.

[Groucho2004]

I think it's the way avs+ is designed.
The plugins+ directory is "only" for the avs+ system plugins, the plugins directory is for the user plugins. "Theoricaly", you shouldn't put plugins in the plugins+ directory. As it's where are supposed to be the system plugins, it's (from my point of view) normal to have this directory read first.
Here's (http://avisynth.nl/index.php/AviSynth%2B#AviSynth.2B.27s_Plugin_Autoloader) how the whole thing works in AVS+. In addition to the plugins that ship with AVS+ you're supposed to put the ones that won't work with classic Avisynth for some reason into the "+" directories.

[tormento]

I usually put 2.6 plugins in + dir and 2.5 in normal.

[real.finder]

Here's (http://avisynth.nl/index.php/AviSynth%2B#AviSynth.2B.27s_Plugin_Autoloader) how the whole thing works in AVS+. In addition to the plugins that ship with AVS+ you're supposed to put the ones that won't work with classic Avisynth for some reason into the "+" directories.

I think this will be better

PluginDir2_5 in Software/Avisynth in HKEY_LOCAL_MACHINE
PluginDir2_5 in Software/Avisynth in HKEY_CURRENT_USER
PluginDir+ in Software/Avisynth in HKEY_LOCAL_MACHINE
PluginDir+ in Software/Avisynth in HKEY_CURRENT_USER

[jpsdr]

Question about the float format. Is there some kind of "rule/limitation", or are value totaly free, and it's normal to have a pixel with value for exemple of 10e-8 or +1e10 or -5000000 ?

[Groucho2004]

Question about the float format. Is there some kind of "rule/limitation", or are value totaly free, and it's normal to have a pixel with value for exemple of 10e-8 or +1e10 or -5000000 ?Do you mean the limits of the float data type? They are defined in "float.h":
#define FLT_DIG 6 /* # of decimal digits of precision */
#define FLT_EPSILON 1.192092896e-07F /* smallest such that 1.0+FLT_EPSILON != 1.0 */
#define FLT_GUARD 0
#define FLT_MANT_DIG 24 /* # of bits in mantissa */
#define FLT_MAX 3.402823466e+38F /* max value */
#define FLT_MAX_10_EXP 38 /* max decimal exponent */
#define FLT_MAX_EXP 128 /* max binary exponent */
#define FLT_MIN 1.175494351e-38F /* min positive value */
#define FLT_MIN_10_EXP (-37) /* min decimal exponent */
#define FLT_MIN_EXP (-125) /* min binary exponent */
#define FLT_NORMALIZE 0
#define FLT_RADIX 2 /* exponent radix */
#define FLT_ROUNDS 1 /* addition rounding: near */

[jpsdr]

No, i mean the limit of the value a pixel can have in a float format.
I should take a look at the avisynth code which convert float to 8bits (if i found it), i think i'll have the answer i'm looking for.

[pinterf]

Float is 0.0 to 1.0 for all planes. Usually not clamped. But the range may change if it is against silent standards, e.g. -0.5 to 0.5 for U and V planes. Now there is no danger of change unless zillions of plugins appear supporting it

[feisty2]

No, i mean the limit of the value a pixel can have in a float format.
I should take a look at the avisynth code which convert float to 8bits (if i found it), i think i'll have the answer i'm looking for.

there SHOULD be no clamping for floating point samples, to make sure that all intermediate data are mathematically correct..
the only exception would be a floating point mask clip, which should be clamped to 0.0 - 1.0

[Myrsloik]

Float is 0.0 to 1.0 for all planes. Usually not clamped. But the range may change if it is against silent standards, e.g. -0.5 to 0.5 for U and V planes. Now there is no danger of change unless zillions of plugins appear supporting it

Why u use offset float 4 uv? Whyyyyyy?

[LigH]

Because U (Cb) and V (Cr) are defined as "difference away from neutral gray" towards blue/yellow or red/green. So it's quite intuitive. A default chrominance in short integer is unfortunately not 0 (between -128 and 127, as signed short int) but 128; it's unsigned short int (Pascal type: Byte).

[feisty2]

Because U (Cb) and V (Cr) are defined as "difference away from neutral gray" towards blue/yellow or red/green. So it's quite intuitive. A default chrominance in short integer is unfortunately not 0 (between -128 and 127, as signed short int) but 128; it's unsigned short int (Pascal type: Byte).

unsigned short int = uint16_t
unsigned char = uint8_t

[LigH]

Oops ... still have Pascal in my mind. :o

[jpsdr]

Float is 0.0 to 1.0 for all planes.
Ok, i'll work with that. I'm trying to see if i can change nnedi3 to support extended format. So far, results are not good, i have to continue to work/test, but i'm affraid that neural network is trained for range value of 0-255. Every input out of this range will produce improper result. For now, my next step will be to put all values in the 0-255 range, but in float format. So if input is float, multiply by 255, if input is integer more than 8 bits, convert to float by 255*(value/valuemax) (all in float of course).
This will be my next thing to try.

[feisty2]

Ok, i'll work with that. I'm trying to see if i can change nnedi3 to support extended format. So far, results are not good, i have to continue to work/test, but i'm affraid that neural network is trained for range value of 0-255. Every input out of this range will produce improper result. For now, my next step will be to put all values in the 0-255 range, but in float format. So if input is float, multiply by 255, if input is integer more than 8 bits, convert to float by 255*(value/valuemax) (all in float of course).
This will be my next thing to try.

floating point support has been added to vaporsynth nnedi3 since, forever. (I'm actually the one who did it, and the original vaporsynth nnedi3 author then merged my floating point patches)

just do some copy-paste work from vaporsynth nnedi3 and ur good to go

[pinterf]

Why u use offset float 4 uv? Whyyyyyy?
To tell the truth I didnt know. I had no experience or prejudice about these expert formats. That's why I mentioned 'now' because you guys who have the knowledge, can propose the right way, it's not too late.

[jpsdr]

@feisty2
I'm doing the 16bits for now, i'll see for the floats latter.
@pinterf
I don't understand. I thought you said all the planes in floats were 0.0-1.0, so the U/V planes are also the same, or have i missed something (because of some english part i may have misunderstood) ?

[pinterf]

Yes. All planes are 0-1.0 at the moment. But it seems that this is unusual.

[Myrsloik]

Yes. All planes are 0-1.0 at the moment. But it seems that this is unusual.

Use -0.5 to 0.5 for UV and it'll match VS. Will make compatibility so much easier...

[jpsdr]

@feisty2
I have a question, just to ask if it's normal :
In evalFunc_0 you use prescreener >=2 only if pixelsize=1, wich is normal, but the way the tests are made, if you ask prescreener=2 with pixelsize>1, you fall in "no prescreener". Isn't better to go back to "original" instead ? I mean, somehow you have asked for a prescreener. But in that case, the constructor/Init would have to be adjusted according to this.

[feisty2]

@feisty2
I have a question, just to ask if it's normal :
In evalFunc_0 you use prescreener >=2 only if pixelsize=1, wich is normal, but the way the tests are made, if you ask prescreener=2 with pixelsize>1, you fall in "no prescreener". Isn't better to go back to "original" instead ? I mean, somehow you have asked for a prescreener. But in that case, the constructor/Init would have to be adjusted according to this.

didn't quite get what you were saying... (not sure what you meant by "go back to original")
anyways, pscrn could only be 0 or 1 for high bitdepth samples (as new prescreeners were trained for byte inputs). 2 is illegal for them.

[pinterf]

I note that GScriptClip is incompatible with avs+ mt, with mode 2 not work at all and with mode 3 gave random scriptcilp errors
This was not forgotten, but I cannot help with it right now, could you post a test script and a hint what is special in GScriptClip (I have read some info about it but it is still not clear for me, never used)

[jpsdr]

@pinterf
Use -0.5 to 0.5 for UV and it'll match VS. Will make compatibility so much easier...
Let me know what you think you'll be doing in final, and if you're doing -0.5/+0.5 for UV, confirm that a PlaneToY will still produce 0.0/1.0.

@feisty2
Ok.
I mean, actualy in the code, from what i've understood, if you've set a value of 2 for the prescrener with high bitdepth, it switches to 0. Maybe it may switche to 1. Just a little thought.

[feisty2]

@feisty2
Ok.
I mean, actualy in the code, from what i've understood, if you've set a value of 2 for the prescrener with high bitdepth, it switches to 0. Maybe it may switche to 1. Just a little thought.

it should pop out an error message, not silently switch to another mode without notifying the user

[pinterf]

@pinterf

Let me know what you think you'll be doing in final, and if you're doing -0.5/+0.5 for UV, confirm that a PlaneToY will still produce 0.0/1.0.

Good catch, I was thinking about the same theoretical problem. I wonder if VS has special treatments for this in ShufflePlanes, but I don't think so.

[Myrsloik]

Good catch, I was thinking about the same theoretical problem. I wonder if VS has special treatments for this in ShufflePlanes, but I don't think so.

Any float value is always valid (except nan and inf) so no special handling at all. If the user wants weird luma they can have it.

There's also no guarantee that the luma channel stays in the 0-1 range so it doesn't matter.

[real.finder]

This was not forgotten, but I cannot help with it right now, could you post a test script and a hint what is special in GScriptClip (I have read some info about it but it is still not clear for me, never used)


ColorBars(width=640, height=480, pixel_type="yv12")
srestore (http://avisynth.nl/images/Srestore_27h.avsi)
Prefetch(6)


it used in Srestore and others

[jpsdr]

Well, let me know the final result of float decision, it will decide if i work or not to use float format in nnedi3.
The luma (or grey) plane in nnedi3 is clipped to 0.0-1.0 (and chroma planes are clipped to -0.5/+0.5). So, if you split an YUV format in 3 Y,U,V planes, and feed them separately in nnedi3, and the splitted plane U,V are in -0.5/+0.5 instead of 0.0-1.0 usual grey plane, your result will be messed up. So, if a splitted/extracted planar chroma plane is not shifted to match an usual Y/grey plane, you can't properly use nnedi3 in float format without adding a parameter to specify the the range of the float of the "grey" planar.
For now, even with the code of vapoursynth, i'm still unable to make it work with 16 bits.... :angry:

[feisty2]

Well, let me know the final result of float decision, it will decide if i work or not to use float format in nnedi3.
The luma (or grey) plane in nnedi3 is clipped to 0.0-1.0 (and chroma planes are clipped to -0.5/+0.5). So, if you split an YUV format in 3 Y,U,V planes, and feed them separately in nnedi3, and the splitted plane U,V are in -0.5/+0.5 instead of 0.0-1.0 usual grey plane, your result will be messed up. So, if a splitted/extracted planar chroma plane is not shifted to match an usual Y/grey plane, you can't properly use nnedi3 in float format without adding a parameter to specify the the range of the float of the "grey" planar.
For now, even with the code of vapoursynth, i'm still unable to make it work with 16 bits.... :angry:

NO!!!
WHYYYYYYYYYYYY DID YOU DO THAT???
overflow is practically nonexistent to floating points, the MAIN reason to use floating point samples is to avoid clipping to have a more mathematically correct result.

CLIPPING IS TOXIC

[pinterf]

ColorBars(width=640, height=480, pixel_type="yv12")
srestore (http://avisynth.nl/images/Srestore_27h.avsi)
Prefetch(6)


it used in Srestore and others
Could you try this one?
GRunT-1.0.2 (avisynth+ and 2.6, x86 and x64) (http://www.mediafire.com/file/77uromd0j0z56ap/GRunT-1.0.2tst.7z)
I suppose I only made it self register as a NICE_FILTER for Avisynth+.
That means that it doesn't run under avs 2.5x.
And if I had work with it, I made a 64bit version also.

[jpsdr]

NO!!!
WHYYYYYYYYYYYY DID YOU DO THAT???

Heu... Sorry, i've done nothing yet, take a look at your nnedi3.cpp lines 338, 641.
Did i misunderstood something ?

[feisty2]

Heu... Sorry, i've done nothing yet, take a look at your nnedi3.cpp lines 338, 641.
Did i misunderstood something ?

https://github.com/dubhater/vapoursynth-nnedi3/blob/master/src/nnedi3.cpp
correct link

[jpsdr]

Ok, thanks. But i'm still struggling with integer for now... ;)

Edit :
If you have any idea/comment about the potentiel issue i've posted in the nnedi3 thread.

[real.finder]

Could you try this one?
GRunT-1.0.2 (avisynth+ and 2.6, x86 and x64) (http://www.mediafire.com/file/77uromd0j0z56ap/GRunT-1.0.2tst.7z)
I suppose I only made it self register as a NICE_FILTER for Avisynth+.
That means that it doesn't run under avs 2.5x.
And if I had work with it, I made a 64bit version also.

it show random things in the top of frame
http://i.imgur.com/3GSte8V.png

[pinterf]

Yes, there is garbage. I was blind or I don't know. But it seems to be a very hard problem, at least haven't seen those avisynth internal parts that I have to debug now. First I'm struggling with the source of those hieroglyphs, which show: "ERROR: I don't know what 'current_frame' means." (Put this line at the beginning of the script: SetLogParams("log.txt", LOG_DEBUG))
And why it appears: ScriptClip errors (exception text with source module and line number) are overlayed on the current frame.
For some reason when formatting the error message, the C++ variadic parameter list (va_list) becomes corrupt between calls or after using vsnprintf. But only in this case. Now I'm reinstalling my VS2015.
But that is only the problem of the weird error message rendering. The real problem is the scope of variables, this plugin sets global paramers run-time, and this is somehow different in AVS+ than is classic AVS.

[pinterf]

As Avisynth+ project is demanding modernized filters, my next port will be masktools2.

Meantime I had a look at tp7's masktools project. As it was mentioned, there is a 16 bit branch, which is in quite a good shape, a lot of filters are basically ready, supporting stacked and 'doublewidth' hacked formats. This latter is basically the normal 16 bit way, how Avisynth+ works. This branch is a little behind over the latest masktools b2 release.

The extensions are 16 bit only, so the 10, 12, 14 bit support have to be inserted in. For practicing I have modded mt_binarize16 to have 10-14 bits, as the easiest part. After several hours it is still very hard for me to understand the framework, very general, very C++, macro magics, and those are somehow producing a working avisynth filter.

My goal will also be to have one filter name that works for all video formats, so there will be no separate mt_binarize and mt_binarize16.

[Motenai Yoda]

Shouldn't be faster starting off Firesledge's dither_tools mt functions? also there is CLExpr which can be a good point to add heterogeneous computing too

[Gavino]

The real problem is the scope of variables, this plugin sets global paramers run-time, and this is somehow different in AVS+ than is classic AVS.
Does the standard ScriptClip work correctly under mt?
Since GScriptClip is essentially a wrapper around the original function, its multithreading characteristics should be the same. However it does have some differences which may or may not be significant - current_frame is made a global variable, and the function may (depending on parameters) start a new scope level with env->PushContext().

How do variables work under mt?
I don't see how functions like ScriptClip, which relies on 'current_frame', can ever be thread-safe (unless somehow each thread has its own set of variables?)

[pinterf]

Does the standard ScriptClip work correctly under mt?
Since GScriptClip is essentially a wrapper around the original function, its multithreading characteristics should be the same. However it does have some differences which may or may not be significant - current_frame is made a global variable, and the function may (depending on parameters) start a new scope level with env->PushContext().

How do variables work under mt?
I don't see how functions like ScriptClip, which relies on 'current_frame', can ever be thread-safe (unless somehow each thread has its own set of variables?)
Standard ScriptClip works under mt, and the filter itself is NICE_FILTER, but I don't know how the derived mt behaviour is figured out by the content.
As for the variables, there is a TLS (thread-local storage) for each thread. They have their own context of local and global variables. When a thread-locale variable is not found in the there, it looks out for the core level variables.
See here (https://github.com/pinterf/AviSynthPlus/blob/MT/avs_core/core/ScriptEnvironmentTLS.h#L51)

The problem that I don't understand, that why I have "end of file reached without matching }" in the debug log.

EDIT:
There was a bug when an avisynth exception came from the thread-local environment, the variadic sprintf-like formatting was not correct, and it would corrupt the stack or the exception rollback mechanism somehow.
-- end of edit --

The error chain is thrown from this srestore.avsi line:
Try { global Srestore_InstanceNumber = Srestore_InstanceNumber+1 } catch(err) { global Srestore_InstanceNumber = 1 }


Original error comes from the try block. The error lines are growing as the exceptions are re-thrown back to one level.

Then follows the "file end reached" error and this is what I don't understand.

And the log:
ERROR: I don't know what 'Srestore_InstanceNumber' means.
---------------------------------------------------------------------
ERROR: I don't know what 'Srestore_InstanceNumber' means.
(C:/Program Files (x86)/AviSynth/plugins/SRestore_27.avsi, line 80)
---------------------------------------------------------------------
ERROR: Script error: end of file reached without matching }
---------------------------------------------------------------------
ERROR: Script error: end of file reached without matching }
((null), line 50, column 0)
---------------------------------------------------------------------
ERROR: Script error: end of file reached without matching }
((null), line 50, column 0)
(C:/Program Files (x86)/AviSynth/plugins/SRestore_27.avsi, line 320)
---------------------------------------------------------------------
ERROR: Script error: end of file reached without matching }
((null), line 50, column 0)
(C:/Program Files (x86)/AviSynth/plugins/SRestore_27.avsi, line 320)
(.\s1.AVS, line 8)

[pinterf]

New dev build
Download: Avisynth plus r2347 (http://www.mediafire.com/file/h1znw337movpwpu/avsplus-r2347.7z)

20161222 r2347dev

CombinePlanes: setting target to default RGBP(A)/YUV(A), when inputs are greyscale and no source planes are given
Decision is made by the target plane characters, if they are like R,G,B then target video format will be planar RGB
Same logic applies for YUV.
Example:
Y1, Y2 and Y3 are greyscale clips
Old, still valid: CombinePlanes(Y1, Y2, Y3, planes="RGB", source_planes="YYY", pixel_type="RGBP8")
New: CombinePlanes(Y1, Y2, Y3, planes="RGB") # result: Planar RGB
Fix: ScriptClip would show garbage text when internal exception occurs instead of the showing human readable error message

[vcmohan]

I tried script clip and ran into problem.My input is colorbars. I was continuously getting some large line number on top of the input image. I found that prefetch is the cause of this. After commenting out this I could run script clip correctly.

[pinterf]

Still getting large numbers with r2347? Or did your Scriptsclip sample work for you in MT before this latest build?

[vcmohan]

I was using r2294

[vcmohan]

Now I am on r2347. With the statement prefetch(4) , I get an error message now readable as "invalid arguments to ....". But commenting out prefetch the script runs fine without any error.

[pinterf]

That will be difficult. Visible error message is only the first stage of a possible fix. Now my test is srestore. I made a modification (post 2347) that affects global variables in scriptclip runtime. Now it does not display the exception, finally goes on, but is turning into deadlock somewhere in the handling of the multithread guard mutexes. And I think this is the real problem to be solved.

Can you strip your script to a short one that is still showing the error?