Avisynth+

[jpsdr]

Just when i finaly found it...
Thanks anyway.
Now, i have to figure out why... why... why... the compiler don't want to compile minmax.h anymore, throwing me a lot of errors about the template T ! :confused:
And i have to implement a test for detecting VS version, because i still want to be able to compile with VS2010, which the avx intrinsic don't allow anymore...
But first, i have to be able to build with VS2015. I can workaround minmax with #define macros, but i would better understand why sudenly it don't want to build anymore...

[pinterf]

Just when i finaly found it...
Thanks anyway. Now, i have to figure out why... why... why... the compiler don't want to compile minmax.h anymore, throwing me a lot of errors about the template T ! :confused:
The types inside min, max or std::clamp are not similar. All parameters have to be the same int, short or float (or pixel_t in templates).

PS, I have cleaned up the resizers a bit (after you found unused avx path test code) and added a real avx2 path, I bet it is still not as fast as zimg, but at least it got cleaner. Just have commited, check my MT repo.

[jpsdr]

The types inside min, max or std::clamp are not similar. All parameters have to be the same int, short or float (or pixel_t in templates).

The problem is that i haven't touched the code where minmax is used... So for now, i have absolutely no idea why there is suddenly this error.:(

PS, I have cleaned up the resizers a bit (after you found unused avx path test code) and added a real avx2 path, I bet it is still not as fast as zimg, but at least it got cleaner. Just have commited, check my MT repo.
Argh....... After all that hard work... Nooooooooooooooooooooooooo...:sly:

zimg... If it works with direct full square ... (argh... don't remember the word now), it will be impossible to multi-thread by splitting.

And a more general question.
In image/video processing, the 1rst idea (at least, for me) which comes to mind when talking of multi-threading is splitting the picture, not multiply it.
So, my question is : Why when introducing multi-threading in avisynth, nobody seems to have thought at the 1rst obvious idea, internaly MT the core filters by splitting the picture,
instead of trying to run several GetFrame at the same time...?
Filters that are pure memory transfert (like doubleweave, or TurnL/R) may not benefit of MT (there is more chances of memory bandwith ... again don't remember the correct word ... issue,struggle), but the others may.
So... Why...?

[pinterf]

8 bit was not affected, and the changes are not huge. Avx and avx2 is basically the same, only their templates are different you dont have to look for other differences

[jpsdr]

Ok, thanks.

[jpsdr]

The minmax.h issue is solved : I must remove the #include within resample.cpp otherwise compiler throws me errors.

[TheFluff]

No, jpsdr, you didn't suddenly come up with the obvious solution that nobody else has considered. There are reasons nobody uses slice based threading. The most obvious one is that if you're working with video you already have your input split up in neat discrete chunks for you: they're called frames. Splitting these chunks further just adds complexity for no real gain at all. I'm pretty sure it's also worse performance wise (for many reasons, especially for spatial filters where you might have to handle overlapping slices) but I haven't benchmarked it.

[LigH]

Even worse, slices will limit the motion vector search for redundancies the codec could have used to increase efficiency.

[ajp_anton]

ArrayGet(test_array,0)Opinions?Why not
test_array(0)
?

[Stephen R. Savage]

Why not
test_array(0)
?

Wouldn't that conflict with a function named test_array? AVS does not prevent creating variables with the same name as functions.

[pinterf]

Wouldn't that conflict with a function named test_array? AVS does not prevent creating variables with the same name as functions.
Yes, there would be conflict.
Final solution should be something like that

black = [0,128,128]
test_array = [ ["ciao", black], ["hello",[1,1,2.718281]], ["null", []] ]
test_array = test_array + ["dummy"]
item = test_array[0]
item2 = test_array["hello"]
e = item2[2]


Provided the [ and ] symbols are not reserved.
Or { and }. Language experts will decide.

[Gavino]

Provided the [ and ] symbols are not reserved.
Or { and }. Language experts will decide.
[ and ] are currently used only for nestable block comments [* ... *].
Although technically possible, I recommend not using { and } as these are already well established as code block delimiters, whereas [] is familiar from other languages as indicating array indexing.

You could argue for using a different symbol, like { ... }, for array creation (as in C/C++ and Java), but I would prefer [] for both indexing and creation. There is no problem in distinguishing the two contexts in the parser even if the same symbol is used for both.

On a wider note...
You have so far shown examples of how the array features would be used in the script language.
From the plugin writer's point of view, I would also like to see a list of proposed API changes.
I imagine there will be changes to the public interface of AVSValue and possibly new ScriptEnvironment functions.

[pinterf]

On a wider note...
You have so far shown examples of how the array features would be used in the script language.
From the plugin writer's point of view, I would also like to see a list of proposed API changes.
I imagine there will be changes to the public interface of AVSValue and possibly new ScriptEnvironment functions.
I agree with [ and ].
I also wonder how can I do that with keeping backward compatibility. Anyway, I will return to the problem later, this will definitely need more time than that day I spent on it so far.

[Gavino]

I agree with [ and ].
I also wonder how can I do that with keeping backward compatibility.
Since [ and ] are currently illegal in scripts (except for [* and *]), what problem do you see with backward compatibility?

[pinterf]

Since [ and ] are currently illegal in scripts (except for [* and *]), what problem do you see with backward compatibility?
Sorry, I was thinking forward on AVSValue things and the possible interface additions.

[Gavino]

Sorry, I was thinking forward on AVSValue things and the possible interface additions.
Yes, that's certainly an area where careful thought will be required (although at the moment I can't see any specific problems).

[stranno]

I'm getting a "display error" with r2294 using ProRes 422LT source. Pre-2000 releases work fine. I have seen r2003+ releases support high bit depth colorspace so i'm not sure whats the problem.

Edit: I'm using FFMS2 2.23.

[pinterf]

I'm getting a "display error" with r2294 using ProRes 422LT source. Pre-2000 releases work fine. I have seen r2003+ releases support high bit depth colorspace so i'm not sure whats the problem.

Edit: I'm using FFMS2 2.23.
Pre r2294 versions did not support 10,12 and 14 bit depths. Could you please try with the latest (http://forum.doom9.org/showthread.php?p=1784354#post1784354) version?

[tuanden0]

I have a problem with Avisynth Plus r2294, can someone help me :scared:

I coppied the Avisynth.dll to System32 but i still error :(

http://i.imgur.com/IsdqpbX.png

[pinterf]

If you have installed 32 bit avisynth+ and have a 64 bit windows, the avisynth.dll must be placed to the SysWOW64 folder.
On 32 bit windows, system32 is O.K. for 32 bit avisynth.
But the installer should do this instead of you.
I suppose, there is another avisynth.dll somewhere, can it be set in avspmod?

[tuanden0]

If you have installed 32 bit avisynth+ and have a 64 bit windows, the avisynth.dll must be placed to the SysWOW64 folder.
On 32 bit windows, system32 is O.K. for 32 bit avisynth.
But the installer should do this instead of you.
I suppose, there is another avisynth.dll somewhere, can it be set in avspmod?

Yah, my fault, i forgot that :p:p

Thank you very much :cool:

[ajp_anton]

Wouldn't that conflict with a function named test_array? AVS does not prevent creating variables with the same name as functions.Yes, there would be conflict
But as mentioned, you can already create those conflicts with variable names. What makes array-name conflicts any different? Either you ignore the problem (allow conflicts), or you do something about it (throw error when conflicts happen, or require ()-brackets for functions), but you do the same thing with variable names and array names (aren't arrays also some kind of "variables"?).

Though I agree that []-brackets are better for arrays.

[Aktan]

I have an interesting problem. I have a Lagarith source AVI in RGB32 colorspace with the resolution of 350x240. If I open this AVI with AVS+, I get distorted video. From Info(), I noticed the video pitch is wrong. Then I realized 350 is not mod 4. Is this a known problem?

[Reel.Deel]

I have an interesting problem. I have a Lagarith source AVI in RGB32 colorspace with the resolution of 350x240. If I open this AVI with AVS+, I get distorted video. From Info(), I noticed the video pitch is wrong. Then I realized 350 is not mod 4. Is this a known problem?

What AVS+ version are you using?

[Aktan]

What AVS+ version are you using?

r1576, x64

Edit: Well I guess it's an old version? lol.

[Reel.Deel]

r1576, x64

IIRC someone reported a similar problem way back. I believe it was fixed, try updating to the latest version (http://avisynth.nl/index.php/AviSynth%2B#Downloads). Report your findings :).

Edit: maybe this? https://github.com/AviSynth/AviSynthPlus/issues/36

[Aktan]

IIRC someone reported a similar problem way back. I believe it was fixed, try updating to the latest version (http://avisynth.nl/index.php/AviSynth%2B#Downloads). Report your findings :).

Thanks, will do!

Edit: r2294 works fine =)

[Stephen R. Savage]

But as mentioned, you can already create those conflicts with variable names. What makes array-name conflicts any different? Either you ignore the problem (allow conflicts), or you do something about it (throw error when conflicts happen, or require ()-brackets for functions), but you do the same thing with variable names and array names (aren't arrays also some kind of "variables"?).

Though I agree that []-brackets are better for arrays.

There are no function pointers, so I believe AVS will use the presence of the parentheses token to identify a function name instead of a variable.

[Gavino]

There are no function pointers, so I believe AVS will use the presence of the parentheses token to identify a function name instead of a variable.
It does indeed, but in principle it could be extended to try x(1) as an array indexing if it finds that there is no function named x - just as now, when it finds plain 'x' and there is no variable called x, it tries to treat it as a parameterless function call x().

However, I don't think that would be a good idea and the availability of [] makes it unnecessary.

[jpsdr]

A little question.
If i am with an YUVA 4:4:4 mode (for exemple), will IsYV24() still be true ?

[pinterf]

A little question.
If i am with an YUVA 4:4:4 mode (for exemple), will IsYV24() still be true ?
No. There is an Is444() which is bit-depth independent and also works for YUVA.
When using my latest Avisynth.h, this Is444 call is mapped silently to IsYV24 if the relevant avisynth host has no Is444() implemented.

[jpsdr]

Ah... Ok i've seen

So, it means there is no 4:1:1 with alpha channel ?

[pinterf]

Ah... Ok i've seen

So, it means there is no 4:1:1 with alpha channel ?
No. We are lucky then :)

[Reel.Deel]

Can we add a straightforward way to get the chroma width and height? Right now we have to do something like UToY8().Width(). Maybe something like Width(chroma=true) or uv=true. Thoughts?

[pinterf]

Avisynth arrays status report:

Implemented empty arrays and [] syntax for array creation


clip=ffms2(film).ConvertBits(16)

empty_array = []
empty_array_2 = empty_array
#n3 = empty_array_2.ArrayGet(0) # array index out of range error!

black_yuv_16 = [0,32768,32768] # or black_yuv_16 = Array( 0,32768,32768)
grey_yuv_16 = [32768,32768,32768]
white_yuv_16 = [65535,32768,32768]
# getting deeper: nested arrays
aSelectColors = [\
["black", black_yuv_16],\
["grey", grey_yuv_16],\
["white", white_yuv_16],\
["empty", empty_array]\
]
test_array = [99, 1.0, "this is a string"] # mixed types
test_array2 = [199, 2.0, "This is a string"]

n = ArraySize(test_array) # 3
n2 = ArraySize(empty_array_2) # 0

clip = clip.SubTitle("Array size = " + String(n) +\
" Empty array size = " + String(n2) +\
" [0]=" + String(ArrayGet(test_array,0)) + \
" [1]=" + String(ArrayGet(test_array,1)) + \
" [2]=" + ArrayGet(test_array,2))

black=blankClip(clip, length=1, colors = ArrayGet(aSelectColors,"black")) # value lookup by name
white=blankClip(clip, length=1, colors = aSelectColors.ArrayGet("white")) # value lookup by name
grey=blankClip(clip, length=1, colors = [32768,32768,32768]) # or direct array

blackwhite = black+grey+white+black+grey+white
n=3
clip = clip.Trim(0,n) + blackwhite + clip.Trim(n+1,499)
clip


Back to work. It would be nice to have indexing with []

[mcjordan]

3> limiter.cpp
3>C:\AviSynthPlus\avs_core\filters\limiter.cpp(107): error C3861: '_mm_max_epu16': identifier not found
3>C:\AviSynthPlus\avs_core\filters\limiter.cpp(108): error C3861: '_mm_min_epu16': identifier not found
...
//min and max values are 16-bit unsigned integers
inline void limit_plane_uint16_sse4(BYTE *ptr, unsigned int min_value, unsigned int max_value, int pitch, int height) {
__m128i min_vector = _mm_set1_epi16(min_value);
__m128i max_vector = _mm_set1_epi16(max_value);
BYTE* end_point = ptr + pitch * height;

while(ptr < end_point) {
__m128i src = _mm_load_si128(reinterpret_cast<const __m128i*>(ptr));
src = _mm_max_epu16(src, min_vector);
src = _mm_min_epu16(src, max_vector);
_mm_store_si128(reinterpret_cast<__m128i*>(ptr), src);
ptr += 16;
}
}
...
This break compilation process ?! Help?

[pinterf]

3> limiter.cpp
3>C:\AviSynthPlus\avs_core\filters\limiter.cpp(107): error C3861: '_mm_max_epu16': identifier not found
3>C:\AviSynthPlus\avs_core\filters\limiter.cpp(108): error C3861: '_mm_min_epu16': identifier not found
...

This break compilation process ?! Help?
Indeed, until I update it, put
#include <smmintrin.h> // for sse41
at the top of limiter.cpp

[mcjordan]

Thank you, pinterf! Works like a charm!

[Gavino]

It would be nice to have indexing with []
I would do this by changing the parser's ParseOOP() function - in the two places it currently calls ParseFunction(), it would instead call a new function ParseIndex(), which would look like this:
PExpression ScriptParser::Parseindex(PExpression context)
{
PExpression result = ParseFunction(context);
while (tokenizer.IsOperator('[') {
tokenizer.NextToken();
result = new ExpIndex(result, ParseConditional());
Expect(']');
}
return result;
}

This allows things like:
- x.f(2)[3], where f is a function returning an array (no need to put brackets round x.f(2))
- a[1]["white"], where a is a multi-dimensional array
- a[1].f(2), equivalent to f(a[1], 2)

ExpIndex is a new Expression subtype whose Evaluate() function calls the built-in ArrayIndex function.

[pinterf]

I would do this by changing the parser's ParseOOP() function - in the two places it currently calls ParseFunction(), it would instead call a new function ParseIndex(), which would look like this:
PExpression ScriptParser::Parseindex(PExpression context)
{
PExpression result = ParseFunction(context);
while (tokenizer.IsOperator('[') {
tokenizer.NextToken();
result = new ExpIndex(result, ParseConditional());
Expect(']');
}
return result;
}

This allows things like:
- x.f(2)[3], where f is a function returning an array (no need to put brackets round x.f(2))
- a[1]["white"], where a is a multi-dimensional array
- a[1].f(2), equivalent to f(a[1], 2)

ExpIndex is a new Expression subtype whose Evaluate() function calls the built-in ArrayIndex function.
Thanks, I'm doing it with inserting ParseOOP an additional '[' check, that can call ParseFunction with either . or [, plus the context. If it works, I try to separate it, to be nice (like your proposed ParseIndex.
All other things are done, now I'm doing compound index formats e.g. a[1,2,3]; a[1][2][3] is working already

[jpsdr]

I've looked on post #2484, but don't find an answer.

If i have a planar YUV mode with alpha channel, and i'm doing the following :

vu = env->Invoke("UtoY8",v).AsClip();
vv = env->Invoke("VtoY8",v).AsClip();
v = env->Invoke("ConvertToY8",v).AsClip();

Where is my alpha channel ?
Is it still on v ?
If still on Y, is doing the following, putting everything back :

AVSValue ytouvargs[3] = {vu,vv,v};
v=env->Invoke("YtoUV",AVSValue(ytouvargs,3)).AsClip();

If not, what should i do when handling this case ?

[pinterf]

I've looked on post #2484, but don't find an answer.

If i have a planar YUV mode with alpha channel, and i'm doing the following :

vu = env->Invoke("UtoY8",v).AsClip();
vv = env->Invoke("VtoY8",v).AsClip();
v = env->Invoke("ConvertToY8",v).AsClip();

Where is my alpha channel ?
Is it still on v ?
If still on Y, is doing the following, putting everything back :

AVSValue ytouvargs[3] = {vu,vv,v};
v=env->Invoke("YtoUV",AVSValue(ytouvargs,3)).AsClip();

If not, what should i do when handling this case ?
UtoY8, VtoY8 and ConvertToY8 functions create greyscale image.

Then there became too much color formats after having Planar RGB and A channel.
Instead of making AtoY8, RtoY8, GtoY8, BtoY8, there is a PlaneToY(plane_string) in Avisynth+ that creates a greyscale clip from the relevant channel.

E.g. PlaneToY("A), PlaneToY("R")
PlaneToY accepts any plane identifier (Y,U,V,A,R,G,B)

PlaneToY("U") is equivalent to UToY8
PlaneToY("V") is equivalent to VToY8
PlaneToY("Y") is equivalent to ConvertToY (ConvertToY8)

All the above functions keep the current bitdepth, the Y8 ending is just because UToY and VToY was already existing functions.

Regarding the back conversion, YtoUV only has U,V and an optional Y clip parameter. To answer on your question, it seems, there is no function for putting A channel back to a YUVA clip.

I always wanted to make a general way mixing together arbitrary planes, something like ShufflePlanes in VapourSynth.

Albeit there is MergeRGB and MergeARGB for packed and planar RGB, it seems that making the same for YUV is missing.
On more reason for implementing a general MergePlanes or something like that. Now I'm considering that.

[jpsdr]

Ok. Meaning that for now, in nnedi3 i can only support YUVA 4:4:4, because for YUV other than 4:4:4 i have to split the planes to process them, and pull back after.

[pinterf]

Ok. Meaning that for now, in nnedi3 i can only support YUVA 4:4:4, because for YUV other than 4:4:4 i have to split the planes to process them, and pull back after.
What is the reason that you have to split the planes then put them together? Bacause nnedi3 can only work internally with identical plane dimensions?

[jpsdr]

Because there is shift to make on chroma only for other modes than 4:4:4, and use of turnleft/right, so need to separate planes in 4:2:2 to have things done properly.
But, this is only for nnedi3_rpow indeed, not nnedi3.

[jpsdr]

Just to be sure, according from what i've understood, IsYUY2 is not only for 8 bits.

[pinterf]

Just to be sure, according from what i've understood, IsYUY2 is not only for 8 bits.
YUY2 is a non-friendly format, and only exists as 8 bit.

[jpsdr]

Ok, thanks for all of these tips.
Now, just 'waiting' (and it will happen when it happens...) for a way to put back alpha channel in YUV mode.

[pinterf]

Ok, thanks for all of these tips.
Now, just 'waiting' (and it will happen when it happens...) for a way to put back alpha channel in YUV mode.

Preliminary information on some future avisynth plus features.
Driven by the popular demand :) I prepared some new stuffs for those who are mixing planes as a daily routine.

Since it is still not released, you can comment it freely and make wish-list.

20161110 Avisynth plus additions

New functions

AToY8 , same as PlaneToY("A") for planar RGBA or YUVA
RToY8 , same as PlaneToY("R") for planar RGB
GToY8 , same as PlaneToY("G") for planar RGB
BToY8 , same as PlaneToY("B") for planar RGB

They work the same way as UToY8 and VToY8 and ConvertToY did.
The functions convert to greyscale, keeping the original bit-depth, not only 8 bits!
Y8 naming was kept, because UToY and VToY already existed.

Extended function syntax
old: YToUV(clip clipU, clip clipV [, clip clipY ] )
new: YToUV(clip clipU, clip clipV [, clip clipY [, clip clipA] ] )

YToUV accepts optional alpha clip after Y clip

Example
U = source.UToY8()
V = source.VToY8()
Y = source.ConvertToY()
A = source.AddAlphaPlane(128).AToY8()
# swaps V, U and A, Y
YToUV(V,U,A,Y).Histogram("levels").Info().RemoveAlphaPlane()


New function
CombinePlanes(clip1 [,clip2, clip3, clip4], string planes [, string source_planes, string pixel_type, string sample_clip])

Combines planes of source clip(s) into a target clip.
Similar to ShufflePlanes in Vapoursynth.

clip sample_clip (optional)
If sample_clip is given, target clip properties are copied from that clip
If no sample_clip is provided, then clip1 provides the template for target clip

string pixel_type (optional)
An optional pixel_type string (e.g."YV24", "YUV420PS", "RGBP8") can override the base video format.

clip clip1, ... clip4
If the source clip count is less than the given planes defined, then the last available clip is used as a source for all later planes

string planes
the target plane order (e.g. "YVU", "YYY", "RGB")
missing target planes will be undefined in the target

string source_planes (optional)
the source plane order, defaulting to "YUVA" or "RGBA" depending on the video format

Source clips can even be mixed from greyscale, YUV, YUVA, planar RGB(A), the only rule that the relevant source plane character should match with the clip format, respectively.

Example#1
#combine greyscale clips into YUVA clip
source=source.AddAlphaPlane(128)
U8 = source.UToY8()
V8 = source.VToY8()
Y8 = source.ConvertToY()
A8 = source.AToY8()
CombinePlanes(Y8, U8, V8, A8, planes="YUVA", source_planes="YYYY", sample_clip=source) #pixel_type="YUVA420P8"

Example#2
# Copy planes between planar RGB(A) and YUV(A) without any conversion
# useful if you have a filter that accepts only YUV input
# yuv 4:4:4 <-> planar rgb
source = last.ConvertBits(32) # 4:4:4
cast_to_planarrgb = CombinePlanes(source, planes="RGB", source_planes="YUV", pixel_type="RGBPS")
# get back a clip identical with "source"
cast_to_yuv = CombinePlanes(cast_to_planarrgb, planes="YUV", source_planes="RGB", pixel_type="YUV444PS")

Example#3
#create a black and white planar RGB clip using Y channel
#source is a YUV clip
grey = CombinePlanes(source, planes="RGB", source_planes="YYY", pixel_type="RGBP8")

Example#4
#copy luma from one clip, U and V from another
#source is the template
#sourceY is a Y or YUV clip
#sourceUV is a YUV clip
clip = CombinePlanes(sourceY, sourceUV, planes="YUV", source_planes="YUV", sample_clip = source)

[ajp_anton]

"RToY8" etc, don't they already exist in "ShowRed" etc?
And since they all end with Y8, does this mean they only work for 8bit?