Avisynth+

[pinterf]

This whole story with the arrays was more than 3 years ago. It's time to re-test, next package will come with and without arrays.

[Gavino]

It can't be a NOP as it has results on encoding speed usually better than put any value.
But does it give different results than not using it at all?

Perhaps your results just show that the default value is actually optimal.

[real.finder]

can someone make fmtconv port for avs+? https://forum.doom9.org/showthread.php?p=1907352#post1907352

[MeteorRain]

I don't know other that trying it.

So I tried it. Having 'a' as type, the filter would not be possible to be used on non-AVS-newvalue build.

Checking the avs core I noticed that it's possible to manipulate (i.e. abuse) param name to support array hinting.

For example, dft has this:
c[sstring]s[ssx]s[ssy]s[sst]s.

If we can somehow do:
c[sstring]s[ssx]s[ssy]s[sst]s[sstring()]f[ssx()]f[ssy()]f[sst()]f
then we can make the filter working under both versions.

In legacy version, because param names has () in it, we know it's not a valid name and you can't assign data to it by name. To support assigning values by order, obviously we can keep array declaration at the end of param list.

In modern version, the core now knows that these 4 parameters are array of floats and will accept them as both single string or array of floats. Weak type arrays can be flagged as "[name()]."

So if user writes
v.dfttest("a", "b")
you get [v, "a", "b", -, -] in both versions.

And if user writes
v.dfttest([1.0], [2.0])
you get [v, [1.0], [2.0], -, -] in newer versions.

Within filter, simply check if args[1].isArray() to know if user passed in an array or a string.

What do you think about this proposal?

==========

EDIT: I'm generating parameter hinting like this:

for (auto &&p : params)
{
...
if (p.IsOptional) {
ss << '[' << p.Name << ']';
if (p.IsArray) {
ss_arrays << '[' << p.Name << "()]" << type_name;
type_name = 's';
}
}
ss << type_name;
}
return ss.str() + ss_arrays.str();

c[ftype]i[sigma]f[sigma2]f[pmin]f[pmax]f[sbsize]i[smode]i[sosize]i[tbsize]i[tmode]i[tosize]i[swin]i[twin]i
[sbeta]f[tbeta]f[zmean]b[f0beta]f[nlocation]s[alpha]f[slocation]s[ssx]s[ssy]s[sst]s[ssystem]i[sfile]s[sfile2]s
[dither]i[y]i[u]i[v]i[opt]i[nlocation()]i[slocation()]f[ssx()]f[ssy()]f[sst()]f

[MeteorRain]

Within dfttest, I noticed that, when nstring="0,0,20,40" I get the following error:

Filter error: GetPlaneHeightSubsampling called with supported plane.

Coming from

const int height = proc_height >> vi.GetPlaneHeightSubsampling(b);
const int width = vi_src.width >> vi.GetPlaneWidthSubsampling(b);

where b == 0.

[MeteorRain]

Also within dfttest, threads=0 and threads=1 returns different results, i.e. bit non-identical.

[pinterf]

Within dfttest, I noticed that, when nstring="0,0,20,40" I get the following error:

Filter error: GetPlaneHeightSubsampling called with supported plane.

Coming from

const int height = proc_height >> vi.GetPlaneHeightSubsampling(b);
const int width = vi_src.width >> vi.GetPlaneWidthSubsampling(b);

where b == 0.
Then it probably never worked. 0,1,2 should be translated to actual PLANAR_Y, PLANAR_U, PLANAR_V constants.

[LigH]

It can't be a NOP as it has results on encoding speed usually better than put any value.

Did you also test not using SetMemoryMax at all (opposing to calling it once, but without a value)?

[pinterf]

When your SetMemoryMax parameter is significantly larger than the actual maximum value reported by avsmeter then all results should be identical within an error margin.
Multiple Avsmeter sessions can report fps values within a range. Just have measured a simple script, five runs, no mt, results varied from 6.53 to 6.69. (2,5%)

[tormento]

Did you also test not using SetMemoryMax at all (opposing to calling it once, but without a value)?
Well, now I tried on another, heavier, script and:

SetMemoryMax() 5.83 fps
no SetMemoryMax 5.69 fps

I mean... has with AVS+ and x64 any matter anymore at this point to put SetMemoryMax(xxx)?

[LigH]

A random spread of results is not surprising. Hardware interrupts, file caching, ... if you want to do serious speed tests, don't run each test only once and trust the one result, instead take the minimum of several attempts with the same circumstances.

[magnetite]

I was wondering about something. Some of these filters were built using different compilers, and I'm not sure which compiled version of a filter I should use. In the archive of say, Rgtools 0.98, it lists a clang version and a non-clang version. Does one offer better performance than the other?

[pinterf]

It depends on the usage, RgTools 24 modes are basically 24 different filters. One can be optimized better with clang other modes with msvc.
If you are a poweruser and always measure everything and you find that the mode N which you are frequently using is 4% quicker with clang than with ms version then you can make your decision on clang.
When you are not sure then choose a version with a more sympathic name :)

[MeteorRain]

Yes, they have different ways of optimization and even depth of optimizations.
As I explained before, for example clang would optimize on lower level, so it's possible what you write (even assembly intrinsics) doesn't match what you get, because compiler is smarter than you and emit even better code.
While for example MSVC doesn't do this, and if you write stupid intrinsics it compiles as is.

So it's possible you'll end up with some filters running faster on clang, and some others are faster on msvc.
Only you would know how fast they run.

[wonkey_monkey]

I find that I get better results on the second/third and subsequent runs of Avsmeter. I always discard the first result.

[tormento]

I find that I get better results on the second/third and subsequent runs of Avsmeter. I always discard the first result.
AVSMeter doesn't give the full pie but a slice only.

If you look at my benchmarks, when real encoding is involved, x264/5 can be limited by the cpu time/memory/whatsoever consumed by the script itself.

If it's FPS you are looking for, after an AVSMeter bench, you should go for real ones.

[tormento]

A random spread of results is not surprising.
I took median results.

[Groucho2004]

Remember that AVSMeter is primarily a troubleshooting tool for the frameserving part of an encoding process. For example, if you think your encoding is going too slow, it will help you determine if it's the encoding settings or the script being the bottleneck (or both :scared:).

[wonkey_monkey]

AVSMeter doesn't give the full pie but a slice only.

If you look at my benchmarks, when real encoding is involved, x264/5 can be limited by the cpu time/memory/whatsoever consumed by the script itself.

If it's FPS you are looking for, after an AVSMeter bench, you should go for real ones.

I use it mainly for optimising my filters.

[magnetite]

It depends on the usage, RgTools 24 modes are basically 24 different filters. One can be optimized better with clang other modes with msvc.
If you are a poweruser and always measure everything and you find that the mode N which you are frequently using is 4% quicker with clang than with ms version then you can make your decision on clang.
When you are not sure then choose a version with a more sympathic name :)

I'm someone with very little knowledge of programming other than the complete basics. Sort of like an every day user of Windows instead of the people at Microsoft who programmed it.

[videoh]

You don't have to be a power user or coder, magnetite (love the nick). Just try both ways in your case and see which you like better. Maybe that's all pinterf was trying to say. Lot's of things are empirical because the theory is too hard or things depend on too many factors. Keep us informed about your projects.

[mcjordan]

Dear Pinterf, i have another try to compile your last build of AviSynthPlus (VC2019),
but this time I have two fatal errors that compromise compilation:

3>C:\git\AvisynthPlus_pinterf\avs_core\filters\conditional\conditional_functions.cpp(140,85): error C2653: 'GetPropertyAsArray': is not a class or namespace name
3>C:\git\AvisynthPlus_pinterf\avs_core\filters\conditional\conditional_functions.cpp(140,79): error C2065: 'Create': undeclared identifier

Аs I said earlier I'm not familiar with modern C/C++ programming.
I would be very grateful if you could help me again.

[pinterf]

You are unlucky. This dev branch is still under heavy changes. Pull often. If the problem persists even after I considered my work finished I'll check it.
Edit: I don't know when you had your last successful build, maybe you could regenerate the vs solution from cmake

[mcjordan]

Thank you, Pinterf. Maybe I'm too impatient, but your build is the best.
Happy оrthodox Easter!

[pinterf]

There is no such thing as the best. We know the best. Ha-ha. Long live Avisynth the covideo processing scripting tool

[real.finder]

Is it possible to know if there are duplicate functions in avsi files?

[pinterf]

Just saying that KNLMeansCL (and TMM2 mod by chikuzen) have to be recompiled for the next Avisynth+ since they are is using the non-finalized IScriptEnvironment2 avs+ interface.
KNLMeansCL is using SetFilterMTMode for MT mode hint (sets MT_MULTI_INSTANCE) instead of using the usual cache hints technique.
So it crashes right after adding its functions (AvisynthPluginInit3).
if (env->FunctionExists("SetFilterMTMode")) {
static_cast<IScriptEnvironment2*>(env)->SetFilterMTMode("KNLMeansCL", MT_MULTI_INSTANCE, true);


If there are other plugins that are using IScriptEnvironment2 directly let us know. (But I think you'll recognize that quite soon, even AVSMeter will list it as unusable)

@Groucho2004: could you prepare avsmeter for checking for AVISYNTH_INTERFACE_VERSION 8

[pinterf]

Is it possible to know if there are duplicate functions in avsi files?
If SetLogParams("stderr", 4) is not listing the duplicates (like it does for similarly named DLL functions) than not.
(Or replace stderr with a file name)

[Groucho2004]

@Groucho2004: could you prepare avsmeter for checking for AVISYNTH_INTERFACE_VERSION 8Sure. Are you skipping AVISYNTH_INTERFACE_VERSION 7 or will it serve a different purpose?

[qyot27]

7 exists, er, existed. It just happened about four years late and was true for about a month (it was bumped to 7 between 3.5.0 and 3.5.1, due to the needs of having to declare the SOVERSION for the POSIX builds (https://github.com/AviSynth/AviSynthPlus/commit/40900dc1c54c14ea9f188c7242b88d464d067a44)).

[Groucho2004]

7 exists, er, existed. It just happened about four years late and was true for about a month (it was bumped to 7 between 3.5.0 and 3.5.1, due to the needs of having to declare the SOVERSION for the POSIX builds (https://github.com/AviSynth/AviSynthPlus/commit/40900dc1c54c14ea9f188c7242b88d464d067a44)).Yeah, I only just noticed.

[real.finder]

If SetLogParams("stderr", 4) is not listing the duplicates (like it does for similarly named DLL functions) than not.
(Or replace stderr with a file name)

they not, only one from dll is listed, I asked because when I Analyze some avsi files in order to add HBD to them I note both EdgeCleaner and MC_Spuds has StarMask() with completely differently code but same function name!

[Groucho2004]

I asked because when I Analyze some avsi files in order to add HBD to them I note both EdgeCleaner and MC_Spuds has StarMask() with completely differently code but same function name!So just rename one of them.

[StainlessS]

Suggest either EdgeCleaner_StarMask() or MC_Spuds_StarMask() [or even both].

[real.finder]

that what I will do, but I want to know if there are any others hidden in other avsi functions

[Groucho2004]

that what I will do, but I want to know if there are any others hidden in other avsi functionsThe only way to check this would be to write a small tool that enumerates all functions in a group of avsi files and checks for dups.

Thinking about it, that might be something worth adding to AVSMeter checking not just functions in avsi files but also checking them against internal/external DLL functions.

Any thoughts about the usefulness of this?

[real.finder]

The only way to check this would be to write a small tool that enumerates all functions in a group of avsi files and checks for dups.

Thinking about it, that might be something worth adding to AVSMeter checking not just functions in avsi files but also checking them against internal/external DLL functions.

Any thoughts about the usefulness of this?

I think it is useful if the user has several avsi files, avs+ already has something for DLL functions but make it in AVSMeter will be a useful thing for older avs+ and avs 2.6

[Groucho2004]

I think it is useful if the user has several avsi filesI suppose that there could even be duplicate functions within a single avsi.

avs+ already has something for DLL functionsBut it does not check functions in avsi files against DLL functions, does it?

[real.finder]

But it does not check functions in avsi files against DLL functions, does it?

you mean like Vinverse which has both dll and avsi? yes avs+ don't check this case

[Groucho2004]

you mean like Vinverse which has both dll and avsi? yes avs+ don't check this caseI don't know about Vinverse. However, it's certainly possible for an internal or external (plugin) function having the same name as a function in a loaded avsi in which case I would assume that the DLL function takes precedence.

[StainlessS]

Any thoughts about the usefulness of this?
Yes, Like it.

I would assume that the DLL function takes precedence.
Nope, Script func overrides dll, dll overrides builtin.

Wiki:- http://avisynth.nl/index.php/Plugins#Plugin_Autoload_and_Name_Precedence
Plugin Autoload and Name Precedence

It is possible to put all plugins and script files with user-defined functions or (global) variables in an autoload directory,
from which all files with the extension .AVSI and .DLL are loaded at startup, then unloaded and reloaded dynamically as the script needs them.

.AVSI scripts in this directory should only contain function definitions and global variables; they should not have a main processing section
(else strange errors may occur). It also is not recommended to put other files in that directory.

The directory is stored in the registry. You can use double-clicking a .REG-file with the following lines to set the path (of course inserting your actual path):

REGEDIT4

[HKEY_LOCAL_MACHINE\SOFTWARE\Avisynth]
"plugindir2_5"="c:\\program files\\avisynth 2.5\\plugins"

The order in which function names take precedence is as follows:

User-defined function (always have the highest priority)
Plugin (external) function (higher priority than built-in)
Built-in (internal) function

Inside those groups the function loaded last takes precedence; there is no error in a namespace conflict.
It follows that there is no harm in loading the same plugin more than once.[dubious – discuss]


I remember Kassandro dll had a Difference() function, for some reason it took precedence over my script function of same name despite above list (maybe in v2.58).

[Groucho2004]

Nope, Script func overrides dll, dll overrides builtin.I see, who'd a thunk it?

[pinterf]

another out of the range float problem


ColorBars(width=640, height=480, pixel_type="yv12")
Convertbits(32)
mt_lutxy(last,Invert(),"x x y - 9.9 * +", use_expr=1)
#~ ConvertBits(8)
Converttorgb24

it's ok with ConvertBits(8) but with Converttorgb24() it's not

I'd say it's the typical case of garbage in garbage out.

In this case ConvertToRGB24() is the shortcut for ConvertToYUV444().ConvertToPlanarRGB().ConvertBits(8).ConvertToRGB24()
The problem occurs during 32 bit float 444->RGB conversion.

When you limit Y to 0..1, it all becomes O.K.
Expr("x 0 < 0 x 1.0 > 1 x ? ?","","")
For the same reason, if you convert to lower bit depths, this limitation is automatically done since integer accepts values between 0 and 2^N -1, and out-of-range values are clamped during the conversion.

Note, that in your example at least half of the image have Y<0, even in the-9.0 range, which is way outside of the 0..1.0 range, and of course results in undefined result during YUV->RGB matrix conversion.

[real.finder]

I'd say it's the typical case of garbage in garbage out.

In this case ConvertToRGB24() is the shortcut for ConvertToYUV444().ConvertToPlanarRGB().ConvertBits(8).ConvertToRGB24()
The problem occurs during 32 bit float 444->RGB conversion.

When you limit Y to 0..1, it all becomes O.K.
Expr("x 0 < 0 x 1.0 > 1 x ? ?","","")
For the same reason, if you convert to lower bit depths, this limitation is automatically done since integer accepts values between 0 and 2^N -1, and out-of-range values are clamped during the conversion.

Note, that in your example at least half of the image have Y<0, even in the-9.0 range, which is way outside of the 0..1.0 range, and of course results in undefined result during YUV->RGB matrix conversion.

but it also ok with
ColorBars(width=640, height=480, pixel_type="yv12")
Convertbits(32)
mt_lutxy(last,Invert(),"x x y - 9.9 * +", use_expr=1)
#~ ConvertBits(8)
Limiter(0,1,-0.5,0.5)
Converttorgb24

[pinterf]

I wonder what's the case with z_ConvertFormat?
Or one could change the order: first convert to 8 bits then to rgb but I chose this processing order to have the more precision for yuv->rgb conversion.

[pinterf]

Does Avisynth have similar thing for RGB like ColorYUV(analyze=true?)

[real.finder]

I wonder what's the case with z_ConvertFormat?
Or one could change the order: first convert to 8 bits then to rgb but I chose this processing order to have the more precision for yuv->rgb conversion.

what about do clamped first? isvideofloat? Limiter(0,1,-0.5,0.5).ConvertToYUV444().ConvertToPlanarRGB().ConvertBits(8).ConvertToRGB24() : ConvertToYUV444().ConvertToPlanarRGB().ConvertBits(8).ConvertToRGB24()

at least only for ConvertToRGB24 since it used in preview in avspmod and others

[StvG]

Does Avisynth have similar thing for RGB like ColorYUV(analyze=true?)
RGBAdjust(analyze=true)

[pinterf]

RGBAdjust(analyze=true)
Thanks, seems that there are too many filters for my memory capacity :)

[pinterf]

what about do clamped first? isvideofloat? Limiter(0,1,-0.5,0.5).ConvertToYUV444().ConvertToPlanarRGB().ConvertBits(8).ConvertToRGB24() : ConvertToYUV444().ConvertToPlanarRGB().ConvertBits(8).ConvertToRGB24()

at least only for ConvertToRGB24 since it used in preview in avspmod and others
Well, I think it was just a good thing, otherwise we could not catch that the mentioned expression is wrong.