AviSynth+ thread Vol.2

[StainlessS]

Thanks RD, fixed.

@ EternalStudent, the filter (linked by Reel.Deel) was updated 10 days ago.

[Nuihc88]

Other than looking at video output...how do people debug Avisynth+ use? I haven't found a log file I can enable, or a debug registry key to set.

Only reliable way i have found to debug SVP's AviSynth-filter usage, is to write my own AVS-scripts for SVPflow filters (https://github.com/Nuihc88/SVPlite) from scratch and then edit line per line. For tracking what Avisynth+ is doing internally, you'll need to use several programs, each of which could be giving misleading output due to several interface versions being used across them. Unless you are willing to play with the source code or attach debuggers, nothing else besides AviSynth+'s built-in script-interface is likely to generate usable diagnostic output.

I'm an SVP4 user (Smooth Video Project) and it uses Avisynth+, but seems to have issues with newer versions. I've found the SVP event log, but not a lot is mentioned there when things go wrong.

If i remember correctly, there was a compatibility issue with SVP and newer AVS+ versions due to a change in interface version around v3.6.0, since SVP used AvsF built for old interface version to retain backwards compatibility with ffdshow. So you'll likely need to use newer AvsF binaries with newer AVS+ or older AVS+ binaries with the AvsF bundled with SVP. This old 3.5.2 test build (https://drive.google.com/file/d/1EY1wgsh7pQD7kqj_2suD--nSj12hILX4/view) should work well with AvsF built for either interface version, while still having a more modern feature set and being one of the best performing and most stable AVS+ builds i have tested, however it has a few use scenario specific deadlock and memory leak issues, which have since been fixed in v3.7.2 line.

I found a debug registry key for AvsF, but I don't know if that'll show me everything happening inside Avisynth+ or not. The AvsF dev said they build a debug Avisynth+ version themselves and set a breakpoint. That's a bit more than I was hoping to do.

AvsF's debug feature can sometimes provide clues as to what is going on, but you'll need to be at least somewhat familiar with inner workings of both AvsF & AVS+ to interpret it at all.

I just found and downloaded a few older wrapper tools that appear to test dependencies and give resource usage for a script. I haven't tried running them yet on the SVP created AVS. I have a feeling it won't be that easy. That they use some stored value from MPC-HC to know what to play.

If SVP is configured to use AvsF, the media file will be loaded through AvsFilterSource() function in the script-file. You won't need to point it to a specific file.

To be clear, I'm not writing an AVS of my own. But am trying to help debug sporadic silent failures I ran across when upgrading a package myself (3.5.1 to 3.7.2). Where SVP reported itself disabled midway through the 2nd or 3rd episode of a TV series, and the video output still plays but not changed anymore. Kind of learning about all the parts by breaking them, and hoping to help the SVP dev(s) out with my efforts (or Avisynth+ if it's their bug).

Silent failure during real-time playback could be due to almost anything, but most likely it's due to some conditional setting in SVP profiles menu. I would recommend first checking whether you are able to reproduce the issue while using a custom script with SVP remote control interface disabled, thus bypassing SVP GUI options entirely.

[DTL]

May be add square hyperbolic zoneplate generator as internal source of test pattern ? https://github.com/DTL2020/hpzp (internal calculation in float 'narrow' range 16.0..235.0 - can output any bitdepth from float to uint8). It creates 'edges aligned and conditioned' to level 16 (video black in 8bit) rectangular buffer so it can be infinitely extended with AddBorders() with colour #10101010 without creating of 'unconditioned' stepping and adding more ringing in processing.
It may be used for testing scalers and some other processing (like quality of sub-sample shifting and so on).
In its simple form it is square pattern (though in theory can have different H and V frequency scale to be rectangular without going to out-of-band frequencies and mirror itself). So it require only 1 parameter of render size (width or height). The only some additional work is require for 'auto-scale' or end-frequency control param (currently manually adjusted when render size changes).

As addition to ColorBars internal test pattern generators.

Currently I create .raw file with executable and read into scripts via RawSourcePlus plugin.

[Ceppo]

QUESTION:
I see here (http://avisynth.nl/index.php/Internal_functions#Functions_for_frame_properties) that there is no field-matching property. Should I just use _PictType, and set it to B/U/C/P/N? Or I'm going to blow up some filter by doing so?
EDIT:
If there is no field matching, and I'm not mistaken, can a _match property be added?

[real.finder]

QUESTION:
I see here (http://avisynth.nl/index.php/Internal_functions#Functions_for_frame_properties) that there is no field-matching property. Should I just use _PictType, and set it to B/U/C/P/N? Or I'm going to blow up some filter by doing so?
EDIT:
If there is no field matching, and I'm not mistaken, can a _match property be added?

_PictType is for mpeg codecs IIRC so that not ok here

and you can add new property with the name you want, but it's better to see if someone has another opinion

edit: tivtc in vs use PROP_TFMMATCH, PROP_TFMD2VFilm, PROP_TFMPP and PROP_TFMField https://github.com/dubhater/vapoursynth-tivtc/blob/1713095068d18a2fe93598aaabd7e53e12163e03/src/TDecimate.cpp#L1515 so yes, made anything you want here :)

[Ceppo]

Tomorrow I will get into it, thanks :)

[StainlessS]

Parser error [maybe], Bug report here:- https://forum.doom9.org/showthread.php?p=1967744#post1967744

[Ceppo]

Does someone know if there is a limit to the number of props that you can add to the frame?

[Dogway]

Does somebody know what's going on in here? I get different values when iterative summing and a simple multiplication.

add = 0.641099
num = 0
for (i = 1, 177, 1) {
num = num + add }
subtitle(string(num))
and
subtitle(string(0.641099*177))

The former seems to be more accurate but I suppose slower.

[wonkey_monkey]

The looped addition is less accurate. It's losing precision due to the limited accuracy of floats (rule of thumb is approx 7 decimal digits) with each operation.

[Dogway]

Thanks wonkey_monkey. I guess this is something ubiquitous because either Excel or Win7 calculator were closer to the looped addition. I wanted to use the multiplication either way so no prob.

[LigH]

Please read about the IEEE Standard for Floating-Point Arithmetic (https://en.wikipedia.org/wiki/Floating-point_arithmetic) (IEEE 754 (https://en.wikipedia.org/wiki/IEEE_754)). There are standard number formats with 32 bit (single precision), 64 bit (double precision) and even 80 bit internally (extended precision); but none is absolutely accurate due to the facts that the precision of the mantissa is limited at all, and the conversion between the decimal and dual number system may require infinite fractions in dual for some finite fractions in decimal (when any negative power of 5 is involved).

AviSynth may only use single precision as this is sufficient for most calculations with video dimensions, color values, and even audio samples as results (the "precision", in terms of significant/available bits, of integer formats is usually even a lot lower).

[Dogway]

Thanks LigH, I read about it some time ago (https://forum.doom9.org/showthread.php?p=1950751#post1950751) but it still puzzles me because a multiplication is nothing more than a looped addition, unless it's doing some bitshifts with rationals, also every software giving out different values despite being under the same standard.


On another note, is this a bug?:
ExtractY()
CombinePlanes(last, planes="YYY", source_planes="YYY",pixel_type="YUV444")
I want to copy Y plane to U and V, only managed it to work with the following.
ExtractY()
CombinePlanes(last, last, last, planes="YUV", source_planes="YYY",pixel_type="YUV444")

[VoodooFX]

"YYY" for planes is probably a 'typo' in wiki (http://avisynth.nl/index.php/CombinePlanes):


string planes = ""
The target plane order (e.g. "YVU", "YYY", "RGB")

[Dogway]

Thanks VoodooFX changing to "YVU" worked, I still fail to fully comprehend this filter.
CombinePlanes(last, planes="YVU", source_planes="YYY",pixel_type="YUV444")

[Reel.Deel]

"YYY" for planes is probably a 'typo' in wiki (http://avisynth.nl/index.php/CombinePlanes):

I don't think it's a typo.

ExtractY()
CombinePlanes(last, planes="YYY", source_planes="YYY", pixel_type="YUV444")

"YYY" in this case means that any plane that is not defined in the planes parameter is set to 0. Likewise, planes="YUY" means that the Y and U planes will be copied from the luma plane of the source clip and since the V plane is not defined it will be set to 0. I will add this info to the docs, if there are no objections.

Same story for RGB:

ExtractY()
CombinePlanes(last, planes="RRR", source_planes="YYY", pixel_type="RBGP")

The green and blue planes of the resulting clip will be set to 0.

[VoodooFX]

I don't think it's a typo.

ExtractY()
CombinePlanes(last, planes="YYY", source_planes="YYY", pixel_type="YUV444")

"YYY" in this case means that any plane that is not defined in the planes parameter is set to 0. Likewise, planes="YUY" means that the Y and U planes will be copied from the luma plane of the source clip and since the V plane is not defined it will be set to 0.
Do you have an example where planes="YYY" "works"?

[Reel.Deel]

Do you have an example where planes="YYY" "works"?

What do you mean works? Like a use case? Of the top of my head I don't but the planes parameter works as I described it.

[Dogway]

I think I understand now:

source_planes="YYY"
|||
|||
vvv
planes= "YUV"

Except for the null exception Reel.Deel explained.

EDIT: To make it a reference post, I tested with two clips and the above doesn't stand (another exception), you need to do the shuffle with the clips.
CombinePlanes(src, last, src, planes="YUV", source_planes="YUV")
This means copy Y and V from src, keep U from last.

[Reel.Deel]

I think I understand now:

source_planes="YYY"
|||
|||
vvv
planes= "YUV"

Except for the null exception Reel.Deel explained.

That's correct.

[VoodooFX]

What do you mean works? Like a use case?
Yes, practical use case. That's what I meant by 'typo' as there is no practical use case for it [maybe I don't see it] then no need to mention it in wiki.
I think "YYY" should be mentioned in source_planes parameter.

[Reel.Deel]

Yes, practical use case. That's what I meant by 'typo' as there is no practical use case for it [maybe I don't see it] then no need to mention it in wiki.
I think "YYY" should be mentioned in source_planes parameter.

I don't have a practical use case and I agree mentioning "YYY" can be a confusing if the mapping behavior is not fully explained. I'll add to the docs when I get a chance.

[wonkey_monkey]

Thanks LigH, I read about it some time ago (https://forum.doom9.org/showthread.php?p=1950751#post1950751) but it still puzzles me because a multiplication is nothing more than a looped addition, unless it's doing some bitshifts with rationals

Exactly. Multiplication in a computer is not looped addition. That's way too inefficient (and it wouldn't work on non-integers anyway). It pretty much is bitshifts (the mantissas are treated as integers and integer multiplied).

https://www.gamedeveloper.com/programming/in-depth-ieee-754-multiplication-and-addition

[Reel.Deel]

EDIT: To make it a reference post, I tested with two clips and the above doesn't stand (another exception), you need to do the shuffle with the clips.
CombinePlanes(src, last, src, planes="YUV", source_planes="YUV")
This means copy Y and V from src, keep U from last.

Correct, the last supplied clip is used for the remaining planes. So if you want a specific plane to come from a particular clip, the clips must be provided in the correct order. Let's take your "copy Y plane to U and V" use case but also with an alpha.

clip1 = Blankclip(pixel_type="Y8").Subtitle("Clip1")
alpha = Blankclip(pixel_type="Y8").Subtitle("Alpha")

CombinePlanes(clip1, clip1, clip1, alpha, planes="YUVA", source_planes="YYYY", pixel_type="YUVA444")

The first 3 planes (YUV) are taken from clip1 and the A plane from the alpha1 clip. If it would of been specified as CombinePlanes(clip1, alpha, planes="YUVA", ...) it would mean that only the first plane (Y) would be taken from the first clip and the remaining planes from the second clip.

[LigH]

Multiplication in a computer is not looped addition.

A loop of inaccurate calculations adds up rounding errors to a much larger sum than the rounding error of one direct multiplication. You may remember the "generation effect" of re-encoding video multiple times with a lossy algorithm and changing quantization factors, which increases the opacity of DCT artifacts (blocks and edge ringing).

[wonkey_monkey]

Just an idea I had and thought I'd put here in case anyone wants to pick it up: ordered dithering for integer output formata could be added fairly easily to Expr. A 4x4 ordered dither could be done as a lookup to a 4-entry __m128 or __m256 table (based on current y coordinate mod 3) and a single add.

[LigH]

Disadvantage: Bayer dither is pretty ugly due to obvious patterns and banding. Every kind of error distribution dithering (even simple Jarvis-Judice-Ninke, Sierra Lite or Atkinson) would look better. Of course, SIMD streaming is harder here.

[FranceBB]

Disadvantage: Bayer dither is pretty ugly due to obvious patterns and banding. Every kind of error distribution dithering (even simple Jarvis-Judice-Ninke, Sierra Lite or Atkinson) would look better. Of course, SIMD streaming is harder here.

Yep, I generally almost always use Floyd Steinberg error diffusion and when I'm not using it, I'm using the Sierra-2-4A error diffusion. There are also the Stucki and Atkinson error diffusion techniques which are also good.

[Gh@nz]

Good day, gentlemans.
Can someone explaine how to connect a second processor?

OS Windows Enterprise LTSC 2021 x64 21H2
Dell T7810: Xeon E5 2669 x 2 + 64Gb RAM
Avisynth+ 3.7.2 (20220317) + FFTW v3.3.10 x32-64 + x264 aMod (DJATOM) Haswell var.

[takla]

Good day, gentlemans.
Can someone explaine how to connect a second processor?

OS Windows Enterprise LTSC 2021 x64 21H2
Dell T7810: Xeon E5 2669 x 2 + 64Gb RAM
Avisynth+ 3.7.2 (20220317) + FFTW v3.3.10 x32-64 + x264 aMod (DJATOM) Haswell var.

Maybe try Process Lasso (https://bitsum.com/)

[Gh@nz]

Maybe try Process Lasso (https://bitsum.com/)

Well, Avisynth+ 3.5.1 use both processors.
Whats wrong with Avisynth+ 3.7.2 or with me)))?

[Solved]

[Selur]

4:1:1 to 4:2:0 interlaced conversion throws:
Convert: Input ChromaPlacement only available with 4:2:0 or 4:2:2 sources.
see: https://forum.doom9.org/showthread.php?t=184197

[DTL]

BlankClip(pixel_type="YV411")
Info()
ConvertToYV12(ChromaInPlacement="DV",interlaced=true)


Throws error about 4:2:0 input only in both 3.7.0 and 3.7.2 versions.
Only

BlankClip(pixel_type="YV411")
Info()
ConvertToYV12(interlaced=true)

is working.

The bug looks like with 411 and _ChromaLocation property is set. And most of colour Convert* functions (except for ConvertToY() at least) fail now.

To reproduce:

BlankClip(pixel_type="YV411")
propSet("_ChromaLocation", 0)
ConvertToYV12(interlaced=true)


The only working Convert with such input stream looks ConvertToY() .

[Balling]

BTW, Spline144 is not ported from http://www.wilbertdijkhof.com/SplineResize_v02.zip and no 64 bit dll in the link.

[FranceBB]

BTW, Spline144 is not ported from http://www.wilbertdijkhof.com/SplineResize_v02.zip

I know a "certain lady" (https://forum.doom9.org/showthread.php?t=175187) who's gonna be happy if Spline144Resize() will ever make it to the core...

[ajp_anton]

Why does Subtitle treat "\n" as newline only if lsp is set? It feels unnecessary to add lsp=0 to pretty much all my Subtitle calls.

[Dogway]

Is there a way for Histogram(mode="classic") to output waveform in PC levels? Or maybe inherit from frameprops. It would save a lot of processing power by skipping a range conversion in the curve graphs of my filters, specially in preview mode in AvsPmod.

[ajp_anton]

Feature request:

Expr("x[X,Y]")
where X and Y can be other than just plain written integers. Even "2 1 -" fails. Would be awesome to be able to use variables like frameno and even pixels from other clips to build a displacement map. Don't know if it's worth it trying to implement float support with some kind of interpolation from surrounding pixels. Maybe just round to the nearest integer.

Also would be nice to remove the -width < X < width and -height < Y < height constraints, as it's already able to handle out-of-frame pixels.

I know this would kind of explode the amount of things one can do with Expr, but does that matter? The user doesn't need to use everything that it's capable of. This would allow the user to, in theory, recreate most plugins within this expression, and even though it might not be the most efficient method, it's better than nothing if one wants to do something that no plugin can do. Coming back to the float/int comment above, even if the coordinate is rounded to an int, one could implement one's own interpolation method if needed.

[Reel.Deel]

Is there a way for Histogram(mode="classic") to output waveform in PC levels? Or maybe inherit from frameprops. It would save a lot of processing power by skipping a range conversion in the curve graphs of my filters, specially in preview mode in AvsPmod.

No, Histogram(mode="classic") is hardcoded to TV levels. Even the "midpoint" line is at 125.5. (https://github.com/AviSynth/AviSynthPlus/issues/268#issuecomment-1048546724) Only recently, "levels" mode outputs PC levels when the input is RGB. Maybe ask pinterf, it would be nice to have an option in Histogram to output PC levels.

[Dogway]

Yes, mid-grey in TV levels corresponds to 125.5. It naturally shifts back to 127.5 when converted to PC levels.
The issue is that graphs in TV levels makes it hard to manipulate (overlay, etc).
Not sure I want to bother pinterf as I think he is semiretired.

[kedautinh12]

Avs+ r3682
https://gitlab.com/uvz/AviSynthPlus-Builds

[StainlessS]

Feature Request for Avs+,

Log to base n.

Can calc nDigits(int) in script to nearly 10,000,000 for base BASE where BASE = 10, using

BASE=10
Int(Log(Max(n,1))/Log(BASE))+1

but craps out due to imprecise calcs in 32 bit float.

test,

Function nDigits(Int n) { Return Int(Log10(Max(n,1)))+1 } # Equivalent for +ve int:- Strlen(String(n)) : Log10 v2.60+ [v2.58, Log10(n) = Log(n)/Log(10) ]


Function nDigitsB(Int n,Int "Base") { Base=Default(Base,10) Assert(2 <= Base <= 36,"nDigits: Bad Base") Return Int(Log(Max(n,1))/Log(Base))+1 }

E=0

# Error MisMatch just before 10,000,000

For(i=0,2000000000) {
n1=nDigits(i)
n2=nDigitsB(i)
if(n1!=n2) {
E=E+1
RT_DebugF("%i] n1=%d n2=%d E=%d",i,n1,n2,E,name="TestBUG: ")
}
if(i% 100000 == 0) {
RT_DebugF("%i] E=%d",i,E,name="PROGRESS: ")
}
}

MessageClip(String(E,"Done E=%.0f"))


Of course there are other tricky bits, eg -ve numbers, [EDIT: or those that we might want as unsigned, eg $FFFFFFFF]
denary uses '-' sign and magnitude, whereas others do not.

Anyways, Log to base n via 64bit double would help.
Tanks :)

[Dogway]

Yep, overall more precision would help (specially for all the matrix calculations), but it might be difficult to implement.

[StainlessS]

but it might be difficult to implement.

The feature requested because of one log divided by another with only 32 bit precision,
Log to base n, if done in 64bit avoids the problem.
If you mean implement 64 bit floats in AVS, I would not hold out too high an expectaion there,
AVSValue [C/C++] is a union of several types, biggest being 32 bit, to add a 64 bit double
would increase size of AVSValue by 4 bytes, and break a whole helluva lot of existing plugins, which only
expect the current sized AVSValue, crashes galore.
A lot of plugins [ALL probably ] would need be recompiled with the new headers with increased AVSValue size, and
you could never mix old/new plugins or old/new avisynth. [return value from a plugin is an AVSValue, so are arguments to a plugin]
Also, some less than brilliant code [maybe some of mine, but I think/hope not, use fixed size for allocating float arrays for buffers,
its hard {some may say impossible} to fit and 8 byte double redefined as float, into a 4 byte gap].

Doubt if anything like that will appear in the next few weeks. [or decades :EDIT: but there are some clever guys here, so who knows.]

[LigH]

I guess a generic logarithm function could quickly be added via plugin compiled in a language which can calculate it in high precision internally and then return a single precision float compatible to AviSynth ... :cool:

[Dogway]

Yes I have done before the divide by Log(base) to get a log() over a different base. Want to think it worked for my case.
I think it shouldn't be less difficult to implement than current log() or log10(). I mean there are examples of 64-bit intermediate so it can be implemented MulDiv (http://avisynth.nl/index.php/Internal_functions#MulDiv)

In that regard I would like to have similar functions for multiply in 64-bit Mul() and maybe Sum()

[StainlessS]

Doggy, Actually I'm gonna change my mind a bit, x64 AVS+ must [I presume] have facility to store 64 bit pointer in AVSValue [eg pointer to clip {or safe pointer to clip, or whatever it is}],
so maybe for x64 not quite so difficult, maybe. There would still be a lot of 'hidden' dangers though, way more likely than the Y2K whotsit that never really materialised.
[EDIT: Dont know why, but avs x64 did not even cross my mind only x86 avs, even though I only use x64 avs myself, go figure]

added via plugin compiled in a language which can calculate it in high precision internally and then return a single precision float compatible to AviSynth
Yep, thats exactly what I wanted but as builtin function, I could easily add it to eg RT_Stats, whenever I get around to going back to it.

[qyot27]

I mean, maybe this is the migraine talking, but if it really needed to be cordoned off from the existing AVSValue, couldn't a new AVSValue2 structure or something like it be possible that allows for all the bigger types (heck, some CPU arches can do quad precision, or at least have it listed even if there's few - or no - existing examples of code making use of it) without posing a compatibility issue with old plugins?

[cretindesalpes]

Can calc nDigits(int) in script to nearly 10,000,000 for base BASE where BASE = 10, using
BASE=10
Int(Log(Max(n,1))/Log(BASE))+1
but craps out due to imprecise calcs in 32 bit float.

This problem can be addressed in a different way, using 100% integer code:
n = Max (n, 1) # n assumed positive, and stored as Int
d = 0
for (i = 0, 31)
{
if (n > 0)
{
d = d + 1
}
n = n / base
}
return d
If at some point n is converted to floating point, results won’t be accurate for large numbers because the 32-bit floating point mantissa is 23 bits only (+1 implicit lead bit). So all integers in the +/-2^24 range can be represented exactly. Out of this range, only a few of them are exact. Therefore using a dedicated 64-bit log+base function won’t help much if the input is 32-bit float.

[StainlessS]

I guess then its best if I just implement nDigitsB() or similar name, as plug func, with some additional control, number base, and on whether to treat
-ve input arg n as unsigned[extra care for 0x80000000]. Also have to decide on whether '-' for base 10 -ve numbers is counted as a digit or not, + maybe some other tricky bits.

I guess for my current immediate needs, I'll just use something like eg Strlen(RT_NumberString(n ,base=BASE,width=0))
where I just wanna know how long printed variable types are, so as to produce a formatted output listing.
{but converting to a string for something like this, has long been repulsive to me due to AVS not having string garbage collection}

Also I'll just use eg StrLen(String(v)) for Float and Bool.

And, for RT_Stats DBase type 'bin' {an unsigned 8 bit int} can just use nDigitsB() as it currently stands.

{I wanna be implementing user configured format for generic DB records listing, eg print a particular field as dec or hex [EDIT: or binary], and with self adjusting formatted width for each field <after scanning for longest field data over entire DBase record range>}


RT_NumberString(int ,int "base"=10, int "width"=0)
First arg is an integer to convert to a number base/radix string.
Base, (10, 2 -> 36), is the number base or radix, eg 2 == Binary, 8 == Octal, 10 == Denary/Decimal, 16 == Hexadecimal.
The default of 10 (decimal) will just convert a number to its decimal string equivalent possibly with a '-' minus sign.
All number bases with the exception of decimal, will be unsigned form, ie -1 to hexadecimal will produce "FFFFFFFF",
(the sign is in the digits rather than as separate 'sign and magnitude' used in decimal representation).
The digits used for the base are, "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ". Binary uses first two; decimal, the first 10;
Hexadecimal the first 16; base 36 all 36 digits.
Width, (0, 0 -> 32) is the minimum width of the returned string.
eg RT_NumberString(255,16,4) returns "00FF".
To convert a Float to a decimal string use Avisynth native "String()" func.


Nifty bit of code there C, wish I still had my Knuth Volume 1 [long since left on a train somewhere], so as to check out
if thats where you got it. :)