AviSynth+ thread Vol.2

[Dogway]

@real.finder: Yes Clip properties are read only. I got the vs cheating, good one, not sure I understood the getparity() behaviour.

Also I have experienced some slowdowns in ex_edge() when replaced the inverse convolution (a few sums ands substractions) with an 'abs'. Is 'abs' optimized in Expr()? I found this question in StackOverflow (https://stackoverflow.com/questions/664852/which-is-the-fastest-way-to-get-the-absolute-value-of-a-number) which shows a few ways to avoid ternaries, one of the fastest is to use assembly or Intel intrinsics.

[wonkey_monkey]

Exrp's abs is a single fully vectorised instruction. It should be much faster than a few adds and subtracts.

[jpsdr]

And if i'm not wrong, on floating point abs is just setting the sign bit to 0, so, you can't do faster...

[Dogway]

Benchmarks for hprewitt:

425 fps Prefetch(4) 368 fps Prefetch(6)
Expr("x[1,1] A^ x[1,0] 2 * B^ x[1,-1] C^ x[-1,1] D^ x[-1,0] 2 * E^ x[-1,-1] F^ x[0,1] 2 * G^ x[0,-1] 2 * H^ " \
+"A B C + + D - E - F - abs F H C + + D - G - A - abs max","")
395 fps Prefetch(4) 376fps Prefetch(6)
Expr("x[1,1] A^ x[1,0] 2 * B^ x[1,-1] C^ x[-1,1] D^ x[-1,0] 2 * E^ x[-1,-1] F^ x[0,1] 2 * G^ x[0,-1] 2 * H^ " \
+"A B C + + D - E - F - A G D + + C - H - F - max F E D + + C - B - A - F H C + + D - G - A - max max","")


EDIT:Also I found a few (many) inconsistencies with the ConvertTo functions. You might want to have a look at them maybe.

1. All arguments names must be declared except for the first one (ie. "ConvertToXXX(mat,interlaced=false,ChromaInPlacement=cplacea,chromaresample=krn,ChromaOutPlacement=cplace)"
2. Interlaced and matrix arguments are swapped for the ConvertToYUVXXX family compared to ConvertToYV411, ConvertToPlanarRGB and ConvertToRGBXX.
3. ConvertToPlanarRGB, ConvertToRGBXX and ConvertToYUVXXX don't accept ChromaInPlacement argument for other than 420 inputs.
4. ConvertToYUVXXX doesn't accept ChromaOutPlacement argument for other than 420 outputs (in other words ConvertToYUV422, etc don't have ChromaOutPlacement argument)

[FranceBB]

ConvertToYUVXXX don't accept ChromaInPlacement argument for other than 420 inputs.

4. ConvertToYUVXXX doesn't accept ChromaOutPlacement argument for other than 420 outputs (in other words ConvertToYUV422, etc don't have ChromaOutPlacement argument)

Well, I get your point, but I think this was done because technically there's nothing like top_left alignment for anything other than 4:2:0. I mean, you can't have 4:2:2 Type 2 and 4:4:4 Type 2 as they're not standard. On this sentence "they're not standard" I would actually like to clarify myself before drawing any conclusion: I don't know if they're really not described as a standard, however I couldn't find the description of them anywhere and, as far as the delivery from third party companies is concerned, whenever I receive stuff like HDR movies in Apple ProRes or Motion JPEG2000 IMF etc in 4:2:2 or 4:4:4, they're always always always with the old classic MPEG-2 chroma placement.

Now we could ask ourselves a question: shall we introduce support for Chroma Placement for everything even if it's not formally described for the sake of having everything nice and dandy (i.e consistent)?

I don't know the answer, honestly, but I'd like to see what others think about this and where we should go.

[Dogway]

Wouldn't that be the use case for the Default() function?
We daily deal with non-standard messy clips to try to fix them, so if you know what you are doing you should be able to use a chroma placement conversion instead of a workaround.

Not that I care too much since the ConvertTo functions are being superseded by fmtconv, avsresize or my own solutions, just trying to help here.

[Dogway]

I think I found a new bug in Expr variables, possibly since the last variable bugfix in Expr() (test 11 I think).
Expr(last,"x[-1,1] A@ x[0,1] B@ x[1,1] C@ x[-1,0] D@ x[1,0] E@ x[-1,-1] F@ x[0,-1] G@ x[1,-1] H@ + + + + + + + 0.125 * W^
A B dup1 dup1 min swap2 max
C D dup1 dup1 min swap2 max
E F dup1 dup1 min swap2 max
G H dup1 dup1 min swap2 max
swap2 swap1 swap6 dup1 dup1 min swap2 max
swap2 swap1 swap4 dup1 dup1 min swap2 max
swap3 swap1 swap7 dup1 dup1 min swap2 max
swap6 swap1 swap5 dup1 dup1 min swap2 max
swap3 swap1 swap2 dup1 dup1 min swap2 max
swap3 swap1 swap6 dup1 dup1 min swap2 max
swap7 swap1 swap4 dup1 dup1 min swap2 max
swap2 swap1 swap5 dup1 dup1 min swap2 max
swap2 swap1 swap7 min
swap4 swap1 swap3 max
swap3 swap1 swap4 min
swap2 max
dup1 dup1 min swap2 max
E^ D^ H^ 50 <= x D E clip x ?","")

It has to do with remaining elements in stack and ternaries. If I explicitly assign a var to the last element it works as in:
E^ D^ H^ A^ A 50 <= x D E clip x ?","")

[FranceBB]

Wouldn't that be the use case for the Default() function?
We daily deal with non-standard messy clips to try to fix them, so if you know what you are doing you should be able to use a chroma placement conversion instead of a workaround.


Well fair enough, as long as Ferenc is willing to add them and no one has an issue with them being non standard, I don't see a problem with adding them. :)

[pinterf]

I think I found a new bug in Expr variables, possibly since the last variable bugfix in Expr() (test 11 I think).
Expr(last,"x[-1,1] A@ x[0,1] B@ x[1,1] C@ x[-1,0] D@ x[1,0] E@ x[-1,-1] F@ x[0,-1] G@ x[1,-1] H@ + + + + + + + 0.125 * W^
A B dup1 dup1 min swap2 max
C D dup1 dup1 min swap2 max
E F dup1 dup1 min swap2 max
G H dup1 dup1 min swap2 max
swap2 swap1 swap6 dup1 dup1 min swap2 max
swap2 swap1 swap4 dup1 dup1 min swap2 max
swap3 swap1 swap7 dup1 dup1 min swap2 max
swap6 swap1 swap5 dup1 dup1 min swap2 max
swap3 swap1 swap2 dup1 dup1 min swap2 max
swap3 swap1 swap6 dup1 dup1 min swap2 max
swap7 swap1 swap4 dup1 dup1 min swap2 max
swap2 swap1 swap5 dup1 dup1 min swap2 max
swap2 swap1 swap7 min
swap4 swap1 swap3 max
swap3 swap1 swap4 min
swap2 max
dup1 dup1 min swap2 max
E^ D^ H^ 50 <= x D E clip x ?","")

It has to do with remaining elements in stack and ternaries. If I explicitly assign a var to the last element it works as in:
E^ D^ H^ A^ A 50 <= x D E clip x ?","")
Eeer, this example is a little bit too long, I'm lost, what is the real problem? Should it give error and it does not, or works not as expected? I can see a W^ which is not used later but for the first sight this is all I can see. I don't know what to expect.

[Dogway]

Yes sorry, I tried to narrow down the expression to the minimum, but less than that it worked.

I rechecked again, and went from all past test versions, 12, 17, 19 and 20, this time it worked in all of them included test 20 so it seems like an obscure memory bug included recently. Read here (https://forum.doom9.org/showthread.php?p=1955158#post1955158).

I will try to pinpoint since what version it happens, most likely something related to long stack elements like in my example.

[pinterf]

Yes sorry, I tried to narrow down the expression to the minimum, but less than that it worked.

I rechecked again, and went from all past test versions, 12, 17, 19 and 20, this time it worked in all of them included test 20 so it seems like an obscure memory bug included recently. Read here (https://forum.doom9.org/showthread.php?p=1955158#post1955158).

I will try to pinpoint since what version it happens, most likely something related to long stack elements like in my example.
Thanks, I've got the access viola, no more help needed atm.

[Dogway]

Boulder was getting it with test14, so I don't think newer commits did affect that. For the time being I will work around it with the explicit var.

[pinterf]

I think the crash reason (addressing of an array's minus 1st element) was there even in the previous versions but somehow they were hidden. Lately I was not able to reproduce the addressing problem with any release version, only with my debug build.
Internal optimization of ternary operator when 'swap' was used in the condition/truecase/falsecase expressions did not like each other. Still testing.

[Dogway]

wow, well you got further than I thought. In some occasions I was also getting some errors when using swap or swap2 as ternaries outputs. Don't remember where but could be related.

Also a question, is there a limit for float values that can't get stored? In Adaptive Sharpen there's a part in the edgemask that goes up to 200.0 float or so. I need to scale it down and then back to pass it to another expression.


EDIT: BTW YPlaneMin can't return negative values (for floats)?

[pinterf]

New build: Avisynth+ 3.7.1 test build 21 (20211021) (https://drive.google.com/uc?export=download&id=1eExZZoR17JCx8G0wfp49Vfha9awz-6I9)
- Allow propGetXXX property getter functions called as normal functions, outside runtime
By default frame property values are read from frame#0 which index can be overridden by the offset parameter

Example:
Colorbars()
PropSet(last, "hello", 1) # Set to 1 for all frames
# Override to 2 with runtime function except for frameNo=1
ScriptClip("""if(current_frame!=1) {propSet("hello",2)}""")
n0 = propGetInt("hello") # same as propGetInt("hello",offset=0)
# or get the frame property from the Nth frame
n1 = propGetInt("hello",offset=1)
n2 = propGetInt("hello",offset=2)
# n0 and n2 is 2 (overridden in runtime)
# n1 will be 1 (keeps global setting)
SubTitle("n0/n1/n2=" + "{n0}/{n1}/{n2}".Format)

- Add parameter string "ChromaOutPlacement" in ConvertToYV16 and ConvertToYUV422 similar to YV12/420 conversions
4:2:2 conversions now allow ChromaInPlacement and ChromaOutPlacement parameters
"left" ("mpeg2") and "center" ("mpeg1", "jpeg").
Note 1: "top_left" and "dv" is still valid only for 4:2:0
Note 2: "mpeg2" (sale as "left") was so far the default for 4:2:0 and 4:2:2 sources as well.
- Source code: use common YUV-RGB conversion matrix values and generation throughout the project
Was: constants and calculations and inline code here and there.
New: YUY2 RGB conversions now allow matrix "PC.2020" and "Rec2020" (as a side effect)
- 4:2:0 conversions: ChromaInPlacement and ChromaOutPlacement parameters: (see http://avisynth.nl/index.php/Convert)
add "top_left" (new)
add "center" and "jpeg" (as an alternative to "mpeg1"), "left" (as an alternative to "mpeg2")
plus fixing Dogway's Expr issue (ternary+swap or variable store).

[FranceBB]

New build: Avisynth+ 3.7.1 test build 21 (20211021) (https://drive.google.com/uc?export=download&id=1eExZZoR17JCx8G0wfp49Vfha9awz-6I9)
- Allow propGetXXX property getter functions called as normal functions, outside runtime


Thank you so much for this! :D

[kedautinh12]

Thank you so much for this! :D

Reel.Deel will complain about only thanks :D

[Reel.Deel]

Reel.Deel will complain about only thanks :D

I only complain about your thanks because there was some threads that only had less than 15 posts and 60% of the thread were from you just saying "thanks" ... That was before Wilbert deleted those useless post :) ... and yet here you are again just adding noise.


New build: Avisynth+ 3.7.1 test build 21 (20211021) (https://drive.google.com/uc?export=download&id=1eExZZoR17JCx8G0wfp49Vfha9awz-6I9)
Thank you pinterf, I know you're a gentleman and always update the wiki but I'll get around to it in the next couple of days. My overworked 2 weeks are finally coming to an end.

[pinterf]

Thank you pinterf, I know you're a gentleman and always update the wiki but I'll get around to it in the next couple of days. My overworked 2 weeks are finally coming to an end.
Good observation :), no wiki on test build changes. I have planned updating wiki; now these changes seem to be final, readme changelogs can be used to do the additions. If you deal with it I thank you.

OFF:
Running season is finished for me with a 108km 3200m+ trail running race two weeks ago, you can imagine that I'm uber relaxed since then :) So I've got a bit more time nowadays.

[StainlessS]

Dont know if Deputy Doggy's edit was spotted [no mention in posted changelog].


EDIT: BTW YPlaneMin can't return negative values (for floats)?

EDIT: Is it supposed to. [if so, what then is the expected min/max limit on YPlaneMin, YPlaneMax, YPlaneMinMaxDiff]

EDIT:

Blankclip(pixel_type="YV12")
ConvertBits(32)
#info
Levels(0.0,1.0,1.0,-2.0,2.0)
SSS="""
y=YPlaneMin
RT_Debugf("%d] Y=%f",current_frame,Y)
S=RT_string("%d] Y=%f",current_frame,Y)
Subtitle(s)
return last
"""
Scriptclip(SSS)


https://i.postimg.cc/D0kLzBfb/I39587872-00.jpg (https://postimages.org/)
16.0/255.0 = 0.062745

EDIT:
Coring=False shows 0.0

Blankclip(pixel_type="YV12")
ConvertBits(32)
#info
Levels(0.0,1.0,1.0,-2.0,2.0,coring=false)
SSS="""
y=YPlaneMin
RT_Debugf("%d] Y=%f",current_frame,Y)
S=RT_string("%d] Y=%f",current_frame,Y)
Subtitle(s)
return last
"""
Scriptclip(SSS)

https://i.postimg.cc/J00VGcTM/a-00.jpg (https://postimages.org/)

EDIT: Ok, saw your next post answer, thanx P.

[pinterf]

Dont know if Deputy Doggy's edit was spotted [no mention in posted changelog].
Spotted but the solution is postponed.

Unfortunately YPlaneMin has an optional 'threshold' parameter which is a percentage, stating how many percent of the pixels are allowed above or below minimum. (Wiki)

The calculation is done by building a histogram internally.

Unfortunately 32 bit float is not a well behaving discrete type like a 10 bit format with pixels from 0 to 1023, from which we can build histograms.

So 32 bit float formats are translated (digitized) to 16 bits internally (e.g. 0..65535 for luma) and histogram is created from this converted values. This is where the clamp occurs. Or else how could we establish easily for which pixel value exceed we the 5% percent.

A possible solution is to make a real min-max when threshold is exactly zero. Planned feature.

[kedautinh12]

I meet problem when use avs cuda ver in megui. When i was indexed video .webm with directshowsource cuda ver, i meet error: "invalid index plugin". I returned avs cpu ver and problem was gone

Video sample: https://drive.google.com/file/d/1xnIgefEeOd4_gbtO2Z45E_cNqYBdMkyM/view?usp=sharing

[pinterf]

Theoretically plugins are not affected and do not differ from normal x64 version other than normal is still xp-compatible, but avisynth dll which can accept cuda filters is not. Is it the complete error message?
EDIT:
My machine was not able to read .webm with DirectShowSource (unsupported)
Using FFMS2 the clip was opened properly.

[Dogway]

Thanks a lot for the update!

So 32 bit float formats are translated (digitized) to 16 bits internally (e.g. 0..65535 for luma)
Maybe this has to do with the issue I was having, is there a limited working dynamic range for 32-bit float? It could also be useful to know if we can work with absolute HDRI's.

Also I searched in the InternalFunctions, there's no such thing as propCopy() right? to copy properties between one clip and another. (sorry to keep you busy :( )

[pinterf]

The project is request-driven, no need to excuse :)

[pinterf]

New build
Avisynth+ 3.7.1 test build 22 (20211022) (https://drive.google.com/uc?export=download&id=1M36wJoTtk15o_D3ihkhEx7FHk0DoQMaS)
There was a fix in the new propGetxxx feature.
New things:
- New function: propCopy(clip, clip [,bool 'merge'])
Copies the frame properties of the second clip to the first.
Parameter 'merge' (default false):
when false: exact copy (original target properties will be lost)
when true: keeps original properties, appends all parameters from source but overwrite if a parameter with the same name already exists.
- xxxPlaneMin xxxPlaneMax, xxxPlaneMinMaxDifference:
- 32 bit float formats: when threshold is 0 then return real values instead of 0..1 (chroma -0.5..0.5) clamped histogram-based result
- for threshold 0 they also became a bit quicker for 8-16 bit formats (~10% on i7-7700)

[Dogway]

So fast, you are in a roll! Now I have to undo my workarounds (Get+Set) hehe

Do you if it's possible to allow Format() to read array items? It's something that bothers me a lot to have to write Format(" x "+string(val[n])+" +")

[kedautinh12]

Theoretically plugins are not affected and do not differ from normal x64 version other than normal is still xp-compatible, but avisynth dll which can accept cuda filters is not. Is it the complete error message?
EDIT:
My machine was not able to read .webm with DirectShowSource (unsupported)
Using FFMS2 the clip was opened properly.

yeah, my machine can read .webm with DirectShowSource in avs+ cpu ver (use both AviSynth.dll, DevIL.dll, DirectShowSource.dll cpu ver). But can't read .webm with DirectShowSource in avs+ cuda ver (use both AviSynth.dll, DevIL.dll, DirectShowSource.dll cuda ver) and meet notice error in Megui: "Unable to render the file. You probably don't have the correct filters installed"

[guest]

I have to ask a pretty basic question, but is there a noticeable advantage in using the CUDA "build" ?

[pinterf]

It can accept specially written cuda plugins, that's all.
The other difference is the xp compatibility. (Until Visual Studio 2019 exist on my machine)

[pinterf]

yeah, my machine can read .webm with DirectShowSource in avs+ cpu ver (use both AviSynth.dll, DevIL.dll, DirectShowSource.dll cpu ver). But can't read .webm with DirectShowSource in avs+ cuda ver (use both AviSynth.dll, DevIL.dll, DirectShowSource.dll cuda ver) and meet notice error in Megui: "Unable to render the file. You probably don't have the correct filters installed"
Then it seems not an avisynth problem. Check MEGUI configuration and plugin folders or whatever it needs. Does it use its own Avisynth or the centrally installed one. Check if plugins are seen by AviSynth invoked by MEGUI. Try your script from avsmeter64 or virtualdub2 or avspmod. Wheck if your MEGUI is 32 or 64 bits. These are my ideas.

[StainlessS]

Would there be any further complications for the MeGUI Avisynth Wrapper thingy with the CUDA whotsit ?

[VoodooFX]

@pinterf
Do you get notifications from your GitHub repositories? I've made few posts at the issues...

[pinterf]

Yes, I receive notifications. Sometimes too frequently :). Which one was forgotten?

[FranceBB]

Yes, I receive notifications. Sometimes too frequently :).

If only your Avisynth contributions counted towards your Jira :p
Employee of the month, every month! :D

[VoodooFX]

Yes, I receive notifications. Sometimes too frequently :). Which one was forgotten?
Sometimes me too, if I don't have anything to answer then I add that reaction emote so user would know that I've seen a post. :)

At your masktools and AvsInpaint.

[pinterf]

For avsinpaint I surely did not get message. The other one is familiar.

[csd79]

Hi Everyone!

Using the MRestoreVect function I get the error message: Error reading source frame 0: Avisynth read error: Filter Error: Filter attempted to break alignment of VideoFrame. The script:

frange = 2

raw = FFMS2(source="src.mkv").Trim(26855,27012)
super = MSuper(raw, pel=2)
vectors = MAnalyse(super, blksize=32, search=5, delta=frange, multi=true)

MStoreVect(vectors)

# 2nd script, for variable values see above
MRestoreVect(AVISource("mv.avi"))

MDegrainN(raw, super, last, frange, thSAD=400, thSAD2=150)

I'm using AS+ 3.7.0 and mvtools 2.7.45, x64 versions. Vectors are saved using Lagarith RGBA.

I get the same error with different sources: UHD 10bit YV12 and SD 8bit YV12.

Any suggestions?

[pinterf]

Hi Everyone!

Using the MRestoreVect function I get the error message: Error reading source frame 0: Avisynth read error: Filter Error: Filter attempted to break alignment of VideoFrame. The script:

frange = 2

raw = FFMS2(source="src.mkv").Trim(26855,27012)
super = MSuper(raw, pel=2)
vectors = MAnalyse(super, blksize=32, search=5, delta=frange, multi=true)

MStoreVect(vectors)

# 2nd script, for variable values see above
MRestoreVect(AVISource("mv.avi"))

MDegrainN(raw, super, last, frange, thSAD=400, thSAD2=150)

I'm using AS+ 3.7.0 and mvtools 2.7.45, x64 versions.

I get the same error with different sources: UHD 10bit YV12 and SD 8bit YV12.

Any suggestions?
Avisynth+ is right. MRestoreVect issue. It must be made a bit smarter.
(It wants to create a subframe starting on unaligned memory position)

[csd79]

Avisynth+ is right. MRestoreVect issue. It must be made a bit smarter.
(It wants to create a subframe starting on unaligned memory position)

OK, thank you for the clarification.

[MysteryX]

A syntax feature that's long-overdue in Avisynth (unless it got added and I didn't know) is default parameter values.


function MyFunc(bool "param" = True)
{
}


instead of

function MyFunc(bool "param")
{
param = Default(param, True)
}

[MysteryX]

ConvertToYUV444 ...

chromaresample allows setting any resizer, but is there a way to specify b and c parameters of Bicubic? Or are the default b=1/3 c=1/3 highly recommended over using b=0 c=0.5 ?

[wonkey_monkey]

Or are the default b=1/3 c=1/3 highly recommended over using b=0 c=0.5 ?

b=1/3, c=1/3 will soften the image. b=0, c=0.5 is more preferable in that it doesn't change the pixels under any "null" transformation.

This choice of defaults has previously caused issues: https://forum.doom9.org/showthread.php?p=1849901

[MysteryX]

Yet there's currently no way of upsampling using b=0 c=0.5 with ConvertToYUV444 -- which means that function should be avoided altogether? Or use Spline36.

That's the kind of feature that's way overdue in Avisynth.

[wonkey_monkey]

Spline16 will probably be closer to Bicubic.

[MysteryX]

Spline16 will probably be closer to Bicubic.
It depends on the usage. Spline16 is an unbalanced sharpening kernel. (Default Bicubic is unbalanced blurring kernel)

I'm looking for numerically-accurate data for better processing in YUV444, and Bicubic(0, .5) is the most numerically accurate.

Spline36 looks pretty similar to Bicubic(0, .5)

[poisondeathray]

ConvertToYUV444 ...

chromaresample allows setting any resizer, but is there a way to specify b and c parameters of Bicubic? Or are the default b=1/3 c=1/3 highly recommended over using b=0 c=0.5 ?

Another option would be avsresize

z_ConvertFormat (pixel_type=, resample_filter_uv = , filter_param_a_uv = , filter_param_b_uv= ) ...

[DTL]

It depends on the usage. Spline16 is an unbalanced sharpening kernel. (Default Bicubic is unbalanced blurring kernel)

I'm looking for numerically-accurate data for better processing in YUV444, and Bicubic(0, .5) is the most numerically accurate.


Avisynth converters are of medium quality for everyday consumer use. And using simple linear resizers can not fix the design bugs of subsampled colour systems. For some better quality you anyway need highly non-linear transforms. So if built-in quality is not enough - it is better use custom designed script with planes separating/combining or external plugin. The only simple way to avoid these bugs is not going from good 4:4:4 to subsampled.

[MysteryX]

In my script I'm offering 3 upsampling methods: bicubic(0, .5), nnedi3 and ChromaFeconstructor_faster

In any case, built-in methods should allow configuring b and c parameters when upsampling. It wouldn't cause any backwards incompatibility. Meanwhile I can use FMTC to do the job.

btw is there a nnedi3 resampling script written in Avisynth?

[kedautinh12]

Here:
https://github.com/realfinder/AVS-Stuff/blob/Community/avs%202.6%20and%20up/nnedi3_resize16.avsi