Avisynth+

[FranceBB]

Is it the same Avisynth version (my second test build) which you had troubles with?

Nope, it's AviSynth+ 3.5 (r3106, 3.5, i386) (3.5.0.0)

Next test please

https://github.com/pinterf/AddGrainC/releases/tag/v1.8.1

Original parameters restored, simd noise blocks use now unaligned load (possible solution to FranceBB's crash?)

You did it. Now it works fine.
Thanks. :)

[MeteorRain]

Had some time looking at mvtools and masktools, they are pretty huge and hard to start. I may try to run QTGMC and MCTD and find the hottest path and see if there's anything I can improve.

Masktools, it's possible I'll start from F2's float version and improve from there. So, likely you'll see neo_masktools in a few weeks if everything goes right.

Another thing that I'm looking is dfttest. dfttest-avs has some WIN32 APIs, and there's vapoursynth-dfttest ported by holy. Maybe it's a good idea to combine them into one that would support AVS on Linux. As usual, I'll find a chance to remove YUY2 support, if there's any. And will change the plugin name if it breaks backward compatibility.

[pinterf]

You mean, mvtools-float? It is full floating point I think, even the hottest path: the SAD calculation.
Latest avs mvtools can do float as well, I separated the motion vector clip format from the format of the clip to process. Do motion vector search on 8 or 10-16 bits and use the results on whatever bit depth clip you have. As mv search is the bottleneck, speed penalty is minimal.
Avs mvtools has finer granularity in block sizes 2-3-6-8-12-16-24...64, vs stayed at original sizes but it has even 128 sized blocks. Avs mdegrain supports 16 bit output from 8 bit inputs.
Vs versions have much nicer code, my next move would be drop yuy2 support, drop stacked (probably there are people who don't even know what it is ) support and re-do the internal MT. Just because this complex codebase is non-maintanable. Four or five years ago I had to keep compatibility with the ancient software infrastructure, but by 2020 things has been changed.

[Stereodude]

If you get really bored > 8-bit support for MCTD would be nice. :)

[MeteorRain]

Thanks, pinterf, for the information. I may take a look at the differences between the 2 before deciding what to do. My plan was to take mvsf as a starting point, gradually adding back functions from the avs version. Considering float version having a higher precision I wonder if it makes sense to have a uint16 internal process variation -- more precise than uint8 but faster than float or double?

At this point, internal MT (C++PSTL), AVX2(Intrinsics), HBD(Template functions) and dual synth interface(wrapper) are the main reasons I'm refactoring these code. I'd like to see MCTD running on AVS+ Linux with zero glitch and at a reasonable speed. A side effect for dual synth interface is of course hopefully reducing the chance of market fragmentation.

BTW I just rewrote the temporal median into dual synth interface early this week. Benchmark speed is amazing with AVX2.

Stereodude, that's definitely what I'm approaching.

[pinterf]

It is even very hard to decide whether 8 or 16 bit clip based motion vector search is better. SAD calculation and comparison is the key, I think it is overkill to use double SADs (note that a 32 bit float may have even less precision at these scales than 32 bit int, double should be used. In classic integer SAD mvtools versions 8 bit SAD code is handcrafted asm which is borrowed from the x264/265 project and I think it has reached its optimization limit. Since the typical blocksize is 8x8 or 16x16 avx2 is not even considered there. Motion vectors however are of greater precision in F2's version, and can cover of greater length, classic mvtools is using s7.8 short precision for them. Still I don't think that it's a deal-breaker difference. But the outer filters that are using the result of the core motion vector search can be optimized, but since the bottleneck is mv search, moving a part from SSE2 to AVX2 can result in <1% benefit if any and even less if looking at a whole qtgmc process. And another difference between integer SAD based vs and avs mvtools: vs version is always using 64 bit for summing up SAD (mainly because of supporting 128x128 blocksize @16bit @3 planes needed it), avs mvtools is using 64 bits ("bigsad_t") when it is needed. Thus avs code is a bit more complex, but is possibly quicker at 8 bit clips.

But since you are not tied to the historical things and I can see clearly that you like the "art" side of programming rather than living with ancient hacks, if I were you I'd take F2's code, as you have said. I can imagine that it would even take less time than to understand and kick away a hack then another hack and another in an infinite loop.

[tormento]

Things are getting interesting here. :)

[MeteorRain]

But since you are not tied to the historical things and I can see clearly that you like the "art" side of programming rather than living with ancient hacks.

:devil: My rule is if there's an ancient alternative running for old folks I'd be happy to ditch the historical part. For example, for those who still need ancient fft3dfilter options they can keep using that, for those who don't, they can migrate to neo_ variant. The fact that my update still keeps pure C and SSE/SSE2 tells a lot about my point of supporting old folks -- if there's no significant overhead.

I'll take your information as a food for though. Let's see if I'll be able to work out something.

[pinterf]

Pure C is essential as a reference. How I hate reverse engineering those many hundred lines inline assembler (which is then able to run faster from C with modern compilers) Other than that I'd target only sse4.1, avx2 and avx512 options

[MeteorRain]

IKR! IT() is one of them and good god that's pure MMX code core functions. Took me days just to extract C code out of MMX, so much pain there.

[tormento]

Somethings buzzes in my mind since yesterday afternoon: what sense has SetMemoryMax now, at least on x64 systems?

We have all at least 8GB of RAM, wouldn't be easier to let AVS+ allocate the RAM it needs with no useless declarations?

[LigH]

Imagine running several processes in parallel ...

[real.finder]

IKR! IT() is one of them and good god that's pure MMX code core functions. Took me days just to extract C code out of MMX, so much pain there.

you can try a new challenge http://forum.doom9.net/showthread.php?p=1882073#post1882073 :D

also Reel.Deel said that the default for the original VerticalCleaner is 2, in RgTools the default is 1.
also here http://avisynth.nl/index.php/RgTools/VerticalCleaner

so RgTools should do same (should back to 2)

[tormento]

Imagine running several processes in parallel ...
Well, tell me a good value for SetMemoryMax and how is it respected.

I use SetMemoryMax(8000) simply because with 16GB of ram, I really don't know what value to put.

From Wiki:

In Avisynth+ this limit for default Memory Max is 1024MB for 32 bits and 4096MB on the x64 version

Well, 8000 seems to be valid too.

@pinterf?

[StainlessS]

SetMemoryMax, Max amount for frame cache, how many frames you reckon you is gonna need cachin' for your mega massive script ?
8000, is probably way overkill [I would think].

Anyways, I aint never played with 4K never mind 8K, but why not try test with (8000) cache, and your script, and then again with (4000, or maybe 2000), any speed diff ?

Setting too high a cache does nothing good, IanB has made many/(well some) posts about overkill where people allocated ridiculous amounts of mem, just
because they felt the need to justify purchase of their huge RAM sticks.

Several Processes In Parallel, might want to limit current script to NOT use all available RAM, because you want to also run another few scripts at same time,
limiting current script usage ensures that subsequent scripts also have some ram to play with.

EDIT Give your script loads of RAM with typical max RAM usage script, and see what RAM usage AvsMeter shows, then set to a little above what AvsMeter shows.

EDIT:
Where are people getting the crazy idea that a big SetMemoryMax is ever a good idea.

I have searched the forum numerous times looking for any fud and misinformational posts and I do not find any, all I find are the occasional post advising to turn it down. and lots of posts with people using insanely high values and having problems.

SetMemoryMax just sets the size of the Avisynth frame cache!

It does not control any other memory usage!

The frame cache only needs to be just large enough to hold the temporal requirements of the script. If a script has no temporal requirements, i.e. a frame that is required more than once in some part of the script then the cache is completely useless and may prevent some other memory usage from being able to malloc the memory that it needs to work.

A 1920x1080 YV12 video frame is ~3meg, as RGB32 it is 8meg, you can hold 64 such frames in 512meg. YV12 720x480 frames are only 0.5meg you can hold 1024 in 512 meg.

It takes some resources to manage the frame cache so doing a SetMemoryMax(2048) for a SD video and having 4000 odd buffers in play is just stupid and will actually slow the script processing down a little bit.

Advise :- SetMemoryMax, Turn it down!

[tormento]

Anyways, I aint never played with 4K never mind 8K, but why not try test with (8000) cache, and your script, and then again with (4000, or maybe 2000), any speed diff ?
To tell you the truth, I am currently limited by video memory, as I extensively use both CUDA and OpenCL. I have to limit multithreading to the number of threads that don't saturate my video card which, unfortunately, has 3GB of DDRx only.

On 4k I have to use 1 thread only, as a single frame process can saturate it. On 2k I can go up to 6 thread.

I will make some trials with AVSMeter and tell you actual used resources.

I know the post of IanB but it was referred to ancient AviSynth. I don't know what happened since AviSynth+ and if his considerations are still valid, higher color spaces given.

[StainlessS]

I don't know what happened since AviSynth+ and if his considerations are still valid
Well HBD frames are of course bigger (size x2 for 16 bit and x4 float) compared to sizes given above by IanB, Avs+ is actually less greedy on cache requirements than Avs Std,
so for same frame size would require less cache than avs std. 32 bit still limited in available memory, I assume you are using x64 else SetMemoryMax(8000) would not even work.
By the way, SetMemoryMax() on its own returns the size that avs is using.

Maybe something like

m=SetMemoryMax()
blankclip
Subtitle(String(m,"MemMax = %.0f"))


would show the default for your memory size. [with old x86 avs std, was I think half available|free RAM limited to max 500MB].

EDIT: You might even try a SetMemoryMax(8000) before above script, to see if Avs+ actually sets the value requested, or limits it assuming that you are daft.
(It probably does not limit it, assuming that you are not daft)

[StainlessS]

Default Memory Max on my machine with 12GB RAM, x86 1024MB, x64 3055MB [seems a bit high but I guess deliberate in case of HBD, probably fine if you got plenty RAM].
x64, 3055MB Looks to be about Available RAM / 3 [9295 / 3 = 3098MB]

Output from AvsInit.avsi (x86)

00000251 0.00000000 [3108] AvsInit:
00000252 0.00005326 [3108] AvsInit: Auto load plugins script ENTRY
00000253 0.00010580 [3108] AvsInit:
00000254 0.00377458 [3108] AvsInit_ShowInfo:
00000255 0.00380937 [3108] AvsInit_ShowInfo: AvsInit_Version = 1.10
00000256 0.00385103 [3108] AvsInit_ShowInfo: RT_Stats Version = 2.00Beta12
00000257 0.00389197 [3108] AvsInit_ShowInfo: SysInfo Version = 0.115000
00000258 0.00393473 [3108] AvsInit_ShowInfo: VersionString = AviSynth+ 3.5 (r3072, master, i386)
00000259 0.00397531 [3108] AvsInit_ShowInfo: SetMemoryMax = 1024
00000260 0.00401733 [3108] AvsInit_ShowInfo: GScript Available = AVS+
00000261 0.00405900 [3108] AvsInit_ShowInfo: OS Bitness = 64
00000262 0.00410175 [3108] AvsInit_ShowInfo: Avisynth Bitness = 32
00000263 0.00414487 [3108] AvsInit_ShowInfo: WorkingDir = C:\VideoTools\AvisynthRepository\AVSPLUS_x86\plugins\
00000264 0.00418545 [3108] AvsInit_ShowInfo: ProcessName = C:\Program Files (x86)\DAUM\PotPlayer\PotPlayerMini.exe
00000265 0.00422784 [3108] AvsInit_ShowInfo: ParentProcessName = PSPad.exe
00000266 0.00426914 [3108] AvsInit_ShowInfo: OSVersionString = Windows 7 (x64) Service Pack 1.0 (Build 7601)
00000267 0.00431045 [3108] AvsInit_ShowInfo: OSVersionNumber = 6.100000
00000268 0.00435139 [3108] AvsInit_ShowInfo: CPUName = Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz / Yorkfield (Core 2 Quad) 6M
00000269 0.00439305 [3108] AvsInit_ShowInfo: Cores = 04:04 (Phy:Log)
00000270 0.00443436 [3108] AvsInit_ShowInfo: CPU Extensions = MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1
00000271 0.00447747 [3108] AvsInit_ShowInfo: Total Memory = 12221MB
00000272 0.00452023 [3108] AvsInit_ShowInfo: Avail Memory = 9161MB'
00000273 0.00456153 [3108] AvsInit_ShowInfo: Screen Res = 1920x1080
00000274 0.00460175 [3108] AvsInit_ShowInfo: Screen BitsPerPixel = 32
00000275 0.00464414 [3108] AvsInit_ShowInfo: Time = Saturday 11 April 2020 11:29:21[GMT Standard Time]
00000276 0.00468544 [3108] AvsInit_ShowInfo: User TEMP Dir = C:\Users\LiquoriceRoot\AppData\Local\Temp
00000277 0.00472674 [3108] AvsInit_ShowInfo: ComSpec = C:\Windows\system32\cmd.exe
00000278 0.00476805 [3108] AvsInit_ShowInfo: Computer Name = PLEX-P2
00000279 0.00480899 [3108] AvsInit_ShowInfo: User Name = LiquoriceRoot


EDIT: And x64

00000543 0.00000000 [1952] AvsInit:
00000544 0.00003333 [1952] AvsInit: Auto load plugins script ENTRY
00000545 0.00006703 [1952] AvsInit:
00000546 0.00352278 [1952] AvsInit_ShowInfo:
00000547 0.00355792 [1952] AvsInit_ShowInfo: AvsInit_Version = 1.10
00000548 0.00359596 [1952] AvsInit_ShowInfo: RT_Stats Version = 2.00Beta12
00000549 0.00363364 [1952] AvsInit_ShowInfo: SysInfo Version = 0.115000
00000550 0.00366661 [1952] AvsInit_ShowInfo: VersionString = AviSynth+ 3.5 (r3072, master, x86_64)
00000551 0.00369886 [1952] AvsInit_ShowInfo: SetMemoryMax = 3055
00000552 0.00373111 [1952] AvsInit_ShowInfo: GScript Available = AVS+
00000553 0.00376444 [1952] AvsInit_ShowInfo: OS Bitness = 64
00000554 0.00379705 [1952] AvsInit_ShowInfo: Avisynth Bitness = 64
00000555 0.00382893 [1952] AvsInit_ShowInfo: WorkingDir = C:\VideoTools\AvisynthRepository\AVSPLUS_x64\plugins\
00000556 0.00386154 [1952] AvsInit_ShowInfo: ProcessName = C:\NON-INSTALL\VDUB\VDUB2\VirtualDub64.exe
00000557 0.00389379 [1952] AvsInit_ShowInfo: ParentProcessName = explorer.exe
00000558 0.00392639 [1952] AvsInit_ShowInfo: OSVersionString = Windows 7 (x64) Service Pack 1.0 (Build 7601)
00000559 0.00395828 [1952] AvsInit_ShowInfo: OSVersionNumber = 6.100000
00000560 0.00399125 [1952] AvsInit_ShowInfo: CPUName = Intel(R) Core(TM)2 Quad CPU Q9550 @ 2.83GHz / Yorkfield (Core 2 Quad) 6M
00000561 0.00402386 [1952] AvsInit_ShowInfo: Cores = 04:04 (Phy:Log)
00000562 0.00405683 [1952] AvsInit_ShowInfo: CPU Extensions = MMX, SSE, SSE2, SSE3, SSSE3, SSE4.1
00000563 0.00408907 [1952] AvsInit_ShowInfo: Total Memory = 12221MB
00000564 0.00412132 [1952] AvsInit_ShowInfo: Avail Memory = 9295MB'
00000565 0.00415393 [1952] AvsInit_ShowInfo: Screen Res = 1920x1080
00000566 0.00418617 [1952] AvsInit_ShowInfo: Screen BitsPerPixel = 32
00000567 0.00421987 [1952] AvsInit_ShowInfo: Time = Saturday 11 April 2020 11:38:25[GMT Standard Time]
00000568 0.00425211 [1952] AvsInit_ShowInfo: User TEMP Dir = C:\Users\LiquoriceRoot\AppData\Local\Temp
00000569 0.00428400 [1952] AvsInit_ShowInfo: ComSpec = C:\Windows\system32\cmd.exe
00000570 0.00431624 [1952] AvsInit_ShowInfo: Computer Name = PLEX-P2
00000571 0.00434921 [1952] AvsInit_ShowInfo: User Name = LiquoriceRoot
00000572 0.00438146 [1952] AvsInit_ShowInfo:


EDIT: SetMemoryMax(8000) on my system does indeed set memory Max to 8000.

[Groucho2004]

I will make some trials with AVSMeter and tell you actual used resources.

I know the post of IanB but it was referred to ancient AviSynth. I don't know what happened since AviSynth+ and if his considerations are still valid, higher color spaces given.
The proper way to do it with AVS+:

Put your SetMemoryMax() line on top of the script.
Put "SetLogParams("stdout", LOG_WARNING)" on the next line.
Run AVSMeter(64) and watch out for this warning:
"Caches have been shrunk due to low memory limit. This will probably degrade performance. You can try increasing the limit using SetMemoryMax()"

Increase the value for SetMemoryMax() until the warning disappears.

[Selur]

Trying to using ScriptClip with Subtitle to see the current frame number.
I see the frame number, but the align I gets ignored, it's always at the default align=7, even if I use align=4 in my code:
LoadCPlugin("I:\WORKSP~2\Hybrid\debug\32bit\AVISYN~1\ffms2.dll")
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
# loading source: F:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\YUV422_sample.avi
# input color sampling YUY2
# input luminance scale tv
FFVideoSource("F:\TESTCL~1\files\INTERL~1\INTERL~1\YUV422~1.AVI",cachefile="E:\Temp\avi_1deca8d5573c8cb8cd8489678b764f38_853323747_1_0.ffindex",fpsnum=25,colorspace="YUY2")
# current resolution: 720x576
# filtering
# color modifications
Levels(0,1.50,255,0,255)
# adding frame number
ScriptClip("Subtitle(string(current_frame),align=4)")
# Reinterlacing since interlaced output is requested
ConvertToYV12(matrix="Rec601", interlaced=true)
PreFetch(8)
return last
I also tried:
ScriptClip("""Subtitle(string(current_frame),align=4)""")
but I don't see where I go wrong.

-> Can someone tell me how to properly use ScriptClip to show the current frame number in a way that the alignment works?

Thanks

Cu Selur

Ps.: using Avisynth+ 32bit r3106

[StainlessS]

# Stack Overhead Subtitle Text, with optional FrameNumber shown.
Function TSub(clip c,string Tit,Bool "ShowFrameNo",Int "Col"){
c.BlankClip(height=20,Color=Default(Col,0))
(Default(ShowFrameNo,False))?ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))"""):Trim(0,-1).Subtitle(Tit)
Return StackVertical(c).AudioDubEx(c)
}

Colorbars

SHOW=True

TIT="Hello World"

#TSub(TIT,SHOW)

ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""))""")

Prefetch(4)


EDIT: Crap, you are right, align does not work with Prefetch, I guess I forgot about that when I was installing same version Avs as you.

Dont work.

Colorbars
TIT="Hello World"
#ScriptClip("""Subtitle(String(current_frame,"%.f] """+Tit+""""),align=4)""")
ScriptClip("Subtitle(string(current_frame),align=4)") # EDIT: Or shorter without additional subtitle string, still BUG
BUG=True # <<<<<<<< TOGGLE BUG
(BUG) ? Prefetch(4) : NOP
Return Last


EDIT: No align bug in v3.5 r3072.

[real.finder]

try with https://forum.doom9.org/showthread.php?p=1906016#post1906016

seems fine

[Selur]

@real.finder: I can confirm that version works fine. :)

[videoh]

BTW I just rewrote the temporal median into dual synth interface early this week. Thank you, MeteorRain, for giving the code under MIT license. Fabulous. Hoping it can help to eliminate code duplication from DGSource(). Built it in VS17 and it runs fine. Really great stuff!

Have to study it closely, though, because like pinterf it's pure C over here. CUDA has C syntax. :p

[MeteorRain]

My pleasure!

[pinterf]

@real.finder: I can confirm that version works fine. :)
This version is still under development, sometimes I'm leaking an actual test version to have feedback whether common functionalities are still working which will happen next week perhaps.

[tormento]

sometimes I'm leaking an actual test version
I use always your bleeding edge releases. :D

[Nuihc88]

And another Avisynth+ test build, please test it thorougly, I need more testers than myself :)

EDIT: link removed until I check real.finder's bug report
EDIT: test build 2nd edition:
https://drive.google.com/open?id=1iZMtqMsHSQNSehAZdyx1C5FZ7PTd-4qw


I just came here to say that this is the fastest (lowest latency) and most stable AviSynth(+) build for real-time usage that i have ever used. Even with the Neo fixes i never would have expected to see latencies or performance this good.

Under 'average stats' MadVR is reporting latencies between 0.2 - 4 ms, averaging <2ms under load with a relatively heavy SVP script, while playing back a 12bit FullHD HEVC encode on a 1.5 decades old computer; while at idle it would be sitting at ~0.15ms...

It's like there's no longer any bottle-necking on the AviSynth side of things; everything just runs incredibly smoothly with no seek glitching and even frame-drop recovery is near-instantaneous.
This build may have rendered AviSynth.dll based performance tweaking obsolete for anything other than synthetic benchmarks.

[tormento]

And another Avisynth+ test build
Would you please put prerelease too on GitHub?

I find them easier to find and get notifications too :D

:thanks:

[real.finder]

another out of the range float problem


ColorBars(width=640, height=480, pixel_type="yv12")
Convertbits(32)
mt_lutxy(last,Invert(),"x x y - 9.9 * +", use_expr=1)
#~ ConvertBits(8)
Converttorgb24

it's ok with ConvertBits(8) but with Converttorgb24() it's not

[pinterf]

There are 3 conversions there, to 444, to 8 bit then to rgb. I'm not sure in the order. I wonder which one fails?

[tormento]

Default Memory Max on my machine with 12GB RAM, x86 1024MB, x64 3055MB [seems a bit high but I guess deliberate in case of HBD, probably fine if you got plenty RAM].

I did really simple tests, my target is max FPS only.

SD material

Script:SetMemoryMax(xxxx)
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\eseguibili\media\DGDecNV_x64\DGDecodeNV.dll")
DGSource("E:\in\DVD 1_43 Paradiso + inferno\inferno.dgi",ct=12,cb=12,cl=4,cr=4)
CompTest(1)
SMDegrain (tr=4, thSAD=500, refinemotion=false, n16=true, mode=0, contrasharp=false, PreFilter=4, truemotion=false, plane=4, chroma=true)
Prefetch(6)
AVSMeter 2.9.8 (x64), 2012-2020, (c) Groucho2004
AviSynth+ 3.5 (r3132, master, x86_64) (3.5.0.0)

Number of frames: 1568
Length (hh:mm:ss.ms): 00:01:02.720
Frame width: 712
Frame height: 552
Framerate: 25.000 (25000/1000)
Colorspace: YV12
SetMemoryMax(128)
FPS (min | max | average): 2.726 | 1000000 | 23.45
Process memory usage (max): 1057 MiB
Thread count: 77
CPU usage (average): 73.6%
SetMemoryMax(256)
FPS (min | max | average): 2.630 | 1000000 | 23.61
Process memory usage (max): 1061 MiB
Thread count: 77
CPU usage (average): 73.9%
SetMemoryMax(512)
FPS (min | max | average): 3.295 | 1000000 | 23.65
Process memory usage (max): 1066 MiB
Thread count: 77
CPU usage (average): 73.8%
SetMemoryMax(1000)
FPS (min | max | average): 3.101 | 1000000 | 23.65
Process memory usage (max): 1064 MiB
Thread count: 77
CPU usage (average): 73.8%
SetMemoryMax(2000)
FPS (min | max | average): 2.826 | 1000000 | 23.65
Process memory usage (max): 1067 MiB
Thread count: 77
CPU usage (average): 73.9%
SetMemoryMax(4000)
FPS (min | max | average): 3.313 | 1000000 | 23.56
Process memory usage (max): 1062 MiB
Thread count: 77
CPU usage (average): 73.6%
SetMemoryMax(8000)
FPS (min | max | average): 3.515 | 1000000 | 23.51
Process memory usage (max): 1062 MiB
Thread count: 77
CPU usage (average): 73.7%

HD material

Script:SetMemoryMax(xxxx)
SetFilterMTMode("DEFAULT_MT_MODE", 2)
LoadPlugin("D:\eseguibili\media\DGDecNV_x64\DGDecodeNV.dll")
DGSource("E:\in\1_40 Alla ricerca di Nemo\nemo.dgi")
CompTest(1)
SMDegrain (tr=4, thSAD=500, refinemotion=false, n16=true, mode=0, contrasharp=false, PreFilter=4, truemotion=false, plane=4, chroma=true)
Prefetch(6)
AVSMeter 2.9.8 (x64), 2012-2020, (c) Groucho2004
AviSynth+ 3.5 (r3132, master, x86_64) (3.5.0.0)

Number of frames: 1456
Length (hh:mm:ss.ms): 00:01:00.727
Frame width: 1920
Frame height: 1080
Framerate: 23.976 (24000/1001)
Colorspace: YV12
SetMemoryMax(128)
FPS (min | max | average): 1.308 | 454542 | 11.16
Process memory usage (max): 1243 MiB
Thread count: 77
CPU usage (average): 66.3%
SetMemoryMax(256)
FPS (min | max | average): 1.946 | 285712 | 11.50
Process memory usage (max): 1241 MiB
Thread count: 77
CPU usage (average): 67.8%
SetMemoryMax(512)
FPS (min | max | average): 1.772 | 232556 | 12.43
Process memory usage (max): 1341 MiB
Thread count: 77
CPU usage (average): 69.4%
SetMemoryMax(1000)
FPS (min | max | average): 1.690 | 270268 | 12.38
Process memory usage (max): 1353 MiB
Thread count: 77
CPU usage (average): 69.0%
SetMemoryMax(2000)
FPS (min | max | average): 1.892 | 277780 | 12.17
Process memory usage (max): 1344 MiB
Thread count: 77
CPU usage (average): 69.0%
SetMemoryMax(4000)
FPS (min | max | average): 1.716 | 294115 | 12.16
Process memory usage (max): 1336 MiB
Thread count: 77
CPU usage (average): 68.6%
SetMemoryMax(8000)
FPS (min | max | average): 2.071 | 232560 | 12.24
Process memory usage (max): 1355 MiB
Thread count: 77
CPU usage (average): 68.9%
SetMemoryMax()
FPS (min | max | average): 1.830 | 263160 | 12.38
Process memory usage (max): 1358 MiB
Thread count: 77
CPU usage (average): 69.2%
Well, it seems we have a winner: SetMemoryMax(512). :D

SetMemoryMax have impact on encoding speed too.

HD material (the same above)

x264_x64.exe --output-depth 8 --crf 20.0 --preset slow --tune animation --profile high --level 4.1 --keyint 240 --aq-mode 2 --sar 1:1 --output "nemoxxxx.mkv" --frames 1456 --demuxer y4m --stdin y4m

SetMemoryMax(256): 2.99 FPS
SetMemoryMax(512): 8.95 FPS
SetMemoryMax(1000): 8.28 FPS
SetMemoryMax(2000): 8.44 FPS
SetMemoryMax(4000): 8.65 FPS
SetMemoryMax(8000): 8.80 FPS
SetMemoryMax(): 9.12 FPS

Setting tr=6 in SMDegrain and repeating the three bests:

SetMemoryMax(512): 6.52 FPS
SetMemoryMax(8000): 6.68 FPS
SetMemoryMax(): 6.83 FPS

And, as I perhaps suffer from OCD, I redid the test for SD.

SD material (the same above)

x264_x64.exe" --output-depth 8 --crf 20.0 --preset slow --profile high --level 4.1 --keyint 240 --aq-mode 2 --sar 64:45 --output infernoxxxx.mkv --frames 1568 --demuxer y4m --stdin y4m

tr=4

SetMemoryMax(512): 18.79 FPS
SetMemoryMax(8000): 18.83 FPS
SetMemoryMax(): 19:15 FPS

tr=6

SetMemoryMax(512): 13.30 FPS
SetMemoryMax(8000): 14.22 FPS
SetMemoryMax(): 13.40 FPS

Fluctuations are so small that, apart SetMemoryMax(256), we can't tell who is the real winner. Better to minimize occupied RAM or to leave lot of space for AVS to work?

Anyway, I will leave SetMemoryMax(8000).

[StainlessS]

What is your default [EDIT: x64] SetMemorymax ? [and installed RAM]
Avs+ default is probably gonna be a good all around choice.

Latest updates to Avs+ and Mvtools etc seem to be kicking ass as far as speed goes (nice one guys).

EDIT:

BlankClip.Subtitle(String(SetMemoryMax)) # using x86[1024MB] and x64[???] app

[tormento]

What is your default [EDIT: x64] SetMemorymax ? [and installed RAM]
8000 with 16GB installed on x64 Windows 10 enterprise.
Avs+ default is probably gonna be a good all around choice.
I agree. That's why I keep on using it. :D


BlankClip.Subtitle(String(SetMemoryMax)) # using x86[1024MB] and x64[???] app

Do you want me to execute that? You need some patience as I am encoding a loooooong job :D

[StainlessS]

Do you want me to execute that?
Nope, no point really, just shows Avs default setMemoryMax, and as you are encoding, so available ram reduced and will show
what is available for 2nd instance of Avs+ if you were to create one.

When I said "What is your default", I meant result of that blankclip script which shows the avs+ default, not what you set YOUR chosen SetMemoryMax at.
My default was about 3550MB on 12GB ram/9GB avail .

[tormento]

Nope, no point really
Isn't the AVSMeter a good indicator of how much RAM is used?

The funny thing about SetMemoryMax() is I can't find any reference of what it does when no parameter is given.

Now I am encoding almost everything with SetMemoryMax() instead of SetMemoryMax(8000) as it seems to give a little better speed for the reels and the script I encode.

[MeteorRain]

Do we have arrays as parameters in AVS+ now? Or are we still limited by passing strings? I'm looking at dfttest, and while vapoursynth version has moved to arrays, the original version still uses strings.

[StainlessS]

tormento,
UsedMB=SetMemoryMax() or UsedMB=SetMemoryMax(0) returns the currently used amount, so SetMemoryMax() without assigning to anything is a NOP, well does nothing useful.


SetMemoryMax

SetMemoryMax(amount)
Sets the maximum memory that AviSynth uses (in MB) to the value of amount. Setting to zero just returns the current Memory Max value. In the 2.5 series the default Memory Max value is 25% of the free physical memory, with a minimum of 16MB.
The default Memory Max is also limited to 512MB.
In Avisynth+ this limit for default Memory Max is 1024MB for 32 bits and 4096MB on the x64 version
DefaultMemoryMax = minimum(physical_memory / 4, secondary_memory_max_limit)
for classic Avisynth see the table below

Free memory <64 128 256 512 1024 2048 3072
Default Max 16 32 64 192 448 512 512

In some versions there is a default setting of 5MB, which is quite low. If you encounter problems (e.g. low speed) try to set this values to at least 32MB. Too high values can result in crashes because of 2GB address space limit.
Return value: Actual MemoryMax value set.

Examples:

SetMemoryMax(128)

SetMemoryMax

SetMemoryMax(amount)
Sets the maximum memory that AviSynth uses (in MB) to the value of amount. Setting to zero just returns the current Memory Max value. In the 2.5 series the default Memory Max value is 25% of the free physical memory, with a minimum of 16MB.
The default Memory Max is also limited to 512MB.
In Avisynth+ this limit for default Memory Max is 1024MB for 32 bits and 4096MB on the x64 version
DefaultMemoryMax = minimum(physical_memory / 4, secondary_memory_max_limit)
for classic Avisynth see the table below

Free memory <64 128 256 512 1024 2048 3072
Default Max 16 32 64 192 448 512 512

In some versions there is a default setting of 5MB, which is quite low. If you encounter problems (e.g. low speed) try to set this values to at least 32MB. Too high values can result in crashes because of 2GB address space limit.
Return value: Actual MemoryMax value set.

Examples:

SetMemoryMax(128)

http://avisynth.nl/index.php/Internal_functions#SetMemoryMax

MeteorRain,
Do we have arrays as parameters in AVS+ now?
[EDIT: Below how it was, apparently is changed as per later posts]
Arrays cannot be implemented for script so long as Avs+ supports v2.58 dll loading. [ AvisynthPluginInit2() ]

Arrays have always been usable within C/CPP dll.

So cannot call script or dll function from within script using any arrays.

Exception is where dll accepts any 0 or more, OR 1 or more, args of a type where avisynth converts to array when calling dll (or builtin).
eg

c2=SomeDllFunction(clip, Int1, int2, int3, int4, int5,"Hello") # dll gets array of 5 int.

You cannot create an array in script, nor can a script function accept an array.

From Framesel() source

env->AddFunction("FrameSel", "ci*[SCmd]s[Cmd]s[Show]b[Ver]b[Reject]b[Ordered]b[Debug]b[Extract]i", Create_FrameSel, 0);
// The AddFunction has the following paramters:
// AddFunction(Filtername , Arguments, Function to call,0);

// Arguments is a string that defines the types and optional nicknames of the arguments for you filter.
// c - Video Clip
// i - Integer number
// f - Float number
// s - String
// b - boolean
// . - Any type (dot)
// Array Specifiers
// i* - Integer Array, zero or more
// i+ - Integer Array, one or more
// .* - Any type Array, zero or more
// .+ - Any type Array, one or more
// Etc


So FrameSel dll script documentation prototype as so

FrameSel(Clip, int F1, ... , int Fn, string "scmd"="",string "cmd"="", bool "show"=false, bool "ver"=false,bool "reject"=false,bool "ordered"=true,bool "debug"=false, int "Extract"=1)

Where dll function has variable number of arguements of a type ie Array, then any arg following the array must be of a different type [otherwise Avs does not know where variable number
of args ends] OR, MUST be called using name of the arg following the variable number of args.
eg

SomeFunction(int Int1, ... ,int IntN, Int "Fred"=0)

If above must be called with fred arg, then must call as eg

x=SomeFunction(int1,int2,int3,int4,Fred=42) # Must be called with Fred=42 otherwise 42 added to IntN array and Fred assigned default 0.

[qyot27]

Do we have arrays as parameters in AVS+ now? Or are we still limited by passing strings? I'm looking at dfttest, and while vapoursynth version has moved to arrays, the original version still uses strings.
On Windows, no. Unless you're using a custom build where you passed -DNEW_AVSVALUE to the CXXFLAGS.

On POSIX, yes. NEW_AVSVALUE is on by default on non-Windows.

[pinterf]

On Windows, no. Unless you're using a custom build where you passed -DNEW_AVSVALUE to the CXXFLAGS.

On POSIX, yes. NEW_AVSVALUE is on by default on non-Windows.
Yes, for arrays and other forecoming features I recommend reading the first ~400 lines:
https://github.com/pinterf/AviSynthPlus/blob/master/distrib/Readme/readme_history.txt

edit: if I remember well, in NEW_AVSVALUE build colors can be passed to BlankClip in array-style as well.

[MeteorRain]

So is the code same for getting values from arrays?
_args[idx].IsArray() -> _args[idx].ArraySize() -> _args[idx][i].AsInt()

[pinterf]

Yes. But even if parameter definition allows arrays, for length=1 the parameter will appear as a single non-array of a specific value
(covering such cases when parameter list of variable number of elements is only a single integer for example and it is not put into an array, it remains as a single integer AVSValue)

Plus: even is such single values are not IsArray, ArraySize will report 1, and you can access it as [0].

For example here, in ConditionalSelect, args[2] can be an array of clips or a single clip.
https://github.com/pinterf/AviSynthPlus/blob/master/avs_core/filters/conditional/conditional.cpp#L228

NEW_AVSVALUE considerations:
Arrays in the above form cannot be named parameters, nor can they be provided as the new [] syntax.
Unnamed or named (and even zero sized) array can appear as specific parameter.

With new AVSValue concept arrays can have arbitrary deepness, with mixed type of elements, and are always "deep copy"-ed (except on the plain C interface)

Handling and parameter list
https://github.com/pinterf/AviSynthPlus/blob/master/avs_core/filters/source.cpp#L419
https://github.com/pinterf/AviSynthPlus/blob/master/avs_core/filters/source.cpp#L2020

Definition (using specific 'a' as parameter type):
As NEW_AVSVALUE is no in the mainstream Windows build, this option is not really documented other than in my readme_xxx.txt.

Original BlankClip definitions
"[]c*[length]i[width]i[height]i[pixel_type]s[fps]f[fps_denominator]i[audio_rate]i[stereo]b[sixteen_bit]b[color]i[color_yuv]i[clip]c
[]c*[length]i[width]i[height]i[pixel_type]s[fps]f[fps_denominator]i[audio_rate]i[channels]i[sample_type]s[color]i[color_yuv]i[clip]c

Extra BlankClip definitions, allow calling with e.g. BlankClip(colors = [4095, 12, 2311])

[]c*[length]i[width]i[height]i[pixel_type]s[fps]f[fps_denominator]i[audio_rate]i[stereo]b[sixteen_bit]b[color]i[color_yuv]i[clip]c[colors]a
[]c*[length]i[width]i[height]i[pixel_type]s[fps]f[fps_denominator]i[audio_rate]i[channels]i[sample_type]s[color]i[color_yuv]i[clip]c[colors]a

[MeteorRain]

I understand that NEW_AVSVALUE is not yet on mainstream Windows, but it is expected to be on Windows at a later time, right?

Regarding to prefix 'a', it doesn't have a type that means it's weak typed, right? So basically I can pass anything in and the filter is responsible for checking the type.

And what happens if I put 'a' in function declaration and filter is loaded by a version not supporting arrays? Do I have to keep 2 DLLs for different AVS version?

And --
(covering such cases when parameter list of variable number of elements is only a single integer for example and it is not put into an array, it remains as a single integer AVSValue)
I assume this does not apply to 'a' parameters, right? If I mark a parameter as 'a' and user pass a 1 element array, I should always get IsArray() true, right?

[pinterf]

I understand that NEW_AVSVALUE is not yet on mainstream Windows, but it is expected to be on Windows at a later time, right?

Regarding to prefix 'a', it doesn't have a type that means it's weak typed, right? So basically I can pass anything in and the filter is responsible for checking the type.

Yes, only requirement that it should be an array.

For an 'a' typed parameter parser will allow explicitely defined constant (using [ and ]) or an array typed variable or function return value.


And what happens if I put 'a' in function declaration and filter is loaded by a version not supporting arrays? Do I have to keep 2 DLLs for different AVS version?

I don't know other that trying it.


I assume this does not apply to 'a' parameters, right? If I mark a parameter as 'a' and user pass a 1 element array, I should always get IsArray() true, right?
Yes, such parameters can even have zero length (like a = [], I think there are examples on that in the readme)

[tormento]

UsedMB=SetMemoryMax() or UsedMB=SetMemoryMax(0) returns the currently used amount, so SetMemoryMax() without assigning to anything is a NOP, well does nothing useful.
It can't be a NOP as it has results on encoding speed usually better than put any value.

[feisty2]

So is the code same for getting values from arrays?
_args[idx].IsArray() -> _args[idx].ArraySize() -> _args[idx][i].AsInt()

this is horrifying, it should be simplified to

for (auto x : _args[key])
// do ur thang

[MeteorRain]

Feisty2, all the API here that we've been talking about would have to maintain enough compatibility all the way back to the date when C++03 was not even a thing.

My wrapper would take and return a std::vector which then should support all the fancy stuff ( for (auto &&item : array_param) ) but then the underlying code -- those doing heavy-lifting, would have to use an interface where technology is much outdated.

[real.finder]

[EDIT: Below how it was, apparently is changed as per later posts]
Arrays cannot be implemented for script so long as Avs+ supports v2.58 dll loading. [ AvisynthPluginInit2() ]

I wonder if avs+ could use some wrapper for AvisynthPluginInit2 to not interfere with avs+ features like HBD and Arrays

[feisty2]

adding begin() and end() doesn't harm compatibility, these are the "magic" functions that automatically enable range-for syntax in c++11 and onwards, in older standards, they are simply regular member functions and do not correspond to any magic syntax.

[StainlessS]

It can't be a NOP as it has results on encoding speed usually better than put any value.

Well it aint supposed to do anything other than return the used value.

I wonder if avs+ could use some wrapper for AvisynthPluginInit2 to not interfere with avs+ features like HBD and Arrays
Well I think that the problem was that could not previously support arrays [EDIT: for script] whilst still supporting v2.58 plugins, judging by above posts that problem seems to have been overcome.

https://www.youtube.com/watch?v=PPvRsLWlDXw

Says it all :)