AviSynth+ thread Vol.2

[DTL]

3.7.5 release again missed from first post. Also 3.7.4 dated of 2024 instead of 2025.

[tebasuna51]

Originally Posted by DTL

3.7.5 release again missed from first post. Also 3.7.4 dated of 2024 instead of 2025.

Modified first post (owner pinterf not qyot27), now Current link go always to the last version.

[pinterf]

Originally Posted by tebasuna51

Modified first post (owner pinterf not qyot27), now Current link go always to the last version.

Thanks. I forgot about it. Sometimes I completely disconnect from forums, or else they just suck me in, leaving less time for my actual tasks (this time also Avisynth-related).
By the way, looking at the download counts, the Windows ARM version has more downloads than the XP-specific one. I suppose this is just out of curiosity, since plugin support is quite limited, to say the least.

[FranceBB]

Originally Posted by pinterf

looking at the download counts, the Windows ARM version has more downloads than the XP-specific one. I suppose this is just out of curiosity, since plugin support is quite limited, to say the least.

Speaking of the XP specific version, the installer is installing the "wrong" ImageSeq.dll in plugins+ and DevIL.dll in system32 instead of the XP specific ones. After swapping them with the ones from the "files only" option under the "legacy" subfolder everything worked perfectly again.

About ARM, Windows on ARM has very limited adoption but the laptops are definitely out there. A friend of mine tried it. It's definitely faster than running the x86 version translated on the fly to ARM, but the problem with the ARM version of Avisynth is that you can't have the ARM frameserver run natively and the x86 incompatible plugins run via the translation layer, rather everything has to be ARM (running natively) or x86 (running via the translation layer). You can't mix and match, which is a bummer. By the way, my use case for ARM was different as I was trying to see if I could save some bucks on AWS by running c6g.2xlarge instances (8c/8th 16GB of RAM) powered by a Graviton 2 ARM host instead of using the c6i.2xlarge instances (8c/8th 16GB of RAM) powered by an Intel Xeon 8375C x86_64 host.

$0.3648 (x86) - $0.2918 (ARM) = $0.073 per hour which corresponds to a 20% saving / reduction.

So, if the ARM version manages not not be 20% slower in encoding it will save some pennies on my Avisynth farm.

[qyot27]

Originally Posted by FranceBB

Speaking of the XP specific version, the installer is installing the "wrong" ImageSeq.dll in plugins+ and DevIL.dll in system32 instead of the XP specific ones.

There is no DevIL.dll in the xp installer.

Clearly one of the libraries in the DevIL dependency chain has issues even when you use -T v141_xp to build all of them.

but the problem with the ARM version of Avisynth is that you can't have the ARM frameserver run natively and the x86 incompatible plugins run via the translation layer, rather everything has to be ARM (running natively) or x86 (running via the translation layer). You can't mix and match, which is a bummer.

ARM64X support isn't quite ready in the toolchain. Even when support for ARM64X (aka ARM64EC) lands, I think it's fairly obvious there's a different 800lb. gorilla to contend with there and why you still wouldn't be able to mix and match architectures, with a handful of possible exceptions.

Not to mention whether it applies to a library loading other libraries (like AviSynth+ and its relation to its plugins) vs. an application loading a library (like FFmpeg loading the AviSynth+ core) is something I don't think I've seen a definitive answer to. What I remember of the tests I attempted is that it didn't work in either case, but that was sometime between July-October of last year.

[DTL]

While testing different SIMD versions of processing functions some feature of SetMaxCPU() found with VirtualDub: It does not switch CPU type if 'reload (F2)' script. To really switch CPU type and SIMD functions, full VirtualDub process restart is required.

I do not not know how it works with other editors with avisynth.dll loading like AVSpmod.

Test script:

SetMaxCPU("something")
version
Info()

pinterf reply is normal feature of Windows .dlls loading (and current implementation in AVS+ ?) - https://github.com/AviSynth/AviSynth...ent-2868872022

[Selur]

As a side note: It really is annoying that sometimes default plugins are in plugins/plugins+/plugins64/plugins64+ when using files_only.
(I ended up loading those plugins manually from a separate folder, to not having to check where the plugins are located on the release.)
=> I'm fine, but not to confuse others, maybe just stick with 'plugins' and that's it.

[LigH]

Maybe a function that accepts a file name only and looks in the two folders of the matching bitness on its own?

[tormento]

Is there any good samaritan to build AVS+ version of DFTTest2?

I've asked WolframRhodium who kindly refused.

Wouldn't be so bad to have a GPU accelerated version of DFTTest.

@pinterf? @anyone?

Thanks for any effort.

[qyot27]

Originally Posted by Selur

As a side note: It really is annoying that sometimes default plugins are in plugins/plugins+/plugins64/plugins64+ when using files_only.
(I ended up loading those plugins manually from a separate folder, to not having to check where the plugins are located on the release.)
=> I'm fine, but not to confuse others, maybe just stick with 'plugins' and that's it.

I had to go back and check old releases, because I had no idea what this was referring to.

Only 3.5.0 deviates from the norm concerning the plugins[+|64|64+] naming, and given this post from shortly after release, it appears that I hadn't prepared a filesonly package for 3.5.0, so it was added after the fact, presumably by pinterf. GitHub seems to have done something to the file timestamps for anything before mid-December 2021, so I can't see exactly when in relation to the release that the 3.5.0 filesonly was uploaded.

All of the other release filesonly archives follow the same pattern, because I just copy the contents of the Output/ directories MSVC creates directly into an AviSynthPlus_VERSION_DATE-filesonly directory, under the relevant build architecture (x64, x64-xp, x86, x86-xp). There have been a couple of times the architecture name differed (3.4.0 used 'amd64' and 'i686', 3.7.0 used a slightly more verbose 'x86-64|-32[_xp]'), but that was it.

arm64 test builds were present in the packages for 3.7.0, 3.7.1, and 3.7.2, but was skipped in 3.7.3, IIRC because it was languishing with virtually zero discussion around it (logical, because it was *only* in the filesonly package and did not have an installer).

arm64 returned in 3.7.4 and 3.7.5 once we actually did introduce an installer, but - as I've mentioned in other posts - part of that mainlining of the WinARM installer was changing how we build for that platform, so the arm64 directory on 3.7.4 and 3.7.5 is laid out completely differently from the earlier test builds.

[VoodooFX]

One script had me scratching my head for half an hour because it was producing unexpected, weird effects, all due to a missing dot...

Why this doesn't produce error, it ignores everything on the line after missing dot and outputs RGB:

BlankClip()
m=BlankClip().trim(0,-1)ConvertToY8
m

This produce error:

m=BlankClip().trim(0,-1)ConvertToY8
m

This outputs Y8:

BlankClip().trim(0,-1)ConvertToY8

[wonkey_monkey]

IIRC, if the parser runs into something that otherwise be unexpected, it assumes it's the start of a new statement.

So your first code is equivalent to:

BlankClip() # implicit last
m=BlankClip().trim(0,-1)
ConvertToY8 # this converts the implicit last cliip
m

The second is equivalent to:

m=BlankClip().trim(0,-1)
ConvertToY8 # no implicit last; error
m

And the third is equivalent to:

BlankClip().trim(0,-1)
ConvertToY8

It also means stuff like the following is valid:

a=1b=2
foo=3bar=10foo
a=1a2a

[FranceBB]

Happy 25th birthday, Avisynth!
(May 19th 2000 - May 19th 2025)

[Gavino]

Originally Posted by wonkey_monkey

IIRC, if the parser runs into something that otherwise be unexpected, it assumes it's the start of a new statement.

That's correct.
To clarify, 'unexpected' here means a character that cannot legally continue the current statement.

It also means stuff like the following is valid:

a=1b=2
foo=3bar=10foo
a=1a2a

That produces the error 'I don't know what "a2a" means'.

[Jamaika]

Has avisynth been permanently removed from ffmpeg?
https://github.com/FFmpeg/FFmpeg/com...5b75147eac79ec

[ryrynz]

How is "Move avisynth and dvdvideo under external libraries group" removal exactly?

[Jamaika]

Hmm... Only I was adding statically to ffmpeg and was counting on plugin modifications in this twenty-fifth anniversary. Now it looks like two separate projects.

[LigH]

In my scope you are the only person trying to link AviSynth statically into ffmpeg...

[Jamaika]

Originally Posted by LigH

In my scope you are the only person trying to link AviSynth statically into ffmpeg...

I am surprised that the l-smash add-on for avisynth is static with ffmpeg
https://github.com/HomeOfAviSynthPlu...orks/releases/


https://github.com/HomeOfAviSynthPlu...76b7a4ec60b850
Modified ffmpeg? Probably yes.

[DTL]

Some sad information for Personal Computer users with current AVS+ filters and frame cache design. I asked a question at stackoverflow https://stackoverflow.com/questions/79633556/ the answer is: We still can not use all 32/64bit address space for software design if we need high performance computing. The only usable with best performance is address space of the size of CPU cache. It is only about 10 MBs of 10ths of MBs typically on top consumer CPUs (except Xeon MAX with HBM RAM onboard).

As answers shows - there still no standard way for software to skip data writing to RAM after generation inside the CPU core. (maybe some hacks with cache coherency control still exist ?) . Even if it is temporal data exchange in between AVS filters and no more consumer of this data exists in the future. It is the use case of a simple filter chain (without splitting of output to several consuming filters).

This causes the main performance impact: Each frame in AVS cache has its own virtual address space range. This means total address space used by AVS is much larger than CPU cache size if we process SD frames totally cacheable even with several threads running. If we only have 1 source buffer and 1 dest buffer to process in between all filters in a chain (of unlimited length).

But if each frame buffer of each intermediate frame in AVS+ cache has its own buffer and number of frames is Nxthreads_number for each connection between filters - this causes the total amount of virtual address space used to be much larger than the CPU cache size.

And with cycling processing of all frame buffers CPU produces data for each buffer and stores it first in some levels of fast cache. But production rate may be much higher than the cachelines storage rate to RAM via slow bus. This causes a quick exhausting of free cache lines and total CPU stall doing nothing and waiting for old frames to be downloaded from cache to RAM. CPUs do not have the right to overwrite still not saved to RAM cache lines with different virtual address data. This will cause data loss for later reading from these addresses.

The only legal way to increase performance in such host design - to send overwrite instructions to the same virtual addresses. This will legally update data in cache at the full write cache performance and do not cause CPU stall of the execution. The rest of data may be slowly downloaded into slow host RAM if time slots on the host RAM bus are present. Typically it is temporal data in between filters and no need for anyone more so can be skipped completely after the sink filter reads it all from cache.

This means general ideas to AVS filters and core design:
1. Use in-place transform filter design if possible (non-resizing and non-spatial non-temporal filters like Levels/Tweak/ColorYUV).
2. Stop using many frames cache (make an option for script writers ?) in between chained filters if possible and reuse a low number of allocated buffers (better only 2 buffers - source and dest and reassign at the next filter call for 1 input and 1 output filter or send pointer to trans in place filter). Need to test in debugger what happens if we set Prefetch(N,1) or even Prefetch(N,0) and make a script with a sequence of transform in place filters.
3. Make transform in place copies of filters where possible and instruct the filter manager of AVS core to use these filters with a single virtual address space allocated buffer in all filter chain while possible.

Also if there will be no usable (hacky) solution it looks we need to send request to intel and AMD for new instruction extension at least for SIMD like _mXX_loadu_invd(addr) of simply load data from addr and invalidate cache line with this addr. Though it is easy to do only for cache line aligned loads and if load full cache line only (so usable for AVX512 with 64bytes load and from L1D only with 64 byte cache line size).

I still hope some general solution exists like high-level operation of free(addr) or free_invd(addr) with also invalidating all cachelines containing addresses from memory-allocated area (with page-size granularity of 4KB). Though it may be natural if user of high-level language calls free() all data is now can be skipped and every free() call may also invalidate cache and save CPU from store not yet saved data from that memory region virtual addresses. This performance issue is really OS-design level with memory services to applications. It is strange if there is no solution yet.