AviSynth+ thread Vol.2

[guest]

Avs+ r3682
https://gitlab.com/uvz/AviSynthPlus-Builds

Hi ked,

Not sure what's going on with this build, but I have had major decoding problems on 1 PC, but every other PC it has worked fine.

Had to revert back to r3661, on that 1 PC...strange behaviour :sly:

[kedautinh12]

Hi ked,

Not sure what's going on with this build, but I have had major decoding problems on 1 PC, but every other PC it has worked fine.

Had to revert back to r3661, on that 1 PC...strange behaviour :sly:

You need report here
https://gitlab.com/uvz/AviSynthPlus-Builds/-/issues

[guest]

You need report here
https://gitlab.com/uvz/AviSynthPlus-Builds/-/issues

Yeah, I can't be bothered signing up, just to do that...I found a fix, so I'm happy.

[LigH]

But others may be left sad.

[guest]

But others may be left sad.

LigH, it's interesting how you quite often find some of my posts (anywhere in the Forum), and leave a very "thought provoking" comment :)

In my post about r3682, I mentioned that it only had a problem on 1 PC (so far), and going back to r3661 solved the problem, however, on further thought, it maybe an AVX thing, as the Xeon 5680's don't support AVX, and therefore, I will have several PC's that r3682 will probably have the same issue.

So even though I wont be reporting this "officially", I'm sure someone will pickup on this problem, and either give an explanation, or better still, fix it (for all the older PC's out there)

[FranceBB]

Scenario: I index a file flagged as TFF but that is actually BFF.

Question: Why when I use "AssumeBFF()" the frame property value _FieldBased isn't updated and stays to 2, Top Field First?

Here the indexing says _FieldBased(2) so Top Field First
https://i.imgur.com/2GFk93H.png

Here I use AssumeBFF() but the value of _FieldBased stays to 1, so Top Field First:

https://i.imgur.com/KUhYItx.png

That is until I set it myself with: PropSet("_FieldBased", 1)

https://i.imgur.com/0BIZ0cj.png



I generally nuke all frame properties with propclearall() but in our staff the collective decision was to make frame properties work, so I'm trying to make them work...

So... is this expected / the right behavior? And in case: why?

[LigH]

Interlaced clips are not generally field based. This property means something different: The result after "SeparateFields()" is field based because previously woven fields are now separate frames.

[FranceBB]

Interlaced clips are not generally field based. This property means something different: The result after "SeparateFields()" is field based because previously woven fields are now separate frames.

I know, that was in "Avisynth terms" before frame properties were introduced.
The "FieldBased" and "FrameBased" meant something else entirely and were about what you're describing, in fact you had AssumeFrameBased() and AssumeFieldBased(), but this "_FieldBased" has nothing to do with AssumeFieldBased(), it's a frame property provided by the indexer in an FFMpeg-like style, hence the nightmare and why I generally start every script with: propclearall() to go back to the good old Avisynth.

[StvG]

Keep in mind that AssumeBFF/AssumeTFF/AssumeFieldBased... are clip properties while _FieldBased is frame property.

[FranceBB]

Gotcha.
So the old Avisynth clip properties won't update the new ffmpeg frame properties and that's the expected behaviour 'cause it's the user himself that has to populate them with propset (), I see...
I guess I'll shy away from frame properties once again

[qyot27]

Okay, so after so many months of putting this off, I have some performance comparisons between different CPU architectures.

Performance benchmarks for AviSynth+ on PowerPC and ARM (Apple Silicon)

The test benches are:
Power Mac G5 Quad, 8GB of DDR2 SDRAM, 7200rpm hard drive, Mac OS X 10.5 "Leopard"
Power Mac G5 Quad, 8GB of DDR2 SDRAM, SSD in USB/FW enclosure, booted over USB 2.0, Debian (Ports) sid ppc64
M1 Mac Mini, 8GB LPDDR4X Unified memory, Internal SSD, macOS 12 "Monterey"
(Control system) Intel Core i5-9400, 64GB DDR4 SDRAM, NVMe M.2 SSD, Ubuntu 22.04

There currently are no SIMD optimizations for AltiVec (PowerPC), or NEON (ARM), so those architectures are essentially at SetMaxCPU("none") anyway. There's also some difference in the compilers used: the M1 Mac Mini uses the default AppleClang, the G5 in OS X uses GCC 7, while the G5 in Debian and the Core i5 both used GCC 12. And because of the difference in architectures, the Version() output does differ slightly in horizontal resolution.

Single-thread Script:
Version(5000).ConvertToYUV420()

Multi-thread script:
SetFilterMTMode("DEFAULT_MT_MODE", MT_NICE_FILTER)
Version(5000).ConvertToYUV420()
Prefetch(x)

Testing:
avs2yuv test.avs -o /dev/null

Build types:
Default build, equivalent to no outstanding optimzations (GCC -O0, in effect)
-O3, which is the highest(?) level of optimization GCC provides in a single preset and includes tree vectorization

+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| | G5 Quad, OS X | G5 Quad, Debian | M1 Mac Mini | Core i5-9400 (SetMaxCPU = none) | Core i5-9400 |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Single thread | 66.98 fps | 40.02 fps | 477.51 fps | 353.41 fps | 840.73 fps |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Single thread, -O3 | 260.32 fps | 319.96 fps | 2285.78 fps | 1373.73 fps | 7110.95 fps |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Multi-thread=4 | 172.41 fps | 150.31 fps | 1744.24 fps | 1387.84 fps | 3343.81 fps |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Multi-thread=4, -O3 | 639.01 fps | 1143.41 fps | 7897.31 fps | 5505.51 fps | 30668.71 fps |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Multi-thread=6 | - | - | - | 1990.05 fps | 4603.13 fps |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Multi-thread=6, -O3 | - | - | - | 6838.58 fps | 26036.46 fps |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Multi-thread=8 | - | - | 2233.69 fps | - | - |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
| Multi-thread=8, -O3 | - | - | 9821.22 fps | - | - |
+---------------------+---------------+-----------------+-------------+---------------------------------+--------------+
Yeah, the obvious problem above is the whole 'running Debian in an enclosure through a USB 2.0 port', rather than directly through the SATA interface inside of the Mac. I do wonder if the rather poor -O0 numbers on Debian reflect that issue somewhat, even if the -O3 runs don't.

To be fair, the Power Mac G5 Quad was released in 2005, the M1 Mac Mini in 2020, and the Core i5 machine was built in 2019 (initially with only 16GB of RAM, upgraded to 64GB at the end of 2021). The G5 also can take a maximum of 16GB, which I may eventually do, so I'm not exactly sure how that might affect later results for the same test.

The M1 is a heterogenous chip, with 4 performance cores and 4 efficiency cores. Running Prefetch(4) was both to make it comparable to the G5 being a quad core, but it also demonstrates the core topology in the M1, as I'm guessing it prefers the performance cores first and the efficiency cores second, as evidenced by the much smaller jump in performance going from 4 threads to 8 threads (not nothing, but not anywhere near like having 8 performance cores).

Version().ConvertToYUV420() is also a really, really synthetic and unscientific benchmark, and not really representative of importing actual footage and filtering it. But since Version() is often one of the first scripts someone runs, why not?

[real.finder]

I did test with 12400F in win11 with Avs+ r3682 by asd-g (G S in gitlab) https://gitlab.com/uvz/AviSynthPlus-Builds/-/blob/44dd302916f1097f7a5deb69d65f9b0963cc654b/x64/AviSynth.dll

SetMaxCPU("none")
Version(5000).ConvertToYUV420()

PS Z:\> .\avs2yuv test.avs - > NULL
Avs2YUV 0.30 (https://github.com/DJATOM/avs2yuv/releases/download/0.30/avs2yuv.exe)
Script file: test.avs
Resolution: 384x104
Frames per sec: 24
Total frames: 5000
Progress Frames FPS Elapsed Remain
[100.0%] 4999/5000 521.71 0:00:09 0:00:00
Started: Sat Sep 10 07:13:17 2022
Finished: Sat Sep 10 07:13:26 2022
Elapsed: 0:00:09

faster than M1 :D

edit: with avs2yuv test.avs -o NUL is even faster

PS Z:\> .\avs2yuv test.avs -o NUL
Avs2YUV 0.30
Script file: test.avs
Resolution: 384x104
Frames per sec: 24
Total frames: 5000
Progress Frames FPS Elapsed Remain
[100.0%] 4999/5000 651.68 0:00:07 0:00:00
Started: Sat Sep 10 09:04:01 2022
Finished: Sat Sep 10 09:04:08 2022
Elapsed: 0:00:07

To be fair, the Power Mac G5 Quad was released in 2005

I wonder what will PS3 CELL in Linux will do

edit2: btw, is there are a way to get GCC -O3 in windows without losing msvc plugins support?

[qyot27]

I did test with 12400F in win11 with Avs+ r3682 by asd-g (G S in gitlab) https://gitlab.com/uvz/AviSynthPlus-Builds/-/blob/44dd302916f1097f7a5deb69d65f9b0963cc654b/x64/AviSynth.dll

SetMaxCPU("none")
Version(5000).ConvertToYUV420()

PS Z:\> .\avs2yuv test.avs - > NULL

edit: with avs2yuv test.avs -o NUL is even faster
The first run was redirecting to a file named NULL, the second was actually going to the null device.

I wonder what will PS3 CELL in Linux will do
You'd probably be limited by what version of what distro you could run on it, as the only ones I was aware of are long dead or would have dropped support for the PS3 by now. Gentoo or one of the *BSDs (yes, I know, not Linux) might be the only really viable contemporary options left at this point.

GCC also dropped support for the SPE side of the Cell architecture in GCC 10, so you'd either have to make do with only the PPE (essentially a regular dual-core PowerPC, derived from the POWER4 like the G5 was), or you would have to go back to GCC 9 and hope it could optimize across the entire unit.

edit2: btw, is there are a way to get GCC -O3 in windows without losing msvc plugins support?
Nope. The C++ ABIs are incompatible with each other. Unless there's some way of wrapping C++ dlls in a C interface bridge, but I'm not holding my breath.

[qyot27]

edit2: btw, is there are a way to get GCC -O3 in windows without losing msvc plugins support?
What is possible, though, is having AviSynth+GCC live alongside the regular MSVC builds, with only one of them active at a time. 64-bit* builds of FFmpeg, x264, or avs2yuv will accept either one. GCC plugins can be stored in their own plugins_gcc and plugins64_gcc directories, which keeps them from conflicting with the MSVC ones.

*32-bit requires a completely separate build; when I've put up FFmpeg or mpv builds, that's what the -avsgcc suffixed ones are.

On Windows, I have symlinks to AviSynth.dll in system32 and syswow64 instead of the real .dlls, which I have in the typical AviSynth+ installation path. Then I can have many different versions there, with suffixes on them, and if I need to test one, I just change out which one is the active AviSynth.dll that the symlinks are pointed at:

$ ls -l
total 122980
-rwxrwxrwx 2 qyot27 qyot27 5811200 Mar 17 22:27 AviSynth32.dll
-rwxrwxrwx 2 qyot27 qyot27 351744 Dec 21 2008 avisynth32.dll.258
-rwxrwxrwx 2 qyot27 qyot27 5355520 Jan 11 2021 AviSynth32.dll.370
-rwxrwxrwx 2 qyot27 qyot27 6279168 Dec 29 2021 AviSynth32.dll.371
-rwxrwxrwx 2 qyot27 qyot27 15084112 Mar 20 22:43 AviSynth32.dll.372gcc
-rwxrwxrwx 2 qyot27 qyot27 378368 May 17 2016 avisynth32.dll.bak
-rwxrwxrwx 1 qyot27 qyot27 13509136 Feb 21 2021 AviSynth32.dll.staticgcc
-rwxrwxrwx 2 qyot27 qyot27 6542336 Mar 17 22:24 AviSynth64.dll
-rwxrwxrwx 2 qyot27 qyot27 5968384 Jan 11 2021 AviSynth64.dll.370
-rwxrwxrwx 2 qyot27 qyot27 6443520 Dec 28 2021 AviSynth64.dll.371
-rwxrwxrwx 2 qyot27 qyot27 19356916 Mar 20 22:43 AviSynth64.dll.372gcc
-rwxrwxrwx 2 qyot27 qyot27 19267536 Dec 24 2021 AviSynth64.dll.gcc
-rwxrwxrwx 2 qyot27 qyot27 14604864 Mar 21 2019 AviSynth64.dll.gcc2831
-rwxrwxrwx 1 qyot27 qyot27 2047488 Jan 11 2021 DevIL32.dll
-rwxrwxrwx 1 qyot27 qyot27 2300928 Jan 11 2021 DevIL64.dll
drwxrwxrwx 1 qyot27 qyot27 0 Jan 1 2022 docs
drwxrwxrwx 1 qyot27 qyot27 4096 Jan 1 2022 Examples
drwxrwxrwx 1 qyot27 qyot27 12288 Jan 1 2022 FilterSDK
drwxrwxrwx 1 qyot27 qyot27 4096 Jan 1 2022 License
drwxrwxrwx 1 qyot27 qyot27 40960 May 29 14:16 plugins
drwxrwxrwx 1 qyot27 qyot27 4096 Mar 20 14:54 plugins+
drwxrwxrwx 1 qyot27 qyot27 4096 May 20 23:11 plugins64
drwxrwxrwx 1 qyot27 qyot27 4096 Mar 20 14:53 plugins64+
drwxrwxrwx 1 qyot27 qyot27 4096 Mar 20 23:40 plugins64_gcc
drwxrwxrwx 1 qyot27 qyot27 4096 Mar 20 23:40 plugins_gcc
-rwxrwxrwx 2 qyot27 qyot27 170739 Dec 31 2021 readme_history.txt
-rwxrwxrwx 1 qyot27 qyot27 100640 Jun 19 2020 readme.txt
-rwxrwxrwx 2 qyot27 qyot27 23234 Jul 5 2018 'Setup Log 2018-07-05 #001.txt'
-rwxrwxrwx 2 qyot27 qyot27 259173 Nov 29 2020 'Setup Log 2020-11-29 #001.txt'
-rwxrwxrwx 2 qyot27 qyot27 336204 Jan 1 2022 'Setup Log 2022-01-01 #001.txt'
-rwxrwxrwx 1 qyot27 qyot27 356473 Jan 1 2022 unins000.dat
-rwxrwxrwx 1 qyot27 qyot27 1256527 Jan 1 2022 unins000.exe

[real.finder]

Unless there's some way of wrapping C++ dlls in a C interface bridge

something like LoadPluginEx.dll that used to load AviSynth v1.0x and v2.0x plugins in avs 2.5? anyway if we can get the speed of GCC -O3 then it worth the workaround

[Reel.Deel]

New test build (r3689): https://gitlab.com/uvz/AviSynthPlus-Builds

[wonkey_monkey]

Is there a way to override an internal function but still call the original version when required? E.g.:


function FlipVertical(clip c) {
return c.FlipVertical
}


As you'd imagine, this causes AviSynth to crash due to the recursive nature of the function, but is there, or could there be, some prefixed version of FlipVertical than can be called from within the function instead?

It would be useful to implement an environment-wide override for a built-in function, for example to workaround this oversight from way back when: http://forum.doom9.net/showthread.php?p=1849921

[EternalStudent]

Very cool you're testing on one of the newer Mac's with ARM! I used to mess around with a NVIDIA Jetson way back when (pre-public release), and wished there was more general support for arm64 hardware.

But on to my question... Are there any Avisynth+ logs I can look in, or a debug mode I can enable? I see a bit of text in a log, but it's very terse. And I'm not seeing an event connected with my problems.

I'm using SVP 4 and manually installed a newer Avisynth+ (the latest release 3.7.2). I'm having periodic visual hangs, but haven't found a log that mentions an issue. Nothing interesting/new in SVP's event log, nor Windows 10's event viewer, etc.

I'd estimate weekly or more, I see a video just stop changing on screen for 5-10 seconds, but the audio is still playing back fine. If I'm using other programs they continue working, but clicking on the player seems to cause the system to try to fix the issue. Which makes the whole UI hang for 2-3 seconds (mouse cursor stops moving), then suddenly it'll work again... and playback video frames at higher speed trying to get the video to match the audio position. No other crashes or instability is noticed so I think it's SVP related, and Avisynth+ seems a logical place to get more information.

I realize this is an atypical use case for the program. And that they package a specific older version, though I can't seem to figure out how to use that copy after my manually installed version. So if I revert it'll be to some random thing I've downloaded, not whatever their tested version is.

Last I looked they were using a pretty old version, like the first after starting development again. And I couldn't find a labeled version that printed the same text in the log (website said 3.5.1, but log said 3.5...it's been a bit since I last tried to downgrade so I might be remembering wrong).

System details... Intel i7-4790k, 32GB RAM, NVIDIA 1080 ti, Windows 10 64-bit, all with latest "release" updates and drivers. Virtualization features are enabled (HyperX) to allow WSL2 to function, but I haven't been using it lately. GeForce Experience is installed, and it's doing the video driver updates to the gaming driver (not studio version). Using MPC-HC + AviSynth Filter + (internal) LAV Filters for playback. Worst resource usage (CPU) is often 30-50% from the video playback, and GPU often 10%. Also have Firefox and low resource games running, but I've seen the video hangs with a variety of videos playing, games, and websites. And I'm not playing DRM videos on Firefox (mostly text documentation today).

[StainlessS]

Sounds to me like you could do with a re-install of W10, or revert to a previous good image (Macrium Reflect or whatever).
Nobody else seems to have your problems, I have pretty similar setup except for i7-8700, GTX 1070 and latest Standard studio driver.

There is a free version of Macrium Reflect (search for "Macrium Reflect free offline install") otherwise you get a downloader exe.
Can make USB bootable drive from Macrium Tools menu, thats the only part of Macrium that i use.

Can also try FoxClone if you like, is USB bootable cloner based on Linux, (Macrium can have problems with certain linux setups if you multi-boot),
can clone FAT/FAT32/NTFS, and Ext2/Ext3/Ext4 filesystems. [EDIT: I'de down the Std version, Edge version is Beta I think]

Good idea to make a known good image that is easily go-backable to.


EDIT: Here the Nvidia ADVANCED driver search page:- https://www.nvidia.com/Download/Find.aspx?lang=en-uk#
Latest Standard GeForce 10 x64 WHQL Studio = v472.84 / 13 Dec 2021
(I don't use the Game Drivers, I've had probs with them in the past - EDIT: Try install of Standard driver before re-install W10)

EDIT: Macrium Reflect (Free) x64 on FileHorse:- https://www.filehorse.com/download-macrium-reflect-64/

FoxClone:- https://www.foxclone.com/

EDIT: Also note from D9 thread "AMD, Intel and Nvidia driver issues and last recommended version"
Is there perhaps an error in OP PDF version number for 472.47 (there are two of them).
https://i.postimg.cc/nsv4CzC0/a3.jpg (https://postimg.cc/nsv4CzC0)

And from here [Official Advanced Driver Search | NVidia]:- https://www.nvidia.com/Download/Find.aspx?lang=en-uk#
https://i.postimg.cc/SjXvxLmr/b4.jpg (https://postimg.cc/SjXvxLmr)

472.84 from 2nd image seems to be latest standard WHQL Studio Driver, and given same date as the latest 472.47 in pdf, ie 13 Dec 2021.

EDIT: To Below, thanks :)

You are right, I correct it soon, some problems now.

[Reel.Deel]

Are there any Avisynth+ logs I can look in, or a debug mode I can enable? I see a bit of text in a log, but it's very terse. And I'm not seeing an event connected with my problems.


Try using the logging facility: http://avisynth.nl/index.php/SetLogParams

[flossy_cake]

Are there any frame-accurate source filters for avisynth that support DXVA2-copyback hardware accelerated decoding like LAV Video Decoder seemingly does? Can Avisynth be configured to use LAV Video Decoder?

When opening .avs files with MPC-HC x64:


If I use DirectShowSource() I get 2 instances of external LAV Video Decoder in the system tray: 1 instance says DXVA2-copyback is active, the other not.


If I use LWLibavVideoSource() I get only 1 instance of external LAV Video Decoder in the tray and it says DXVA2-copyback is not active.

I've read that DirectShowSource() isn't frame accurate.

Trying to understand how this all works -- I thought "source filter" was separate from "transform filter" but it seems in Avisynth world a source filter can do both.

[DTL]

"If I use DirectShowSource() I get 2 instances of external LAV Video Decoder in the system tray: 1 instance says DXVA2-copyback is active, the other not."

May be try to construct DirectShow graph manually in the GraphEdit/GraphStudioNext to check and be sure what is happen inside DirectShow part of Windows first and load saved .grf file into DirectShowSource() to prevent graph from auto-building ? (If it can save from auto-adjustment of graph after loading in DirectShowSource ?). You need to construct typical file playback graph using your required data decoder and delete final renderer (after checking if everything can playback OK) and save as .grf file. May be start from Render Media File command for auto graph building and adjust it if it not looks good.

Allowing GraphBuilder to auto-build DirectShow graph may sometime cause very wierd results like loading too much redundant modules and so on. May be it try to insert LAV Video Decoder twice to make colour conversion or something other not needed op. So first is real DXVA processing and second is just pass-through.

[StainlessS]

There is [EDIT: quite old] DirectShowSource2() [avss.dll] which is frame accurate but only supports video, no audio.
(No idea if DXVA2-copyback or whatever)
https://forum.doom9.org/showthread.php?t=134275

EDIT: avss.dll is from Haali media splitter, apparently there is updated Dss2Mod,

do search on it in D9 or Google.

EDIT: On Wiki:- http://avisynth.nl/index.php/DSS2mod

[guest]

New test build (r3689): https://gitlab.com/uvz/AviSynthPlus-Builds

Did someone forgot to "us" about this...

New test build (r3820)
https://gitlab.com/uvz/AviSynthPlus-Builds

[flossy_cake]

There is [EDIT: quite old] DirectShowSource2() [avss.dll] which is frame accurate but only supports video, no audio.
(No idea if DXVA2-copyback or whatever)
https://forum.doom9.org/showthread.php?t=134275

EDIT: avss.dll is from Haali media splitter, apparently there is updated Dss2Mod,

do search on it in D9 or Google.

EDIT: On Wiki:- http://avisynth.nl/index.php/DSS2mod

Thanks, but it looks a little old -- last update was from 2014. Would that really work with modern files? What is everyone using these days?

I was recommended LWLibavVideoSource which seems fine but the cache file generation takes forever, which DirectShowSource() doesn't suffer from. Also with LWLibavVideoSource I have to manually join the audio with something like AudioDub(LWLibavVideoSource(),LWLibavAudioSource()) -- am I doing this right? Should I be doing something else?

And it seems I was mistaken about the meaning of "frame accurate" -- I thought this meant the pixel values were decoded accurately. Seems that is referring to seek accuracy, which isn't that important to me (unless it interferes with other filters like TDecimate which rely on frame counters when eg. specifying manual overrides of certain ranges of frames).

[flossy_cake]

So first is real DXVA processing and second is just pass-through.

Yeah that occurred to me and I tested with 4k60HDR file which my system can only play smoothly if DXVA2 decoding is active, and it still stutters even though the 1st instance of LAV is saying DXVA2-copyback is active.

Anyway thanks I will check out my graph configuration. I have K-Lite codec pack installed, not sure if that's interfering.

edit: it seems k-lite already installed "Win7DSfiltertweaker" for me which I can use to try and control what decoders are used by DirectShowSource()

edit2: after investigating with GPU-Z which shows "video engine load" it seems that DirectShowSource() is infact using DXVA2-copyback decoding (via LAV). I don't know why it's stuttering on that video though. A separate issue is that "mediainfo" type metadata isn't getting passed to the renderer (in my case MadVR) so MadVR doesn't know it's a HDR video and tonemapping is wrong (on it's ctrl+J debug screen it says "best guess" instead of "says upstream").

edit3: the double instance of LAV Video Decoder seems to be due to MPC-HC being configured to use its own internal version of that filter, and unticking it inside MPC-HC options makes the 2nd instances disappear, except for LAV splitter which still has 2 instances. Framerate is still stuttery though. Nevermind, I guess DirectShowSource() just can't handle 4k60HDR. So for Avisynth I am limited to about 4k30 SDR it seems, which covers 99% of my videos so it's not a big deal. A bigger issue is the lack of metadata being passed to MadVR which could result in inaccurate colours since MadVR has to guess based on other metrics like resolution, framerate etc.

[flossy_cake]

A separate issue is that "mediainfo" type metadata isn't getting passed to the renderer (in my case MadVR) so MadVR doesn't know it's a HDR video and tonemapping is wrong (on it's ctrl+J debug screen it says "best guess" instead of "says upstream").

This seems to be the biggest issue I'd like to solve -- are there any Avisynth functions which can tell the Avisynth source filter to pass on metadata (like matrix, primaries, SDR/HDR etc.) so that when MadVR gets the pixel data from Avisynth it doesn't have to use "best guess"?

[StainlessS]

Thanks, but it looks a little old
I imagine that DirectShowSource (v1) looks even older.

Seems that is referring to seek accuracy, which isn't that important to me (unless it interferes with other filters like TDecimate which rely on frame counters when eg. specifying manual overrides of certain ranges of frames).
Where filters rely on frame accurate seeking, it will become important to you. [because the result may be garbled video]

[kedautinh12]

You know original directshowsource very old and don't update too long although avs+ still up to date

[gispos]

Did someone forgot to "us" about this...

New test build (r3820)
https://gitlab.com/uvz/AviSynthPlus-Builds

This repo contains Clang builds of AviSynthPlus.

Unfortunately I get error messages every now and then with these builds.
AvsPThumb (a delphi program) cannot create a clip every now and then. I have never had this problem with the official versions, but I have never tested a 'Clang' version before.
I don't know if it's the clang compiler or not.

Unfortunately I can't provide any exact information about the error, but I just wanted to post it here, even if it doesn't make much sense.

[kedautinh12]

This repo contains Clang builds of AviSynthPlus.

Unfortunately I get error messages every now and then with these builds.
AvsPThumb (a delphi program) cannot create a clip every now and then. I have never had this problem with the official versions, but I have never tested a 'Clang' version before.
I don't know if it's the clang compiler or not.

Unfortunately I can't provide any exact information about the error, but I just wanted to post it here, even if it doesn't make much sense.

Can you report to Asd-g??
https://gitlab.com/uvz/AviSynthPlus-Builds/-/issues

[qyot27]

This seems to be the biggest issue I'd like to solve -- are there any Avisynth functions which can tell the Avisynth source filter to pass on metadata (like matrix, primaries, SDR/HDR etc.) so that when MadVR gets the pixel data from Avisynth it doesn't have to use "best guess"?
What versions of AviSynth+ and LSMASHSource are you using?

Open the script in either FFmpeg to see the info printout, or play it back in mpv and throw the file stats up on screen by pressing i.

[flossy_cake]

What versions of AviSynth+ and LSMASHSource are you using?

The lastest one, just downloaded it a couple weeks ago and learning it for the first time.

Open the script in either FFmpeg to see the info printout, or play it back in mpv and throw the file stats up on screen by pressing i.

Test file (mkv): https://4kmedia.org/lg-daylight-hdr-uhd-4k-demo/

Opening the mkv directly in MPC-HC, which is configured to use LAV splitter & LAV video decoder: https://lensdump.com/i/1FME3o (everything is good)

Opening the mkv via an Avisynth script in MPC-HC (the script uses LWlibavVideoSource or DirectShowSource to open the mkv): https://lensdump.com/i/1FMkN9 (everything is wrong, MadVR can't see matrix, primaries, & EOTF).

Running the Avisynth script from ffmpeg comand line: https://lensdump.com/i/1FMCi2

[qyot27]

I said mpv. (https://mpv.io/installation/)

[VoodooFX]

Is there some bug in ConditionalFilter, variables in expression are lost with Prefetch?

https://i.imgur.com/0jDQZes.png

v = ColorBars(width=500, height=200, pixel_type="YV12")
Y = BlankClip(v, pixel_type="Y8", color_yuv=$000000)
w = BlankClip(Y, width=50, height=50, color_yuv=$FFFFFF)
m = Y.Overlay(w)
c1 = Y.Subtitle("1")
c2 = Y.Subtitle("2")
#ConditionalFilter(v, c1, c2, "Y.RT_YPlaneMinMaxDifference(mask=m) > 128")
ConditionalFilter(v, c1, c2, "Y.YPlaneMinMaxDifference() > 128")
Prefetch(2)

EDIT:
"global Y" solves error. Is this expected behavior?

[FranceBB]

I don't wanna be "that guy", but... I'm gonna say it anyway.
In 2017 (and again in 2019) I suggested adding XYZ to Avisynth so that we could handle it without workarounds just like we can handle RGB and YUV.
Aside from a bunch of people promoting VapourSynth (which I will NOT use) I didn't get much feedback.
I'm back in 2022 to see if there has been any progress on this and if there's any will to add this.
The reason is that nowadays it's basically impossible to handle XYZ stuff in Avisynth unless you make a very very very convoluted workflow.
Any XYZ stuff won't be indexed by LWLibavVideoSource() as it says that it's not supported, while FFMpegSource2() will index it but it will also convert it to YUV 4:4:4 16bit planar as the conversion is done internally with 16bit precision, so if you have like a 12bit XYZ you'll always end up with a 16bit YUV.
Now, although this isn't such a big deal for most people, it makes it impossible for those who receive MJPEG2000 files in XYZ to work with those inside Avisynth.
There are other people like Jean Philippe Scotto di Rinaldi (JPSDR here on Doom9) who use XYZ effectively, but had to mark it as fake RGB64 (while it's actually carrying XYZ) so anything that "receives" the XYZ will think it's RGB64 and it takes some workaround to make the whole chain from Avisynth to the encoder to work.

Recently I've got a user called spoRv asking for this too, so clearly there are people who work in XYZ other than me and who would like it to be supported.

I'm gonna quote myself from a reply I gave to a user here on Doom9 in a different topic, but still, it would be awesome to have proper XYZ support in Avisynth.


If you have a DCP, it means that it's a MJPEG2000 4:4:4 XYZ 12bit, however XYZ isn't natively supported in Avisynth (despite my desperate requests to support it dating back to 2017 and which I included in my meme collection https://forum.doom9.org/showthread.php?p=1953469).
There are some people like Jean Philippe who had to work in XYZ, so what they did was to create a fake RGB64 output on his ConvertYUVtoXYZ() function with XYZ inside. Some other people like Hydra/HolyWu/Asd just recognized that XYZ isn't supported in Avisynth and therefore indexers like LWLibavVideoSource() won't output anything and report an error:


https://i.imgur.com/sq24AWJ.png


Other people like Myrsloyk instead added automatic conversion, which is the case for ffms2, which brings me to the final reply of your question: if you index an XYZ 12bit file with FFVideoSource(), it will automatically bring it to 16bit, convert it to YUV and output a YUV 16bit stream.

As you can see here, my input was a MJPEG2000 in UHD XYZ 4:4:4 12bit and after indexing with FFVideoSource() it has become a YUV 4:4:4 16bit planar:

https://i.imgur.com/6j88Bqt.png


This is the Mediainfo of the original file:

Video
ID : 2
Format : JPEG 2000
Format profile : D-Cinema 4k
Format settings, wrapping mode : Frame
Codec ID : 0D010301020C0100-0401020203010104
Duration : 1 h 15 min
Bit rate : 248 Mb/s
Width : 3 996 pixels
Height : 2 160 pixels
Display aspect ratio : 1.85:1
Frame rate : 25.000 FPS
Color space : XYZ
Chroma subsampling : 4:4:4
Bit depth : 12 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 1.147
Stream size : 130 GiB (100%)
Title : Picture Track
Color range : Full



XYZ seen as is through MPV: https://i.imgur.com/46XuSBi.jpg
XYZ indexed by FFVideoSource() in Avisynth which automatically converts to YUV 16bit: https://i.imgur.com/xi8pHH4.jpg

[StainlessS]

@VX, as your post

v = ColorBars(width=500, height=200, pixel_type="YV12")
Y = BlankClip(v, pixel_type="Y8", color_yuv=$000000)
w = BlankClip(Y, width=50, height=50, color_yuv=$FFFFFF)
m = Y.Overlay(w)
c1 = Y.Subtitle("1")
c2 = Y.Subtitle("2")
#ConditionalFilter(v, c1, c2, "Y.RT_YPlaneMinMaxDifference(mask=m) > 128")
ConditionalFilter(v, c1, c2, "Y.YPlaneMinMaxDifference() > 128")
Prefetch(2)

https://i.postimg.cc/vHL42dk8/vx-00.jpg (https://postimages.org/)
Y, No problem for me.

EDIT: You using Neo (or Cuda [EDIT: or XP]) version Avs ?

EDIT: Swapping commented out ConditionalFilter() works Ok too (result looks same as above image).

[VoodooFX]

Y, No problem for me.

EDIT: You using Neo (or Cuda [EDIT: or XP]) version Avs ?

AvS+ 3.7.2 r3661
Maybe you are fooled by the first frame or scrolling in AvsPmod, try to play or encode it.

[StainlessS]

Oopsie Daisy, you are correct, I only loaded first frame in VD2

you could try GConditionalFilter(args="Y,M") [ie Grunt]


v = ColorBars(width=500, height=200, pixel_type="YV12")
Y = BlankClip(v, pixel_type="Y8", color_yuv=$000000)
w = BlankClip(Y, width=50, height=50, color_yuv=$FFFFFF)
m = Y.Overlay(w)
c1 = Y.Subtitle("1")
c2 = Y.Subtitle("2")
GConditionalFilter(v, c1, c2, "Y.RT_YPlaneMinMaxDifference(mask=m) > 128",args="Y,M")
#GConditionalFilter(v, c1, c2, "Y.YPlaneMinMaxDifference() > 128",args="Y,M")
Prefetch(2)

wurx OK

Oopsie Daisy:- https://www.youtube.com/watch?v=brMJfFK9d44

[VoodooFX]

you could try GConditionalFilter(args="Y,M") [ie Grunt]
Thanks for info, I might switch to that.

[TomArrow]

Hello. I'm trying to develop a plugin and running into super strange stuff. My plugin is an import plugin that produces an RGB64 image. When I try to use ConvertToRGB48() on the output I get the error

"Device unmatch: ConvertToRGB48[CPU] does not support [CUDA] frame".

I can assure you that my code does absolutely nothing CUDA-related. The same happens if I output RGB48 and try to convert to RGB64. This does not happen if I try to do this with a BlankClip.

However if I just leave it at outputting RGB64 and open the script without any conversions in VirtualDub, it shows me the image just fine with no issues.

Is this some new stuff? How do I fix this? I reckon the error might lie somewhere else and this might just be a symptom but I find this mystifying.

Edit: With ffmpeg I get access violation. Must be some other issue I guess that's bleeding over.

Edit 2: This actually happens even with an older version of the plugin: http://avisynth.nl/index.php/ImageSequence . If you take the 64 bit plugin and use it with the wildcard input filter for 2 jpeg files and then run ConvertToRGB48() after it, the same error happens, including the access violation in ffmpeg (but not in VirtualDub). I'm at a loss.

Edit 3: On a sidenote, my PC doesn't even have CUDA. It's an AMD APU without a discrete GPU.

Edit 4: Searching around on Google, I ignorantly (without knowing what it even does) tried adding onCPU(1) after the input plugin. Made no difference. Then I tried onCUDA(1) and it said "This AviSynth does not support memory type 2(CUDA)". This is, uh, really strange?

Edit 5: Ok it might have been an access violation that somehow went under the radar for years and nobody noticed. I fixed that, however I downgraded to a lower AVISynth version, so not sure if I'll get to test if it also solved the CUDA thing error.

Edit 6: Ok this problem was completely unrelated to the Access Violation problem. Bottom line is, every version above 3.7.0 (I tested from newest backwards) has this problem with the CUDA stuff.

[gispos]

This repo contains Clang builds of AviSynthPlus.

Unfortunately I get error messages every now and then with these builds.
AvsPThumb (a delphi program) cannot create a clip every now and then. I have never had this problem with the official versions, but I have never tested a 'Clang' version before.
I don't know if it's the clang compiler or not.

Unfortunately I can't provide any exact information about the error, but I just wanted to post it here, even if it doesn't make much sense.
New test build (r3820)
https://gitlab.com/uvz/AviSynthPlus-Builds

Since r3820 there are no more problems so far.
Nice!

[Stereodude]

Is prefetch supposed to slow down sources?

A script consisting of only a source

DGSource("HZLIP.dgi")

I get 550fps per AVSmeter without a prefetch or with a prefetch of 1. With a prefetch of 2 or more it drops to ~60fps. I'm sure you're wondering why I'm using prefetch with just a source. I'm not normally, but I'm trying to figure out if my script was source limited and saw this behavior.

[FranceBB]

Some plugins have their own multithreading that works internally and I'm pretty sure DGSource() is one of them.
For those who have their own multithreading and/or threadpool, Prefetch() will do nothing but harm.
To be absolutely fair, I'm not a big fan of Prefetch as I believe plugins should be multithreading and create their own thread-pool in the best and most efficient way, without relying on Avisynth trying to be smart.
Things used to be much worse back in the days (I'm talking about AVS 2.5 MT) and that came back to bite me in one of my first jobs as encoder for a streaming company when I was hardsubbing stuff as some filters didn't really like the temporal discontinuity introduced by some of the MT Modes.

Anyway, long story short, it's normal to get faster speeds without prefetch for some filters/plugins as it means that they create their own threadpool with their own multithreading (and rightly so).

[Stereodude]

Some plugins have their own multithreading that works internally and I'm pretty sure DGSource() is one of them.
For those who have their own multithreading and/or threadpool, Prefetch() will do nothing but harm.
To be absolutely fair, I'm not a big fan of Prefetch as I believe plugins should be multithreading and create their own thread-pool in the best and most efficient way, without relying on Avisynth trying to be smart.
Things used to be much worse back in the days (I'm talking about AVS 2.5 MT) and that came back to bite me in one of my first jobs as encoder for a streaming company when I was hardsubbing stuff as some filters didn't really like the temporal discontinuity introduced by some of the MT Modes.

Anyway, long story short, it's normal to get faster speeds without prefetch for some filters/plugins as it means that they create their own threadpool with their own multithreading (and rightly so).
FWIW, I did find that after adding another filter to the script that the prefetch at the end didn't slow down DGSource anymore. :confused:

[FranceBB]

FWIW, I did find that after adding another filter to the script that the prefetch at the end didn't slow down DGSource anymore. :confused:

Perhaps the filter you added is single thread and therefore nullified the advantage of DGSource()?

Try with:


DGSource()

Spline64ResizeMT()


###Use whatever parameters you like###


from plugins_JPSDR.dll
I know for sure that his plugins have their own multithreading and create their own threadpool, so you should see better results without Prefetch() compared to with Prefetch, thus confirming my hypothesis.


p.s if you don't get better results without Prefetch, then I'll join you in the :confused: :confused: :confused:

[Boulder]

My guess is that using Prefetch clogs up the lane between the CPU and the GPU in a single line DGSource's case. Have you checked what the GPU usage shows in Task Manager for the AVSMeter process? I've noticed that when launching an encode, it jumps very high at the beginning and then gradually slows down as the encoder does not request frames so fast (because my script has denoising, resizing etc. in it). In my case it's like 3-4% at maximum during the encode phase.

[Stereodude]

r3820 doesn't really do this anymore. I was using r3661 before. No prefetch is 550, prefetch(2) is 535, prefetch(12) is 500.

[DTL]

It may be a bug or issue with internal MT:

I sometime have lost of MT (single thread running) with adding


myclip=some_filter()
return myclip

Prefetch(N)


If simply left internal last clip as output

some_filter()

Prefetch(N)


- the MT of N threads is working. It can not be easy simulated with BlankClip but happens with last filter of MDegrainN (or Weave after MDegrainN) from mvtools for me. May be it depends on MT_MODE somehow too.

It is typically not a great issue because I use return as debug/tuning preview and production script is running without return at the end. But some finding.

[Gavino]

'return' ends the script execution so the Prefetch() is never invoked.

Do this:
return myclip.Prefetch(N)
or

return myclip. \
Prefetch(N)