AviSynth+ thread Vol.2

[pinterf]

AviSynthPlus Builds r4035 with IntelLLVM and Clang build by Asd-g, fix this error (https://forum.doom9.org/showthread.php?p=1994621#post1994621) too
https://drive.google.com/file/d/1OcPJNK_qEuYPTsL6yXvy-FV3XATUx9Io/view?usp=drive_link
What error? You mean there was an error in your previous 3rd party unofficial Avisynth build?

[kedautinh12]

Yeah, fix error from IntelLLVM build

[pinterf]

And what's new in the build? Please write something, what it contains, because just dropping yet another link here is not enough.
Or just keep posting in your 'I'm the quickest one who found a new stuff on the internet' topic.

[kedautinh12]

Did you see "r4035" same with your test 10. Why i put more info when you were post changes in test 10?? I added more info "it contains IntelLLVM and Clang build"
https://forum.doom9.org/showthread.php?p=1994584#post1994584

Modern days, are people lazy for just click and read info?

[tebasuna51]

I have 3 r4035 now:

05/12/2023 02:54 7.252.480 AviSynthClang.dll
05/12/2023 02:54 9.033.216 AviSynthILLVM.dll
02/12/2023 19:48 4.958.720 AviSynthTest10.dll

What is the recommended one and for what?
The changelog is the same.

[kedautinh12]

Test what is faster build and choose, easy. For personal PC, speed is different for one by one build. That mean one build can't always faster than other build so i'm not recommended what build is better one
Example: your PC faster with clang build than otther build, you can choose clang. My PC faster with IntelLLVM, i choose IntelLLVM

[FranceBB]

What is the recommended one and for what?
The changelog is the same.

Strictly speaking, it might not be totally correct, but I go by the following line of reasoning:

- anything Ferenc posts is considered beta and will eventually become stable in Stephen's (qyot) final builds which is what I consider the official releases

- anything everyone else commits in other repositories might or might not be merged upstream and is considered experimental


TL;DR if there's a build, I always pick the Ferenc one.

[VoodooFX]

I think Asd-g doesn't add there unofficial commits, just that those unofficial builds are less tested and may contain weird bugs.

[StvG]

Yeah, fix error from IntelLLVM build

And what's new in the build? Please write something, what it contains, because just dropping yet another link here is not enough.
Or just keep posting in your 'I'm the quickest one who found a new stuff on the internet' topic.

Related - https://github.com/AviSynth/AviSynthPlus/commit/77ac885cfa64cdcec0469a87237a44d3472d8425

[tebasuna51]

My test, a QTGMC() over a interlaced DV:

AVSMeter64 02/12 AviSynthTest10 05/12 AviSynthClang 05/12 AviSynthILLVM
-------------------------- --------------------- --------------------- ---------------------
Frames processed: 40430 (0 - 40429) 40430 (0 - 40429) 40430 (0 - 40429)
FPS (min | max | average): 35.68 | 83.37 | 59.84 35.81 | 95.07 | 58.98 35.44 | 92.10 | 58.25
Process memory usage (max): 382 MiB 370 MiB 373 MiB
Thread count: 29 29 29
CPU usage (average): 28.6% 28.3% 28.3%

Time (elapsed): 00:11:15.583 00:11:25.530 00:11:34.072

Then the last ones are slow for me.

[kedautinh12]

Like i said
https://forum.doom9.org/showthread.php?p=1994786#post1994786

[gispos]

Did you see "r4035" same with your test 10. Why i put more info when you were post changes in test 10?? I added more info "it contains IntelLLVM and Clang build"
https://forum.doom9.org/showthread.php?p=1994584#post1994584

Modern days, are people lazy for just click and read info?
"Avisynth+ builds. This repo contains Clang and IntelLLVM builds of AviSynthPlus."

Now the penny drops.

Nowhere is it pointed out that they are the same versions as from pinterf and only the compilers are different.
I think that was not clear to most people, e.g. not to me, I always wondered why Avisynth builds are created here, and above all what other functions are included in them.

This should perhaps be made clearer on the website, so that I can understand it too. :)

[kedautinh12]

"r4035" is point this is same commits with test 10 cause pinterf was use that word in his release post
https://forum.doom9.org/showthread.php?p=1994584#post1994584

[kedautinh12]

AviSynthPlus Builds r4035 (same commmits with official build for people who lazy to read pinterf release (https://forum.doom9.org/showthread.php?p=1994584#post1994584)) with IntelLLVM and Clang build by Asd-g
https://gitlab.com/uvz/AviSynthPlus-Builds

[guest]

https://gitlab.com/uvz/AviSynthPlus-Builds

This appears to have had a refresh, yesterday !!!

[jpsdr]

Hello.
I followed the instructions for building DirectShowSource plugin provided by Asd-g in his IntelLLVM & Clang builds : downloading baseclasses, adjusting the CMakefile in the plugin directory. It was less complex than expected...
It worked with Visual Studio, but when i'm building baseclasses with LLVM (17.0.6) (LLVM integrated with VS 2019, not IntelLLVM), it fails with an error in wxdebug.cpp.
And trying to use the .lib build with Visual Studio during the LLVM build is not working.
So... How are you able to build baseclasses with clang/LLVM ?

[kedautinh12]

Ask him :D
https://gitlab.com/uvz/AviSynthPlus-Builds/-/issues

[jpsdr]

In case someone has an idea, the error i have :
1>wxdebug.cpp(1087,20): warning : ISO C++11 does not allow conversion from string literal to 'char *' [-Wwritable-strings]
1>wxdebug.cpp(1093,16): warning : ISO C++11 does not allow conversion from string literal to 'char *' [-Wwritable-strings]
1>wxdebug.cpp(1237,16): error : qualified reference to 'CDisp' is a constructor name rather than a type in this context
1>wxdebug.cpp(1237,21): warning : parentheses were disambiguated as redundant parentheses around declaration of variable named 'pp' [-Wvexing-parse]
1>wxdebug.cpp(1237,21): message : add a variable name to declare a 'CDisp::CDisp' initialized with 'pp'
1>wxdebug.cpp(1237,9): message : add enclosing parentheses to perform a function-style cast
1>wxdebug.cpp(1237,21): message : remove parentheses to silence this warning
1>wxdebug.cpp(1237,22): error : no matching constructor for initialization of 'CDisp::CDisp'
1>wxdebug.cpp(1130,8): message : candidate constructor not viable: requires single argument 'clsid', but no arguments were provided
1>./wxdebug.h(303,5): message : candidate constructor not viable: requires single argument 'd', but no arguments were provided
1>wxdebug.cpp(1143,8): message : candidate constructor not viable: requires single argument 'llTime', but no arguments were provided
1>wxdebug.cpp(1174,8): message : candidate constructor not viable: requires single argument 'pPin', but no arguments were provided
1>wxdebug.cpp(1203,8): message : candidate constructor not viable: requires single argument 'pUnk', but no arguments were provided
1>./wxdebug.h(298,7): message : candidate constructor (the implicit copy constructor) not viable: requires 1 argument, but 0 were provided
1>wxdebug.cpp(1101,8): message : candidate constructor not viable: requires at least argument 'll', but no arguments were provided
1>wxdebug.cpp(1238,11): error : member reference type 'CDisp::CDisp' is not a pointer; did you mean to use '.'?
1>wxdebug.cpp(1238,13): error : no member named 'Release' in 'CDisp'
1>Génération du projet "BaseClasses.vcxproj" ends -- FAIL.


EDIT
Asd-g answered me.

[jpsdr]

Out of curiosity, where the DevIL.dll is comming from ?
When i want to build avisynth and i create the project with CMake, the .dll is already here in the Output before i even start any build...:confused:

[pinterf]

I think it is from here: https://openil.sourceforge.net/
But it's put in the project tree:
https://github.com/AviSynth/AviSynthPlus/tree/master/plugins/ImageSeq/lib/

Edit 2:
They are of version 1.7.8
But there exists version 1.8.0
https://openil.sourceforge.net/download.php
Probably those DLLs in ImageSeq/lib (x86/x64) must be replaced, after checking if they are compatible. (unicode or not, dependencies, etc.)

[kedautinh12]

Maybe can add more commits from offical project??
https://github.com/DentonW/DevIL/commits/master/

[flossy_cake]

Has anyone noticed memory leaks when seeking? Especially if CPU is loaded with a heavy filter like QTGMC and I seek back or forward 10 seconds, memory usage can spike up by as much as 1000MB per seek. Seeking by 1 frame in either direction seems to be even worse. After too much seeking I eventually reach the SetMemoryMax value (4GB on my system) after which filters may become slow, eg. QTGMC will become slower once the max memory usage has been reached. I was able to reproduce this with other filters too, and I tried changing source filters to no effect.

Anyone experienced anything like that?

[kedautinh12]

no, i use QTGMC(Preset="Very Slow", Sharpness=0).SelectEven() with prefetch(4) and don't pike up by as much as 1000MB per seek. My memory increased by 1000mb and after the preview and it doesn't pike up anymore

[flossy_cake]

no, i use QTGMC(Preset="Very Slow", Sharpness=0).SelectEven() with prefetch(4) and don't pike up by as much as 1000MB per seek. My memory increased by 1000mb and after the preview and it doesn't pike up anymore

Can you try doing some 1-frame step backwards in a row at a reasonably fast tempo, like a frame step every 200ms (edit: or just try rapidly clicking around in the seek bar close to current position). Especially backwards seems to trigger it, I think because it has to go back and kind of preroll up to the current frame or something. I don't know what video player you are using but in MPC-HC there is a hotkey for "frame-step back".

Prefetch doesn't affect it for me. But it depends how fast I press it. There is a random element to it. Sometimes I seek +/- 10 seconds multiple times and it doesn't spike the memory usage at all. Other times a single skip ahead 10 seconds will shoot it up 1000MB. It seems to be a function of what the current frame is in the clip. If I do Preroll() that seems to make it worse too.

https://c.l3n.co/i/3dJsNM.png

Above is just with
LWLibavVideoSource(clip, repeat=true)
QTGMC(preset="slow")
With this (https://drive.google.com/file/d/1KdE0np8016zr7ovRassdbH7OtbKU7e-a/view?usp=sharing) clip.

edit: I'm using LAV DirectShow filter to open .avs files in MPC-HC, but I tried with default "AVI/WAV file source" (I think it's Microsoft filter) and that leaks too (and has other issues).

[kedautinh12]

I'm using preview of megui and don't have that error

[flossy_cake]

I'm using preview of megui and don't have that error

Ok thanks, I tried AvsPMod just now and same issue after frame stepping backwards with left arrow, or just clicking around in the seek bar multiple times:

https://c.l3n.co/i/3q9K8M.png

I've got two theories:

1. It's somehow CPU load related, as it only happens when CPU is loaded. I can also reproduce it by doing multiple Spline36Resize from 480p->4k->480p to load up the CPU. So it's not QTGMC related. If you have a powerful CPU then maybe you don't experience it because CPU is not loaded up enough.

2. It's something specific to my version of Avisynth.dll (3.7.3 r3936 x64)

[kedautinh12]

Try latest avs+ ver (4035)
https://gitlab.com/uvz/AviSynthPlus-Builds

[flossy_cake]

Try latest ver avs+ (4035)
https://gitlab.com/uvz/AviSynthPlus-Builds

Updated to 4035, same issue.

How much CPU load do you have with QTGMC(preset="slow").Prefetch(4)?

For a 480i clip I have 85% initially then drops to 55% after about 10 seconds once prefetch buffer is full. 4 cores @ 3.2ghz (i5-4570).

[kedautinh12]

I used your video with your script, Memory from 78 to 84 and keep it here even I load frame by frame manual or auto play
https://i.imgur.com/MWucxSl.png

[flossy_cake]

I used your video with your script, Memory from 78 to 84 and keep it here even I load frame by frame manual or auto play

Ok thank you. Must be an issue with Windows 7 memory management.

[pinterf]

Has anyone noticed memory leaks when seeking? Especially if CPU is loaded with a heavy filter like QTGMC and I seek back or forward 10 seconds, memory usage can spike up by as much as 1000MB per seek. Seeking by 1 frame in either direction seems to be even worse. After too much seeking I eventually reach the SetMemoryMax value (4GB on my system) after which filters may become slow, eg. QTGMC will become slower once the max memory usage has been reached. I was able to reproduce this with other filters too, and I tried changing source filters to no effect.

Anyone experienced anything like that?
It's not very likely that you are seeing this because of Win7 memory management.
I have to try to reproduce using the very same environment.
- I'd need the simplest script (no QTGMC, just the resizer; btw. does it happen with a simple ColorBarsHD as the source filter?)
- seeking from MPC-HC only?, or does it happen with AvsPMod as well?
- Which version of LWLibavVideoSource?

[flossy_cake]

I have to try to reproduce using the very same environment.
- I'd need the simplest script (no QTGMC, just the resizer; btw.
does it happen with a simple ColorBarsHD as the source filter?)
- seeking from MPC-HC only?, or does it happen with AvsPMod as well?

In AvsPMod and MPC-HC I am able to reproduce it with:


ColorBarsHD().KillAudio()
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Prefetch(4)


In AvsPmod:

1. Press play and let the video play for a bit
2. Press pause
3. Framestep backwards by rapidly tapping or holding the left arrow key in AvsPMod

https://c.l3n.co/i/3qGOIA.png

[pinterf]

In AvsPMod and MPC-HC I am able to reproduce it with:

Thanks, that info must be enough for me for the holiday week :)

[qyot27]

I think it is from here: https://openil.sourceforge.net/
But it's put in the project tree:
https://github.com/AviSynth/AviSynthPlus/tree/master/plugins/ImageSeq/lib/

Edit 2:
They are of version 1.7.8
But there exists version 1.8.0
https://openil.sourceforge.net/download.php
Probably those DLLs in ImageSeq/lib (x86/x64) must be replaced, after checking if they are compatible. (unicode or not, dependencies, etc.)

A user can drop in 1.8.0 as a replacement for 1.7.8 and it'll still work.

My position has been that we need to remove the DevIL binaries and embedded copy of SoundTouch that we currently use, in favor of just using the one found on the system. It was something that I'd planned on doing back in 2020 just after we'd gotten the Linux port working, but I'd encountered issues in trying to detect things and walked away from it for a while.

In fact, anyone that's used the upstream release of AviSynth+ for macOS has been using latest DevIL in the ImageSeq plugin the whole time. All the non-Windows OSes can use the version from their distro's/OS' repositories, which is typically 1.8.0 by now.

I finally did figure out the proper way to handle both DevIL on Windows as well as grabbing the system copy of SoundTouch by using pkg-config, and confirmed that it does work when building on Windows (even if Windows is supremely weird with all the CMake/pkg-config system detection stuff).

To illustrate this,
Proof of concept. (https://www.mediafire.com/file/i08vu91xvj4715l/static_plugs.7z/file)

Both ImageSeq and TimeStretch are using the current git HEAD of DevIL and SoundTouch rather than the old versions from the source tree.

[pinterf]

In AvsPMod and MPC-HC I am able to reproduce it with:


ColorBarsHD().KillAudio()
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Spline36Resize(3840, 2160)
Spline36Resize(1920, 1080)
Prefetch(4)




As a workaround try setting 1 to default cache mode.

#SetCacheMode(0) # Run until frame 40, then step back 10 times in avspmod, 11th and on back step increases 200MB cache space
SetCacheMode(1) #no problem
.. script follows


An issue was created at github: https://github.com/AviSynth/AviSynthPlus/issues/379

(The video cache size prediction algorithm probably went mad after this manually governed frame order pattern - 1,2,3,4,...25 then back 24-23-22-21, etc.)

[flossy_cake]

As a workaround try setting 1 to default cache mode.


Can confirm that works for me for the colorbars demo in AvsPMod, but breaks a lot of other things in my scripts - freezing on certain seek patterns, performance randomly goes bad after seeking, and my scriptclips become desynchronised where current_frame != previous_frame + 1 (it's never guaranteed with cache mode 0 either, but tends towards synchronicity over time whereas with cache mode 1 it can randomly get suck in desync).

I'm quite happy to just continue using cache mode 0 and limit my seeking behaviour to avoid the issue.

I'm now worried you are going to try to fix cache mode 0 and break all my scripts permanently so I'm kinda regretting mentioning this now :scared:

[pinterf]

This issue will be fixed surely. This part is black magic, such problems usually require 30-40 net working hours to understand, experiment, fix, test.

[flossy_cake]

This issue will be fixed surely. This part is black magic, such problems usually require 30-40 net working hours to understand, experiment, fix, test.

Ok well thank you for putting in the time. Hopefully it doesn't break my Preroll()'d ScriptClips too much :thanks:

[flossy_cake]

This issue will be fixed surely. This part is black magic, such problems usually require 30-40 net working hours to understand, experiment, fix, test.

For backwards compatibility would it be possible to let the user still have access to the current caching behaviour? Something like SetCacheMode(CACHE_FAST_START_OLD) to force legacy behaviour.

[gispos]

I'm now worried you are going to try to fix cache mode 0 and break all my scripts permanently so I'm kinda regretting mentioning this now :scared:
I also hope that the shot doesn't backfire.
Almost none of my scripts run in cache mode 1 and take ages to load. Then it's better to eat up the memory and be fast :)

[FranceBB]

Yep, RAM is there to be allocated. If it needs to be allocated, so be it. :P
My laptop has 64GB while my servers have 128GB, both DDR4 'cause they're from 2016 and 2019 respectively, so it's not really a big deal if a script consumes 3.5 GB of RAM.
Heck, I regularly have UHD encodes using over 12 GB of RAM and I'm totally fine with it.
Even my Windows XP machine with PAE supports 64 GB of RAM (with the only caveat that each process tops up at 2GB, so on some encodes I might have to use MPPipeline to split on each filter so that each one of them has 2GB allocable memory up to 64GB instead of it being 2GB max for the whole thing). :)

[flossy_cake]

That's what I thought too but then I noticed QTGMC becomes slow once memory use hits the SetMemoryMax value. This is only an issue for real-time use as CPU would be pegged at 100 the whole time when transcoding anyway.

I can workaround it by giving QTGMC an extra thread or two, but then it's a bit tricky to find the sweet spot where it's not overutilizing CPU during the period before the memory hits max. Basically I need to keep around 20% CPU overhead for the media player, the remaining 80% can go to QTGMC during realtime playback.

[Boulder]

Is there a measurable performance difference between the two cache modes?

[flossy_cake]

Is there a measurable performance difference between the two cache modes?

As long as you don't seek or use a ScriptClip that rapidly picks frames from different clips the average fps seems to be about the same - measure with ffmpeg.exe:

"c:\program files\ffmpeg\bin\ffmpeg.exe" -i "C:\YourScript.avs" -f null NUL

[DTL]

Is there a measurable performance difference between the two cache modes?

It may greatly depends on lots of ways: OS used, memory subsystem (CPU caches and RAM), scripts and plugins used and settings for each filter, frame size and pixel type for each filter input and output and so on.

But it looks user can not control cache mode for each filter to finetune performance if possible (in the same way as frame-based MT) ?

[ErazorTT]

Could anybody on any 2.7.3 check this simple code, and confirm that blue and green are switched?


clip=BlankClip(color=$0000ff,pixel_type="RGBP8")
StackHorizontal(clip,clip.ShowRed(),clip.ShowGreen(),clip.ShowBlue())


In contrast to what my expection I'm seeing this:
https://i.postimg.cc/2VHf89TZ/show-Channel-status-quo.jpg (https://postimg.cc/2VHf89TZ)

My expection was of course:
https://i.postimg.cc/pmpM9Q11/show-Channel-expectation.jpg (https://postimg.cc/pmpM9Q11)

[flossy_cake]

Could anybody on any 2.7.3 check this simple code, and confirm that blue and green are switched?


clip=BlankClip(color=$0000ff,pixel_type="RGBP8")
StackHorizontal(clip,clip.ShowRed(),clip.ShowGreen(),clip.ShowBlue())


In contrast to what my expection I'm seeing this:
https://i.postimg.cc/2VHf89TZ/show-Channel-status-quo.jpg (https://postimg.cc/2VHf89TZ)

My expection was of course:
https://i.postimg.cc/pmpM9Q11/show-Channel-expectation.jpg (https://postimg.cc/pmpM9Q11)

The issue is RGBP8, since pixel_type=RGB/undefined() works as expected.

The wiki (http://avisynth.nl/index.php/ShowAlpha) says "Returns the selected channel of an RGB32 or RGB24 clip as greyscale." I guess that doesn't include RGBP8 so the function should probably throw an exception if the clip isn't RGB32/24.

[ErazorTT]

should probably throw an exception if the clip isn't RGB32/24.

I don’t think so, since currently ShowGreen returns the Blue channel and ShowBlue returns the Green channel. So a working fix should be very easy.

[flossy_cake]

I don’t think so, since currently ShowGreen returns the Blue channel and ShowBlue returns the Green channel. So a working fix should be very easy.

I could be totally wrong but my guess is the planar data is getting interpreted as interleaved, because the function doesn't check to confirm pixel format is interleaved. In the meantime you could just convert your clip to interleaved and avoid the issue.

[pinterf]

Could anybody on any 2.7.3 check this simple code, and confirm that blue and green are switched?


clip=BlankClip(color=$0000ff,pixel_type="RGBP8")
StackHorizontal(clip,clip.ShowRed(),clip.ShowGreen(),clip.ShowBlue())


In contrast to what my expection I'm seeing this:
https://i.postimg.cc/2VHf89TZ/show-Channel-status-quo.jpg (https://postimg.cc/2VHf89TZ)

My expection was of course:
https://i.postimg.cc/pmpM9Q11/show-Channel-expectation.jpg (https://postimg.cc/pmpM9Q11)
Thanks for the report, it must be a bug. It's more than I can bear :) so expect a fix soon.
EDIT: fixed on git + updated documentation (https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/corefilters/showalpha.html)