View Full Version : AviSynth+ thread Vol.2
Emulgator
11th March 2023, 23:59
...and realize that Avisynth+ has NO audio cache implemented... The feeling is priceless.
For this and for all that cleanup work: Nagyon szépen köszönöm Ferenc !
flossy_cake
12th March 2023, 05:20
DSS should only ever be used as a last resort
DSS just uses whatever source filter has highest merit on your system, in my case that is LAV filters which are excellent and maintained with up to date features by nevcairiel.
But Avisynth's implementation of DSS seems to downsample to 8-bit internally before handing it to LAV, so I can't use it for 10-bit videos even though LAV filters support it. imo DSS deserves an overhaul and should support subtitles as well like DSS2Mod.
DSS is very useful for realtime use as it doesn't have to index/cache anything, instant playback, instant seeking (with LAV that is, which uses ffmpeg).
guest
12th March 2023, 06:36
For this and for all that cleanup work: Nagyon szépen köszönöm Ferenc !
And just in case nobody knows what that means...
"thank you so much" :D
jpsdr
12th March 2023, 15:06
Strange, temporarily I'd try to get rid of all autoloaded plugins, move them away, to make sure if one of them is making avisynth crash when loading. I don't have any other idea. My avsmeter is 3.0.0.4, and happily consumes any avisynth version
Excellent idea, i tested, and... Still the same.
Even with all the autoload directories empty (plugins, plugins+, plugins64, plugins64+), AVSMeter still says "Error, unable to load avisynth.dll" with r3950 clang and IntelLLVM+redist build files.
No issue with the VisualStudio test7 build files.
...:confused:
StvG
12th March 2023, 15:38
Excellent idea, i tested, and... Still the same.
Even with all the autoload directories empty (plugins, plugins+, plugins64, plugins64+), AVSMeter still says "Error, unable to load avisynth.dll" with r3950 clang and IntelLLVM+redist build files.
No issue with the VisualStudio test7 build files.
...:confused:
Download Dependencies_x64_Release.zip (https://github.com/lucasg/Dependencies/releases), load avisynth.dll in DependenciesGui and share the output/check if something is missing.
filler56789
13th March 2023, 01:37
DSS just uses whatever source filter has highest merit on your system,
DirectShowSource can also use ANY source filter /demuxer /decoder /processor available on one's Windows PC —
— just tell DSS() to open a "well-constructed" .grf file.
FranceBB
13th March 2023, 14:08
DirectShowSource can also use ANY source filter /demuxer /decoder /processor available on one's Windows PC —
— just tell DSS() to open a "well-constructed" .grf file.
True.
It used to be extremely useful back in the days when proprietary codecs which didn't have an indexer had a separate .exe that you could install to make such a codec widely available on the system by all applications that were able to use DirectShow. I still remember CanopusHQ being a prime example of this and how I could index it with DirectShowSource() only.
Unfortunately, nowadays, companies are way more reluctant in doing those things and when they release their codecs, it's generally just some "plugin" for the most common non linear editors like AVID Media Composer, Adobe Premiere, Davinci Resolve etc.
Think about Sony Raw, Canon Raw, Blackmagic Raw, Red Raw, Arri Raw etc...
They all work with their proprietary software like Sony Catalyst, ArriRaw Converter etc and their "codec" is more of a "plugin" as it exposes it to non linear editors, but not to Windows, so they can't be decoded using DirectShow.
This meant that over the years DirectShowSource() has lost a bit of popularity in its use, which is a shame, really, as those proprietary codecs (which don't have any open source indexer that can decode them) should really be exposing themselves to Windows via DirectShow...
Oh, if only those company listened to their users... :(
jpsdr
13th March 2023, 19:55
@StvG
I've checked the dependencise of AviSynth.dll and DevIL.dll, both x86 and x64, on both x64 and x86 OS, nothing is missing... :(
pinterf
14th March 2023, 12:48
For this and for all that cleanup work: Nagyon szépen köszönöm Ferenc !
Thanks :)
Anyway, the Avisynth+ audio cached version is uploaded on git, I won't provide test build today for sure. I guess old prehistoric SoxFilter will automagically start working.
But I'm still going to finish my new SoxFilter which is using the latest sox library. It is basically ready but you know, one must nicely arrange and polish the code, document, put it properly to github, try building it under various settings (maybe figure out how to build it under linux) still many days ahead...
Relevant change log part:
20230314 3.7.3 WIP
------------------
- Set automatic MT mode MT_SERIALIZED to
ConvertToMono, EnsureVBRMP3Sync, MergeChannels, GetChannel, Normalize, MixAudio, ResampleAudio
- Add back audio cache from classic Avisynth 2.6.
Believe it or not, audio cache was never ported to Avisynth+
- Make use of avisynth.h constants: CACHE_GETCHILD_AUDIO_MODE and CACHE_GETCHILD_AUDIO_SIZE:
Filters are queryed about their desired audio cache mode through their SetCacheHints (similarly to CACHE_GET_MTMODE).
- Filters can answer CACHE_GETCHILD_AUDIO_MODE with
CACHE_AUDIO: Explicitly cache audio, X byte cache.
CACHE_AUDIO_NOTHING: Explicitly do not cache audio.
CACHE_AUDIO_AUTO_START_OFF: Audio cache off (auto mode), X byte initial cache.
CACHE_AUDIO_AUTO_START_ON: Audio cache on (auto mode), X byte initial cache.
- Default value is CACHE_AUDIO_AUTO_START_OFF.
- Filters can specify the required cache size by returning CACHE_GETCHILD_AUDIO_SIZE.
Default cache size is 256kB.
- For custom audio cache querying example see EnsureVBRMP3Sync::SetCacheHints in source.
How it works:
- Modes CACHE_AUDIO_AUTO_START_OFF (default) and CACHE_AUDIO_AUTO_START_ON are automatic modes.
The decision whether the stream benefits caching or not - and how big the cache
size should be - is made upon continously gathering some statistics on the audio
stream requests (an internal score is maintained).
- when strict linear reading is detected. why bother with a cache,
mode would finally changed to CACHE_AUDIO_AUTO_START_OFF.
- When the requests are continously skipping chunks - a cache might not help;
go with CACHE_AUDIO_AUTO_START_OFF as well.
- When the next sample request is within the cache size, a cache could help:
if audio cache was swithed off Avisynth would turn it into active caching by changing
the working mode to CACHE_AUDIO_AUTO_START_ON.
- Modes CACHE_AUDIO and CACHE_AUDIO_NOTHING are explicitely enable/disable audio cache at a give size.
pinterf
14th March 2023, 14:20
@StvG
I've checked the dependencise of AviSynth.dll and DevIL.dll, both x86 and x64, on both x64 and x86 OS, nothing is missing... :(
Other ideas.
- Virus check engine?
- BTW: Looked at avsmeter (3009) source and avsmeter cannot report such error message: "Error, unable to load avisynth.dll". Is this your exact message? Is it a popup window or just written on the console?
- Can you find relevant entries in Sysem or Application in event viewer eventvwr.exe?
- Do you have ffdshow or some of its dll somewhere in your system?
DTL
14th March 2023, 15:34
The clang and IntelC builds of 3.7.3 test 7 also not work at my Win7 x64. With VirtualDub error is sort of 'AVI import fail'. So it is something with compilers expected. Though the performance of clang builds for some architectures (like AVX512) is much better in compare with MSVC compiler. So it is good to find why clang builds not loads at user side.
"Do you have ffdshow or some of its dll somewhere in your system?"
Yes - ffms2.dll is loaded as plugin in the script and in the current working directory. FFmpegSource2() used as the source.
FranceBB
14th March 2023, 15:54
Anyway, the Avisynth+ audio cached version is uploaded on git, I won't provide test build today for sure. I guess old prehistoric SoxFilter will automagically start working.
https://media.tenor.com/TrAsjYbL720AAAAM/chris-pratt-wow.gif
WOW! A-M-A-Z-I-N-G!
Thank you so, so, so much! :D
Now I can finally say goodbye to FFMpeg once again ehehehehehe
I'm still going to finish my new SoxFilter which is using the latest sox library.
A new sox version too? Is it Christmas already? :D
Thank you, again, a lot, Ferenc, we would be lost without you!! :)
pinterf
14th March 2023, 16:28
The clang and IntelC builds of 3.7.3 test 7 also not work at my Win7 x64. With VirtualDub error is sort of 'AVI import fail'. So it is something with compilers expected. Though the performance of clang builds for some architectures (like AVX512) is much better in compare with MSVC compiler. So it is good to find why clang builds not loads at user side.
"Do you have ffdshow or some of its dll somewhere in your system?"
Yes - ffms2.dll is loaded as plugin in the script and in the current working directory. FFmpegSource2() used as the source.
VirtualDub or VirtualDub2? (I meant ffdshow, not ffms)
Win 7? I forgot, jpsdr are you using Win7, too?
pinterf
14th March 2023, 16:35
Windows 7 vs. LLVM based compilers:
I found this on Intel's site.
https://www.intel.com/content/www/us/en/developer/articles/release-notes/intel-c-compiler-190-for-windows-targets-release-notes-for-intel-system-studio-2019.html#nextgen_sysreq
For the /Qnextgen option to use LLVM Technology only the following are supported:
For Intel64, supported Windows OSes and Visual Studio - only those listed below:
Windows 10
Windows Server 2019
Windows Server 2016 (1607)
Visual Studio 2019 with Windows SDK 10
Visual Studio 2019 Build Tools* with Windows SDK 10
Visual Studio 2017 with Windows SDK 10
Target software requirements
The target platform should be based on one of the following environments:
Microsoft Windows 10 IoT Core*, Microsoft Windows 10*, Microsoft Windows Embedded 8*, Microsoft Windows Embedded 7*
pinterf
14th March 2023, 16:50
And the above was only for two versions behind.
This is from 2022
https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-dpcpp-system-requirements-2022.html
"Windows* 10 (64 bit) Microsoft Windows Server* 2022, 2019, 2016"
Windows 7 or 8 is not mentioned
And this one from 2023
https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-dpcpp-system-requirements.html
Windows* 10, 11 (64 bit) Microsoft Windows Server* 2022, 2019
jpsdr
14th March 2023, 18:33
Yes, i'm using Windows 7.
BTW, I didn't know there was an Intel LLVM version. To make llvm builds of my plugins i'm using the LLVM at the llvm.org (well, their github release), and they work fine under Windows 7.
Ok, the Intel don't even target Windows 7, but what about the clang ?
I could try to build it myself... But don't know how...
Edit:
I think i've figured out, i've tried CMake, it seems to work.
Edit2:
Ohhh... First quick test worked... Now i just have to test with llvm.
pinterf
14th March 2023, 19:13
Forget DirectShow at the moment, uncheck or do not define the build DirectShowSrouce option. Not easy.
Check
https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/contributing/compiling_avsplus.html#id8
and
https://avisynthplus.readthedocs.io/en/latest/avisynthdoc/contributing/compiling_avsplus.html#building-with-microsoft-c-cmake-command-line as well.
My test7 build was with Intel Classic 19.2 (icl), which btw. will be abandoned by Intel in 2023H2.
Then there is the IntelLLVM aka NextGen (icx) which is supported and recommended by Intel from now on.
I just tried them both and added them to the documentation some weeks ago (updated with the latest 2023 versions).
I have Windows 11 though so cannot even tell you about the adventures with Win10 and earlier version.
jpsdr
14th March 2023, 22:41
First, the AVSMeter error message is exactly : "Error: Cannot load avisynth.dll", in the console. My version is 3.0.9.0.
Second, i've build AviSynth with llvm the same way (and with the same compiler) i build my VDub or AviSynth plugins. It didn't work either, exact same behavior.
I build with llvm downloaded on the github, and with VisualStudio 2019 (16.9.26) installed with llvm support but without the llvm compiler provided with VS, Update 9 is the last version it works using "external" installed llvm version.
Very very odd, as my plugins work fine... :confused:
kedautinh12
15th March 2023, 01:50
Anyone had problem with intel build can try install latest Intel® oneAPI DPC++/C++ Compiler Runtime for Windows*
https://www.intel.com/content/www/us/en/developer/articles/tool/compilers-redistributable-libraries-by-version.html
DTL
15th March 2023, 08:34
"For the /Qnextgen option to use LLVM Technology only the following are supported:
For Intel64, supported Windows OSes and Visual Studio - only those listed below:
Windows 10"
It looks AVS+ starting to lost compatibility not only with WinXP but with Win7 too.
jpsdr
15th March 2023, 09:46
Warning : There is 2 things :
- The OS supported by the compiler : Where you can install it.
- The targets supported by the compiler, what you can build for, not the same. Be sure to not mix them.
But what bothers me in my tries, is why the avisynth i've build is not working, when my plugins are, both being build with the exact same compiler...
And the x265.exe i've also build with the same compiler is working under Windows 7.
I'm totaly lost... :confused:
There must be something i'm not seeing in the options...?
DTL
15th March 2023, 09:52
"why the avisynth i've build is not working, when my plugins are, "
Possibly because AVS+ is designed differenly to your plugins. So it mean it starting to lost compatibility with old versions of windows. Sadly it not make compiler warnings or errors and simply fail to load in Win7 ? Or may be as large project as AVS+ have most of warnings and non completely critical stop-errors disabled to make building easier. But it may cause such issues on target operating systems. The debug build also not loads ? Or debug run not tested ?
FranceBB
15th March 2023, 10:01
It looks AVS+ starting to lost compatibility not only with WinXP but with Win7 too.
Nah, it will be something like: MSVC version compatible with literally everything and then Clang/LLVM version compatible only with the "latest and greatest" version of Windows and Windows Server.
To be fair, I don't see MSVC builds going away anytime soon and as long as they're gonna be there, people will have a choice. ;)
DTL
15th March 2023, 10:15
I mean internal design of AVS+ may starting to lost compatibility with Win7. It may be no one day process but someday even MSVC builds will no loads at Win7.
As you see with jpsdr example - some simple software like small plugins still can be built with llvm and run at Win7 and AVS+ as much complex software set already fails.
FranceBB
15th March 2023, 12:04
As you see with jpsdr example - some simple software like small plugins still can be built with llvm and run at Win7 and AVS+ as much complex software set already fails.
I too use his plugins and it's indeed very weird that they work while the AVS core doesn't.
I mean, there must be a reason, surely.
pinterf
15th March 2023, 14:50
I too use his plugins and it's indeed very weird that they work while the AVS core doesn't.
I mean, there must be a reason, surely.
Many things happen when loading a DLL with lots of classes, setting global variables, static initialization, which can be different among compilers.
Even before the any part is run from our written code.
E.g. LLVM clang-cl is trying the keep the same ABI as Microsoft.
And they made it more compatible than Microsoft themselves, when they broke their own ABI :)
https://clang.llvm.org/docs/MSVCCompatibility.html
"Thread-safe initialization of local statics: Complete. MSVC 2015 added support for thread-safe initialization of such variables by taking an ABI break. We are ABI compatible with both the MSVC 2013 and 2015 ABI for static local variables."
Avisynth initialization sequence is a "bit" more serious than most of the plugins'.
So it is not weird at all that plugins work, but Avisynth doesn't.
Or simply there is indeed a rare bug which happens only in special circumstances.
pinterf
15th March 2023, 20:04
Avisynth+ 3.7.3 test 8 (20230315 - r3958) (https://drive.google.com/uc?export=download&id=1BvFA3vNePZVLgtnQ4FiugcHddkricK2m)
Sporting with a good old friend: audio cache. I hope it won't introduce new problems. Basically the same as in classic Avisynth 2.6.
And a fix of a bug introduced in test7 (AvsPMod F5 refresh), thanks Asd-g for the report.
FranceBB
15th March 2023, 21:00
You nailed it, Ferenc, as always!
Sox magically started to work as you predicted. :)
And indeed, now I finally have a meaningful way to go from Stereo 2.0 to 5.1 thanks to the good old Avisynth upmix functions:
https://i.imgur.com/LViYTru.png
https://i.imgur.com/F1b8WQQ.png
kedautinh12
16th March 2023, 00:09
Avs+ r3958
https://gitlab.com/uvz/AviSynthPlus-Builds
guest
16th March 2023, 02:34
Avs+ r3958
https://gitlab.com/uvz/AviSynthPlus-Builds
Not another one...
I will try both, hopefully no issues, this time. :D :cool:
EDIT:- r3958 LLVM is working for me :) :)
flossy_cake
16th March 2023, 05:46
Sorry to keep asking for things, but is there any possibility the Normalize (http://avisynth.nl/index.php/Normalize) filter could be modded to support Normalize2 (http://avisynth.nl/index.php/Normalize2)'s "store the peak level value in an external file and uses a lookup table to do the actual normalizing (for speed)"
:thanks:
jpsdr
16th March 2023, 14:35
I have to test and check something i just tought when back home.
If i remember properly, the officiel release of 3.7.2 have "big" files -> It's a clang build (and now i know why there was "big" file releases, and pinterf's "small" files releases before), and they worked for me. So i have to check fir the "if i remember properly", and if it's the case, find in the git the exact release point this was built with (maybe there is a point where there is a 3.7.2 tag), try to build myself with my current build process, and :
- If it works, it means the commit wich break things can be identified.
- If it doesn't work, it means that it's the llvm version, an old one worked, not a current one.
Edit:
Don't have to wait...
Is the AviSynthPlus_3.7.2_20220317-filesonly.7z found here https://github.com/AviSynth/AviSynthPlus/releases/tag/v3.7.2 build with clang/llvm ? Files are big enough for, but not sure...
pinterf
16th March 2023, 16:01
Is the AviSynthPlus_3.7.2_20220317-filesonly.7z found here https://github.com/AviSynth/AviSynthPlus/releases/tag/v3.7.2 build with clang/llvm ? Files are big enough for, but not sure...
Size can be bigger, it may contain debug info and labels, despite being a release version.
pinterf
16th March 2023, 16:03
And now let's experiment a bit
Avisynth+ 3.7.3 test 9 (20230316 - r3961) (https://drive.google.com/uc?export=download&id=1xFMgLh5lh6kmKPxT5lYGZbpVVBkLyxiK)
20230316 3.7.3 WIP
------------------
- Add initial audio channel mask support (CPP and C interface, script function)
It still belongs to V10 changes (there were only tests since then), but it can be discussed if not.
Technically it is done by using another 18+2 bits in the Clip's VideoInfo.image_type field.
Due to lack of enough bits in this VideoInfo field, the mapping between the original dwChannelMask
and Avisynth's internal values are not 1:1, but all information is kept however.
This is because not 32 but only 18 (strictly: 18+1) bits are defining speaker locations, so
the remaining bits of our existing 'image_type' field can be used for this purpose.
Thus 20 new bits are occupied.
- 1 bit: marks if channel mask is valid or not
- 18 bits for the actually defined WAVE_FORMAT_EXTENSIBLE dwChannelMask definitions
(https://learn.microsoft.com/en-us/windows/win32/api/mmreg/ns-mmreg-waveformatextensible)
- 1 bit for the special SPEAKER_ALL value
Programmers can check AvsChannelMask and AvsImageTypeFlags in avisynth.h and avisynth_c.h
- new C++ interface functions
- Check for existence:
bool VideoInfo::IsChannelMaskKnown()
- Setting:
void VideoInfo::SetChannelMask(bool isChannelMaskKnown, unsigned int dwChannelMask)
Re-maps and stores channel mask into image_type, sets the 'has channel mask' flag as well
Note: this data can be set independently from the actual NumChannels number!
- Retrieving:
unsigned int VideoInfo::GetChannelMask()
- new C interface functions
bool avs_is_channel_mask_known(const AVS_VideoInfo * p);
void avs_set_channel_mask(const AVS_VideoInfo * p, bool isChannelMaskKnown, unsigned int dwChannelMask);
unsigned int avs_get_channel_mask(const AVS_VideoInfo * p);
Like when establishing BFF, TFF and fieldbased flags from 'image_type', technically 'image_type' can
be manipulated directly. See SetChannelMask and GetChannelMask in Avisynth source for
image_type <-> dwChannelMask conversion.
I guess once ffmpeg will support it, it will read (or not read) channel masks such a way.
- new Script functions
bool IsChannelMaskKnown(clip)
int GetChannelMask(clip)
SetChannelMask(clip, bool known, int dwChannelMask) (parameters compulsory, no names must be set)
dwChannelMask must contain the combination of up to 18 positions or 0x80000000 for SPEAKER_ALL.
VfW export rules (included the existing sequence)
1.) OPT_UseWaveExtensible global variable must be 'true'
or
*new*new*new*
if VideoInfo::IsChannelMaskKnown is true, then fill WAVEFORMATEXTENSIBLE struct
2.) *new*new*new*
Is channel mask defined in Avisynth's VideoInfo? (VideoInfo::IsChannelMaskKnown() is true)
Yes -> return VideoInfo::GetChannelMask()
3.) No-> (Channel mask not defined in VideoInfo, guess it or set from variable)
3.1)Guess channel layout:
For 0 to 8 channels there is a predefined 'guess map':
#of channels dwChannelMask
0 0,
1 0x00004, // 1 -- -- Cf
2 0x00003, // 2 Lf Rf
3 0x00007, // 3 Lf Rf Cf
4 0x00033, // 4 Lf Rf -- -- Lr Rr
5 0x00037, // 5 Lf Rf Cf -- Lr Rr
6 0x0003F, // 5.1 Lf Rf Cf Sw Lr Rr
7 0x0013F, // 6.1 Lf Rf Cf Sw Lr Rr -- -- Cr
8 0x0063F, // 7.1 Lf Rf Cf Sw Lr Rr -- -- -- Ls Rs
For 9-18 channels:
sets first 9-18 bits in dwChannelMask
Above:
SPEAKER_ALL (dword msb bit is 1)
3.2) if OPT_dwChannelMask global variable is defined and is different from 0, then use it.
E.g. VirtualDub2 is using VfW, so after opening the script, ended with SetChannelMask(true, $0063F),
one can check the value File|File Info menü, under "compression" line (e.g.PCM, chmask 63f).
SetChannels does not check against NumChannels, so you can set the 7.1 constant for a stereo
if you wish. Microsoft's documentation mentions the cases of what can do az application with
less or more than necessary defined speaker bits.
- What to do about GetChannels, MixAudio, ConvertToMono? To be discussed.
KillAudio will call SetChannelMask(false, 0), nevertheless.
tebasuna51
16th March 2023, 23:19
Thanks for the audio improvement.
Now decoders must suply that info or we can use the exported variables?
About defaults please use a modern one (and more ffmpeg compatible) showed here (https://forum.doom9.org/showthread.php?p=1984523#post1984523).
FranceBB
17th March 2023, 00:59
Thanks for the audio improvement.
+1
Thanks for the new release, Ferenc!
It's always nice to see the audio getting its well deserved space after spending so much time on video. :)
Oh and by the way, 3.7.3 test 9 also works on Windows XP just fine, I just tested it before going to bed. (yep it's 1AM over here, ehehehehe).
Tomorrow I'll test on the Windows Server 2019 farm I have at work too. ;)
jpsdr
17th March 2023, 09:33
I've tested build the 3.7.2 with LLVM, same behavior, so it's relay compiler related, and not the version. As pinterf said, even if biger, the files provided with the release weren't probably llvm build.
pinterf
17th March 2023, 13:02
I've tested build the 3.7.2 with LLVM, same behavior, so it's relay compiler related, and not the version. As pinterf said, even if biger, the files provided with the release weren't probably llvm build.
As Intel say: "may or may not work".
"Note: These OS distributions are tested by Intel or known to work; other distributions may or may not work and are not recommended."
pinterf
17th March 2023, 15:13
I've tested build the 3.7.2 with LLVM, same behavior, so it's relay compiler related, and not the version. As pinterf said, even if biger, the files provided with the release weren't probably llvm build.
Or related to your machine.
I've freshly installed a Win7 Pro x64 (ISO from 2018) under HyperV VM.
Downloaded official Avisynth+ 3.7.2 (with redistributables)
Copied there latest VirtualDub2, avsmeter and avsmeter64, ffmpeg.
Everything worked. My clang and intel from 3.7.3 test7, latest uvz builds, my freshly compiled clang-cl.
Then installed the latest Visual C++ Redistributables. Same success.
DTL
17th March 2023, 17:13
Yes - it looks clang uses some >SSE instructions. The VirtualDub loaded via SDE emulator
set PATH=%PATH%;G:\sde
sde.exe -avx 1 -avx2 1 -emu_fast 1 -fma 1 -sse41 1 -- G:\Distr\VirtualDub-1.10.4-AMD64\veedub64.exe
can load script with x64-clang AVS+ 3.7.3 test7. Slow enough but working. So if it possible to disable AVX and higher usage in clang (also IntelClassic must have switches for it).
May be clang and llvm so visibly faster because they finally start to make use of registerfile of SIMD coprocessor for temporals and function arguments instead of pushing to stack and so on. But old chips do not have registerfile of required size (as AVX it is 256 bytes) and instructions to store and load data from it. So new clang and llvm no more compatible with SSE architecture chips ?
pinterf
17th March 2023, 17:40
Yes - it looks clang uses some >SSE instructions. The VirtualDub loaded via SDE emulator
set PATH=%PATH%;G:\sde
sde.exe -avx 1 -avx2 1 -emu_fast 1 -fma 1 -sse41 1 -- G:\Distr\VirtualDub-1.10.4-AMD64\veedub64.exe
can load script with x64-clang AVS+ 3.7.3 test7. Slow enough but working. So if it possible to disable AVX and higher usage in clang (also IntelClassic must have switches for it).
May be clang and llvm so visibly faster because they finally start to make use of registerfile of SIMD coprocessor for temporals and function arguments instead of pushing to stack and so on. But old chips do not have registerfile of required size (as AVX it is 256 bytes) and instructions to store and load data from it. So new clang and llvm no more compatible with SSE architecture chips ?
I doubt they are incompatible by design. I wonder, that the illegal instruction come from Avisynth code or somewhere from the library. I know that some of my plugins require sse4.1 when compiled with clang. (Where it would be too painful to separate SSE2 and SSE4.1 code to put them into different functions with different pragmas), while with MS I could do it simply by templates.
pinterf
17th March 2023, 17:56
Check cmakelist.txt. I think sse4.1 is set as the minimum for llvm compilation.
DTL
17th March 2023, 18:31
Unfortunately VS2019 debugger not break on the Illegal Istruction exception in AVS (or I still not know how to configure it to break and show disassembly). But my Core2Duo E7500 CPU do have MMX, SSE, SSE2, SSE3, SSSE3, SSE 4.1, EM64T, VT-x and still invalid instruction crash with that builds without SDE emulating up to AVX2.
It definitely something around AVX/2 :
Disabling AVX and AVX2 emulation
sde.exe -avx 0 -avx2 0 -- G:\Distr\VirtualDub-1.10.4-AMD64\veedub64.exe
cause crash with x64_clang build:
Illegal instruction at address = 7fed7947e0a: c5 fc 10 05 9e 17 19 00 c5 fc 11 05 e6 b4 24
Image name: C:\Windows\system32\AviSynth.dll
Offset in image: 0x4a7e0a
IDA shows at disassembly:
.text:00000001804A7E0A vmovups ymm0, cs:ymmword_1806395B0 - it is AVX or AVX2 instruction.
Same as VS2019 debug output : Exception thrown at 0x000007FEC4FF7E0A (AviSynth.dll) in Veedub64.exe: 0xC000001D: Illegal Instruction.
7E0A address from some page offset ?
jpsdr
17th March 2023, 19:17
My quick test PC hasn't AVX, but has SSE4.2. Never tested on my dedicated video processing PC (as it didn't work on quick test), which has AVX2.
I'll test again, but on my Video PC.
Is it possible that despite compile option clang still puts AVX/AVX2 when it shouldn't ? Or some intrinsic code with AVX/AVX2 is in some place it doesn't belong ? Because that's strange that both Intel and clang produce the same result.
Now Visual Studio has allready changed things without telling (the example is the bug in non aligned for AVISource, where VS compiled a not aligned when the code asked for aligned).
DTL
17th March 2023, 19:39
Or some intrinsic code with AVX/AVX2 is in some place it doesn't belong ?
IDA disassembly for clang build shows not anything looking like human-handcrafted program:
sub_1804A7E00 proc near
mov cs:dword_1806F32F0, 10100h
vmovups ymm0, cs:ymmword_1806395B0 < -- crash
vmovups cs:ymmword_1806F3300, ymm0
mov cs:dword_1806F3320, 1030200h
vmovups ymm0, cs:ymmword_1806395D0
vmovups cs:ymmword_1806F3330, ymm0
vmovaps xmm0, cs:xmmword_18052C510
vmovaps cs:xmmword_1806F3350, xmm0
vmovups ymm0, cs:ymmword_1806395F0
vmovups cs:ymmword_1806F3360, ymm0
vmovups ymm0, cs:ymmword_180639610
vmovups cs:ymmword_1806F3380, ymm0
vmovaps xmm0, cs:xmmword_18052C520
vmovaps cs:xmmword_1806F33A0, xmm0
vmovups ymm0, cs:ymmword_180639630
vmovups cs:ymmword_1806F33B0, ymm0
vmovups ymm0, cs:ymmword_180639650
vmovups cs:ymmword_1806F33D0, ymm0
vmovups ymm0, cs:ymmword_180639690
vmovups cs:ymmword_1806F3410, ymm0
vmovups ymm0, cs:ymmword_180639670
vmovups cs:ymmword_1806F33F0, ymm0
vmovups ymm0, cs:ymmword_180639710
vmovups cs:ymmword_1806F3490, ymm0
vmovups ymm0, cs:ymmword_1806396F0
vmovups cs:ymmword_1806F3470, ymm0
vmovups ymm0, cs:ymmword_1806396D0
vmovups cs:ymmword_1806F3450, ymm0
vmovups ymm0, cs:ymmword_1806396B0
vmovups cs:ymmword_1806F3430, ymm0
and so on. It looks like compiler generated block.
pinterf
17th March 2023, 19:49
It's an unrolled loop doing memcpy.
Fortunately I have a non-avx machine at home.
DTL
17th March 2023, 20:19
It is doing memcpy of large enough block. Typical universal memcpy must support from 1 byte to any ?
jpsdr
17th March 2023, 21:04
Tested on my Video PC with AVX2 (and Windows 7), my llvm build works.
DTL
17th March 2023, 21:20
So do llvm build environment have some settings to force disable emitting AVX and later instructions ? Or these builds can be only marked as AVX(2) minimum ?
Users may still have some second-hand multicore multi-chips Xeons systems capable of AVS processing but noAVX even.
pinterf
17th March 2023, 21:30
It is doing memcpy of large enough block. Typical universal memcpy must support from 1 byte to any ?
It is optimized, prechecking the data, then automatically handling multiple processor path, small block, aligned block, large block, overlaps, small one, usually inlined, different technique when amount is known; both MS and LLVM do it.
vBulletin® v3.8.11, Copyright ©2000-2026, vBulletin Solutions Inc.