Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th April 2020, 18:33   #61  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,589
now all them (Except ICL 19.1) seems as fast as v3.3.8 from https://forum.doom9.org/showthread.p...13#post1883313

but ICL 19.1 one is the fastest for now!
__________________
See My Avisynth Stuff
real.finder is offline   Reply With Quote
Old 17th April 2020, 19:45   #62  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,616
Quote:
Originally Posted by HolyWu View Post
For people who want to try binaries built by different compilers, here are the new builds.
Great job!

__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 22nd April 2020, 07:32   #63  |  Link
Zetti
Registered User
 
Join Date: Dec 2015
Posts: 318
Link for the new build is invalid.
Zetti is offline   Reply With Quote
Old 22nd April 2020, 13:04   #64  |  Link
foxyshadis
Angel of Night
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,559
Other than build fixes and the very rare bugfix, the last meaningful change to FFTW was in 2015, so other than hunting down and fixing a bug yourself, a newer build isn't likely to mean anything anyway.
foxyshadis is offline   Reply With Quote
Old 9th May 2020, 05:47   #65  |  Link
almosely
Registered User
 
Join Date: Dec 2006
Location: Germany
Posts: 91
Thanks, HolyWu!

I tested all your versions against each other and against my old compile from Wolfberry (3.3.8 AVX ICC 19.0).

- edit -

I just uploaded Wolfberry's build here, if someone needs it. There are multiple files for different SIMD versions (SSE2, AVX, AVX2) for x64 and x86.

- edit -

What is your version exactly, HolyWu? 3.3.8 with SSE2 and AVX enabled?

Quote:
Originally Posted by http://www.fftw.org/release-notes.html
FFTW 3.3.5
Jul 31, 2016

- New SIMD support:
-- Power8 VSX instructions in single and double precision. To use, add --enable-vsx to configure.
-- Support for AVX2 (256-bit FMA instructions). To use, add --enable-avx2 to configure.
-- Experimental support for AVX512 and KCVI. (--enable-avx512, --enable-kcvi) This code is expected to work but the FFTW maintainers do not have hardware to test it.
-- Support for AVX128/FMA (for some AMD machines) (--enable-avx128-fma)
-- Double precision Neon SIMD for aarch64. This code is expected to work but the FFTW maintainers do not have hardware to test it.
-- generic SIMD support using gcc vector intrinsics.
- Add fftw_make_planner_thread_safe() API.
- fix #18 (disable float128 for CUDACC).
- fix #19: missing Fortran interface for fftwq_alloc_real.
- fix #21 (don't use float128 on Portland compilers, which pretend to be gcc).
- fix: Avoid segfaults due to double free in MPI transpose.
- Special note for distribution maintainers: Although FFTW supports a zillion SIMD instruction sets, enabling them all at the same time is a bad idea, because it increases the planning time for minimal gain. We recommend that general-purpose x86 distributions only enable SSE2 and perhaps AVX. Users who care about the last ounce of performance should recompile FFTW themselves.
I am running Windows 10 Pro (x64). I have an Intel Core i7-3770 CPU with 16 GB DDR3 1333 RAM (Dual-Channel).

My test-script was:

DGSource("test.dgi", crop_l=0, crop_r=0, crop_t=0, crop_b=0, deinterlace=0, use_top_field=true, use_pf=true)

Trim(0, 1000)

RequestLinear(rlim=100, clim=100)

neo_fft3d(sigma=1.0, beta=1.0, bw=32, bh=32, sharpen=0.040, scutoff=0.01, y=3, u=2, v=2, bt=5)

Prefetch(2) # 3,4 and without prefetch

return last

My test-clip was:

1280 x 720 (tennis training at roland garros)

Additionally to the following Intel C++ redists (I updated from 15 to 19 for this test) are all the (latest) versions of MS Visual C++ redists from 2005 to 2019 installed.
Sadly, the newest Intel C++ version (2020 19.1) is not downloadable (Intel-Link is broken).



... so, for me, GCC 9.3 was the fastest and least cpu consuming version of all.

Oh, and I used AviSynth+ 3.5.1 r3106 (x64) and AVSMeter 2.9.9.1 (x64).

- edit -

Meanwhile I got the newest Intel C++ 2020 version from here and made two further tests (as seen in the newer table above). GCC 9.3 is still the fastest and most efficient build.

Last edited by almosely; 12th May 2020 at 22:01.
almosely is offline   Reply With Quote
Old 10th May 2020, 02:38   #66  |  Link
almosely
Registered User
 
Join Date: Dec 2006
Location: Germany
Posts: 91
Could it be possible that the GCC compile is bypassing the injected Spectre and Meltdown protection microcode somehow, while the ICC and MSVC versions don't?

almosely is offline   Reply With Quote
Old 10th May 2020, 08:26   #67  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
I would like to suggest that plugin authors using this library support loading the DLLs from the path environment variable (LoadLibrary searches in path by default), otherwise GUIs are forced to put all plugins and DDLs in the same folder if they want to support avisynth and vapoursynth portable mode. This is extremely messy, there is a workaround using soft links that generally works, it's just not the best possible solution, needs extra code and requires to identify all plugins which use the library, in case of staxrip not easy as it includes 152 plugins:

AddGrainC, adjust, AnimeIVTC, AutoAdjust, Average, AviSynthShader AVSI, AviSynthShader DLL, AvsResize, AVSTP, AWarpSharp2, BM3D, Bwdif, checkmate, CNR2, CNR2, CropResize, CTMF, d2vsource, DAA3Mod, DCTFilter, DCTFilter, DCTFilter-f, Deblock, Deblock, Deblock_QED, DeblockPP7, Decomb, DegrainMedian, DeGrainMedian, DehaloAlpha, DeNoise Histogram, DeNoiseMD, DeNoiseMF, DePan, DePanEstimate, DFTTest, DFTTest, DGDecodeNV, Dither, Dither AVSI, Dither DLL, DSS2mod, edi_rpow2 AVSI, EEDI2, EEDI2, EEDI3, eedi3_resize, EEDI3m, ffms2, FFT3DFilter, FFT3DFilter, FFT3DGPU, FineDehalo, finesharp, FineSharp, FixTelecinedFades, flash3kyuu_deband, FluxSmooth, FluxSmooth, fmtconv, FrameRateConverter AVSI, FrameRateConverter DLL, fvsfunc, G41Fun, GradFun2DB, GradFun2DBmod, havsfunc, HQDeringmod, HQDN3D, HQDN3D, InterFrame, IT, JincResize, JPSDR, KNLMeansCL, Lazy Utilities, LSFmod, L-SMASH-Works, MAA2Mod, masktools2, mcdegrainsharp, mClean, MCTemporalDenoise, MedianBlur2, MiniDeen, MipSmooth, modPlus, MPEG2DecPlus, MSharpen, msmoosh, MT Expand Multi, MultiSharpen, muvsfunc, mvmulti, mvsfunc, mvtools, mvtools2, mvtools-sf, NicAudio, nnedi3, nnedi3 AVSI, nnedi3_rpow2, nnedi3cl, nnedi3x AVSI, Oyster, Plum, psharpen, pSharpen, QTGMC, resamplehq, ResizeX, RgTools, Sangnom, SangNom2, scenechange, SMDegrain, SmoothAdjust, SmoothD2, SmoothD2c, SVPFlow 1, SVPFlow 1, SVPFlow 2, SVPFlow 2, taa, TCanny, TDeint, TDeintMod, TEMmod, TemporalMedian, temporalsoften, TimeCube, TIVTC, TMM2, TNLMeans, TTempSmooth, UnDot, VagueDenoiser, VagueDenoiser, VapourSource, vcfreq, vcmod, vcmove, Vine, vinverse, vsCube, VSFilterMod, W3FDIF, xNLMeans, Yadifmod, yadifmod2, YFRC, znedi3
stax76 is offline   Reply With Quote
Old 10th May 2020, 09:03   #68  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Why can't you keep all plugins in one folder? I mean avisynth is doing the same...

In staxrip you could use the loaddll plugin to load FFTW from anywhere I think.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database

Last edited by ChaosKing; 10th May 2020 at 09:06.
ChaosKing is offline   Reply With Quote
Old 10th May 2020, 09:24   #69  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
If every plugin would only be exactly one file then it would be OK, if there are more files like support DLLs and help files then you need a folder otherwise it's messy.

Every good extension system supports folders:

https://mpv.io/manual/master/#script-location

https://github.com/stax76/mpv.net/bl....md#extensions

Every tool including plugins has its own folder in staxrip.

Quote:
In staxrip you could use the loaddll plugin to load FFTW from anywhere I think.
This might be an alternative to symbolic links, didn't really know it.
stax76 is offline   Reply With Quote
Old 10th May 2020, 10:09   #70  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,616
Quote:
Originally Posted by stax76 View Post
If every plugin would only be exactly one file then it would be OK, if there are more files like support DLLs and help files then you need a folder otherwise it's messy.
Just rename them to follow plugin naming.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 10th May 2020, 10:10   #71  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 2,616
Quote:
Originally Posted by almosely View Post
Could it be possible that the GCC compile is bypassing the injected Spectre and Meltdown protection microcode somehow, while the ICC and MSVC versions don't?
I have disabled all of them... useless here.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 10th May 2020, 10:46   #72  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,361
Quote:
Originally Posted by stax76 View Post
I would like to suggest that plugin authors using this library support loading the DLLs from the path environment variable (LoadLibrary searches in path by default)
Thats a bad idea for security purposes, which opens you up to DLL injection attacks of all sorts.

Its much better to determine where a plugin lives and load it specifically, instead of hoping the right one gets loaded from $somewhere. And not only for FFTW but in general all DLL loading.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 10th May 2020, 11:50   #73  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by stax76 View Post
I would like to suggest that plugin authors using this library support loading the DLLs from the path environment variable (LoadLibrary searches in path by default)
Every (Avisynth) plugin I know that uses FFTW loads the DLL by using LoadLibrary() or LoadLibraryEx().
Am I missing something?
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 10th May 2020, 12:05   #74  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
Quote:
Thats a bad idea for security purposes, which opens you up to DLL injection attacks of all sorts.

Its much better to determine where a plugin lives and load it specifically, instead of hoping the right one gets loaded from $somewhere. And not only for FFTW but in general all DLL loading.
Isn't then the whole path thing flawed altogether? Many tools just call LoadLibrary("avisynth.dll") and portable mode works because LoadLibrary searches path and it's a simple rule many devs agree and appreciate. If it's so bad then why is that the default behavior of LoadLibrary and dotnet? How are you going to make avisynth and vapoursynth portable work without putting all DLLs and executables in one folder creating a giant mess? What if executables require different versions of DLLs like ffmpeg libraries, please note that a GUI like staxrip or hybrid includes up to twenty tools that read avs and vpy and conflicts are almost certain. In staxrip I modify the path env par putting the locations on top, that makes DLL injection attacks impossible, right?

Quote:
Every (Avisynth) plugin I know that uses FFTW loads the DLL by using LoadLibrary() or LoadLibraryEx().
Am I missing something?
There are some, I use soft links to workaround it, probably I miss some, got to check that using the CLI of Dependency Walker, look here:

https://github.com/staxrip/staxrip/b...Class.vb#L1395

Last edited by stax76; 10th May 2020 at 12:11.
stax76 is offline   Reply With Quote
Old 10th May 2020, 13:04   #75  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by stax76 View Post
There are some, I use soft links to workaround it, probably I miss some, got to check that using the CLI of Dependency Walker
You won't find them with DepWalker, they are delay-loaded.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 10th May 2020, 13:06   #76  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
If you want portable, have a look at this.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 10th May 2020, 13:53   #77  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
LoadDLL maybe is cleaner than soft links, I think I will try this, need to look how to do it with python.

I didn't even know delay load is possible, runtime linking has still advantages in regard of versioning?

I use it here:

https://github.com/staxrip/staxrip/b...Server.cpp#L56

nvenc code is similar.

I can wait for bug reports, some tips which else plugin might be affected would be a great help, so far I'm aware of three vapoursynth filters:

DFTTest
BM3D
DCTFilter

Last edited by stax76; 10th May 2020 at 13:59.
stax76 is offline   Reply With Quote
Old 10th May 2020, 14:02   #78  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by stax76 View Post
I didn't even know delay load is possible, runtime linking has still advantages in regard of versioning?
If you use LoadLibrary() in a plugin it's considered delay-load. Loading the DLL won't happen until the function in which you attempt to load the external DLL (fftw) is called.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 10th May 2020, 14:17   #79  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,837
Code:
In case you are not aware of, LoadPlugin in VapourSynth has an altsearchpath parameter which does what you want when set to True.
It would find DFTTest in path but also FFTW being in different path folder than DFTTest?
stax76 is offline   Reply With Quote
Old 10th May 2020, 14:24   #80  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,361
Quote:
Originally Posted by stax76 View Post
If it's so bad then why is that the default behavior of LoadLibrary and dotnet?
Legacy. Many things were designed a long time ago, and one needs to take care today to make them secure.
Microsoft doesn't typically break existing code, instead they expect you do use it differently.

Quote:
Originally Posted by stax76 View Post
How are you going to make avisynth and vapoursynth portable work without putting all DLLs and executables in one folder creating a giant mess?
Security tops your concerns. Just because something is hard is not a reason to throw security out of the window.

If you know where your AviSynth (etc) folder is, which you should, even in portable, then you can just dynamically build pathes, or even use SetDllDirectory/AddDllDirectory (with care, it has its own problems).
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 10th May 2020 at 14:26.
nevcairiel is offline   Reply With Quote
Reply

Tags
fftw, fftw3.dll

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:40.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.