New filter: Fix Telecined Fades

[feisty2]

Originally Posted by Myrsloik

Now you see why asm is kinda your friend.

There's a pile of pitfalls here you missed. And some you found.

1. You compile the whole file with avx2/whatever instrction set you selected. That means avx2 instructions may be emitted for you plain C++ code too. Solve it by placing the avx2 code in a separate file and use different compiler settings.

2. You can't put sse and avx intrinsics in the same file either. For example subps can get turned into vsubps to reduce register copies. So then the sse path needs avx too...

3. Cpuid is horrible in intrinsics, that's why I use asm. Lrn2yasm. Or write code that requires gcc or clang where convenient intrinsics are available. Clang is even available as a simple dropin for visual studio nowadays.

1. got it
2. got it
3. the thing is I can't set it up... yasm doesn't seem to have an installer like clang that gets everything done automatically, also the manual setting-up documentation on the yasm website is obsolete, the visual studio folders it mentioned don't even exist anymore! I could switch to clang but that won't solve the xgetbv problem cuz it's not defined in clang's immintrin.h... and back to square one, I have to set up yasm first which is impossible.

[sl1pkn07]

Originally Posted by feisty2

yeah


blankclip is okay, long as it's not crashing.

Ok, but how? Xd

[feisty2]

Originally Posted by sl1pkn07

Ok, but how? Xd

I separated the source code into 2 files.
just compile "Source.cpp" with "-march=native" and "Source_AVX_FMA.cpp" with "-march=haswell" and it should work, you don't have to edit the source code this time, I added the conditional compilation macros and it would pick "cpufeatures_gnu.hpp" automatically if you're on GCC

test script

import vapoursynth as vs
core = vs.get_core()
clp = core.std.BlankClip(width=1920, height=1080, format=vs.GRAYS, length=20000)
clp = core.ftf.FixFades(clp)
clp.set_output()

[sl1pkn07]

http://stackoverflow.com/questions/6...nstruction-set

if help you

[feisty2]

Originally Posted by sl1pkn07

http://stackoverflow.com/questions/6...nstruction-set

if help you

I know how to do that SIMD extension detecting at runtime and it's already there in the source code, I don't need any help

I'm asking if it's working cuz I'm not sure about that..

[sl1pkn07]

Failed to evaluate the script:
Python exception: No attribute with the name ftf exists. Did you mistype a plugin namespace?
Traceback (most recent call last):
File "src/cython/vapoursynth.pyx", line 1712, in vapoursynth.vpy_evaluateScript (src/cython/vapoursynth.c:34991)
File "/home/sl1pkn07/aplicaciones/vapoursynth-test/fixtelecided-test.vpy", line 4, in 
clp = core.ftf.FixFades(clp)
File "src/cython/vapoursynth.pyx", line 1306, in vapoursynth.Core.__getattr__ (src/cython/vapoursynth.c:28067)
AttributeError: No attribute with the name ftf exists. Did you mistype a plugin namespace?

all:
	  g++ -c -std=c++14 -fPIC -march=native -O2 -pipe -fstack-protector-strong -D_FORTIFY_SOURCE=2 -I. -I/usr/include/vapoursynth -o fixtelecinedfades.o Source.cpp
	  yasm -f elf -m amd64 -DARCH_X86_64=1 -o cpu.o cpu.asm
	  g++ -shared -fPIC -Wl,-O1,--sort-common,--as-needed,-z,relro -o libvsfixtelecinedfades.so fixtelecinedfades.o cpu.o

[feisty2]

Originally Posted by sl1pkn07

Failed to evaluate the script:
Python exception: No attribute with the name ftf exists. Did you mistype a plugin namespace?
Traceback (most recent call last):
File "src/cython/vapoursynth.pyx", line 1712, in vapoursynth.vpy_evaluateScript (src/cython/vapoursynth.c:34991)
File "/home/sl1pkn07/aplicaciones/vapoursynth-test/fixtelecided-test.vpy", line 4, in 
clp = core.ftf.FixFades(clp)
File "src/cython/vapoursynth.pyx", line 1306, in vapoursynth.Core.__getattr__ (src/cython/vapoursynth.c:28067)
AttributeError: No attribute with the name ftf exists. Did you mistype a plugin namespace?

all:
	  g++ -c -std=c++14 -fPIC -march=native -O2 -pipe -fstack-protector-strong -D_FORTIFY_SOURCE=2 -I. -I/usr/include/vapoursynth -o fixtelecinedfades.o Source.cpp
	  yasm -f elf -m amd64 -DARCH_X86_64=1 -o cpu.o cpu.asm
	  g++ -shared -fPIC -Wl,-O1,--sort-common,--as-needed,-z,relro -o libvsfixtelecinedfades.so fixtelecinedfades.o cpu.o

then I think I won't be able to help you with that... sorry
I'm really just not familiar with all that gnu stuff and

1. there's the runtime simd extension detection so the unsupported forcibly-compiled functions won't matter cuz they will never get called.
2. you compiled "Source.cpp" which contains functions that will actually get called with "-march=native", so there should be no unsupported instructions in those functions.

so logically I can't see why your binary failed to work, maybe you should ask others that are more familiar with the whole gnu world..

[feisty2]

anyways, if you really wanna use this plugin,
manually comment out:
line3
line123
line124
line156 - line161
in "Source.cpp",
line10 - line14 in "Shared.hpp"
and compile it and it should work..

[jackoneill]

Python exception: No attribute with the name _____ exists. Did you mistype a plugin namespace?

If you get this error and you know you have the plugin, try to load it manually to find out what's wrong with it:

core.std.LoadPlugin("/your/plugin/here.so")

[sl1pkn07]

@jackoneill the steps for build the plugin whit yasm is correct?

edit:

Failed to evaluate the script:
Python exception: Failed to load /usr/lib/vapoursynth/libvsfixtelecinedfades.so. Error given: /usr/lib/vapoursynth/libvsfixtelecinedfades.so: undefined symbol: _Z24fixfadesGetFrame_AVX_FMAiiPPvS0_P14VSFrameContextP6VSCorePK5VSAPI
Traceback (most recent call last):
File "src/cython/vapoursynth.pyx", line 1712, in vapoursynth.vpy_evaluateScript (src/cython/vapoursynth.c:34991)
File "/home/sl1pkn07/aplicaciones/vapoursynth-plugin-fixtelecinedfades-git/fixtelecined-test.vpy", line 3, in 
clp = core.std.BlankClip(width=1920, height=1080, format=vs.GRAYS, length=20000)
File "src/cython/vapoursynth.pyx", line 1604, in vapoursynth.Function.__call__ (src/cython/vapoursynth.c:33131)
vapoursynth.Error: Failed to load /usr/lib/vapoursynth/libvsfixtelecinedfades.so. Error given: /usr/lib/vapoursynth/libvsfixtelecinedfades.so: undefined symbol: _Z24fixfadesGetFrame_AVX_FMAiiPPvS0_P14VSFrameContextP6VSCorePK5VSAPI

[jackoneill]

Gah, you made me look at the code.

You forgot to compile Source_AVX_FMA.cpp.

[sl1pkn07]

but that is not only for AVX CPU compilant?

Source.cpp -> older CPU
Source_AVX_FMA.cpp -> newer CPU

because Source_AVX_FMA.cpp needs build with -march=haswell, amd my cpu is not capable (don't have AVX or FMA3)

[jackoneill]

Originally Posted by sl1pkn07

but that is not only for AVX CPU compilant?

Source.cpp -> older CPU
Source_AVX_FMA.cpp -> newer CPU

because Source_AVX_FMA.cpp needs build with -march=haswell, amd my cpu is not capable (don't have AVX or FMA3)

If the author did everything right, the code from that file will only be used if the CPU has FMA3. You still have to compile it.

[sl1pkn07]

ok. done

[feisty2]

Originally Posted by sl1pkn07

ok. done

is it working now?

[sl1pkn07]

i think yes, but i need make more test

if all is ok, then i think you can merge the Source{,_AVX_FMA}.cpp again

sorry :S

[feisty2]

Originally Posted by sl1pkn07

i think yes, but i need make more test

if all is ok, then i think you can merge the Source{,_AVX_FMA}.cpp again

sorry :S

no, I can't(shouldn't).
if I do merge Source.cpp and Source_AVX_FMA.cpp into one file, you'll have to compile that file with -march=haswell, and the compiler will probably generate unsupported instructions for the C++ functions..

[feisty2]

Originally Posted by sl1pkn07

i think yes, but i need make more test

if all is ok, then i think you can merge the Source{,_AVX_FMA}.cpp again

sorry :S

I think you should make a PR on GitHub so I can merge those GNU compiling files into the repo

[sl1pkn07]

Then need making more refraction in Source.cpp, because as @jackoneill said,build Source.cpp with march=native and build Source_AVX_FMA3.cpp with march=haswell and merge both in the library, the plugin now load without problem. build

[feisty2]

Originally Posted by sl1pkn07

Then need making more refraction in Source.cpp, because build It with march=native and build source_avx_fma3.cpp with march=haswell and merge both in the library the plugin now load without problem. Like @jackoneill said

what do you mean by "more refraction in Source.cpp"?