Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 2nd April 2019, 16:41   #1  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
FluxSmooth - pfmod

As the title says.
An almost complete rewrite. x86/x64, clang, AVX2.

https://github.com/pinterf/FluxSmooth/releases

Packet contains DLLs compiled with clang (LLVM), and (probably) XP compatible Microsoft Visual CPP compiled DLLs.

Enjoy.

Code:
- (20190402) v1.3, rewrite by pinterf
  - project moved to github: https://github.com/pinterf/FluxSmooth
  - Built using Visual Studio 2017, additional LLVM 8.0 clang support
  - Changed to AVS 2.6 plugin interface
  - x64 build for Avisynth+
  - Added version resource to DLL
  - Removed MMX support, requires SSE2. (Though pure C is still available in the source)
  - Drop all inline assembly, SIMD intrinsics based on C code, SSE2, SSE4.1 and AVX2 optimizations
  - Single DLL, optimizations for different CPU instruction sets are chosen automatically.
  - Reports MT Modes for Avisynth+: MT_NICE_FILTER
  - Added Y, YV411, YV16 and YV24, 10-16 bits 4:2:0, 4:2:2, 4:4:4, planar RGB(A) 8-16 bits support besides existing YV12
  - (YUY2 support with workaround: internally converted to YV16, process and convert back 
    conversion is lossless, but slower than using native YV16)
  - New parameters: bool "luma", bool "chroma" (default true) to disable processing of luma/chroma planes
And an interesting benchmark:
Code:
ColorBars(pixel_type = "YV12")
FluxSmoothST(12,10,opt=0) # 0: C, #1: SSE2, #2: SSE4.1, #3: AVX2
#or:
#FluxSmoothT(12,opt=0) # 0: C, #1: SSE2, #2: SSE4.1, #3: AVX2

# Benchmarks i7-7770 Win10 Avisynth+ r2837 (fps)

# 32 bit LLVM 8.0           32 bit Visual C++ 15.9
#    opt=0  1    2    3     opt=0  1   2    3
# ST 1765 1882 2509 4776    1380 1540 1840 3730 
#  T 1834 3539 6116 12658   1530 3390 5210 10860

# 64 bit LLVM 8.0           64 bit Visual C++ 15.9
# ST 2020 1970 2520 5100    1520 1700 2048  4303 
#  T 2220 3600 6570 12700   2110 3630 5306 13100

Last edited by pinterf; 3rd April 2019 at 05:18. Reason: Benchmark typo in title
pinterf is offline   Reply With Quote
Old 3rd April 2019, 01:42   #2  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Quote:
and (probably) XP compatible
It is Windows XP compatible (up to SSE4.1 of course due to the lack of AVX implementation on XP at OS level).
Link: Image
Many thanks!!
FranceBB is offline   Reply With Quote
Old 3rd April 2019, 09:35   #3  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Thanks!

Btw does AVX-512 optimization require a lot of work/time?
StvG is offline   Reply With Quote
Old 3rd April 2019, 09:50   #4  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
"Can you play piano?" "Dunno, never tried"
Possibly not, but I'd need a tester.
Is it such a bottleneck?
And a question/feedback request: did this version improved your fps in real life scenario and if so, how much?

EDIT:
Test 1.4 w/ AVX512 (AVX512F and AVX512BW required)
https://drive.google.com/open?id=1Uz...NKiEQ-kGq9wcWP

Last edited by pinterf; 3rd April 2019 at 17:33. Reason: version test2
pinterf is offline   Reply With Quote
Old 6th April 2019, 00:47   #5  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
No, it's not a bottleneck.
It's used usually in combination with other filters in real life scenario and there is no difference (< 1%) between this version and the old (SSSE3) for the final fps. This version is ~3x faster when used only FluxSmoothST().

Actually the question of AVX512 was not specific to this plugin but rather as general.

AVX2.clang
Code:
FFVideoSource("source.mkv")
fluxsmoothst()


AVX512.vs
Code:
FFVideoSource("source.mkv")
LoadPlugin(".\FluxSmooth-pfmod-v1.4-test2\x64_xp\FluxSmooth.dll")
fluxsmoothst()


AVX512.clang
Code:
FFVideoSource("source.mkv")
LoadPlugin(".\FluxSmooth-pfmod-v1.4-test2\x64\FluxSmooth.dll")
fluxsmoothst()



A real life scenario - AVX2 vs AVX512:

AVX2
Code:
FFVideoSource("1080p.source.mkv")
z_ConvertFormat(pixel_type="RGBPS",colorspace_op="709:709=>rgb:linear", resample_filter_uv="spline36", cpu_type="avx2")
z_ConvertFormat(854,480,resample_filter="spline36",pixel_type="YV12",colorspace_op="rgb:linear=>709:709",dither_type="error_diffusion", cpu_type="avx2")


AVX512
Code:
FFVideoSource("1080p.source.mkv")
z_ConvertFormat(pixel_type="RGBPS",colorspace_op="709:709=>rgb:linear", resample_filter_uv="spline36", cpu_type="skylake-x")
z_ConvertFormat(854,480,resample_filter="spline36",pixel_type="YV12",colorspace_op="rgb:linear=>709:709",dither_type="error_diffusion", cpu_type="skylake-x")
StvG is offline   Reply With Quote
Old 6th April 2019, 07:24   #6  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
StvG, thanks for the tests (and giving me motivation for experimenting with this new topic).
I was doing my tests with Intel SDE emulator, which is good only for seeing if there is any difference in the results. But I'm not able to do benchmarks, emulation is way too slow.
Actually, Microsoft build (in the xp folder) cannot do AVX512, specifically AVX512BW, so AVX512 parts are missing from that build.
The issue was reported to MS.
pinterf is offline   Reply With Quote
Old 8th April 2019, 19:50   #7  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Thanks for your time.
If you need more benchmarks, I could do them.
StvG is offline   Reply With Quote
Old 25th July 2019, 12:21   #8  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by pinterf View Post
StvG, thanks for the tests (and giving me motivation for experimenting with this new topic).
I was doing my tests with Intel SDE emulator, which is good only for seeing if there is any difference in the results. But I'm not able to do benchmarks, emulation is way too slow.
Actually, Microsoft build (in the xp folder) cannot do AVX512, specifically AVX512BW, so AVX512 parts are missing from that build.
The issue was reported to MS.
It is worth to report issues to Microsoft. I've just got an update on my report that they implemented those missing AVX512 intrinsics in their latest VS 2019 16.2.

https://developercommunity.visualstu...ntrinsics.html
pinterf is offline   Reply With Quote
Old 26th July 2019, 19:03   #9  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 447
Now msvc build (16.2) AVX-512 is ~2% faster than clang build (8.0.1) AVX-512.
But using "opt=3" (AVX2) the same clang build is ~10% faster than the same msvc build.
StvG is offline   Reply With Quote
Old 24th August 2019, 20:13   #10  |  Link
hungryeyes
Registered User
 
Join Date: Apr 2019
Posts: 3
I'm really enjoying this updated version of fluxsmooth.

Last night I had the idea of combining fluxsmooth with Didee & LaTo's LimitedSharpen, MVtools2, and masktools2

I haven't tested it yet, but here's what I came up with:
Code:
super = MSuper()
backward_vectors = MAnalyse(super, isb = true)
forward_vectors = MAnalyse(super, isb = false)
forward_compensation = MFlow(super, forward_vectors, thSCD1=500) # or use MCompensate
backward_compensation = MFlow(super, backward_vectors, thSCD1=500)

sharp=sharpen(1)

# create interleaved 3 frames sequences
interleave(forward_compensation, last, backward_compensation)

FluxSmoothT() # place your preferred temporal (spatial-temporal) denoiser here

selectevery(3,1) # return filtered central (not-compensated) frames only

# detect edges
edge=mt_edge(thY1=0,thY2=255,mode="min/max",U=1,V=1).mt_lut("x 32 / 0.86 ^ 255 *",U=1,V=1)

# add sharp edges back
overlay(sharp, mask=edge, mode="blend", opacity=1)
The edge detection is from LSFMod[1] and the motion compensation stuff is from the examples in the MVtools2 documentation[2].
This is probably far from optimal, however the new FluxSmooth is quite a bit more flexible than the old version!

*
http://avisynth.nl/index.php/LSFmod
http://avisynth.org.ru/mvtools/mvtools2.html#examples

Last edited by hungryeyes; 26th August 2019 at 19:39. Reason: added comments/links
hungryeyes is offline   Reply With Quote
Old 4th September 2021, 02:03   #11  |  Link
ryrynz
Registered User
 
ryrynz's Avatar
 
Join Date: Mar 2009
Posts: 3,646
Oh wow, two years late to the party but so nice to see this. I asked for an updated Fluxsmooth many years ago. Will give Hungryeyes's script a go too.
Pinterf, you're the best.
ryrynz is offline   Reply With Quote
Old 4th September 2021, 02:55   #12  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,153
LSFmod new ver
https://github.com/Dogway/Avisynth-S...mod.v2.9x.avsi
kedautinh12 is offline   Reply With Quote
Old 4th September 2021, 04:46   #13  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
Quote:
Originally Posted by kedautinh12 View Post
I may come off as grumpy but why do you keep posting off topic comments? LSFmod has nothing to do with Fluxsmooth. One is a sharpener and the other is a denoiser. LSFmod does not even use FluxSmooth. Lastly, Dogway has his own thread where he posts any updates that happen, so there is no need to be cross-posting or even worse, posting off topics comments.
Reel.Deel is offline   Reply With Quote
Old 4th September 2021, 04:49   #14  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,153
Quote:
Originally Posted by Reel.Deel View Post
I may come off as grumpy but why do you keep posting off topic comments? LSFmod has nothing to do with Fluxsmooth. One is a sharpener and the other is a denoiser. LSFmod does not even use FluxSmooth. Lastly, Dogway has his own thread where he posts any updates that happen, so there is no need to be cross-posting or even worse, posting off topics comments.
Are you see #10??? He shared link old ver, i just update link new ver. And #11 still tag srcipt in #10. I lazy quote his comment in #10 and you did't seen that???

Last edited by kedautinh12; 4th September 2021 at 04:53.
kedautinh12 is offline   Reply With Quote
Old 4th September 2021, 04:57   #15  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
Quote:
Originally Posted by kedautinh12 View Post
Are you see #10??? He shared link old ver, i just update new ver. I lazy quote his comment in #10 and you did't seen that???
Still off-topic ... so with that mentality are you going to go to every single thread that mentions LSFmod and post a link to the updated version? I hope not.



Quote:
8) No cross posting. Post your message once, to the appropriate forum....

11) Don't post just to increase your number of posts. If you have nothing to say on a certain matter then don't post. The number of posts on this board has no direct relation to the experience of a member.
I'm guilty of #11 by replying here but it's getting out hand.

I miss N2
Reel.Deel is offline   Reply With Quote
Old 4th September 2021, 05:04   #16  |  Link
kedautinh12
Registered User
 
Join Date: Jan 2018
Posts: 2,153
Ah, ok my bad, sr
kedautinh12 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:00.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.