Speedup using MP_Pipeline or splitup

[anton_foy]

Hi!

This script I would like to speed up (file is UHD 3840x2160, 25fps):
### Avisynth+ 3.6.1 (r3300, 3.6, I386) ###
SetFilterMTMode("DEFAULT_MT_MODE", 2)

#Index
FFVideoSource("F:\C1044.mp4")

SetFilterMTMode("FFVideoSource", MT_SERIALIZED)

convertbits(16)

# DENOISE #
neo_dfttest(sigma=16,sigma2=16,f0beta=1.0,tbsize=1,sbsize=120,sosize=90, y=2 , u=3, v=3, threads=4) #threads=4
mergeluma(TemporalSoften(3, 4, 8, mode=2))


# UNBLOW #
clip1 = last.Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false)
clipR = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractR()
clipG = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractG()
clipB = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractB()
RBD = clipB.Overlay(clipR, mode="darken", opacity=1.0)
GRD = clipG.Overlay(RBD, mode="darken", opacity=1.0)
umask= Greyscale().Levels(197*256, 0.76, 65280, 0, 255*256, coring=false, dither=false).fastgaussblur(20)
mt_merge(clip1, GRD, umask, Y=3, U=2, V=2, chroma="copy first", w=-1, h=-1, paramscale="i16")

# TOPSHARP #
shrp=sharpen(1.0)
vid1 = last
emask1 = vid1.mt_edge(mode="prewitt",thy1=0,thy2=65280, paramscale="i16").mt_expand(paramscale="i16").fastgaussblur(10)
result1 = mt_merge(vid1,shrp,emask1, U=2,V=2, paramscale="i16") #tsharp
merge(result1)

# SOFTEN #
blr=mt_expand(paramscale="i16").Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56)
vid = last
shine = vid.Overlay(blr, mode="blend", opacity=0.9)
emask = vid.mt_edge(mode="prewitt",thy1=0,thy2=65280, paramscale="i16").mt_expand(paramscale="i16").mt_expand(paramscale="i16").mt_expand(paramscale="i16").fastgaussblur(20)
result = mt_merge(vid,shine,emask, U=2,V=2, paramscale="i16")
result

#Debanding
neo_f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=true)


#Chroma Upscaling
ConverttoYUV444()

ConvertBits(bits=10, dither=1)
Prefetch(6)

My computer: 16gb RAM, Intel i5-3570k, 3.40Ghz (4 cores/threads)

I have tried with MP_pipeline:
MP_Pipeline("""

### Avisynth+ 3.6.1 (r3300, 3.6, I386) ###
SetFilterMTMode("DEFAULT_MT_MODE", 2)

#Index
FFVideoSource("F:\C1044.mp4")

# ### prefetch: 32, 16
### ###
### platform: Win32

SetFilterMTMode("FFVideoSource", MT_SERIALIZED)
trim(2,6)

convertbits(16)

### platform: win32
### branch 2
### ###

### platform: Win32

### ###

# BEST CHROMA DENOISE #
""")
neo_dfttest(sigma=16,sigma2=16,f0beta=1.0,tbsize=1,sbsize=120,sosize=90, y=2 , u=3, v=3, threads=4) #threads=4
("""
mergeluma(TemporalSoften(3, 4, 8, mode=2))

### platform: win32
### branch 2
### ###

### platform: Win32

### ###

# UNBLOW #
clip1 = last.Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false)
clipR = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractR()
clipG = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractG()
clipB = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractB()
RBD = clipB.Overlay(clipR, mode="darken", opacity=1.0)
GRD = clipG.Overlay(RBD, mode="darken", opacity=1.0)
umask= Greyscale().Levels(197*256, 0.76, 65280, 0, 255*256, coring=false, dither=false).fastgaussblur(20)
mt_merge(clip1, GRD, umask, Y=3, U=2, V=2, chroma="copy first", w=-1, h=-1, paramscale="i16")


# TOPSHARP #
shrp=sharpen(1.0)
vid1 = last
emask1 = vid1.mt_edge(mode="prewitt",thy1=0,thy2=65280, paramscale="i16").mt_expand(paramscale="i16").fastgaussblur(10)
result1 = mt_merge(vid1,shrp,emask1, U=2,V=2, paramscale="i16") #tsharp
merge(result1)

# SOFTEN #
blr=mt_expand(paramscale="i16").Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56)
vid = last
shine = vid.Overlay(blr, mode="blend", opacity=0.9)
emask = vid.mt_edge(mode="prewitt",thy1=0,thy2=65280, paramscale="i16").mt_expand(paramscale="i16").mt_expand(paramscale="i16").mt_expand(paramscale="i16").fastgaussblur(20)
result = mt_merge(vid,shine,emask, U=2,V=2, paramscale="i16")
result

#Debanding
neo_f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=true)

### platform: Win32

### ###

""")
#Chroma Upscaling
ConverttoYUV444()

ConvertBits(bits=10, dither=1)
Prefetch(6)
but MP_Pipeline doesn't like neo_dfttest so when I try to make MP_pipeline exclude neo_dfttest it ignores it and all the greyed colored part below it aswell. Probably I do not use the "exclusion" of neo_dfttest correctly (red part)?

I also tried StainlessS MakeMultiPartScripts.avs to split it into 4 or 3 parts but 4 parts was too heavy to run simultaniously for my comp and 3 parts was not faster than using MTmode.

Any ideas? Thanks!

[FranceBB]

I never really played with Prefect() that much, but, talking about neo_dfttest, if that's the only one that doesn't work with MPPipeline, you could go for the good old DFTTest() (https://github.com/pinterf/dfttest/releases/download/1.9.6/dfttest-v1.9.6.7z):


MP_Pipeline("""

### Avisynth+ 3.6.1 (r3300, 3.6, I386) ###
SetFilterMTMode("DEFAULT_MT_MODE", 2)

#Index
FFVideoSource("F:\C1044.mp4")

# ### prefetch: 32, 16
### ###
### platform: Win32

SetFilterMTMode("FFVideoSource", MT_SERIALIZED)
trim(2,6)

convertbits(16)

### platform: win32
### branch 2
### ###

### platform: Win32

### ###

# BEST CHROMA DENOISE #

ConverttoStacked()

### ###
### platform: Win32

dfttest(sigma=16, tbsize=1, lsb_in=true, lsb=true, Y=true, U=true, V=true, opt=0, dither=0)

### ###
### platform: Win32

ConvertFromStacked()

### ###
### platform: Win32


mergeluma(TemporalSoften(3, 4, 8, mode=2))

### platform: win32
### branch 2
### ###

### platform: Win32

### ###

# UNBLOW #
clip1 = last.Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false)
clipR = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractR()
clipG = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractG()
clipB = Levels(0, 1, 65280, 0, 235*256, coring=false, dither=false).converttorgb64().extractB()
RBD = clipB.Overlay(clipR, mode="darken", opacity=1.0)
GRD = clipG.Overlay(RBD, mode="darken", opacity=1.0)
umask= Greyscale().Levels(197*256, 0.76, 65280, 0, 255*256, coring=false, dither=false).fastgaussblur(20)
mt_merge(clip1, GRD, umask, Y=3, U=2, V=2, chroma="copy first", w=-1, h=-1, paramscale="i16")

### ###
### platform: Win32


# TOPSHARP #
shrp=sharpen(1.0)
vid1 = last
emask1 = vid1.mt_edge(mode="prewitt",thy1=0,thy2=65280, paramscale="i16").mt_expand(paramscale="i16").fastgaussblur(10)
result1 = mt_merge(vid1,shrp,emask1, U=2,V=2, paramscale="i16") #tsharp
merge(result1)

### ###
### platform: Win32

# SOFTEN #
blr=mt_expand(paramscale="i16").Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56).Blur(1.56)
vid = last
shine = vid.Overlay(blr, mode="blend", opacity=0.9)
emask = vid.mt_edge(mode="prewitt",thy1=0,thy2=65280, paramscale="i16").mt_expand(paramscale="i16").mt_expand(paramscale="i16").mt_expand(paramscale="i16").fastgaussblur(20)
result = mt_merge(vid,shine,emask, U=2,V=2, paramscale="i16")
result

### ###
### platform: Win32

#Debanding
neo_f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, mt=true)

### platform: Win32

### ###

""")
#Chroma Upscaling
ConverttoYUV444()

ConvertBits(bits=10, dither=1)
Prefetch(6)



By the way, I didn't cross-check it, but if neo_DFTTest really doesn't work with MPPipeline, then it's worth letting MeteorRain know about this (but I'll leave it up to you).

[anton_foy]

@FranceBB
Thank you :)!! Changing both to dfttest and f3kdb (turned out that also neo_f3kdb did not work with MP_Pipeline) with stacked16 works like a charm! Speed=0.0325x