MP_Pipeline 0.18 - run parts of avisynth script in external processes [2014-04-06]

[Chyrka]

real.finder, that hack must work. Thanks.

[bxyhxyh]

Hello, I wanted to speed up xsharpen with this. With MT version, xsharpen goes at nice 2x-4x speed. But sometimes it gives me corrupted frame.

Can I call it like this to prevent from denoising being inside of it?


mp_pipeline("""
Import("C:\Users\BallGNM\Desktop\denoised.avs")
s16=dither_resize16(6840,4860)
s16.ditherpost(mode=-1).xsharpen(strength=255,threshold=255).Dither_convert_8_to_16()
s16.Dither_limit_dif16 (last, thr=0.7, elast=2.0)
dither_resize16(1120,840).gradfun3(smode=2,lsb_in=true,lsb=false,mode=6,thr=0.5)
### branch 2
### ###
""")
dehalo_alpha_mt()

[tuanden0]

Can someone help me? I'm using avs+.

MP_Pipeline's great, my speed increase 30 ~ 45% and encode time decrease 30% with my script:

MP_Pipeline("""
### platform: win64
LWLibavVideoSource("E:\Test Zone\test.mkv")
AssumeFPS(24000, 1001)
FFT3DFilter(sigma=1.7, bt=1, ncpu=4)
flash3kyuu_deband()
Prefetch(4)
### ###
### platform: win32
Toon(0.25)
TextSub("E:\Test Zone\test.ass", 1)
### ###
""")

Then, I tried this script and my encode time decrease 2/3 with my speed increase 50 ~ 60%. But, sometime it's crashed :confused::confused::

MP_Pipeline("""
### platform: win64
LWLibavVideoSource("E:\Test Zone\test.mkv")
AssumeFPS(24000, 1001)
FFT3DFilter(sigma=1.7, bt=1, ncpu=4)
flash3kyuu_deband()
Prefetch(4)
### ###
### platform: win32
Toon(0.25)
TextSub("E:\Test Zone\test.ass", 1)
Prefetch(4)
### ###
""")

I tried to put Prefetch(4) outside MP_Pipeline but they slower than my 1st script :(:(

[pinterf]

Can someone help me? I'm using avs+.

I tried to put Prefetch(4) outside MP_Pipeline but they slower than my 1st script :(:(
This is a known issue in avs+ since r206x, workaround in progress.

[tuanden0]

@pinterf: :thanks: for the information

[FranceBB]

I'd like to split my script into multiple process in order to avoid the 2 GB limit, 'cause it crashes.

I'm trying to split my script like this:


MP_Pipeline("""

FFVideoSource("op.mxf", fpsnum=24000, fpsden=1001)
Spline64ResizeMT(1920, 1080, threads=4, logicalCores=true, MaxPhysCore=true, SetAffinity=true)
#AmplifyDB(-12.0)
opening=last

### platform: win32
### export clip: opening
### ###

FFVideoSource("ep.mxf", fpsnum=24000, fpsden=1001)
tdeint(mode=2, order=-1, field=-1, mthreshL=6, mthreshC=6, map=0, type=2, debug=false, mtnmode=1, sharp=true, cthresh=6, blockx=16, blocky=16, chroma=true, MI=64, tryWeave=true, link=1, denoise=true, slow=2, opt=4)
Converttoyv12(interlaced=false, chromaresample="Spline64", matrix="Rec709")
opening++trim(2157, 30327)
episode=last

### platform: win32
### export clip: episode
### ###

FFVideoSource("ed.mxf", fpsnum=24000, fpsden=1001)
Spline64ResizeMT(1920, 1080, threads=4, logicalCores=true, MaxPhysCore=true, SetAffinity=true)
#AmplifyDB(-14.0)
episode++trim(0,0)
final=last

### platform: win32
### export clip: final
### ###

#Normalize(0.89, show=false)
#ResampleAudio(48000)
tweak(sat=1.53 , dither=true)

### platform: win32
### ###

dfttest(sigma=64, tbsize=1, lsb_in=false, lsb=false, Y=true, U=true, V=true, opt=3, dither=0)

### platform: win32
### ###

Spline64ResizeMT(3840, 2160, threads=4, logicalCores=true, MaxPhysCore=true, SetAffinity=true)

### platform: win32
### ###

aWarpSharp2(thresh=180, blur=3, type=0, depth=35, depthC=10, chroma=4)

### platform: win32
### ###

f3kdb(range=15, Y=45, Cb=30, Cr=30, grainY=0, grainC=0, sample_mode=2, blur_first=true, dynamic_grain=false, opt=3, mt=true, keep_tv_range=true, input_depth=8, output_depth=8)

### platform: win32
### ###

TextSubmod("sub.ass")

### platform: win32
### ###
""")


I'll let you know if the encode finishes successfully or not.
If it works, MPP it's gonna be really useful to my daily tasks! :D

[FranceBB]

It works, but I don't know how to handle audio now, 'cause MPP doesn't support it.

[real.finder]

It works, but I don't know how to handle audio now, 'cause MPP doesn't support it.

maybe this (http://forum.doom9.org/showthread.php?t=169961) can help you, or you must wait someone can added audio to mpp (https://github.com/SAPikachu/MP_Pipeline/issues/1#issuecomment-269108878), or made another script just for audio

[FranceBB]

@real.finder... uh... I see... I just replied on github as well. As a "workaround", I'm using audio created in a separate avisynth script, created to make a low-res version that was sent out to our subtitling team. Anyway, audio support would be a really useful feature. Unfortunately, I'm a .NET guy and my lazy ass made me forget almost everything about C++ I studied at university and embrace C# time ago. Perhaps, JPSDR / StainlessS / Groucho2004 could take a look at the code and, if interested, add audio support.

[ChaosKing]

Or you could use avisynth 64 bit to avoid the 2gb limit... as far as I can see all your filters are also available for x64.

[Stereodude]

Anyone have a working download link for .18?

[poisondeathray]

Anyone have a working download link for .18?

MP_Pipeline-0.18 mirror

http://www.mediafire.com/file/vsmamxey9llxgxc/MP_Pipeline-0.18.rar

[Stereodude]

MP_Pipeline-0.18 mirror

http://www.mediafire.com/file/vsmamxey9llxgxc/MP_Pipeline-0.18.rar
Thanks!

[Dogway]

I'm trying to load 32-bit plugins in avs+ x64 latest build 2772 without success, I put the content of MP_Pipeline 0.18 64 folder into plugins+64 and try to preview the next script with AvsPmod.v2.5.1_x64 (also tried vdub2).

Did more tests with other code explicitly loading x86 avisynth dll.
MP_Pipeline("""
### platform: win32
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins+\JpegSource.dll")
jpegSource("D:\Profiles\Usuarios\Administrador\Desktop\147.jpg",length=1)

### ###
""")
But I get an error as if I still was using 64-bit avisynth.

MP_Pipeline: Unable to create slave process. Message: Script error: Cannot load a 64 bit DLL in 32 bit Avisynth: 'C:/Program Files (x86)/AviSynth+/plugins64+/firstfoundplugin_x64.dll'.

[Dogway]

I have doing more tests and I'm still unable to mix 32 and 64 bit plugins.
Here's my clean scenario.
Installed avs+ r2772 both x86 and x64, x86 through Avisynth Repository

Current registry paths:
[HKEY_LOCAL_MACHINE\SOFTWARE\Avisynth]
@="D:\\Rip\\workshop\\Front_Ends\\AvisynthRepository\\AVSPLUS_x64"
"PluginDir2_5"="D:\\Rip\\workshop\\Front_Ends\\AvisynthRepository\\AVSPLUS_x64\\plugins"
"PluginDir+"="C:\\Program Files (x86)\\AviSynth+\\plugins64+"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Avisynth]
@="D:\\Rip\\workshop\\Front_Ends\\AvisynthRepository\\AVSPLUS_x86"
"PluginDir2_5"="D:\\Rip\\workshop\\Front_Ends\\AvisynthRepository\\AVSPLUS_x86\\plugins"
"PluginDir+"="C:\\Program Files (x86)\\AviSynth+\\plugins+"

MP_Pipeline x64 is 318Kb v0.18 version:

plugins+ folder:
ConvertStacked.dll
DirectShowSource.dll
ImageSeq.dll
MtModes.avsi
Shibatch.dll
TimeStretch.dll
VDubFilter.dll

plugins64+ folder:
ConvertStacked.dll
DirectShowSource.dll
ImageSeq.dll
MP_Pipeline.dll
MP_Pipeline.dll.slave.exe
MP_Pipeline.dll.win32
MP_Pipeline.dll.win32.slave.exe
MP_Pipeline_readme.avs
MtModes.avsi
Shibatch.dll
SmoothAdjust.dll
TimeStretch.dll
VDubFilter.dll


This is my script loaded in AvsPmod 2.5.1 x64 from this link (https://forum.doom9.org/showthread.php?p=1801816#post1801816).
Plugin Autoload Folder in AvsPmod is set to: C:\Program Files (x86)\AviSynth+\plugins64+

LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\MP_Pipeline.dll")
MP_Pipeline("""
### platform: win32
imageSource("D:\Profiles\Usuarios\Administrador\Desktop\147.jpg")

### ###
""")

and error:

MP_Pipeline: Unable to create slave process. Message: Script error: Cannot load a 64 bit DLL in 32 bit Avisynth: 'C:/Program Files (x86)/AviSynth+/plugins64+/ConvertStacked.dll'.

Trying to explicitly load x86 avisynth:
LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\MP_Pipeline.dll")
MP_Pipeline("""
### platform: win32
### dll: "C:\Windows\SysWOW64\AviSynth.dll"
imageSource("D:\Profiles\Usuarios\Administrador\Desktop\147.jpg")

### ###
""")

MP_Pipeline: Unable to create slave process. Message: Unable to load "C:\Windows\SysWOW64\AviSynth.dll", code = 126

[Stereodude]

The combo works find for me (r2772 and MP 0.18). What program are you trying to load the script with? It looks like you're trying to load your script with a x86 (32-bit) program. This causes AVIsynth+ to try to load your 64-bit script in the x86 version and it fails.

Have you tried loading the script with the 64-bit version of VD2 or AVSmeter?

[Dogway]

My setup is posted above. r2772 yes, MP 0.18 yes 64bit AvsPmod, VD2, avsmeter64... and so on.
I had to fix the registries because AvisynthRepository was messing with them bad time but still no luck.
Now they are:
[HKEY_LOCAL_MACHINE\SOFTWARE\AviSynth]
@="C:\\Program Files (x86)\\AviSynth+\\plugins64+"
"plugindir2_5"="C:\\Program Files (x86)\\AviSynth+\\plugins64"
"plugindir+"="C:\\Program Files (x86)\\AviSynth+\\plugins64+"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\AviSynth]
@="C:\\Program Files (x86)\\AviSynth+\\plugins+"
"plugindir2_5"="C:\\Program Files (x86)\\AviSynth+\\plugins"
"plugindir+"="C:\\Program Files (x86)\\AviSynth+\\plugins+"
In my opinion MP_Pipeline is failing to load avisynth x86 for some reason.
It can't even load an avisynth.dll
### dll: C:\Windows\SysWOW64\AviSynth.dll
MP_Pipeline: Unable to create slave process. Message: Unable to load "C:\Windows\SysWOW64\AviSynth.dll", code = 193
By the way I'm on Win7 SP1 x64 if that makes a difference.

[Yanak]

Hello,

Not sure if it will help anything but I have both x86 and x64 version of avisynth, also on Win7 SP1 x64 , installed via the AviSynthPlus-MT-r2772.exe installer provided by pinterf here (https://github.com/pinterf/AviSynthPlus/releases)

One thing that ticks me in your script is " LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\MP_Pipeline.dll") "
Why program files x86 ?, the x64 version should be in "Program Files", the x86 version of avisynth is the one that should be in Program Files (x86),

according to your reg entries the "plugins+" & " plugins64+ " are located at the same place for both x86 and x64 version, both in Program Files (x86), seems strange, unless i'm not seeing clear tonight it looks like both your avisynth installs are locate at the same place in program files x86

I checked and i do not have any "plugins64+" folder in the x86 version at "C:\Program Files (x86)\AviSynth+", only a "plugin+" folder is present there.

My reg entries :
[HKEY_LOCAL_MACHINE\SOFTWARE\AviSynth]
@="C:\\Program Files\\AviSynth+"
"plugindir2_5"="C:\\Program Files\\AviSynth+\\plugins64"
"plugindir+"="C:\\Program Files\\AviSynth+\\plugins64+"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\AviSynth]
@="C:\\Program Files\\AviSynth+"
"plugindir2_5"="C:\\Program Files\\AviSynth+\\plugins"
"plugindir+"="C:\\Program Files (x86)\\AviSynth+\\plugins+"

MP-pipeline files :
MP_Pipeline.dll
MP_Pipeline.dll.slave.exe
MP_Pipeline.dll.win32
MP_Pipeline.dll.win32.slave.exe
MP_Pipeline_readme.avs
All located inside the x64 version of avisynth, here : "C:\Program Files\AviSynth+\plugins64+"

My x64 version of avisynth is installed at C:\Program Files\AviSynth+\
My x86 version of avisynth is nstalled at C:\Program Files (x86)\AviSynth+\



also with avisynth+ you don't need to call the Mp_pipeline dll at the start of the script , it is auto loaded so this line should not even be needed in the script.
Tested also within the 2.5.1 x64 version of AvsPmod that is, unless i make an error, the same as yours, there again no need to use the " LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\MP_Pipeline.dll") ", only this to load a pic .

MP_Pipeline("""
### platform: win32
imageSource("F:\My Pic.png")
### ###
""")

Like i said not sure it it will help with anything, maybe you can compare and find something that is odd.

Good luck, took me a while before getting MP_pipeline to work, hopefully the forum good souls helped me at this time :)

[Dogway]

Thanks for the help. I just preferred to have all my plugins in the same location for practical purposes, I investigated if this made a difference (like not taking advantage of 64bit features) and it seems it doesn't. The location of 64-bit executables is unrelated to their performance (despite some saying otherwise). BUT it could be MP_Pipeline wasn't designed in this particular way, I should have a look at what line 193 says in the source. I use MP_Pipeline since years ago, but recently just switched to avs+ x64 and after some hurdles this issue is pretty much the only thing irking me.
I'm finishing a project so I will have a deeper look in a few days. Maybe installing though the official installer this time.

[Yanak]

I see, maybe it is not liking it this way indeed, i have not looked into the code plus I am not a coder so i don't know much more than this sadly.

The very last thing that come to my mind and might help figure out thing is in avspmod to only do this as script, without using the LoadPlugin("C:\Program Files (x86)\AviSynth+\plugins64+\MP_Pipeline.dll") that is not needed anymore with avs+ ( it is needed inside the win32 or win64 loops of mp_pipeline as this nice tool can't auto-load plugins sadly ), anyways :

MP_Pipeline("""
### platform: win32
Version()
### ###
""")

And after the same with win64 variant

MP_Pipeline("""
### platform: win64
Version()
### ###
""")

and see what the preview window returns you for the versions of avisynth+ it is using, this is if it managed to load anything at all with mp_pipeline.

good luck

[Groucho2004]

I had to fix the registries because AvisynthRepository was messing with them bad time but still no luck.I'm assuming you mean that the Universal Installer messed with your registry entries. That shouldn't happen if you set up the batch file correctly. Can you describe the problem in more detail?

[real.finder]

try put those in plugins64 folder not the + one

MP_Pipeline.dll
MP_Pipeline.dll.slave.exe
MP_Pipeline.dll.win32
MP_Pipeline.dll.win32.slave.exe


and those in plugins

MP_Pipeline.dll
MP_Pipeline.dll.slave.exe
MP_Pipeline.dll.win64
MP_Pipeline.dll.win64.slave.exe


and you can try my edit too https://forum.doom9.org/showthread.php?p=1746479#post1746479

and make sure that you dont use teracopy or whatever since it did some problems back then for my friend, to check that try to load avs dll using ### dll: from other location

[FranceBB]

I just wanna report that Planar RGB 16bit (RGBP16) produces a green clip. Converting it back to yuv in the same block "solves" the issue.

MP_Pipeline("""

FFVideoSource("I:\Production\RAW\test8K.mkv")

### ###

Crop(0, 0, -0, -630)

### ###

AddBorders(0, 314, 0, 314)

### ###

Spline64Resize(848, 480)

### ###

trim(1081, 5220)

### ###

ConvertBits(16)

### ###

ConvertToPlanarRGB()

### ###

Cube("C:\Programmi\AviSynth+\LUTs\void_null.cube", cpu=1, fullrange=false)

Converttoyuv420()

""")

https://i.imgur.com/qhCegup.png

MP_Pipeline("""

FFVideoSource("I:\Production\RAW\test8K.mkv")

### ###

Crop(0, 0, -0, -630)

### ###

AddBorders(0, 314, 0, 314)

### ###

Spline64Resize(848, 480)

### ###

trim(1081, 5220)

### ###

ConvertBits(16)

### ###

ConvertToPlanarRGB()

Cube("C:\Programmi\AviSynth+\LUTs\void_null.cube", cpu=1, fullrange=false)

Converttoyuv420()

### ###

""")

https://i.imgur.com/o85cRBo.png

[Stereodude]

I just wanna report that Planar RGB 16bit (RGBP16) produces a green clip. Converting it back to yuv in the same block "solves" the issue.
I've also noticed it doesn't handle passing at least some of the new AVS+ formats between "blocks" correctly. Not all of them fail in such an obvious manner though.

[FranceBB]

Any chance for someone to pickup the project, work on it, make it Avisynth+ compatible (modern high bit depth formats) and support audio?
It would be a very good thing to have it working properly.
It's been years since the last release...
Even just audio support would be a big plus.

[Myrsloik]

Any chance for someone to pickup the project, work on it, make it Avisynth+ compatible (modern high bit depth formats) and support audio?
It would be a very good thing to have it working properly.
It's been years since the last release...
Even just audio support would be a big plus.

Why is it still needed when you have 64bit binaries and threading? I'm curious...

[FranceBB]

Why is it still needed when you have 64bit binaries and threading? I'm curious...

X86 only Avisynth is still very much popular so are hybrid workflows.
It could be because of plugins that have not been ported yet or perhaps because of people having dual boot OS which includes a relatively modern x64 OS and an x86 one. I've seen many people having Windows 7 x64 and XP x86. I myself have Windows 10 Pro x64, however I still have my Win XP x86 machine up and running and having completely separate workflows (x86 and x64) would be a compatibility nightmare 'cause I know that x64 variants may be slightly different. Besides, it's still very common to switch between Avisynth versions, that's why Groucho's Universal Avisynth Installer was very welcome when he released it. In other words, having x86 filters loading inside x64 Avisynth or Avisynth x86 being able to allocate more than 2 GB of memory it's still very useful, especially in a world where high bit depth and ultra high resolutions are common. An example? Satellite feeds in FULL HD are still interlaced, but you would almost definitely run out of RAM if you use QTGMC on a x86 system. And... sure, there are other deinterlacers like tdeint, yadif, Bob() itself and so on, but they're beyond what I'm trying to say here. What I'm trying to say is that we're not living in an utopia in which everything is x64, perfectly multi-threaded, 16bit planar and we're living in a nirvana. Just look at 16 bit stacked and interleaved: years and years after the planar regular high bit depth support inside avisynth, stacked and interleaved are still very much alive due to the lack of compatibility with some older plugins that have never been ported.

Anyway, that's my opinion on why it's relevant. :)

[goorawin]

Why is it still needed when you have 64bit binaries and threading? I'm curious...
It is still anywhere from 10% to 40% quicker than Avisynth 64bit with multi treading.
It just takes sometime to set it up for your system and your script.
And yes you can still include audio outside the pipeline script, which works well.
Not ideal but better than nothing.

[real.finder]

It is still anywhere from 10% to 40% quicker than Avisynth 64bit with multi treading.
It just takes sometime to set it up for your system and your script.
And yes you can still include audio outside the pipeline script, which works well.
Not ideal but better than nothing.

audio inside is needed for cases like this https://forum.doom9.org/showthread.php?p=1671402#post1671402 (mvtools2)

aside from that adding avs+ high bit depth support should be in C api (like x264, ffmpeg and avspmod) to avoid losing older avs support since mpp can load different avs in every block by ### dll: path

[poisondeathray]

It is still anywhere from 10% to 40% quicker than Avisynth 64bit with multi treading.
It just takes sometime to set it up for your system and your script.
And yes you can still include audio outside the pipeline script, which works well.
Not ideal but better than nothing.

Depends on the script,

I've had 80-90% slower with mp_pipeline

[tormento]

Now that pinterf has released MP_Pipeline with Avisynth+ colorspace support, would somebody explain me what are the differences between MP_Pipeline and Prefetch?

[gpower2]

Now that pinterf has released MP_Pipeline with Avisynth+ colorspace support, would somebody explain me what are the differences between MP_Pipeline and Prefetch?

I think that Prefetch uses new threads inside the same AviSynth process, while MP_Pipeline creates new processes all together.

But someone with more experience on the subject could give more details.

[MeteorRain]

Assume simplest scenario:

source().deinterlace().denoise().deband()

If you run that with MT prefetch(4), you might get:

Thread1: source().deinterlace().denoise().deband() → for frames 0, 4, 8, 12...
Thread2: source().deinterlace().denoise().deband() → for frames 1, 5, 9, 13...
Thread3: source().deinterlace().denoise().deband() → for frames 2, 6, 10, 14...
Thread4: source().deinterlace().denoise().deband() → for frames 3, 7, 11, 15...

If you run that with MP and put each filter into its own process, you get:

Process1: source() ↓
Process2: deinterlace() ↓
Process3: denoise() ↓
Process4: deband() →

Multi threading has restrictions, such as (1) the filter must be thread-safe to properly function (2) the filter must be non-temporal or have internal historical data cache to run fast (3) you have enough memory to hold the historical cache.

Multi process however, doesn't require filters to be thread-safe, and does not break request order (unless you branch, which is not generally recommended).

[MeteorRain]

For "historical data" part, fortunately I was rewriting fft3dfilter and it requires historical data so I can share some experience.

In fft3dfilter, for the parameter bt = X, a total of X frames will be examined for each output frame. If bt=5, then [prev2 prev cur next next2] will be examined to produce cur_output.

The original caching mechanism simply store the last (bt+2) frames and replace the oldest frame with the newest one.

Now, if you run it in multi thread, it's possible that prefetch would fetch frame #5 #6 #7 #8 at the same time.
The filter would then fetch and process its 4 neighbors for each request.
#5 ← [3 4 6 7]
#6 ← [4 5 7 8]
#7 ← [5 6 8 9]
#8 ← [6 7 9 10]

If you simply keep the cache within threads, obviously you'll have to waste time re-computing frames. To increase the efficiency we'd use a shared caching system, where it caches a few historical data and shares with all requests.

#existing_cache = [1 2 3 4 5 6]
#5 ← [7]
#existing_cache = [2 3 4 5 6 7]
#6 ← [8]
#existing_cache = [3 4 5 6 7 8]
#7 ← [9]
#existing_cache = [4 5 6 7 8 9]
#8 ← [10]
#existing_cache = [5 6 7 8 9 10]

That works great until prefetch issues frame requests in random order:

#existing_cache = [1 2 3 4 5 6]
#8 ← [7 9 10]
#existing_cache = [5 6 7 8 9 10]
#5 ← [3 4]
#existing_cache = [9 10 3 4 5 6 7]
#7 ← [8]
#existing_cache = [4 5 6 7 8 9]
#6 ← []
#existing_cache = [9 4 5 6 7 8]

Note the cache miss for #3 #4 and #8, unless we increase the cache size to 8.

The more thread you create, the more cache space you need to prevent a cache miss. Thus increasing the memory usage (possibly by a lot).

[tormento]

At the end of the day, a simple script with SMDegrain would have no speed benefit, as everything is inside the avsi, right?

[MeteorRain]

Theoretically you can split the image into half and put them into multiple processes. I run QTGMC in 2 processes by cutting it half and let each process do half the work.

[Stereodude]

Theoretically you can split the image into half and put them into multiple processes. I run QTGMC in 2 processes by cutting it half and let each process do half the work.
I've split UHD footage into 4 ~FHD segments run them concurrently with MCTD (in 4 different MPP segments) and put them back together into a single UHD image. I made them each slightly larger than FHD to make them overlap at the segment edges for things that move through the frame and cropped them.

like:LoadPlugin("C:\HDTV Tools\MP_Pipeline\x64\MP_Pipeline.dll")

MP_Pipeline("""

### platform: win64
source=LWLibavVideoSource("UHD_source.mkv").crop(0,280,-0,-280)

source

### export clip: source
### prefetch: 16, 8
### ###
### platform: win32
SetMemoryMax(1280)

lefttop=source.crop(0,0,-1856,-736).MCTemporalDenoise(settings="low", radius=3)

lefttop

### export clip: lefttop
### pass clip: source
### prefetch: 16, 8
### ###
### platform: win32
SetMemoryMax(1280)

leftbot=source.crop(0,736,-1856,-0).MCTemporalDenoise(settings="low", radius=3)

leftbot

### export clip: leftbot
### pass clip: source, lefttop
### prefetch: 16, 8
### ###
### platform: win32
SetMemoryMax(1280)

righttop=source.crop(1856,0,-0,-736).MCTemporalDenoise(settings="low", radius=3)

righttop

### export clip: righttop
### pass clip: source, lefttop, leftbot
### prefetch: 16, 8
### ###
### platform: win32
SetMemoryMax(1280)

source.crop(1856,736,-0,-0).MCTemporalDenoise(settings="low", radius=3)

### pass clip: lefttop, leftbot, righttop
### prefetch: 16, 8
### ###
### platform: win64

rightbot=last

left=stackvertical(lefttop.crop(0,0,1920,-64),leftbot.crop(0,64,1920,-0))
right=stackvertical(righttop.crop(64,0,-0,-64),rightbot.crop(64,64,-0,-0))

stackhorizontal(left,right)

### prefetch: 20, 10
### ###

""")

MPP also can speed up heavy serial processing. Like QTGMC followed by srestore followed by MCTD. You put each one in it's own MPP segment and it is faster than when done in a conventional script.

like:LoadPlugin("C:\HDTV Tools\MP_Pipeline\x64\MP_Pipeline.dll")

MP_Pipeline("""

### platform: win64
LoadPlugin("C:\HDTV Tools\DGDecNV\DGDecodeNV.dll")

DGSource("Video_2.dgi")


### prefetch: 16, 8
### ###
### platform: win64
SetMemoryMax(1536)

QTGMC( Preset="Slower", InputType=0, SourceMatch=3, Lossless=2, Sharpness=0.2, EZKeepGrain=0.0, ShowSettings=false )

### prefetch: 16, 8
### ###
### platform: win64

srestore()

### prefetch: 16, 8
### ###
### platform: win32
SetMemoryMax(1280)

MCTemporalDenoise(settings="low", radius=3)

### prefetch: 16, 8
### ###

""")

MPP also supports adjustable thunking (threaded chunking). You can set the chunk size (in frames) and the number of simultaneous chunks. I've rarely used this.

[MeteorRain]

I've split UHD footage into 4 ~FHD segments run them concurrently with MCTD (in 4 different MPP segments) and put them back together into a single UHD image. I made them each slightly larger than FHD to make them overlap at the segment edges for things that move through the frame and cropped them.

Yup, that's exactly how I use MCTD on my server CPU (low freq slow single core). Except that I use it on HD content, splitting images into 2 976x1080 including 16px overscan, and crop and stick them together.

[Stereodude]

Yup, that's exactly how I use MCTD on my server CPU (low freq slow single core). Except that I use it on HD content, splitting images into 2 976x1080 including 16px overscan, and crop and stick them together.
Maybe I'm too generous, but I've used 64 pixels of overlap. I've not done it much. I've subsequently gotten MCTD working in 64-bit, but I have yet not checked to see if it can handle UHD footage well or not. The 32-bit side will eventually pass a UHD image though MCTD, but it's REALLY slow.

I suspect MCTD could be heavily optimized by someone who really understood what it's doing in each of the many plugins it's using and consolidating it into a single plugin with only the essential bits and pieces, but that's so far beyond me...

It's quite slow and also stuck at 8-bit color only (and will probably always be). :( So far, I've not found a better noise reduction plugin though.

[gispos]

So far, I've not found a better noise reduction plugin though.
+1
The only filters that I still use and that are comparable are the denoise filters in Resolve 16

[tormento]

So far, I've not found a better noise reduction plugin though.
It's a long time, since I jumped to x64, that I don't give it a try. Is latest version still 1.4.20 from Lato? Are all the x64 plugins available?

[Stereodude]

It's a long time, since I jumped to x64, that I don't give it a try. Is latest version still 1.4.20 from Lato?
Yes

Are all the x64 plugins available?
More or less...

MysteryX put together this list of plugins looking at high bit depth support.
Supports high-bit-depth

- MVTools: yes
- MaskTools2: yes
- RgTools: yes
- FFT3Dfilter: yes
- AddGrainC: yes (?)
- DeBlock: yes
- DctFilter: yes

- TTempSmooth: NO
- EEDI2: NO
- SangNom: NO -- Note: VapourSynth version doesn't use SangNom
- GradFun2db: NO
Using this list I couldn't find a x64 build of Sangnom, but I found all the rest in x64. However, I haven't actually had MCTD complain about SangNom being missing. I'm not sure when it uses it, but it doesn't with the NR presets I'm using within MCTD.

[real.finder]

same story for awarpsharp and awarpsharp2 there are SangNom2 but it need to add SangNom in the same dll like awarpsharp2 https://forum.doom9.org/showthread.php?p=1655648#post1655648

[Stereodude]

same story for awarpsharp and awarpsharp2 there are SangNom2 but it need to add SangNom in the same dll like awarpsharp2 https://forum.doom9.org/showthread.php?p=1655648#post1655648
From a quick review of the script, it looks like SangNom2 could be substituted in place of SangNom in MCTD. Just change the two calls on line 1024 of the MCTD .AVSI to call SangNom2 instead.

Edit: I updated the script to use SangNom2 and posted it here (https://forum.doom9.org/showthread.php?p=1906491#post1906491).

[tormento]

MysteryX put together this list of plugins looking at high bit depth support.

To sum up x64 releases:

- MVTools: found
- MaskTools2: found
- RgTools: found
- FFT3Dfilter: found. What about replacing with Neo_FFT3D (https://forum.doom9.org/showthread.php?p=1905291#post1905291), with proper script changes?

- SangNom: Sangnom2, this (https://github.com/tp7/SangNom2/releases) the latest?
- AddGrainC: is this (https://forum.doom9.org/showthread.php?p=1655346#post1655346) the latest?
- DeBlock: this (https://github.com/mysteryx93/Avisynth-Deblock/releases)?
- DctFilter: this (https://github.com/chikuzen/DCTFilter/releases)?

- TTempSmooth: where?
- EEDI2: where?
- GradFun2db: where?

[real.finder]

even if there are MaskTools2 with x64 and HBD, scripts need update to make sure it can work with HBD, Especially the mt_lut* things

[MeteorRain]

tormento: Please allow some time before using neo_fft3d in production. Well it has been through some tests but I'm still working on it during the days.

(Good news is I managed to write a new dual synth wrapper, inspired by feisty2, and have just moved fft3d onto this new platform. Looking good so far, so I may release a new version in a few days.)

[Stereodude]

- SangNom: Sangnom2, this (https://github.com/tp7/SangNom2/releases) the latest?
- AddGrainC: is this (https://forum.doom9.org/showthread.php?p=1655346#post1655346) the latest?
- DeBlock: this (https://github.com/mysteryx93/Avisynth-Deblock/releases)?
- DctFilter: this (https://github.com/chikuzen/DCTFilter/releases)?
Sure, those will work.

- TTempSmooth: where?
- EEDI2: where?
- GradFun2db: where?
You can find them here (http://avisynth.nl/index.php/AviSynth%2B_x64_plugins).

[tormento]

tormento: Please allow some time before using neo_fft3d in production. Well it has been through some tests but I'm still working on it during the days.
I always hope someone will write a OpenCL version :o

[tormento]

You can find them
:thanks: