SMDegrain fails at Interleaved2planar

[MysteryX]

I've been using SMDegrain successfully with SD sources.

Now I'm trying it on a HD source. Not sure what's the difference... but this time it throws this error:

"There is no function named 'Interleaved2planar'"

Searching online, it appears Interleaved2planar is part of SSETools.dll which is part of RemoveGrain 0.9, and all the links are down.
This page needs to be updated for broken links (http://avisynth.nl/index.php/Removegrain)

I really can't stay what's the difference between this video and the others before, but I can't get it to work nor find a working link for SSETools.dll

Any help would be appreciated!

[Groucho2004]

I really can't stay what's the difference between this video and the others before
You're probably using RGTools which only supports planar formats.
As for the link for SSETools - It's probably part of the original plugin package for QTGMC() (first post in the QTGMC thread).

[StainlessS]

Here, removegrain_25_dll_20050501-v0.9.zip with source (~225KB):- http://www.mediafire.com/download/dxfd76bhsbwm540/removegrain_25_dll_20050501-v0.9.zip


SSE2Tools_ORDERED_Function_List

There follows a list of all function names together with CPP style argument specifiers that inform
Avisynth the argument types and optional names. Optional arguments have square brackets surrounding
their name as in [name] and are followed by a type specifier character that gives the type.
Unnamed arguments are not optional. eg "cc[arg1]b[arg2]i" would be two compulsory unnamed clip args,
followed by optional 'arg1' of type bool and optional 'arg2' of type int.

# Argument type specifier strings.
c - Video Clip
i - Integer number
f - Float number
s - String
b - boolean
. - Any type (dot)
# Array Specifiers
i* - Integer Array, zero or more
i+ - Integer Array, one or more
.* - Any type Array, zero or more
.+ - Any type Array, one or more
# Etc
###################################


AmplifyDifference "cc[factor]f[factorU]f[factorV]f[grey]b"
Difference "cc[frames]i[tolerance]i[above]b[yuv]b"
Interleaved2Planar "c[grey]b"
LimitChange "cci[limitU]i[limitV]i"
Planar2Interleaved "c[grey]b"
Quadruple "c[grey]b"
ReduceFluctuations "c[limit]i[limitU]i[limitV]i[reduceflicker]b"
Shrinkby2 "c[grey]b"
TransformColor "cffffffffff"
Truncate "cii[lowerU]i[upperU]i[lowerV]i[upperV]i"


EDIT: Kassandro had a nasty habit of including 'secret' or undocumented filters, functions, and arguments.
Dont know if above are all documented, I seem to recall Didee querying Kassandro on his VideoProcessing forum
about 'secret' arguments to some functions in SSETools, thought I saved the pages but cant seem to find them.

EDIT: Above FunctionList generated by RT_Stats included script.
EDIT: Added the descriptor to above FunctionList (for non devs).

[MysteryX]

I really don't get it. Here's a script. It works on 288p videos and requires a bunch of extra filters on 1080p videos. Also, on 1080p, contrasharp=true causes an instant crash.


P="Encoder\"
Import(P+"AviSynthMT.avsi")
LoadPlugin(P+"LSMASHSource.dll")
LoadPlugin(P+"Dither.dll")
Import(P+"Dither.avsi")
LoadPlugin(P+"MVTools2.dll")
LoadPlugin(P+"masktools2.dll")
Import(P+"smdegrain.avsi")
LoadPlugin(P+"rgtools.dll")

file="288p.mpg"
LWLibavVideoSource(file, cache=False)
AudioDub(LWLibavAudioSource(file, cache=False))
Dither_convert_8_to_16()
SMDegrain(thsad=2000, contrasharp=true, lsb_in=true, lsb_out=true)
DitherPost()
#Prefetch(8)


288p and 480p videos work. 720p and 1080p videos fail !?

If I turn off contrasharp, then it has none of that madness, which means it's taking a different processing path for 480p than for 720p content.

[Groucho2004]

Also, on 1080p, contrasharp=true causes an instant crash.
You should know this by now - "causes an instant crash" is useless.

Run the script through AVSMeter. Report the error message.

[Groucho2004]

If I turn off contrasharp, then it has none of that madness, which means it's taking a different processing path for 480p than for 720p content.
Which is well documented.

[MysteryX]

You should know this by now - "causes an instant crash" is useless.

Run the script through AVSMeter. Report the error message.
OK. Here's the error


Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
Module: C:\GitHub\NaturalGroundingPlayer\NaturalGroundingPlayer\bin\Debug\Encoder\masktools2.dll
Address: 0x01E689DB


Which is well documented.
Vague answers like this aren't very useful.

These extra plugin dependencies are definitely not well documented in the SMDegrain page.

[StainlessS]

I seem to recall that (some version of) Masktools had some kind of fix for Access Violations, not too long ago.

[Groucho2004]

OK. Here's the error


Exception 0xC0000005 [STATUS_ACCESS_VIOLATION]
Module: C:\GitHub\NaturalGroundingPlayer\NaturalGroundingPlayer\bin\Debug\Encoder\masktools2.dll
Address: 0x01E689DB

What Stainless wrote, I also seem to recall some fix related to that. Try search.

Vague answers like this aren't very useful.

These extra plugin dependencies are definitely not well documented in the SMDegrain page.
Search for the term "HD" on the avisynth.nl page, look at the functions it uses for SD/HD in smdegrain.avsi. Takes 3 minutes to see the differences and consequently the dependencies.

[MysteryX]

I seem to recall that (some version of) Masktools had some kind of fix for Access Violations, not too long ago.
I've been searching using Google and can't find it.

It's not on the Wiki page so that will need to be updated. The RemoveGrain 0.9 section also needs to be updated.

[StainlessS]

How bout this:- http://forum.doom9.org/showthread.php?p=1657879#post1657879

Have to dash now to catch bus, but D9 Advanced search "masktools violation".

[MysteryX]

I found this one and it works
http://forum.doom9.org/showthread.php?p=1726387#post1726387

[MysteryX]

The one I just posted doesn't crash but causes problems somewhere else: "masktools only support planar YUV colorspaces"

The one you posted still crashes.

It crashes on both AviSynth+ and AviSynth 2.6

[MysteryX]

btw, how the heck do you use RemoveGrainSSE2.dll ? RemoveGrainS.dll is the only one that works for me.

Documentation says it requires MSVCR80.DLL

Searching about that DLL says that installing Microsoft Visual C++ 2005 Service Pack 1 Redistributable Package MFC Security Update (https://www.microsoft.com/en-us/download/details.aspx?id=26347) is the solution.

That still doesn't work.

[StainlessS]

The 'S' version is static linked, as opposed to eg SSE2, SSE3 whatever. (I heard that avoid SSE3 is a good idea).

[MysteryX]

To avoid the crash, I can change this line

if4 = (w > 1099 || h > (lsb_in ? 1199 : 599)) ? true : false
to

if4 = false

For HD sources after KNLMeans, however, the noise is subtle and there is far less blurring than with SD sources so it may be best without.

The MT deadlock would be more difficult to pin-point; but then it works on SD sources.

[MysteryX]

OK... SMDegrain on its own works on HD sources.

Part of the problem seems to be that I was running out of memory.

The other part is that... the script runs FASTER in single-threaded mode!


file="Input.mov"
LWLibavVideoSource(file, cache=True)
AudioDub(LWLibavAudioSource(file, cache=True))
Dither_convert_8_to_16()
Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y", lsb_in=true, tv_range=false)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=true, mode=0)
Dither_resize16nr(Width, Height/2, kernel="Spline36", csp="YV24")
KNLMeansCL(D=2, A=2, h=1.1, cmode=true, device_type="GPU", device_id=0, lsb_inout=true)
Dither_resize16nr(Width, Height/2, kernel="Spline36", csp="YV12")
DitherPost(mode=6)
SMDegrain(thsad=2000, prefilter=2)
#Prefetch(4)


Prefetch(4)

FPS (min | max | average): 0.040 | 33406 | 0.416
Memory usage (phys | virt): 1294 | 1443 MiB
Thread count: 29
CPU usage (average): 31%


Single thread

FPS (min | max | average): 0.199 | 0.522 | 0.490
Memory usage (phys | virt): 1249 | 1366 MiB
Thread count: 25
CPU usage (average): 24%

There seems to be something between SMDegrain and KNLMeans where they don't like each other in MT mode. KNLMeans runs as MT_SERIALIZED


btw I'm using the 2016-01-24 mod version of SMDegrain.

[real.finder]

fixed http://pastebin.com/tTcXNNtw

it was copy past issue in YUY2 in avs 2.6 support code

[MysteryX]

fixed http://pastebin.com/tTcXNNtw

it was copy past issue in YUY2 in avs 2.6 support code
Thanks, now it's working!

Btw I just realized two things.

First, this is stupid. I'm converting from RGB back to YV12 and then to YV24!?

Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=true, mode=0)
Dither_resize16nr(Width, Height/2, kernel="Spline36", csp="YV24")


Second, KNLMeans somehow runs MUCH faster if I move it just before converting back from RGB to YUV! And the quality is considerably better, too.

Here's my script


file="Input1080p.mov"
LWLibavVideoSource(file, cache=True)
AudioDub(LWLibavAudioSource(file, cache=True))
Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y", noring=true, tv_range=false)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
KNLMeansCL(D=2, A=2, h=1.1, cmode=false, device_type="GPU", device_id=0, lsb_inout=true)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", noring=true, lsb=true, output="YV12")
SMDegrain(thsad=2000, prefilter=2, contrasharp=true, lsb_in=true, lsb_out=true)
DitherPost(mode=6)
InterFrame(Cores=1, Tuning="Smooth", NewNum=60, NewDen=1, GPU=true)
Prefetch(4)


Performance:

Frames processed: 747 (0 - 746)
FPS (min | max | average): 0.118 | 129914 | 3.987
Memory usage (phys | virt): 1301 | 1399 MiB
Thread count: 32
CPU usage (average): 40%


Then I have a question about SMDegrain's thsad parameter. Documentation says to keep it low. I find it to work well with 2000 but don't see much difference when I change it. 2000 has worked for me with both SD and HD sources so far but I don't know whether it's the optimal value.

I find SMDegrain to work VERY WELL after KNLMeans. What value do you generally use or recommend for thsad?


Edit: shifting the KNLMeans order gave better result on my HD content but leaves more noise on SD content so it's doing something else

[MysteryX]

To partly answer my question about thsad, I just did a test without KNLMeans and it blurred out the output a LOT!!

But by running KNLMeans first, there doesn't seem to be much difference between 1000 or 2000 and the output looks much better than running either one.

[real.finder]

I don't have good gpu for KNLMeans, and using server most of time

I don't use anything more than thSAD=400, in most cases something more or less than 100

[MysteryX]

I don't have good gpu for KNLMeans, and using server most of time

I don't use anything more than thSAD=400, in most cases something more or less than 100
Well... the reason my last script ran faster is because the KNLMeans line wasn't being executed! Nothing like a good night of sleep to leave dumbness behind.

It just means SMDeblock is doing a much better job than KNLMeans in that case. That being said, I can use KNLMeans by setting prefilter=4.

I did an image comparison between thSAD=200 and thSAD=2000, and honestly the difference was very minor. Which prefilter is being used makes a MUCH bigger difference.

With prefilter=4 and thSAD=200, the quality is slightly better and it respects more the shapes than prefilter=2 distorting some lines, but it is also grainier. With thSAD=2000, there is just very slightly less grain. The difference is very minor but enough to make it feel less grainy.

[MysteryX]

Still having issues with MT. KNLMeans and SMDegrain really don't like each other.

Here I'm running KNLMeans as a prefilter for SMDegrain (prefilter=4)

file="Input.MOV"
LWLibavVideoSource(file, cache=True)
AudioDub(LWLibavAudioSource(file, cache=True))
Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y", noring=true, tv_range=false)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", noring=true, lsb=true, output="YV12")
SMDegrain(thsad=200, prefilter=4, contrasharp=true, lsb_in=true, lsb_out=true)
DitherPost(mode=6)
InterFrame(Cores=1, Tuning="Smooth", NewNum=60, NewDen=1, GPU=true)
#Prefetch(4)


Single-thread performance

FPS (min | max | average): 0.193 | 10.94 | 1.935
Memory usage (phys | virt): 1214 | 1230 MiB
Thread count: 28
CPU usage (average): 16%

4 threads performance

FPS (min | max | average): 0.026 | 111355 | 0.374
Memory usage (phys | virt): 1441 | 1508 MiB
Thread count: 32
CPU usage (average): 31%

Removing InterFrame from the script gives the same thing.

Changing KNLMeans from MT mode 3 to 2 makes it slightly better (and also removing InterFrame; I'm at the memory limit)

FPS (min | max | average): 0.098 | 123076 | 0.607
Memory usage (phys | virt): 1734 | 1854 MiB
Thread count: 29
CPU usage (average): 26%

[Reel.Deel]

Here I'm running KNLMeans as a prefilter for SMDegrain (prefilter=4)

file="Input.MOV"
LWLibavVideoSource(file, cache=True)
AudioDub(LWLibavAudioSource(file, cache=True))
Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y", noring=true, tv_range=false)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", noring=true, lsb=true, output="YV12")
SMDegrain(thsad=200, prefilter=4, contrasharp=true, lsb_in=true, lsb_out=true)
DitherPost(mode=6)
InterFrame(Cores=1, Tuning="Smooth", NewNum=60, NewDen=1, GPU=true)
#Prefetch(4)



Just out of curiosity, why are you converting From YUV to RGB and back to YUV? In your script both the input matrix and output matrix are 709. The only thing that is doing is converting the levels from PC to TV. For that I recommend you use SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971).

[MysteryX]

Just out of curiosity, why are you converting From YUV to RGB and back to YUV? In your script both the input matrix and output matrix are 709. The only thing that is doing is converting the levels from PC to TV. For that I recommend you use SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971).
Does SmoothAdjust support Stack16 output? If not, then while compressing 0-255 colors into 16-235 I'm losing quite a bit of data.

I see LSB support was added in v2 but it's not clear what parameter it is. HQ perhaps?

So basically, what command from SmoothAdjust would be the equivalent of what I wrote above to convert PC->TV with LSB output?

[Reel.Deel]

Does SmoothAdjust support Stack16 output? If not, then while compressing 0-255 colors into 16-235 I'm losing quite a bit of data.

Yes it does and it also self-registers as a MT_NICE_FILTER. Read the docs.

I see LSB support was added in v2 but it's not clear what parameter it is. HQ perhaps?


All 16-bit filters have a 16 at the end.


So basically, what command from SmoothAdjust would be the equivalent of what I wrote above to convert PC->TV with LSB output?

Something like this:
...
Dither_convert_8_to_16()
SmoothLevels16(preset="pc2tv")
SMDegrain(thsad=200, prefilter=4, contrasharp=true, lsb_in=true, lsb_out=true)
....

[MysteryX]

OK it's not liking the combination of certain plugins.


Dither_convert_8_to_16()
SmoothLevels16(preset="pc2tv")
SMDegrain(thsad=200, prefilter=4, lsb_in=true, lsb_out=true)
DitherPost(mode=6)
Prefetch(4)


Performance

FPS (min | max | average): 0.242 | 116922 | 1.656
Memory usage (phys | virt): 1246 | 1299 MiB
Thread count: 30
CPU usage (average): 28%

Single-thread performance

FPS (min | max | average): 0.736 | 1.839 | 1.760
Memory usage (phys | virt): 819 | 867 MiB
Thread count: 26
CPU usage (average): 25%

This isn't crawling as much but I'm still better off disabling MT here

[Reel.Deel]

OK it's not liking the combination of certain plugins.


Dither_convert_8_to_16()
SmoothLevels16(preset="pc2tv")
SMDegrain(thsad=200, prefilter=4, lsb_in=true, lsb_out=true)
DitherPost(mode=6)
Prefetch(4)


Performance

FPS (min | max | average): 0.242 | 116922 | 1.656
Memory usage (phys | virt): 1246 | 1299 MiB
Thread count: 30
CPU usage (average): 28%

Single-thread performance

FPS (min | max | average): 0.736 | 1.839 | 1.760
Memory usage (phys | virt): 819 | 867 MiB
Thread count: 26
CPU usage (average): 25%

This isn't crawling as much but I'm still better off disabling MT here

Here's my results. The only difference is I'm using DGSource as a source filter and I'm using Dogway's official SMDegrain script. Also can you please post what MTModes you're using.

Script:
SetFilterMTMode("DGSource", 3)
SetFilterMTMode("KNLMeansCL", 3)
SetFilterMTMode("DEFAULT_MT_MODE", 2)

DGSource("00010.dgi") # 1920x1080 yv12
Trim(0, 1000)
Dither_convert_8_to_16()
SmoothLevels16(preset="pc2tv")
SMDegrain(thsad=200, prefilter=4, lsb_in=true, lsb_out=true)
DitherPost(mode=6)
#Prefetch(x)

MT disabled:

Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 1.147 | 6.359 | 2.365
Memory usage (phys | virt): 727 | 855 MiB
Thread count: 27
CPU usage (average): 8%
Time (elapsed): 00:07:03.320

Prefetch(2)
Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.965 | 36.57 | 4.896
Memory usage (phys | virt): 995 | 1125 MiB
Thread count: 29
CPU usage (average): 16%
Time (elapsed): 00:03:24.442

Prefetch(4)
Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.513 | 123042 | 6.915
Memory usage (phys | virt): 1271 | 1413 MiB
Thread count: 31
CPU usage (average): 24%
Time (elapsed): 00:02:24.748

Prefetch(6)
Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.416 | 553690 | 8.786
Memory usage (phys | virt): 1428 | 1606 MiB
Thread count: 33
CPU usage (average): 40%
Time (elapsed): 00:01:53.933


Prefetch(8)
Frames processed: 1001 (0 - 1000)
FPS (min | max | average): 0.165 | 474591 | 9.022
Memory usage (phys | virt): 1581 | 1729 MiB
Thread count: 36
CPU usage (average): 54%
Time (elapsed): 00:01:50.952
As you can see MT seems to be working pretty well. I have a 6-core processor, and with the script above the Prefetch 'sweet spot' seems to be around 6*. With Prefetch(8) it uses more resources without much benefit. The prefetch value highly depends on the complexity of the script. On complex scripts I tend to get the best performance using Prefetch(4), on easier scripts I get better performance using higher Prefetch values. It's always a good idea to test the performance with different Prefetch settings. Sometimes less is more.

*On my 4-core CPU the 'sweet spot' is around 4, but as always, YMMV.

[MysteryX]

Yeah I always set the threads to match the CPU cores. I have 8 cores. Any other number (such as 7 or 9) decreases performance due to thread synchronization penalties.

With your MT modes, it works. I got this... until it randomly crashed after 2 minutes.

FPS (cur | min | max | avg): 1.087 | 0.106 | 70862 | 2.678
Memory usage (phys | virt): 1584 | 1677 MiB
Thread count: 35
CPU usage (current | average): 54% | 58%


Here's my MT mode definition file

###########################################################################
# DO NOT OPEN THIS DOCUMENT IN GOOGLE TRANSLATE OR BAD THINGS WILL HAPPEN #
###########################################################################
# General info:
# 1. Do not include core filters in the list
# 2. Specify the lowest mode that produces correct output with any settings.
# For example if something works with MT_MULTI_INSTANCE on some parameters and MT_SERIALIZED with other, MT_SERIALIZED should be used
# You can of course specify this in comments so people who do want to get MAXIMUM PERFORMANCE can modify the script themselves.
# 3. Do not specify MT modes for runtime filters. Behaviour of runtime filtering with mt is not very well known yet.
# 4. In case it isn't obvious, NICE_FILTER - mt mode 1, MULTI_INSTANCE - mt mode 2, SERIALIZED - mt mode 3. Variables are defined by avs+ core.
##################################################################################
#Another Lists
#https://gist.github.com/tp7/8899021
##################################################################################
# Official avs+ filters. Newer versions will register themself so you should never force any of these modes
############################################
#rgtools
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)

SetFilterMTMode("removegrain", MT_NICE_FILTER)
SetFilterMTMode("repair", MT_NICE_FILTER)
SetFilterMTMode("verticalcleaner", MT_NICE_FILTER)
SetFilterMTMode("clense", MT_NICE_FILTER)
#medianblur
SetFilterMTMode("medianblur", MT_MULTI_INSTANCE)
SetFilterMTMode("medianblurtemporal", MT_MULTI_INSTANCE)
#masktools
SetFilterMTMode("mt_invert", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_binarize", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_inflate", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_deflate", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_inpand", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_expand", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_lut", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_lutxy", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_lutxyz", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_luts", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_lutf", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_lutsx", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_lutspa", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_merge", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_logic", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_convolution", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_mappedblur", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_makediff", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_average", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_adddiff", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_clamp", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_motion", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_edge", MT_MULTI_INSTANCE)
SetFilterMTMode("mt_hysteresis", MT_MULTI_INSTANCE)
SetFilterMTMode("sangnom2", MT_SERIALIZED) #can run with 2 but you should specify threads=1 then,
# otherwise too many threads will get created without any benefit
SetFilterMTMode("average", MT_NICE_FILTER)
SetFilterMTMode("TMaskCleaner", MT_MULTI_INSTANCE)
SetFilterMTMode("checkmate", MT_NICE_FILTER)
SetFilterMTMode("Deblock", MT_NICE_FILTER)
SetFilterMTMode("msharpen", MT_MULTI_INSTANCE)
SetFilterMTMode("TColorMask", MT_NICE_FILTER)
SetFilterMTMode("Vinverse", MT_MULTI_INSTANCE)
SetFilterMTMode("Vinverse2", MT_MULTI_INSTANCE)

#tp7's update: removed sangnom2 and vinverse lines since they cannot run with mode 1 correctly
# (updated versions can but those will register themselves anyway), also removed core filters

#########################################
# Core filters (because putting it here is easier than modifying the source and firing a PR)
SetFilterMTMode("LanczosResize", MT_SERIALIZED) #Tested On 2016/03/06
##################################


#########################################
# External filters (guessed, might not be very stable)
##################################
SetFilterMTMode("dfttest", MT_SERIALIZED) #same story as with sangnom2
SetFilterMTMode("AddGrainC", MT_MULTI_INSTANCE)
SetFilterMTMode("FFT3dGPU", MT_SERIALIZED)

#############################################
# External Filters (tested by Reel.Deal)
#############################################
# Testing info:
# 1. All processing plugins have been tested with ColorBars(pixel_type="YV12") as the source. Plugins that do not support YV12 have been tested accordingly.
# 2. AVSMeter is used to check optimal performance and VDub/MPC-HC is used to check correct output.
#############################################
# Source plugins
SetFilterMTMode("DGSource", MT_SERIALIZED) #DGDecNV 2046 2014/01/01 - GPU decoding
SetFilterMTMode("DSS2", MT_SERIALIZED) #DSS2 2.0.0.13 2014/11/10
SetFilterMTMode("FFAudioSource", MT_SERIALIZED) #FFMS2 V2.22 2015/10/12
SetFilterMTMode("FFImageSource", MT_SERIALIZED) #FFMS2 V2.22 2015/10/12
SetFilterMTMode("FFFFmpegSource2", MT_SERIALIZED) #FFMS2 V2.22 2015/10/12
SetFilterMTMode("FFMS2", MT_SERIALIZED) #FFMS2 V2.22 2015/10/12
SetFilterMTMode("FFVideoSource", MT_SERIALIZED) #FFMS2 V2.22 2015/10/12
SetFilterMTMode("FRIMSource", MT_SERIALIZED) #FRIMSource V1.26 2016/01/16
SetFilterMTMode("JpegSource", MT_SERIALIZED) #JpegSource ---> 2014/04/19
SetFilterMTMode("LSMASHAudioSource", MT_SERIALIZED) #LSMASHSource r877 2016/03/09
SetFilterMTMode("LSMASHVideoSource", MT_SERIALIZED) #LSMASHSource r877 2016/03/09
SetFilterMTMode("LWLibavAudioSource", MT_SERIALIZED) #LSMASHSource r877 2016/03/09
SetFilterMTMode("LWLibavVideoSource", MT_SERIALIZED) #LSMASHSource r877 2016/03/09
SetFilterMTMode("RawSource", MT_SERIALIZED) #RawSource26 2046 2013/08/26

# Processing plugins
SetFilterMTMode("aBlur", MT_NICE_FILTER) #aWarpSharp2 ---> 2012/03/28 - Odd, with certain Prefetch values mode 1 is
# slightly better with other values mode 2 is slightly better???
SetFilterMTMode("aSobel", MT_NICE_FILTER) #aWarpSharp2 ---> 2012/03/28 - Same as aBlur
SetFilterMTMode("aWarp", MT_NICE_FILTER) #aWarpSharp2 ---> 2012/03/28 - Same as aBlur
SetFilterMTMode("aWarp4", MT_NICE_FILTER) #aWarpSharp2 ---> 2012/03/28 - Mode 1/2/3 perform very similar and have
# no real inprovement over single threaded.
SetFilterMTMode("aWarpSharp", MT_NICE_FILTER) #aWarpSharp2 ---> 2012/03/28 - Same as aBlur
SetFilterMTMode("aWarpSharp2", MT_NICE_FILTER) #aWarpSharp2 ---> 2012/03/28 - Same as aBlur
SetFilterMTMode("DeCross", MT_SERIALIZED) #DeCross v0.0.02 2004/02/01 - Mode 1/2 hang up and crash. Mode 3 with
# Prefetch >1 is much less effecient than single threaded.
SetFilterMTMode("frfun7", MT_MULTI_INSTANCE) #frfun7 rev6 2006/05/10 - With mode 1 the output is corrupted.
# This plugin works well by itself but it's really slow with other filters. (2013 version seems to work as mode 1)
SetFilterMTMode("gradfun2db", MT_MULTI_INSTANCE) #gradfun2db v1.0 2006/03/15 - With mode 1 the output is corrupted,
# random artifacts when playing and/or seeking.
SetFilterMTMode("MosquitoNR", MT_SERIALIZED) #MosquitoNR v0.10 2013/03/14 - It also works with mode 2,
# but you have to set MosquitoNR(threads=1) so it does not create any unnecessary threads.
SetFilterMTMode("StaticNoiseC", MT_NICE_FILTER) #StaticNoiseC ---> 2011/01/08 - Mode 1/2 performance is almost identical.
SetFilterMTMode("SVSuper", MT_MULTI_INSTANCE) #SVPflow 4.0.0.128 2016/02/12 - Documentation states "all filters now
# compatible with MT mode 1"
SetFilterMTMode("SVAnalyse", MT_MULTI_INSTANCE) #SVPflow 4.0.0.128 2016/02/12 - Documentation states "all filters now
# compatible with MT mode 1"
SetFilterMTMode("SVSmoothFps", MT_MULTI_INSTANCE) #SVPflow 4.0.0.128 2016/02/12 - Documentation states "all filters now
# compatible with MT mode 1"
SetFilterMTMode("tcanny", MT_MULTI_INSTANCE) #tcanny v1.0 2009/06/03 - With mode 1 the output is corrupted,
# random artifacts when playing and/or seeking.
SetFilterMTMode("tdtrans", MT_MULTI_INSTANCE) #tcanny v1.0 2009/06/03 - Same as tcanny
SetFilterMTMode("TCannyMod_TCannyMod", MT_NICE_FILTER) #TCannyMod v1.1.1 2016/04/10
SetFilterMTMode("TCannyMod_GBlur", MT_NICE_FILTER) #TCannyMod v1.1.1 2016/04/10
SetFilterMTMode("TCannyMod_EMask", MT_NICE_FILTER) #TCannyMod v1.1.1 2016/04/10
SetFilterMTMode("TEdgeMask", MT_MULTI_INSTANCE) #TEdgeMask v0.9 2005/01/20 - Same as tcanny
SetFilterMTMode("TEMmod", MT_MULTI_INSTANCE) #TEMmod v0.2.0 2013/08/12 - Same as tcanny
SetFilterMTMode("TMM", MT_MULTI_INSTANCE) #TMM v1.0 2007/04/06 - Mode 1 randomly crashes and mode 2/3 are not very efficient.
SetFilterMTMode("TNLMeans", MT_MULTI_INSTANCE) #TNLMeans v1.0.3 2007/08/28 - Mode 1 is extremely slow
SetFilterMTMode("UnsharpHQ", MT_MULTI_INSTANCE) #UnsharpHQ v0.5 2014/01/19 - Mode 1 creates a few extra threads without any benefit.
SetFilterMTMode("VideoScope", MT_NICE_FILTER) #VideoScope v1.2 2004/05/01 - Mode 1/2 performance is almost identical.
SetFilterMTMode("flash3kyuu_deband", MT_MULTI_INSTANCE) #flash3kyuu_deband v1.5.1 2012/04/07 - Also works with mode 1 if set mt=False
# flash3kyuu_deband 2.0pre2 2012/12/03 - Crashes with mode 1 even with mt=False
#########################################
# Filters from Dither 1.25.0. Tested by Firesledge (not extensively though)
SetFilterMTMode("DitherPost", MT_SERIALIZED)
SetFilterMTMode("SmoothGrad", MT_NICE_FILTER)
SetFilterMTMode("Dither_box_filter16", MT_NICE_FILTER)
SetFilterMTMode("Dither_bilateral16", MT_NICE_FILTER)
SetFilterMTMode("Dither_limit_dif16", MT_NICE_FILTER)
SetFilterMTMode("Dither_resize16", MT_NICE_FILTER)
SetFilterMTMode("Dither_out", MT_NICE_FILTER)
SetFilterMTMode("Dither_removegrain16",MT_NICE_FILTER)
SetFilterMTMode("Dither_repair16", MT_NICE_FILTER)
SetFilterMTMode("Dither_median16", MT_NICE_FILTER)
SetFilterMTMode("Dither_add16", MT_NICE_FILTER)
SetFilterMTMode("Dither_sub16", MT_NICE_FILTER)
SetFilterMTMode("Dither_max_dif16", MT_NICE_FILTER)
SetFilterMTMode("Dither_merge16", MT_NICE_FILTER)
#########################################
SetFilterMTMode("DGDecode_mpeg2source", MT_SERIALIZED) #seems to work fine as 1 although MT_SERIALIZED is recommended for almost all source filters
SetFilterMTMode("ColorMatrix", MT_NICE_FILTER) #read note2
SetFilterMTMode("NNEDI3", MT_MULTI_INSTANCE)
SetFilterMTMode("eDeen", MT_SERIALIZED) #1 and 2 gave errors
SetFilterMTMode("Deen", MT_SERIALIZED) #2 randomly gave errors, or corrupted output
SetFilterMTMode("VagueDenoiser", MT_MULTI_INSTANCE)
SetFilterMTMode("TFM", MT_MULTI_INSTANCE) #2 is faster. 1 crashes randomly.
SetFilterMTMode("TDecimate", MT_SERIALIZED) #1 gave error, 2 was slower than 3
SetFilterMTMode("FTurnLeft", MT_NICE_FILTER) #FTurn is deprecated since it's already included in the avs+ core
SetFilterMTMode("FTurnRight", MT_NICE_FILTER) #FTurn is deprecated since it's already included in the avs+ core
SetFilterMTMode("nnedi3ocl", MT_MULTI_INSTANCE)
SetFilterMTMode("Yadif", MT_MULTI_INSTANCE)
#note1: Can't use mode 1 or 2 on any filters if encoding 1080i files(480i works perfectly fine).
#FFVideoSource runs a few seconds, however, it always crashes.
#As far as I know, the only workaround seems to be running a deinterlacer right after the video source and running it as "MT_SERIALIZED",
#than running any desired filters after. Filters after the deinterlacer, seem to be able use any mode.
#note2: tried multiple files, seems to corrupt video even when it is the only filter in a script.
#tried mode 1, 2, and 3, none worked. however it works fine if MT isn't enabled.
#should ColorMatrix still be in the list? as a notice? or should it be removed?
#########################################
SetFilterMTMode("FRFun3d", MT_NICE_FILTER)
SetFilterMTMode("ContinuityFixer", MT_NICE_FILTER)
SetFilterMTMode("ReferenceFixer", MT_NICE_FILTER)
SetFilterMTMode("SmoothLevels", MT_NICE_FILTER)
SetFilterMTMode("SmoothTweak", MT_NICE_FILTER)
SetFilterMTMode("SmoothCurve", MT_NICE_FILTER)
SetFilterMTMode("SmoothCustom", MT_NICE_FILTER)
SetFilterMTMode("SmoothTools", MT_NICE_FILTER)
SetFilterMTMode("SmoothLevels16", MT_NICE_FILTER)
SetFilterMTMode("SmoothTweak16", MT_NICE_FILTER)
SetFilterMTMode("SmoothCurve16", MT_NICE_FILTER)
SetFilterMTMode("SmoothCustom16", MT_NICE_FILTER)
SetFilterMTMode("SmoothTools16", MT_NICE_FILTER)
SetFilterMTMode("TDeint", MT_MULTI_INSTANCE) # Mode 1 creates artifacts
SetFilterMTMode("yadifmod", MT_NICE_FILTER)
SetFilterMTMode("yadifmod2", MT_NICE_FILTER)

#########################################
# Filters from MVTools v2.6.0.5 (from http://forum.doom9.org/showthread.php?p=1386559#post1386559 ) Tested On 2016/03/06
SetFilterMTMode("MSuper", MT_MULTI_INSTANCE)
SetFilterMTMode("MAnalyse", MT_MULTI_INSTANCE)
SetFilterMTMode("MFlowFps", MT_MULTI_INSTANCE)
SetFilterMTMode("MRecalculate", MT_MULTI_INSTANCE)
#########################################
#Filters from EEDI2 v0.9.2 (from http://forum.doom9.org/showthread.php?t=173238 ) Tested On 2016/03/06
SetFilterMTMode("EEDI2", MT_MULTI_INSTANCE)

SetFilterMTMode("KNLMeansCL", MT_SERIALIZED)

#keep it the last line please
if (FunctionExists("avstp_set_threads")) {
# this isn't actually optimal, because it will also disable avstp threads if running
# on a single-threaded filter chain
avstp_set_threads(0, 1) #disable threading in dither
}

[Reel.Deel]

With your MT modes, it works. I got this... until it randomly crashed after 2 minutes.


Don't know what to tell you. Just finished a 5 minute test with Prefetch(6) and no problems. Same script as before:

Frames processed: 2881 (0 - 2880)
FPS (min | max | average): 0.536 | 237295 | 9.318
Memory usage (phys | virt): 1432 | 1604 MiB
Thread count: 33
CPU usage (average): 41%
Time (elapsed): 00:05:09.195

Yeah I always set the threads to match the CPU cores. I have 8 cores. Any other number (such as 7 or 9) decreases performance due to thread synchronization penalties.

Does your CPU actually have 8 physical cores or do you mean 4 cores, 8 threads? Anyways, my point is, blindly setting Prefetch can sometimes be detrimental. A bit of testing on each script can go a long way. :)

[MysteryX]

Does your CPU actually have 8 physical cores or do you mean 4 cores, 8 threads? Anyways, my point is, blindly setting Prefetch can sometimes be detrimental. A bit of testing on each script can go a long way. :)
4 dual-cores, 8 virtual cores.

I always test. It generally works best with a thread per core. Unless I'm running out of memory and need to decrease. In some cases 4 threads might give nearly the same performance. But I rarely get any benefit from 6 or 10 threads. It works best with multiples: 2, 4 or 8.

If you have 6 cores, then I suppose you get the best performance from 3 or 6 threads.

[MysteryX]

I did more testing to identify the source of slowdown. It only happens when lsb=true.

It turns out the line causing the slowdown from 2.2 to 1.6fps is this

SetFilterMTMode("DitherPost", MT_SERIALIZED)


I had set that to MT_SERIALIZED because otherwise DitherTools crashed occasionally.

Here it's working best with DitherPost and Dither_resize16 set to MT_MULTI_INSTANCE while the other dither functions are fine with MT_NICE_FILTER. I'll see if I get more dither crashes and try to work around it without setting DitherPost to MT_SERIALIZED

Unfortunately, now I'm getting dither.dll crashes again. Even with all dither functions set to MULTI_INSTANCE.

[MysteryX]

The 'S' version is static linked, as opposed to eg SSE2, SSE3 whatever. (I heard that avoid SSE3 is a good idea).
Does the S version include SSE2 optimizations? I have VC++ 2005 Redistributable installed (both x86 and x64) but still can't get SSE2 version to run.

btw where is the source code for that one? edit: found it here (http://www.removegrain.de.tf/) but the download link is broken. It returns a 0 byte file.

Edit: it appears RemoveGrain isn't being used in that patched smdegrain script anyway!

[StainlessS]

Sorry, no idea if the static linked version is SSE1, SSE2, or whatever, I only ever use the SSE2 dll version.

[real.finder]

Does SmoothAdjust support Stack16 output? If not, then while compressing 0-255 colors into 16-235 I'm losing quite a bit of data.

I see LSB support was added in v2 but it's not clear what parameter it is. HQ perhaps?

So basically, what command from SmoothAdjust would be the equivalent of what I wrote above to convert PC->TV with LSB output?

with this http://pastebin.com/vz6Ewuvv by TV_range=false you don't have to convert PC->TV

[MysteryX]

with this http://pastebin.com/vz6Ewuvv by TV_range=false you don't have to convert PC->TV
Saw that and gave it a try but it didn't work. Perhaps it took tv_range=false for both input and output, and thus didn't change anything to the range?

[real.finder]

Saw that and gave it a try but it didn't work. Perhaps it took tv_range=false for both input and output, and thus didn't change anything to the range?

yes, it for both, it just don't do TV->PC by Dither_Luma_Rebuild(S0=1.0,c=0.0625) in super_search, so you can use full (aka PC) range if you set tv_range to false

[MysteryX]

yes, it for both, it just don't do TV->PC by Dither_Luma_Rebuild(S0=1.0,c=0.0625) in super_search, so you can use full (aka PC) range if you set tv_range to false
I see; and then I can convert to TV range afterwards. It just avoids intermediary conversion.

[real.finder]

I see; and then I can convert to TV range afterwards. It just avoids intermediary conversion.

yes, or even encode in pc range

Dither_Luma_Rebuild do TV->PC just for super_search, and it do this for the dark parts of the real clip to be less filtered (http://forum.doom9.org/showthread.php?p=1548814#post1548814)

[MysteryX]

yes, or even encode in pc range

Dither_Luma_Rebuild do TV->PC just for super_search, and it do this for the dark parts of the real clip to be less filtered (http://forum.doom9.org/showthread.php?p=1548814#post1548814)
I don't know about encoding in PC range... not all players are handling that properly. I'm also not finding the ffmpeg or x264 setting to encode in full range; or does it detect it automatically?

[real.finder]

I don't know about encoding in PC range... not all players are handling that properly. I'm also not finding the ffmpeg or x264 setting to encode in full range; or does it detect it automatically?

x264 detect it automatically for input or you can use --input-range "pc" and for encode (also auto by default) --range "pc" and you have also --fullrange