SVP-like frame interpolation?

[raffriff42]

I notice that in several places in the script different functions are written as (example)
EM.ChangeFPS instead of
ChangeFPS(EM,), or ChangeFPS(clip = EM,)

There is no difference in performance, as far as I know.
http://avisynth.nl/index.php/Grammar

The fourth form is an alternate syntax called "OOP notation" in AviSynth:
expression . function_name ( argument_list ) is equivalent to
function_name ( expression , argument_list )

However, occasionally there is a need for the normal, non-"OOP" form; for example, if a filter accepts multiple clips in the left-most set of arguments, implicit Last may insert itself as the first clip when not intended. When that happens, the output is obviously wrong; it's not a subtle performance hit. (except possibly in the case of ScriptClip) (http://forum.doom9.org/showthread.php?t=168698)

It's only an issue where implicit Last is involved. With explicit variables, both forms are equivalent.

...and ChangeFPS(clip = EM,...) will not work. There is no argument named 'clip' - it's a unnamed argument.

[StainlessS]

FrameRateConverter() script is for Avs+ only, but there is also a Median() function in Median.dll v2.5 plugin.


Median "c+[CHROMA]b[SYNC]i[SAMPLES]i[DEBUG]b"
MedianBlend "c+[LOW]i[HIGH]i[CHROMA]b[SYNC]i[SAMPLES]i[DEBUG]b"
TemporalMedian "c[RADIUS]i[CHROMA]b[DEBUG]b"


So maybe should use "ModPlus_Median()" instead (to explicitly name the required dll).
http://avisynth.nl/index.php/Plugins#Plugin_Autoload_and_Conflicting_Function_Names

EDIT: Leastwise I take it that Median exists in the modPlus dll.
EDIT: Yes it does:- http://www.avisynth.nl/users/vcmohan/modPlus/modPlus.html
EDIT: Would not be surprised if filter 'Median()' exists in more than those two dll's.

[Groucho2004]

The last incarnation of the script requires AviSynth+, and this pushes it over the edge for me.What's wrong with AVS+? It should work on your SSE CPU:
if(CMAKE_SIZEOF_VOID_P EQUAL 4)
# VC++ enables the SSE2 instruction set by default even on 32-bits. Step back a bit.
add_definitions("/arch:SSE")
#add_definitions("/arch:SSE2") # Better use this one, it's 2016 now
endif()

[pinterf]

What's wrong with AVS+? It should work on your SSE CPU:
if(CMAKE_SIZEOF_VOID_P EQUAL 4)
# VC++ enables the SSE2 instruction set by default even on 32-bits. Step back a bit.
add_definitions("/arch:SSE")
#add_definitions("/arch:SSE2") # Better use this one, it's 2016 now
endif()

My builds override this flag. I always published SSE2-only builds, to have optimized functions (mainly because of high bit depths that had less SSE2 optimized filters) for C-only parts. No one complained so far.

[Groucho2004]

My builds override this flag. I always published SSE2-only builds, to have optimized functions (mainly because of high bit depths that had less SSE2 optimized filters) for C-only parts.Seems reasonable. Manolito's CPU (Athlon XP, I think) does not support SSE2 and above.

[manolito]

I followed the AVS+ threads by Utim and pinterf loosely, I tried AVS+ a few times (on the ancient desktop where it didn't work and on the newer laptop where it did). My conclusion so far is very clear:

All the new stuff which AVS+ offers does not do anything for me. I do not need 64-bit, I do not need MT, I do not need the high bit depths.
But I do need an AviSynth build which is stable (looking at the AVS+ thread all I can say is that this is work in progress, looks far from stable to me).
I absolutely do need to use older 32-bit filters like the VDub filters.
And I do need to use it in a variety of older software (without switching versions - even if Groucho made this quite convenient).

So it looks like I will stick with the plain vanilla AviSynth versions for some time...


Cheers
manolito

[manolito]

Manolito's CPU (Athlon XP, I think) does not support SSE2 and above.

No, it is an Intel Celeron (Coppermine) 1.1 GHz. Hitting the 1 GHz limit was a milestone at that time. And I do not have an upgrade path for the old desktop because the big tower only accomodates BAT mainboards, not ATX. I did upgrade to the latest available BAT MoBo, but that was in the year 2000...

And I have to say that this is by far the most reliable computer I ever had. The original machine is from 1993, it started as an AMD 486DX266. I upgraded it constantly, but the power supply is still the original one, and my drive A: is still a 5.25" floppy drive. And with my highly optimized WinXP installation it only feels slow when I try to play HD videos... :D


Cheers
manolito

[MysteryX]

Which part of the script requires Avisynth+? Unless modPlus.dll requires Avisynth+ but it should work with v2.6 too

I decided last-minute to compile ConditionalMT with XP support -- glad it serves someone.


EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(92)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)


Let's split this up.


EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
EM = EM.mt_expand(mode= mt_circle(zero=true, radius=1))
EM = EM.mt_binarize(92)
EM = EM.Blur(.6)
EM = EM.BicubicResize(C.Width, C.Height)


Which of these lines fail? BicubicResize and Blur are very unlikely. mt_expand or mt_binarize are most likely the ones crashing .. but I wonder why. Neither take blksize as parameter.

[burfadel]

I found there was no speed difference with the different syntax including the different reference to EM.

Could this have something to do with it?
## "B" - Blending, "BHard" - No blending
BHard = ChangeFPS(C, NewNum, NewDen)
B = ConvertFPS(C, NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## Convert masks to desired frame rate
EM = ChangeFPS(EM, NewNum, NewDen)
Sc = ChangeFPS(SC, NewNum, NewDen)

M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true)
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true) : M

Aren't you mixing changefps and convertfps which work slightly differently?

[MysteryX]

Aren't you mixing changefps and convertfps which work slightly differently?
Blending for artifact masking, ChangeFps for scene changes and skipped bad scenes.

[manolito]

Which part of the script requires Avisynth+? Unless modPlus.dll requires Avisynth+ but it should work with v2.6 too

You are right, contrary to what vcmohan himself says about his plugin it does work with v2.6...
for 2.5 not available for 2.6 Not Available for 32 & 64 bit avisynth+ modPlus
But compared to the old RemoveGrain command there is quite a speed sacrifice. I think I will stick with RemoveGrain...

I decided last-minute to compile ConditionalMT with XP support -- glad it serves someone.

I have no use for AviSynth MT so I took out ConditionalReaderMT and replaced it with the standard ConditionalReader command.

Which of these lines fail? BicubicResize and Blur are very unlikely. mt_expand or mt_binarize are most likely the ones crashing .. but I wonder why. Neither take blksize as parameter.

It is this line:
EM = EM.BicubicResize(C.Width, C.Height)

For SD sources and BlkSize 32 the resized mask is very small, maybe there is a problem within AviSynth to resize such a small mask back to the original size?? No idea, but my workaround sure fixes it.


Cheers
manolito

[StainlessS]

W=4
H=4
Colorbars(Width=W,Height=H,Pixel_Type="YV12").KillAudio
BicubicResize(1920,1280)


Error message (perhaps referring to chroma size) [have seen this error before, was confused by it].
Resize: source image too small for this resize method. Width=2, Support=2.

EDIT: W=6 H=6 works ok.

EDIT: YV12, I think that error can only happen if source dimensions (as per Manolito) less than 192.
(192 / 32 / 4.0) = 1.5, * 4 = 6, so less than 192 is error [Round(191.9999 / 32 / 4.0) rounds down to 1.0, so 1 * 4 = 4 = error] .

EDIT: If YV24, then same resize error if W=2 H=2 [W=3 H=3 OK], so for YV24 error if less than 192 / 2 = 96.

EDIT: YV12 BilinearResize with source W=4 H=4 OK, Bilinear needs 2 (chroma) samples minimum, BicubicResize needs 3 (chroma) samples.

EDIT: Fixed some cockups.

EDIT: For YV411, (not suggesting that anyone should use that), then minimum width with that script would be 384 else error (assuming that it would work at all with YV411).

[MysteryX]

Median indeed is slower than RemoveGrain -- a LOT.

RemoveGrain: 636fps @ 24% CPU
Median: 30fps @ 82% CPU

I edited the script to use Median with presets "slower|slow", RemoveGrain with presets "normal|fast", and none with preset "faster". It is done with Eval so that you don't need a reference to a library you're not using.

Output="over" image brightening issue is fixed, as well as another similar mask issue.

I fixed Avisynth+'s conditional filters code so the standard ConditionalFilter will work with MT mode in the next release of AVS+; thus I removed ConditionalFilterMT.

This code is tweaked for YV24 sources; use that for testing.

TODO:
- Mask luma is different between YV12 and YV24; it needs to be normalized (Pinterf)
- SkipOver only works in 8-bit, as the 16-bit value of AverageLuma is 220x higher and needs to be normalized (Pinterf)
- ChangeFps on the mask somehow causes the CPU to stall at 20% (Pinterf)
- Release the next version of AVS+ with conditional MT (Pinterf)

After some testing, I'm not seeing any difference at all between prefilters. I might just go back to RemoveGrain(22); even RemoveGrain(21) is still a *LOT* faster than Median. I'm open to suggestions.


# Frame Rate Converter
# Version: 24-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2
## Prefilter: modPlus for presets "slow|slower", otherwise rgtools
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Preset - The speed/quality preset [slower|slow|normal|fast|faster]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ Output - Output mode [auto|inter|none|mask|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only
## none=ConvertFPS only; mask=mask only; over=mask as cyan overlay for debugging
##
## @ MaskStr - The artifact masking strength (1 to 100). This will apply a gamma curve on the mask
## before processing, where 100 applies no gamma curve and 5 applies 0.05 gamma (Default=5)
##
## @ MaskSAD - Artifact masking strength for bad motion, 0 to disable (Default=190)
##
## @ thSCD1 - MSCDetection scene detection treshold 1 (Default=500)
##
## @ thSCD2 - MSCDetection scene detection treshold 2 (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. Does not work in Avisynth+ MT mode. (Default=48)
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", bool "FrameDouble", string "Preset",
\ int "BlkSize", int "BlkSizeV", string "Output", float "MaskStr", int "MaskSAD", int "thSCD1", int "thSCD2", int "SkipOver")
{
Output = Default(Output, "auto")
FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
Preset = Default(Preset, "normal")
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
MaskStr = Default(MaskStr, 5)
MaskSAD = Default(MaskSAD, 190)
#MaskOcc = Default(MaskOcc, 64)
thSCD1 = Default(thSCD1, 500)
thSCD2 = Default(thSCD2, 150)
SkipOver = Default(SkipOver, 48)

Assert(Preset == "slower" || Preset == "slow" || Preset == "normal" || Preset == "fast" || Preset == "faster",
\ "FrameRateConverter: Preset must be slower, slow, normal, fast or faster")
Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
Assert(MaskStr > 0 && MaskStr <= 100, "FrameRateConverter: MaskStr must be between 1 and 100")
Assert(SkipOver >= 0 && MaskStr <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

# Performance settings: slower, slow, normal, fast, faster
Recalculate = preset == "slow" || preset == "normal"
prefilter = (preset == "slower" || preset == "slow") ? "Median(C, uu = true, vv = true)" :
\ (preset == "normal" || preset == "fast") ? "C.RemoveGrain(21)" : "C"
DCT = preset == "slower" ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## jm_fps interpolation
prefiltered = C.RemoveGrain(21) #Eval(prefilter)
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16) # all levels for MAnalyse
super = Prefilter != "C" ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2)

## "EM" - error or artifact mask
EM = MaskSAD > 0 ? C.MMask(bak, ml=MaskSAD, kind=1) : BlankClip(C) # kind=SAD
EMfwd = MaskSAD > 0 ? C.MMask(fwd, ml=MaskSAD, kind=1) : EM # kind=temporal blending
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
#EMocc = MaskOcc > 0 ? C.MMask(bak, ml=MaskOcc, kind=2).mt_inpand() : BlankClip(C) # kind=occlusion
EM = MaskSAD > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
#EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=0.5, mode="lighten", pc_range=true) : EM
SkipEM = EM

## mask strength
EM = EM.Levels(0, MaskStr / 100.0, 255, 0, 255, coring=false)

EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(92)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)

## "Sc" - scene detection
Sc = thSCD1 > 0 && thSCD2 > 0 ? C.MSCDetection(bak, thSCD1=thSCD1, thSCD2=thSCD2) : BlankClip(EM)
Sc = SkipOver > 0 ? ConditionalFilter(SkipEM, BlankClip(EM, color=color_white), Sc, "AverageLuma()", ">", string(SkipOver)) : Sc

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true)
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true) : M

R = (StrCmpi(Output, "auto")==0) [** auto: artifact masking *]
\ ? (FrameDouble ? Interleave(C, M) : M)
\ : (StrCmpi(Output, "flow")==0) [** flow: interpolation only *]
\ ? Flow
\ : (StrCmpi(Output, "none")==0) [** none: ConvertFPS only *]
\ ? B
\ : (StrCmpi(Output, "mask")==0) [** mask: mask only *]
\ ? EM
\ : (StrCmpi(Output, "over")==0) [** over: mask as cyan overlay *]
\ ? mt_merge(
\ Flow.Overlay(MergeRGB(BlankClip(EM), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(EM, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true)
\ : Assert(false, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|over)")
return R
}

[burfadel]

I didn't realise median was so slow, but it makes sense. It's like mode 4 of Removegrain but more advanced. It's probably overkill. Removegrain mode 22/21 is very fast by nature, but probably a little weak and inprecise for best results, which is why median had nicer output.

Maybe you could try something like:
prefiltered = C.minvar(lx=120, ty=100) #Eval(prefilter)

Minvar is part of the same package as Median (modplus).

Small values for lx and ty as larger values would be slower, it's for the computing of global variance. I just chose 45 and 35 as an example, these can be higher or lower (wouldn't go too low though).

EDIT: I originally had the lx and ty a little smaller, but chose higher numbers because it is very fast.

[manolito]

OK, since the ToDo list only contains stuff for AVS+ (pinterf), I decided that this script is static for my needs.

I did the same to this MysteryX script which I did a while ago to the original JohnMeyer script. I modified the interface so now it is a regular fps converter which can be used like ConvertFPS or ChangeFPS. The only exposed params are FPS, Preset and BlkSize. (Update: Two additional exposed parametes "Dct" and "Mask")

I call it "mx_fps" just like the previous "jm_fps" (which seems to stick...). In my tests it works beautifully with all kinds of sources. Only slightly slower than jm_fps, but better results with many source clips, and so far never worse than jm_fps. Looks like a winner to me...

mx_fps.avsi
# Frame Rate Converter
# Version: 2-June-2019
# By Etienne Charland aka MysteryX
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Pinterf is the one who spent the most time working on the core libraries, adding features and fixing bugs
# Slightly simplified user interface and code cleanup by manolito
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### mx_fps
### Changes the frame rate with interpolation and fine artifact removal.
##
## YV12/YUY2
## Requires: FrameRateConverter.dll, MaskTools2, MvTools2, GRunT, RemoveGrain, FFTW3.dll (in the system32 or syswow64 folder) for Dct values other than 0
##
## @ fps - The new framerate.
##
## @ Preset - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize - The block size. Supported values are 8, 16 and 32.
## Defaults for 4/3 video of height:
## 0-359: 8
## 400-1199: 16
## 1200-2160: 32
##
## @ BlkSizeV - The vertical block size. (default = BlkSize)
##
## @ Output - Output mode [auto|flow] (default = auto)
## auto=normal artifact masking; flow=interpolation only
##
## @ Prefilter - Specified a custom prefiltered clip. (default = C.RemoveGrain(22))
##
## @ Mask - Enable artifact masking (default = true)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255. Smaller = stronger.
## 0 to disable artifact masking. (default = 100)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (default = 105)
##
## @ SkipTrh - The treshold where a block is counted for the skip mask, between 0 and 255. Smaller = stronger.
## Must be smaller (stronger) than MaskTrh. (default = 55)
##
## @ BlendOver - Try fallback block size when artifacts cover more than specified treshold, or 0 to disable.
## If it fails again, it will revert to frame blending. (default = 60)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## or 0 to disable. (default = 120)
##
## @ Stripes - How to deal with stripes [none|skip|blend] (default=blend)
##
## @ Dct - Overrides DCT parameter (default: Fast=0, Normal=0, Slow=1)
## Useful values are 0, 4 and 1.
##
## @ BlendRatio - Changes the blend ratio used to fill artifact zones. 0 = frame copy and 100 = full blend.
## Other values provide a result in-between to eliminate ghost effects. Default = 40.
##
## Presets
## Fast: Basic interpolation
## Normal: Fast + prefilter + MSuper on prefilter + MRecalculate
## Slow: Normal + DCT=1
##

function mx_fps(clip C, float "fps", string "Preset", int "BlkSize", int "Dct", bool "Mask")
{
Preset = Default(Preset, "normal")
P_SLOW = 1 P_NORMAL = 2 P_FAST = 3
Pset = Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1
Assert(Pset != -1, "mx_fps: 'Preset' must be slow, normal or fast {'" + Preset + "'}")
Mask = Default(Mask, true)
Output = Mask ? "auto" : "flow"
O_AUTO = 0 O_FLOW = 1
OPut = Output == "auto" ? O_AUTO : Output == "flow" ? O_FLOW : -1
Stripes = "blend"
S_NONE = 0 S_SKIP = 1 S_BLEND = 2
Stp = Stripes == "none" ? S_NONE : Stripes == "skip" ? S_SKIP : Stripes == "blend" ? S_BLEND : -1

fps = default(fps, 25.000)
NewNum = int(fps * 1000)
NewDen = 1000
DefH = Max(C.Height, C.Width/4*3)
BlkSize = Default(BlkSize, DefH<360 ? 8 : DefH<1200 ? 16 : 32)
Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "mx_fps: BlkSize must be 8, 16 or 32")
BlkSizeV = BlkSize
MaskTrh = 100
SkipTrh = 55
MaskOcc = MaskTrh > 0 ? 105 : 0
BlendOver = 60
SkipOver = 120
CalcPrefilter = Pset != P_FAST
Prefilter = CalcPrefilter ? C.RemoveGrain(22) : C
Recalculate = PSET <= P_NORMAL
Dct = Default(Dct, PSET == P_SLOW ? 1 : 0)
BlendRatio = 40

## "B" - Blending, "BHard" - No blending
Try {
B = C.ConvertFpsLimit(NewNum, NewDen, ratio=BlendRatio)
}
Catch(Err_Msg) {
B = C.ChangeFps(floor(fps * 3/2)).ConvertFpsLimit(NewNum, NewDen, ratio=BlendRatio)
}
BHard = C.ChangeFps(NewNum, NewDen)
Blank = BlankClip(C.ConvertToY8(), color_yuv=$000000)

## Adjust parameters for different block sizes, causing stronger or weaker masks
blk = Max(BlkSize, BlkSizeV)
MaskTrh = MaskTrh + (blk<=8 ? -20 : blk<=16 ? 0 : blk<=32 ? 20 : 35)
SkipTrh = SkipTrh + (blk<=8 ? -18 : blk<=16 ? 0 : blk<=32 ? 16 : 30)
MaskTrh = Max(Min(MaskTrh, 255), 0)
SkipTrh = Max(Min(SkipTrh, 255), 0)
gam = blk<=8 ? .56 : blk<=16 ? .50 : blk<=32 ? .36 : .14

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad=16, vpad=16, sharp=1, rfilter=4) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad=16, vpad=16, levels=1, sharp=1, rfilter=4) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizev=BlkSizeV, overlap = (BlkSize/4+1)/2*2, overlapv = (BlkSizeV/4+1)/2*2, search=3, dct=Dct)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizev=BlkSizeV, overlap = (BlkSize/4+1)/2*2, search=3, dct=Dct)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizev=BlkSizeV/2, overlap = BlkSize/2>4?(BlkSize/8+1)/2*2:0, overlapv = BlkSizeV/2>4?(BlkSizeV/8+1)/2*2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizev=BlkSizeV/2, overlap = BlkSize/2>4?(BlkSize/8+1)/2*2:0, overlapv = BlkSizeV/2>4?(BlkSizeV/8+1)/2*2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num=NewNum, den=NewDen, blend=false, ml=200, mask=2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.ConvertToY8().MMask(bak, ml=255, kind=1, gamma=1/gam, ysc=255, thSCD2=255) : Blank
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.ConvertToY8().MMask(fwd, ml=255, kind=1, gamma=1/gam, thSCD2=255) : EM
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=.6, mode="lighten", pc_range=true) : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.ConvertToY8().MMask(bak, ml=MaskOcc, kind=2, gamma=1/gam, ysc=255, thSCD2=255).mt_inpand() : Blank
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM

# Last mask frame is white. Replace with previous frame.
EM = EM.DeleteFrame(EM.Framecount-1).Loop(2, EM.Framecount-1)

# Create skip mask
EMskip = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(SkipTrh)

## Create artifact correction mask
Try {
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
}
Catch(Err_Msg) {
Try {
BlkSize = BlkSize/2
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
}
Catch(Err_Msg) {
BlkSize = BlkSize/2
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
}
}

# Mask: Stripes
EMstp = C.StripeMask(blksize=BlkSize, blksizev=BlkSizeV, str=min(SkipTrh*2+20, 255), strf=min(SkipTrh+10, 255), thr=23)
\ .BicubicResize(Round(C.Width/BlkSize)*4, Round(C.Height/BlkSizeV)*4)
\ .ContinuousMask(22)
EMstp = EMstp.BicubicResize(EMstp.Width/2, EMstp.Height/2)
\ .mt_binarize(82)
\ .mt_inpand()
\ .mt_expand(mode= mt_circle(zero=true, radius= Stp==S_SKIP ? 12 : 8))
\ .FRC_GaussianBlur42(Stp==S_SKIP ? 8.0 : 2.8)
\ .BicubicResize(C.Width, C.Height)

## "M" - Apply artifact removal
EM = EM.ChangeFPS(NewNum, NewDen)
EMskip = EMskip.ChangeFPS(NewNum, NewDen)
EMstp = EMstp.ChangeFPS(NewNum, NewDen)
M = mt_merge(Flow, B, EM, luma=true, chroma="process")
M = Stp != S_NONE ? mt_merge(M, Stp == S_SKIP ? BHard : B, EMstp, luma=true, chroma="process") : M

## Apply BlendOver and SkipOver
M2 = SkipOver > 0 ? ConditionalFilterMT(EMskip, B, BHard, "AverageLuma", "<", string(SkipOver)) : B
M = BlendOver > 0 ? ConditionalFilterMT(EMskip, M, M2, "AverageLuma", "<", string(BlendOver)) : M

# Output modes
R= (Oput==O_AUTO) [** auto: artifact masking *]
\ ? M
\ : (Oput==O_FLOW) [** flow: interpolation only *]
\ ? Flow
\ : nop

return R
}


##################################
### Emulate [[VariableBlur/GaussianBlur]]
## For YUV, effective chroma blur varies depending on source
## color subsampling - YUV444 has *more* chroma blur, others less.
##
## @ var - works like GaussianBlur's varY
## @ rad - blur radius (<var> squared); overrides <var>
## @ vvar, vrad - vertical var & rad; default same as horizontal
## @ p - final [[GaussResize]] sharpness. Default 19
## (if > 25, blockiness; if < 15, loss of contrast)
##
## version 2013-10-23 raffriff42
## version 2014-05-31 discrete hor. and vert. args
## version 2017-05-21 bugfix: blockiness
##
function FRC_GaussianBlur42(clip C,
\ float "var", float "rad",
\ float "vvar", float "vrad", float "p")
{
var = Max(0.0, Float(Default(var, 1.0)))
rad = Max(1.0, Float(Default(rad, Pow(var, 0.5))))
var = Pow(Min(Max(0.0, rad), 60.0), 1.9) ## arbitrary max radius = 60

vvar = Max(0.0, Float(Default(vvar, var)))
vrad = Max(1.0, Float(Default(vrad, Pow(vvar, 0.5))))
vvar = Pow(Min(Max(0.0, vrad), 60.0), 1.9)
p = Default(p, 19)

w0 = C.Width
h0 = C.Height
w1 = Round(w0/rad)
h1 = Round(h0/vrad)

B = C.BilinearResize(
\ Min(Max(4, w1 + (w1 % 2)), w0),
\ Min(Max(4, h1 + (h1 % 2)), h0))

B = B.Blur(1.0).Blur(1.0)

return (var<0.01 && vvar<0.01) ? C
\ : (B.Width>8 && B.Height>8) ? B.GaussResize(w0, h0, p=p)
\ : B.BilinearResize(w0, h0)
}



Cheers
manolito


//EDIT//
Here is a complete AIO package with all the required plugins:
https://files.videohelp.com/u/172211/mx_fps%20AIO.zip
Please note that I used older (but stable) plugin versions which also run on old hardware.
This AIO package also contains a version of the script which does not require a SSE2 capable CPU (had to remove the stripe mask feature for this).

//EDIT 2//
Changed the user interface to expose the DCT value which will override the default. Useful if you want to use DCT=4. Also added a "Mask" parameter which makes it easy to enable or disable artifact masking.
NOTE: This script version needs the latest version 1.3 of "FrameRateConverter.dll". It comes with the AIO package.

//EDIT3//
Added the "ConvertFpsLimit" function from the latest FrameRateConverter 1.3

[StainlessS]

Manolito,

Requires: MaskTools2, MvTools2, RemoveGrain and fftw3.dll (in the System32 or SysWOW64 folder)

What is it that needs fftw3.dll, I see nothing that sticks out, and does not seem to be required in earlier mods ?

[manolito]

It is required for DCT=1 in mvtools2 (at least in Fizick's latest version 2.5.11.22). In this script Preset = "Slow" will use "DCT=1", and without fftw3.dll you will get a crash...

Cheers
manolito

[StainlessS]

Cheers Mani, had no idea that it was required, not noted in docs.
There is mention that it uses fftw3 header, and several mentions of fft regards to DCT but no specific requirement mentioned anywhere.
Deleted fftw3.dll from system32 and still worked ok, then remembered something about something also being able to use lib named as
libfftw3f-3.dll, so deleted that too, and crashes, just as you said.

God, its a bit slow with that setting, dont think I'm ever likely to use it.

[manolito]

God, its a bit slow with that setting, dont think I'm ever likely to use it.

Yeah right, but for some sources it does make a real difference...

Cheers
manolito

[MysteryX]

A caveat to be careful about: YV12 and YV24 will have different artifact mask strengths. YV12 will have much weaker mask.

Also, for mask strength, besides adjusting Gamma, the other line that will make a difference is "binarize(92)".

You guys can play around with Gamma and Binarize, in YV12 or YV24, and see what works best for you.

[StainlessS]

A caveat to be careful about: YV12 and YV24 will have different artifact mask strengths. YV12 will have much weaker mask.


If chroma masks cannot be normalized somehow, then perhaps chroma introduces too big a random variation to be useful, as cannot be separately processed/controlled/combined. (or everything converted to YV24, or everything to YV12).

[johnmeyer]

We had a discussion about DCT=1 making a difference, and I did some tests and couldn't see a difference. As I remember, the evidence posted in that thread was not convincing (that DCT=1 helped reduce ME artifacts). The only reason I bring this up is that I'd hate to see lots of effort chasing issues related to DCT=1, especially since, in addition to maybe not making much difference, it is so doggedly slow. Instead, I'm hoping most of the focus will be on enhancing the effort to produce a version that can detect and mask artifacts. I've been waiting for the dust to settle before I jump in and start testing this.

[burfadel]

Would there be any advantage running the script natively as YV24 (that is, using converttoyv24() if not already in YV24)?

Have you looked into minvar function? It should provide the benefit of the median filter without the speed penalty.

[MysteryX]

Any benefit to processing in YV24 will be lost just as soon as you re-convert to YV12, but if you do further processing in YV24, you can see better chroma precision before discarding the data.

But this script so far was tweaked for YV24. We may want to compare YV12, YV24 and Luma-only masks. Who knows -- maybe a Luma-only mask would be more accurate by discarding chroma noise -- only one way to know.

Edit: Mask cannot be applied to only Luma, because it is based on the vectors analysis which takes all planes for motion interpolation.

Right now, YV12 produces stronger mask than YV24.

YV24 / YV12
https://s1.postimg.org/gge0ottu3/Mask_YV24.png (https://postimg.org/image/gge0ottu3/) https://s1.postimg.org/qp6hunhvv/Mask_YV12.png (https://postimg.org/image/qp6hunhvv/)

Did another test using MMask's Gamma. This time YV12 produces weaker mask. YV24 masks just look better in general.
https://s13.postimg.org/v806swshv/Mask_YV24.png (https://postimg.org/image/v806swshv/) https://s13.postimg.org/moc7heurn/Mask_YV12.png (https://postimg.org/image/moc7heurn/)

[MysteryX]

I've done plenty of changes to the artifact removal by tweaking MMask's arguments.

MMask's ml is unecessary because it simply sets to 255 values above that treshold -- which is exactly what mt_binarize is for; thus I set that to 255 and removed MaskSAD parameter.

MMask has a gamma property, which removes the need for another call to Levels. The mask level adjustment function has been removed.

MMask also has ysc which says what to do on scene changes: set everything to 255. Thus, I no longer need MSCDetection.

Mask adjustment settings are now MaskGam (gamma, applied during MMask) and MaskTrh (treshold, applied during mt_binarize).

MaskGam affects SkipOver -- and I believe I've found the right gamma value. If we find no need to change it, I might remove that property.

Because SkipOver now operate on a gamma-adjusted mask, it alters SkipOver behaviors.

After plenty of testing, I've set default to MaskGam=.6, MaskTrh=90, SkipOver=15

I removed Median and "slower" preset, went back to RemoveGrain(22) until someone can prove benefits of doing more than that.

To please manolito, I changed the arguments order for simplicity. First parameters are now: NewNum, NewDen, Preset, BlkSize

Again, settings are tweaked for YV24, and YV12 produces a different mask strength so settings will need to be adjusted.

Because now I know what I'm doing (instead of randomly using old code), the mask now both covers artifacts more precisely and leaves out more areas. I found it to be better in almost every cases.


# Frame Rate Converter
# Version: 25-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast|faster]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only
## none=ConvertFPS only; mask=mask only; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1.
## Higher value means stronger mask. (Default=.6)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255. 0 to disable artifact masking.
## Higher value means stronger mask. (Default=90)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. Does not work in Avisynth+ MT mode. (Default=48)
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "SkipOver")
{
Output = Default(Output, "auto")
FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
Preset = Default(Preset, "normal")
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
MaskGam = Default(MaskGam, .6)
MaskTrh = Default(MaskTrh, 90)
SkipOver = Default(SkipOver, 15)

Assert(Preset == "slow" || Preset == "normal" || Preset == "fast" || Preset == "faster",
\ "FrameRateConverter: Preset must be slow, normal, fast or faster")
Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

# Performance settings: slow, normal, fast, faster
Recalculate = preset == "normal"
prefilter = preset != "faster"
DCT = preset == "slower" ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## jm_fps interpolation
prefiltered = Prefilter ? C.RemoveGrain(22) : C
superfilt = MSuper(prefiltered, hpad = 16, vpad = 16) # all levels for MAnalyse
super = Prefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2)

## "EM" - error or artifact mask
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, ysc=255, gamma=1.0/MaskGam) : BlankClip(C) # kind=SAD
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam) : EM # kind=temporal blending
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
SkipEM = EM

EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
\ .mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(SkipEM, BlankClip(EM, color=color_white), BlankClip(EM),
\ "AverageLuma()", ">", string(SkipOver)) : BlankClip(EM)

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true)
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true) : M

R = (StrCmpi(Output, "auto")==0) [** auto: artifact masking *]
\ ? (FrameDouble ? Interleave(C, M) : M)
\ : (StrCmpi(Output, "flow")==0) [** flow: interpolation only *]
\ ? Flow
\ : (StrCmpi(Output, "none")==0) [** none: ConvertFPS only *]
\ ? B
\ : (StrCmpi(Output, "mask")==0) [** mask: mask only *]
\ ? EM
\ : (StrCmpi(Output, "over")==0) [** over: mask as cyan overlay *]
\ ? mt_merge(
\ Flow.Overlay(MergeRGB(BlankClip(EM), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(EM, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true)
\ : Assert(false, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|over)")
return R
}

[MysteryX]

I was wondering what it would change if I scaled the mask down to 4 pixels per block instead of 1 pixel per block.

Regular
https://s8.postimg.org/ggqos82xd/Mask1.png (https://postimg.org/image/ggqos82xd/)
4 pixels per block
https://s8.postimg.org/4g58rhvip/Mask2.png (https://postimg.org/image/4g58rhvip/)

I'll definitely leave it as it is. This change is not bringing good results: too much false positives and chip-chopping between clips. I'm surprised at the difference though.

[manolito]

Thanks MX for the improvements...

I think there are a few small oversights in the script, could you please check?

Line 54: Wrong default value for SkipOver
Line 80: Recalculate really only for preset "normal", not for "slow"?
Line 82: DCT = 1 for the no longer existing preset "slower" instead of "slow"?


Cheers
manolito

[burfadel]

Did you try:
prefiltered = Prefilter ? C.minvar(lx=120, ty=100) : C

Probably the only adjustment would be xgrid (which automatically sets ygrid if not specified). Default is 5, your can go lower number for weaker, or higher for stronger effect. The more noise that there is, the greater the noise reduction, so I think it's ideal for the prefilter (especially considering its very fast). You can choose higher or lower values for lx and ty (these must be specified). I think they're reasonable values.

It should possibly help to reduce some artifacting as well, if the artifacting is caused by noise as was the case with the improvement with median. This filter is very much faster though, probably faster than Removegrain (or so it seems).

[raffriff42]

Noise is not usually the major issue (for me) -- all the things you see in JM's parade clip are. Motion-detection-busting occlusions, or "antlers" for short :)

Most of the time a simple Blur gives plenty of prefiltering. Why not a prefilter clip argument, as used by SMDegrain (http://avisynth.nl/index.php/SMDegrain)("prefilter") , Interp2 (https://forum.doom9.org/showthread.php?p=1391871#post1391871)("denoised") and probably others, to allow users with noise problems to supply their own denoising? There are many filters available to handle the many types of noise -- high frequency? low frequency? gaussian? mosquito? RF? jitter? flicker? streaking? herringbone?

[burfadel]

Blur blur's edges though, you want to maintain the picture detail whilst removing noise, obviously with minimal speed impact.

[raffriff42]

Most of the time a simple Blur gives plenty of prefiltering...
Blur blur's edges though, you want to maintain the picture detail whilst removing noise, obviously with minimal speed impact.The output of Blur (or any pre-filter) goes to motion analysis, not the main video path.

[burfadel]

I know, but the motion analysis wouldn't be accurate if the edges are not representative of the main clip. You want to motion picture detail, not noise, so a good noise filter that maintains lines would product the most accurate analysis results.

[MysteryX]

Did you try:
prefiltered = Prefilter ? C.minvar(lx=120, ty=100) : C

How about posting a benchmark comparison between RemoveGrain and minvar?

Then you can post a screenshot comparison of improvements you see.

We can make up our mind from there.

[manolito]

Did a couple of tests comparing the old mask functions to the new ones, The only real difference I found is that the new SkipOver threshold is way lower than it used to be in the old version. IMO skipping starts much too early, so far I raised the threshold from 15 to 20 which looks better to me. I'll do more testing...

Cheers
manolito

[MysteryX]

Line 54: Wrong default value for SkipOver
Line 80: Recalculate really only for preset "normal", not for "slow"?
Line 82: DCT = 1 for the no longer existing preset "slower" instead of "slow"?


Thanks, fixing those

I'm also re-adding occlusion masking. On its own, it doesn't do anything, but when added to the raw mask, it does expand the mask to cover larger. I'm tweaking it to alter the mask just the right way.

This time, I'm tweaking settings for YV12 instead of YV24, as MVTools2 will normalize to the YV12 mask levels (although Pinterf's latest version still doesn't do it right).

I'll post the new script soon.

[MysteryX]

Here's the updated code. I'm re-added occlusion masks. Now it is based on YV12 mask levels.

Overall, it seems to be giving great results.

On some clips, however, SkipOver is too high; but if I set it any higher the moose scene doesn't get detected. I found that gamma=.5 works best in most cases but it "may" help reduce the SkipOver discrepancy with a gamma of .4 or .3

There's a bit more work to do on the "SkipOver" part.

MMask's scene detection also seems to trigger too often -- I disabled it and it's working better, but that might require more investigation.


# Frame Rate Converter
# Version: 26-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast|faster]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only;
## mask=mask only; skip=mask used by SkipOver; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255.
## 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. (Default=16)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", int "SkipOver", clip "Prefilter")
{
Output = Default(Output, "auto")
FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
Preset = Default(Preset, "normal")
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
MaskGam = Default(MaskGam, .5)
MaskTrh = Default(MaskTrh, 120)
MaskOcc = Default(MaskOcc, 150)
MaskOcc = MaskTrh > 0 ? MaskOcc : 0
SkipOver = Default(SkipOver, 16)
CalcPrefilter = Defined(Prefilter) || Preset != "faster"
Prefilter = Default(Prefilter, Preset != "faster" ? C.RemoveGrain(22) : C)

Assert(Preset == "slow" || Preset == "normal" || Preset == "fast" || Preset == "faster",
\ "FrameRateConverter: Preset must be slow, normal, fast or faster")
Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
Assert(MaskOcc >= 0 && MaskOcc <= 255, "FrameRateConverter: MaskOcc must be between 0 and 255")
Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

# Performance settings: slow, normal, fast, faster
Recalculate = preset == "slow" || preset == "normal"
DCT = preset == "slow" ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255) : BlankClip(C)
EM = EM.ConvertToY8()
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(100)
EM = EM.mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
EMskipOut = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(EMskip, BlankClip(EM, color=$FFFFFF), BlankClip(EM),
\ "AverageLuma", ">", string(SkipOver)) : BlankClip(EM)
Sc = Sc.mt_binarize(128)

# Display SkipOver value on Output="over
Global GEMskip = EMskip.ChangeFPS(NewNum, NewDen)
FlowOver = Flow.ScriptClip("Subtitle(string(GEMskip.AverageLuma()))")

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

R = (StrCmpi(Output, "auto")==0) [** auto: artifact masking *]
\ ? (FrameDouble ? Interleave(C, M) : M)
\ : (StrCmpi(Output, "flow")==0) [** flow: interpolation only *]
\ ? Flow
\ : (StrCmpi(Output, "none")==0) [** none: ConvertFPS only *]
\ ? B
\ : (StrCmpi(Output, "mask")==0) [** mask: mask only *]
\ ? EM
\ : (StrCmpi(Output, "skip")==0) [** skip: skip mask *]
\ ? EMskipOut
\ : (StrCmpi(Output, "over")==0) [** over: mask as cyan overlay *]
\ ? mt_merge(
\ FlowOver.Overlay(MergeRGB(BlankClip(EM), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
\ : Assert(false, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|over)")
return R
}

[MysteryX]

Code updated. I've done a lot of work to improve SkipOver, to separate important vs unimportant noise. The logic has been changed. I had the problem that the moose creates small but serious artifacts while lighting changes create weak but large artifacts -- so I had to give more importance to small/strong artifacts and discard big/weaker ones.

Scene detection has also been disabled on MFlowFps and MMask which were creating additional skipped frames.

Added output="skip" to view the mask being used by SkipOver.

I'd like to display the "skip" value on each frame with output="over" but haven't figured out how to do so.

[MysteryX]

Code updated. Output="over" now displays the "SkipOver" value of each frame, and masks are now processed in Y8 which results in higher performance.

[MysteryX]

Added "Prefilter" argument to use custom prefilter.

SC mask was in the 16-235 range which resulted in unwanted blending of frames. This has been fixed and results in much better performance.

It plays weird in AVS+, even without MT... it plays a bunch of frames, pauses, then continues playing a bunch of frames. With MT, it stalls at low CPU usage, but first it would be good to debug the issue without MT.

[StainlessS]

MysteryX, no idea if of help but RT_Stats has eg

Last.RT_Subtitle("%d ] %f",current_frame,Last.AverageLuma()) # MUCH faster than subtitle, MONOSPACED fixed size font (10x20)
RT_WriteFile("Bug.log","%d ] %f",current_frame,Last.AverageLuma,Append=True)
RT_DebugF("%d ] %f",current_frame,Last.AverageLuma,name="MysteryX: ") # to DebugView (google)

EDIT: In blue Added

[StainlessS]

MX,


# Display SkipOver value on Output="over
Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
FlowOver = Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",args="Disp_EMskip",Local=True)


Above removes Global entirely and so function is not single instance limited,
new req Grunt, but that should be built-in to Avisynth, along with GScript (GScript is builtin in AVS+, already).

Completely untested.

EDIT: Having multi-instace ability to see side-by-side differences with alternate settings is a good thing to have.
EDIT: I think SRestore() and QTGMC() are screwed in that respect, ie single instance only (Although I think Martin53 may have fixed recent version of SRestore).

[MysteryX]

Agree, GRunT would work better -- but that would add extra dependency only for debug code. I also agree GRunT should be built into AVS+, but even so, it then wouldn't work under AVS 2.6

The code I wrote at least works for a single instance, and is only for debug mode. There's no point in activating debug mode for 2 instances simultaneously.

[StainlessS]

GRunt only used in Debug mode, no globals.

NOTE, Preset Fast is not used anywhere in script (other than Preset string check).


# Frame Rate Converter
# Version: 26-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast|faster]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only;
## mask=mask only; skip=mask used by SkipOver; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255.
## 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. (Default=16)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", int "SkipOver", clip "Prefilter")
{
Output = Default(Output, "auto")
Preset = Default(Preset, "normal")
P_SLOW=0 P_NORMAL=1 P_FAST=2 P_FASTER=3 # NOTE, Preset="Fast" is NOT used anywhere in Script
O_AUTO=0 O_FLOW=1 O_NONE=2 O_MASK=3 O_SKIP=4 O_OVER=5
Pset=Preset=="slow"?P_SLOW:Preset=="normal"?P_NORMAL:Preset=="fast"?P_FAST:Preset=="faster"?P_FASTER:-1
OPut=Output=="auto"?O_AUTO:Output=="flow"?O_FLOW:Output=="none"?O_NONE:Output=="mask"?O_MASK:Output=="skip"?O_SKIP:Output=="over"?O_OVER:-1
Assert(Pset!=-1, "FrameRateConverter: Preset must be slow, normal, fast or faster")
Assert(OPut!=-1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|over)")

FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
MaskGam = Default(MaskGam, .5)
MaskTrh = Default(MaskTrh, 120)
MaskOcc = Default(MaskOcc, 150)
MaskOcc = MaskTrh > 0 ? MaskOcc : 0
SkipOver = Default(SkipOver, 16)
CalcPrefilter = Defined(Prefilter) || Pset != P_FASTER
Prefilter = Default(Prefilter, Pset != P_FASTER ? C.RemoveGrain(22) : C)

Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
Assert(MaskOcc >= 0 && MaskOcc <= 255, "FrameRateConverter: MaskOcc must be between 0 and 255")
Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

# Performance settings: slow, normal, fast, faster
Recalculate = PSet==P_SLOW || PSET==P_NORMAL
DCT = PSet==P_SLOW ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255) : BlankClip(C)
EM = EM.ConvertToY8()
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(100)
EM = EM.mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
EMskipOut = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(EMskip, BlankClip(EM, color=$FFFFFF), BlankClip(EM),
\ "AverageLuma", ">", string(SkipOver)) : BlankClip(EM)
Sc = Sc.mt_binarize(128)

# Display SkipOver value on Output="over
Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
FlowOver = (Oput==O_OVER) ? Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",args="Disp_EMskip",Local=True) : 0

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

R= (Oput==O_AUTO) [** auto: artifact masking *]
\ ? (FrameDouble ? Interleave(C, M) : M)
\ : (Oput==O_FLOW) [** flow: interpolation only *]
\ ? Flow
\ : (Oput==O_NONE) [** none: ConvertFPS only *]
\ ? B
\ : (Oput==O_MASK) [** mask: mask only *]
\ ? EM
\ : (Oput==O_SKIP) [** skip: skip mask *]
\ ? EMskipOut
\ : (Oput==O_OVER) [** over: mask as cyan overlay *]
\ ? mt_merge(
\ FlowOver.Overlay(MergeRGB(BlankClip(EM), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
\ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")
return R
}


Oops, changed GScript to GRunt

EDIT: Untested, I dont have requirements setup.

[StainlessS]

MX, I presume that you are more familiar with the code than I, so take a peek here,
Puts up a load of stacked clips, looks to me like M is wrong, but then I'm not sure what it should look like
(I mean for the weird backwards forwards jumping thing).


# Frame Rate Converter
# Version: 26-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast|faster]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only;
## mask=mask only; skip=mask used by SkipOver; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255.
## 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. (Default=16)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", int "SkipOver", clip "Prefilter",Bool "Debug")
{
Function FRSub(clip c,string tit){ # Auto Convert clips to YV12 if not already (NOP if YV12)
c.ConvertToYV12.Subtitle(tit,align=5)
}
Output = Default(Output, "auto")
Preset = Default(Preset, "normal")
P_SLOW=0 P_NORMAL=1 P_FAST=2 P_FASTER=3 # NOTE, Preset="Fast" is NOT used anywhere in Script
O_AUTO=0 O_FLOW=1 O_NONE=2 O_MASK=3 O_SKIP=4 O_OVER=5
Pset=Preset=="slow"?P_SLOW:Preset=="normal"?P_NORMAL:Preset=="fast"?P_FAST:Preset=="faster"?P_FASTER:-1
OPut=Output=="auto"?O_AUTO:Output=="flow"?O_FLOW:Output=="none"?O_NONE:Output=="mask"?O_MASK:Output=="skip"?O_SKIP:Output=="over"?O_OVER:-1
Assert(Pset!=-1, "FrameRateConverter: Preset must be slow, normal, fast or faster")
Assert(OPut!=-1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|over)")

FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
MaskGam = Default(MaskGam, .5)
MaskTrh = Default(MaskTrh, 120)
MaskOcc = Default(MaskOcc, 150)
MaskOcc = MaskTrh > 0 ? MaskOcc : 0
SkipOver = Default(SkipOver, 16)
CalcPrefilter = Defined(Prefilter) || Pset != P_FASTER
Prefilter = Default(Prefilter, Pset != P_FASTER ? C.RemoveGrain(22) : C)
Debug = Default(Debug,False)

Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
Assert(MaskOcc >= 0 && MaskOcc <= 255, "FrameRateConverter: MaskOcc must be between 0 and 255")
Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

# Performance settings: slow, normal, fast, faster
Recalculate = PSet==P_SLOW || PSET==P_NORMAL
DCT = PSet==P_SLOW ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255) : BlankClip(C)
EM = EM.ConvertToY8()
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(100)
EM = EM.mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
EMskipOut = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(EMskip, BlankClip(EM, color=$FFFFFF), BlankClip(EM),
\ "AverageLuma", ">", string(SkipOver)) : BlankClip(EM)
Sc = Sc.mt_binarize(128)

# Display SkipOver value on Output="over
Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
FlowOver=(Oput==O_OVER||DEBUG) ? Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",args="Disp_EMskip",Local=True) : 0

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

Z_AUTO = (DEBUG||OPut==O_AUTO) ? (FrameDouble ? Interleave(C, M) : M) : 0
Z_FLOW = (DEBUG||OPut==O_FLOW) ? Flow : 0
Z_NONE = (DEBUG||OPut==O_NONE) ? B : 0
Z_MASK = (DEBUG||OPut==O_MASK) ? EM : 0
Z_SKIP = (DEBUG||OPut==O_SKIP) ? EMskipOut : 0
Z_OVER = (DEBUG||Oput==O_OVER)
\ ? mt_merge(FlowOver.Overlay(MergeRGB(BlankClip(EM), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
\ : 0

R=(Debug)
\ ? StackHorizontal(
\ StackVertical(FRSub(Z_AUTO,"Auto"),FRSub(Z_FLOW,"Flow"),FRSub(Z_NONE.ChangeFps(NewNum, NewDen),"None")),
\ StackVertical(FRSub(Z_MASK,"Mask"),FRSub(Z_SKIP,"Skip"),FRSub(Z_OVER,"Over")),
\ StackVertical(FRSub(EM,"EM"),FRSub(SC,"SC"),FRSub(M,"M"))
\ )
\ :(Oput==O_AUTO) ? Z_AUTO [** auto: artifact masking *]
\ : (Oput==O_FLOW) ? Z_FLOW [** flow: interpolation only *]
\ : (Oput==O_NONE) ? Z_NONE [** none: ConvertFPS only *]
\ : (Oput==O_MASK) ? Z_MASK [** mask: mask only *]
\ : (Oput==O_SKIP) ? Z_SKIP [** skip: skip mask *]
\ : (Oput==O_OVER) ? Z_OVER [** over: mask as cyan overlay *]
\ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")
return R
}


Avisource("F:\v\XMen2.avi")

FrameRateConverter(Debug=True)
#FrameRateConverter(Output="Auto")
#FrameRateConverter(Output="Flow")
#FrameRateConverter(Output="None")
#FrameRateConverter(Output="Mask")
#FrameRateConverter(Output="Skip")
#FrameRateConverter(Output="Over")



Could be expanded / modded to find out exactly at what point if goes wrong.

EDIT: M aint right again :(

[MysteryX]

Thanks StainlessS. Preset "faster" was for no prefilter, but considering RemoveGrain(22) is so fast and that it can be customized, I removed that preset.

I added output="raw" to see the raw artifact mask.

Made a few other minor tweaks.


# Frame Rate Converter
# Version: 27-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools (default prefilter), GRunT (output="over")
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only;
## skip=mask used by SkipOver; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255.
## 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. (Default=16)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", int "SkipOver", clip "Prefilter")
{
Preset = Default(Preset, "normal")
P_SLOW=0 P_NORMAL=1 P_FAST=2
Pset=Preset=="slow"?P_SLOW:Preset=="normal"?P_NORMAL:Preset=="fast"?P_FAST:-1
Assert(Pset!=-1, "FrameRateConverter: Preset must be slow, normal or fast")
Output = Default(Output, "auto")
O_AUTO=0 O_FLOW=1 O_NONE=2 O_MASK=3 O_SKIP=4 O_RAW=5 O_OVER=6
OPut=Output=="auto"?O_AUTO:Output=="flow"?O_FLOW:Output=="none"?O_NONE:Output=="mask"?O_MASK:Output=="skip"?O_SKIP:Output=="raw"?O_RAW:Output=="over"?O_OVER:-1
Assert(OPut!=-1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over)")

FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
MaskGam = Default(MaskGam, .5)
MaskTrh = Default(MaskTrh, 120)
MaskOcc = Default(MaskOcc, 150)
MaskOcc = MaskTrh > 0 ? MaskOcc : 0
SkipOver = Default(SkipOver, 16)
CalcPrefilter = Defined(Prefilter) || Pset != P_FAST
Prefilter = Default(Prefilter, C.RemoveGrain(22))

Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
Assert(MaskOcc >= 0 && MaskOcc <= 255, "FrameRateConverter: MaskOcc must be between 0 and 255")
Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

# Performance settings: slow, normal, fast
Recalculate = PSet==P_SLOW || PSET==P_NORMAL
DCT = PSet==P_SLOW ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255).ConvertToY8() :
\ BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = EM.ConvertToY8()
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
OutRaw = EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(100)
EM = EM.mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
OutSkip = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(EMskip, BlankClip(EM, color_yuv=$ff0000), BlankClip(EM, color_yuv=$000000),
\ "AverageLuma", ">", string(SkipOver)) : BlankClip(EM, color_yuv=$000000)

# Display SkipOver value on Output="over
Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
FlowOver = (Oput==O_OVER) ? Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",args="Disp_EMskip",Local=True) : 0

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

R= (Oput==O_AUTO) [** auto: artifact masking *]
\ ? (FrameDouble ? Interleave(C, M) : M)
\ : (Oput==O_FLOW) [** flow: interpolation only *]
\ ? Flow
\ : (Oput==O_NONE) [** none: ConvertFPS only *]
\ ? B
\ : (Oput==O_MASK) [** mask: mask only *]
\ ? EM
\ : (Oput==O_SKIP) [** skip: skip mask *]
\ ? OutSkip
\ : (Oput==O_RAW) [** raw: raw mask *]
\ ? OutRaw
\ : (Oput==O_OVER) [** over: mask as cyan overlay *]
\ ? mt_merge(
\ FlowOver.Overlay(MergeRGB(BlankClip(EM, color_yuv=$000000), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
\ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")
return R
}



Performance is much better in this version than in previous versions. It no longer freezes every few frames, and with Prefetch(8), it stalls at 50% CPU instead of 20%.

Preset="fast"

FPS (min | max | average): 52.66 | 224.8 | 92.41
Memory usage (phys | virt): 535 | 541 MiB
Thread count: 29
CPU usage (average): 56%


Preset="normal"

Frames processed: 749 (0 - 748)
FPS (min | max | average): 8.133 | 146152 | 57.98
Memory usage (phys | virt): 586 | 593 MiB
Thread count: 29
CPU usage (average): 48%


Preset="slow"

FPS (min | max | average): 3.729 | 167031 | 32.70
Memory usage (phys | virt): 461 | 470 MiB
Thread count: 29
CPU usage (average): 38%

[StainlessS]

Ill repost previous one with DEBUG option, really good idea to fully implement this, not fully working, only working for FrameDouble mode.
Need to ensure same framerate for each debug stack window, FRSub() nested function auto converts non YV12 to YV12 (so in one place only).

Take a look, I recommend you get it fully working, will make future development/debugging a whole lot easier.

Posting update now.

EDIT: The flashes are coming from same time at original framerate, ie at frame 10000, flash frame is source frame 5000.

EDIT: Perhaps easier to fail if debug and not FrameDouble.

[MysteryX]

Here are screenshot comparisons.

- FrameRateConverter SkipOver=0
- FrameRateConverter SkipOver=0, dct=1
- Interframe
- Interframe Tuning="smooth"

JM's clip, frame 106
https://s23.postimg.org/oa6zk53if/106-frc0.png (https://postimg.org/image/oa6zk53if/) https://s23.postimg.org/up60gta87/106-frc1.png (https://postimg.org/image/up60gta87/) https://s23.postimg.org/4i4tkuryf/106-inter0.png (https://postimg.org/image/4i4tkuryf/) https://s23.postimg.org/ax3uhiyo7/106-inter1.png (https://postimg.org/image/ax3uhiyo7/)

Frame 377
https://s23.postimg.org/dfpji7kef/377-frc0.png (https://postimg.org/image/dfpji7kef/) https://s23.postimg.org/azsbi3rpj/377-frc1.png (https://postimg.org/image/azsbi3rpj/) https://s23.postimg.org/q9s6pan7r/377-inter0.png (https://postimg.org/image/q9s6pan7r/) https://s23.postimg.org/6gg2wl9tz/377-inter1.png (https://postimg.org/image/6gg2wl9tz/)

Frame 582
https://s23.postimg.org/3nmvck9hj/582-frc0.png (https://postimg.org/image/3nmvck9hj/) https://s23.postimg.org/hvck17m6f/582-frc1.png (https://postimg.org/image/hvck17m6f/) https://s23.postimg.org/9ed1qahhj/582-inter0.png (https://postimg.org/image/9ed1qahhj/) https://s23.postimg.org/8q477cirr/582-inter1.png (https://postimg.org/image/8q477cirr/)

My dreaded "frame 4477"
https://s23.postimg.org/vg3c0c1zb/4477-frc0.png (https://postimg.org/image/vg3c0c1zb/) https://s23.postimg.org/dek72j7yf/4477-frc1.png (https://postimg.org/image/dek72j7yf/) https://s23.postimg.org/7rnu525fr/4477-inter0.png (https://postimg.org/image/7rnu525fr/) https://s23.postimg.org/rax0s5tl3/4477-inter1.png (https://postimg.org/image/rax0s5tl3/)
Frame 4655
https://s23.postimg.org/4aqdftvrb/4655-frc0.png (https://postimg.org/image/4aqdftvrb/) https://s23.postimg.org/4c0b98xl3/4655-frc1.png (https://postimg.org/image/4c0b98xl3/) https://s23.postimg.org/a1gjtk3rb/4655-inter0.png (https://postimg.org/image/a1gjtk3rb/) https://s23.postimg.org/fqwsdv9xj/4655-inter1.png (https://postimg.org/image/fqwsdv9xj/)

[MysteryX]

Frame 6836
https://s23.postimg.org/cy3ktu9l3/6836-frc0.png (https://postimg.org/image/cy3ktu9l3/) https://s23.postimg.org/m75r3yih3/6836-frc1.png (https://postimg.org/image/m75r3yih3/) https://s23.postimg.org/fhz7nxx53/6836-inter0.png (https://postimg.org/image/fhz7nxx53/) https://s23.postimg.org/ptbkglouf/6836-inter1.png (https://postimg.org/image/ptbkglouf/)

Observations:

- The difference between Interframe and FrameRateConverter is HUGE!

- dct=1 brings significant improvements, but causes the CPU to stall. It's not just twice slower, it makes VirtualDub freeze for 10 seconds.

- dct=1 causes a slightly stronger artifact mask, which causes more blending to occur, and thus more double shadows. Settings need to be adjusted consequently.

- SVP/Interframe always appears softer, as if it was applying some level of frame blending on every frame

- On dreaded frame 4477, Interframe shows a "hard blending", followed by that same image mixed with ugly artifacts, which brings an original frame followed by gradually increasing artifacts on a still image.

- To compare scenes with heavy artifacts, I had to set SkipOver=0 otherwise these ugly frames aren't being generated

[johnmeyer]

It looks like you are in the home stretch, so I really need to dig into this so I can provide some feedback before you're all through.

To my surprise, the DCT=1 does indeed seem to provide better results. However, I want to compare the results to a "native" MVTools2 implementation, like that script I posted that people have been using, which is nothing more than the MVTools2 sample code, but with some very "finely tuned" parameters. I think that would provide a better benchmark than Interframe which, based on reading lots of posts, is really not the thing to use as the basis for comparison when you are looking at quality, because SVPFlow/Interframe is mostly about speed, not quality.

[StainlessS]

MX, are you not getting the weird frame flashes that I'm getting (and Selur metions in VS forum) ?
@ FrameDouble, Flash frames @ output frame 10000, are coming from input frame 2500.


Avisource("F:\v\XMen2.avi")
ShowFrameNumber

FrameRateConverter(Debug=True)


I'll make same changes to my version script as yours and repost.

EDIT: Also, debug frame M, @ output frame ~10000 is alternating between input frames ~5000 and ~10000.
Something weird in there somewhere.