SVP-like frame interpolation?

[StainlessS]

Modified similar to your last post, Still getting weird flashes, and M jumps from eg frame 10000, to frame 5000 when flashing.


# Frame Rate Converter
# Version: 27-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools (default prefilter), GRunT (output="over")
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only;
## skip=mask used by SkipOver; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255.
## 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. (Default=16)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", int "SkipOver", clip "Prefilter",Bool "Debug")
{
Function FRSub(clip c,string tit){ # Auto Convert clips to YV12 if not already (NOP if YV12)
c.ConvertToYV12.Subtitle(tit,align=5)
}
Preset = Default(Preset, "normal")
P_SLOW=0 P_NORMAL=1 P_FAST=2
Pset=Preset=="slow"?P_SLOW:Preset=="normal"?P_NORMAL:Preset=="fast"?P_FAST:-1
Assert(Pset!=-1, "FrameRateConverter: Preset must be slow, normal or fast")
Output = Default(Output, "auto")
O_AUTO=0 O_FLOW=1 O_NONE=2 O_MASK=3 O_SKIP=4 O_RAW=5 O_OVER=6
OPut=Output=="auto"?O_AUTO:Output=="flow"?O_FLOW:Output=="none"?O_NONE:Output=="mask"?O_MASK:Output=="skip"?O_SKIP:Output=="raw"?O_RAW:Output=="over"?O_OVER:-1
Assert(OPut!=-1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over)")

FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
MaskGam = Default(MaskGam, .5)
MaskTrh = Default(MaskTrh, 120)
MaskOcc = Default(MaskOcc, 150)
MaskOcc = MaskTrh > 0 ? MaskOcc : 0
SkipOver = Default(SkipOver, 16)
CalcPrefilter = Defined(Prefilter) || Pset != P_FAST
Prefilter = Default(Prefilter, C.RemoveGrain(22))
Debug = Default(Debug,False)

Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
Assert(MaskOcc >= 0 && MaskOcc <= 255, "FrameRateConverter: MaskOcc must be between 0 and 255")
Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

# Performance settings: slow, normal, fast
Recalculate = PSet==P_SLOW || PSET==P_NORMAL
DCT = PSet==P_SLOW ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)
B = FrameDouble ? SelectOdd(B) : B

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255).ConvertToY8() :
\ BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = EM.ConvertToY8()
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
OutRaw = EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(100)
EM = EM.mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
OutSkip = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(EMskip, BlankClip(EM, color_yuv=$ff0000), BlankClip(EM, color_yuv=$000000),
\ "AverageLuma", ">", string(SkipOver)) : BlankClip(EM, color_yuv=$000000)

# Display SkipOver value on Output="over
Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
FlowOver = (Oput==O_OVER||DEBUG) ? Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",args="Disp_EMskip",Local=True) : 0

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

Z_AUTO = (DEBUG||OPut==O_AUTO) ? (FrameDouble ? Interleave(C, M) : M) : 0
Z_FLOW = (DEBUG||OPut==O_FLOW) ? Flow : 0
Z_NONE = (DEBUG||OPut==O_NONE) ? B : 0
Z_MASK = (DEBUG||OPut==O_MASK) ? EM : 0
Z_SKIP = (DEBUG||OPut==O_SKIP) ? OutSkip : 0
Z_RAW = (DEBUG||OPut==O_RAW) ? OutRaw : 0
Z_OVER = (DEBUG||Oput==O_OVER)
\ ? mt_merge(
\ FlowOver.Overlay(MergeRGB(BlankClip(EM, color_yuv=$000000), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
\ : 0

R=(Debug)
\ ? StackHorizontal(
\ StackVertical(FRSub(Z_AUTO,"Auto"),FRSub(Z_FLOW,"Flow"),FRSub(Z_NONE.ChangeFps(NewNum, NewDen),"None")),
\ StackVertical(FRSub(Z_MASK,"Mask"),FRSub(Z_SKIP,"Skip"),FRSub(Z_OVER,"Over")),
\ StackVertical(FRSub(EM,"EM"),FRSub(SC,"SC"),FRSub(M,"M"))
\ )
\ :(Oput==O_AUTO) ? Z_AUTO [** auto: artifact masking *]
\ : (Oput==O_FLOW) ? Z_FLOW [** flow: interpolation only *]
\ : (Oput==O_NONE) ? Z_NONE [** none: ConvertFPS only *]
\ : (Oput==O_MASK) ? Z_MASK [** mask: mask only *]
\ : (Oput==O_SKIP) ? Z_SKIP [** skip: skip mask *]
\ : (Oput==O_RAW) ? Z_RAW [** raw: raw mask *]
\ : (Oput==O_OVER) ? Z_OVER [** over: mask as cyan overlay *]
\ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")
return R
}

Avisource("F:\v\XMen2.avi")
ShowFrameNumber
#return last

FrameRateConverter(Debug=True)
#FrameRateConverter(Output="Auto")
#FrameRateConverter(Output="Flow")
#FrameRateConverter(Output="None")
#FrameRateConverter(Output="Mask")
#FrameRateConverter(Output="Skip")
#FrameRateConverter(Output="Raw")
#FrameRateConverter(Output="Over")


EDIT: is this an error

B = FrameDouble ? SelectOdd(B) : B # B is original rate when FrameDouble
...
## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen) # EM is double original rate when FrameDouble
...
## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
<-original rate when framedouble -->, B Orig, EM is Doublerate

Above in blue, different rates ???

[MysteryX]

Just tried FrameDouble again, and everything is fine. No, I'm not getting any weird flash. What ColorSpace are you using? (YV12 here) What Avisynth version are you using?

Btw I'm trying to split long lines so that it displays nicely in the browser.

Your script is also working fine here, no frame jumps. However, "SC" isn't displaying and "Mask" and "EM" are displaying the same. I believe Mask should be the raw mask. I think it's showing too much to be useful. 4 views would be more than enough.

Or perhaps, on top of "over", to have "debug" which shows 4 views and "debug2" which shows 9 views ... but isn't it starting to be overkill when the script is already ready and working?

One advantage of your "debug" mode is to help others understand what this script is doing under the hood. It took me a while to understand the basic YFRC script.

If you're going to display 9 clips, I'd recommend this order, from top-left to bottom-right: Output, MFlow, Blending, Raw, Mask, Skip --- these 6 are enough. The 3 others are redundant and unnecessary.

If we display 4 clips, Output, Over, Raw, Skip -- or something like that.

But first, what would be the intent? With the script working, it's only about tweaking settings by comparing details. If too many clips are showing up the details of each clip are too small to see and compare.

Edit: OK I see some frames not matching the others. That happens because some clips are 15fps while others are 60fps. If you display a 15fps clip next to a 60fps clip, you're going to see such mismatch.

[StainlessS]

See edit

EDIT: I'm using AVS standard, YV12. With test script given including ShowFrameNumber.

M clip bottom RHS debug is jumping 5000 -> 10000 -> 5000 etc. (original frame numbers)

Only just updated to latest mvtools and masktools, was getting exact same type thing with older versions
(several versions ago)

EDIT: OverKill, not when your trying to find a bug (and its called debug mode).

EDIT: 20 frames flashing, ~ 200KB:- http://www.mediafire.com/file/cj3kwpaq748bwzm/MX.mp4
With Output=Auto.
Selur has been posting about the flashes in vapoursynth forum.

[MysteryX]

I see no problem with AVS 2.6 here... nor with FrameDouble.

I did see a bug though: with FrameDouble, scene changes aren't being applied.

[StainlessS]

FlowX=FrameDouble ? SelectOdd(Flow) : Flow
RT_DebugF("Flow_Rate=%f B_Rate=%f EM_Rate=%f",FlowX.FrameRate,B.FrameRate,EM.FrameRate)
RT_DebugF("BHard_Rate=%f SC_Rate=%f",BHard.FrameRate,SC.FrameRate)

## the FrameRateConverter magic happens
M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
RT_DebugF("M_1 Rate=%f",M.FrameRate)
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M
RT_DebugF("M_2 Rate=%f",M.FrameRate)


Debug output

[3888] RT_DebugF: Flow_Rate=25.000000 B_Rate=25.000000 EM_Rate=50.000000
[3888] RT_DebugF: BHard_Rate=50.000000 SC_Rate=50.000000
[3888] RT_DebugF: M_1 Rate=25.000000
[3888] RT_DebugF: M_2 Rate=25.000000

M_2 Rate=25.0 but that is probably just because the first clip framerate FlowX was 25.0,
The Mt_merge stuff obviously aint working proper there.

I think that mt_merge pair is where the flashes originate, result of first one mangles things up, but result is 25.0 fps
and probably no longer than first clip (FlowX).

EDIT: Re-did above debug output, more than one problem there

EDIT: As when DoubleRate you use SelectOdd(Flow), so should not all of the other clips be SelectOdd,
however, B has already had SelectOdd applied.
Also, would that mean that when NOT double rate, then no SelectOdd should be applied to any, including B.
I'm not sure I understand what its supposed to do so I'm not sure how to correct it.

[MysteryX]

I think the whole "SelectOdd" and "FrameDelete" in the processing parts of the clip are unnecessary. It's only at the end that you interlace source and processed clips. Processed clip will only be generated for such requested frames.

I'll make those changes tomorrow.

Is 60fps working for you, or are both modes failing?

[StainlessS]

I have only been trying it with default doublerate, I'de already reached same conclusion as you about selectodd stuff and already implemented my fix,
seems to be working just fine, except I did not know what to do with this line (so I've left it for you :) )


EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd


here fix, seems to be working just fine, no flashes at all.

# Frame Rate Converter
# Version: 27-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools (default prefilter), GRunT (output="over")
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ BlkSizeV - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only;
## skip=mask used by SkipOver; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255.
## 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. (Default=16)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc",
\ int "SkipOver", clip "Prefilter",Bool "Debug")
{
Function FRSub(clip c,string tit){ # Auto Convert clips to YV12 if not already (NOP if YV12)
c.ConvertToYV12.Subtitle(tit,align=5)
}
Preset = Default(Preset, "normal")
P_SLOW=0 P_NORMAL=1 P_FAST=2
Pset=Preset=="slow"?P_SLOW:Preset=="normal"?P_NORMAL:Preset=="fast"?P_FAST:-1
Assert(Pset!=-1,"FrameRateConverter: 'Preset' must be slow, normal or fast {'"+Preset+"'}")
Output = Default(Output, "auto")
O_AUTO=0 O_FLOW=1 O_NONE=2 O_MASK=3 O_SKIP=4 O_RAW=5 O_OVER=6
OPut=Output=="auto"?O_AUTO:Output=="flow"?O_FLOW:Output=="none"?O_NONE:Output=="mask"?O_MASK:
\ Output=="skip"?O_SKIP:Output=="raw"?O_RAW:Output=="over"?O_OVER:-1
Assert(OPut!=-1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over) {'"+Output+"'}")

FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
BlkSize = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
BlkSizeV = Default(BlkSizeV, BlkSize)
blkmin = Min(BlkSize,BlkSizeV)
MaskGam = Default(MaskGam, 0.5)
MaskTrh = Default(MaskTrh, 120)
MaskOcc = MaskTrh > 0 ? Default(MaskOcc, 150) : 0
SkipOver = Default(SkipOver, 16)
CalcPrefilter = Defined(Prefilter) || Pset != P_FAST # If !Defined(Prefilter) && P_FAST : Prefilter MUST be Input C
Prefilter = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C)
Debug = Default(Debug,False)

Assert(BlkSize==8||BlkSize==16||BlkSize==32,String(BlkSize,"FrameRateConverter: BlkSize must be 8, 16 or 32 {%.f}"))
Assert(BlkSizeV==8||BlkSizeV==16||BlkSizeV==32,String(BlkSizeV,"FrameRateConverter: BlkSizeV must be 8, 16 or 32 {%.f}"))
Assert(MaskGam>0.0&&MaskGam<=1.0,String(MaskGam,"FrameRateConverter: MaskGam must be between 0.0 and 1.0 {%.1f}"))
Assert(MaskTrh>=0&& MaskTrh<= 255,String(MaskTrh,"FrameRateConverter: MaskTrh must be between 0 and 255 {%.f}"))
Assert(MaskOcc>=0&&MaskOcc<=255,String(MaskOcc,"FrameRateConverter: MaskOcc must be between 0 and 255 {%.f}"))
Assert(SkipOver>=0&&SkipOver<=255,String(SkipOver,"FrameRateConverter: SkipOver must be between 0 and 255 {%.f}"))

# Performance settings: slow, normal, fast
Recalculate = PSet==P_SLOW || PSET==P_NORMAL
DCT = PSet==P_SLOW ? 1 : 0

## "B" - Blending, "BHard" - No blending
BHard = C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt,isb=true,blksize=BlkSize,blksizeV=BlkSizeV,overlap=blkmin>8?4:blkmin>4?2:0,search=3,dct=DCT)
fwd = MAnalyse(superfilt,isb=false,blksize=BlkSize,blksizeV=BlkSizeV,overlap=blkmin>8?4:blkmin>4?2:0,search=3,dct=DCT)
fwd = Recalculate ? MRecalculate(super,fwd,blksize=BlkSize/2,blksizeV=BlkSizeV/2,overlap=blkmin>8?2:0,thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super,bak,blksize=BlkSize/2,blksizeV=BlkSizeV/2,overlap=blkmin>8?2:0,thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255).ConvertToY8() :
\ BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = EM.ConvertToY8()
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
OutRaw = EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(100)
EM = EM.mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
OutSkip = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(EMskip,BlankClip(EM, color_yuv=$ff0000),BlankClip(EM,color_yuv=$000000),
\ "AverageLuma", ">", string(SkipOver)) : BlankClip(EM, color_yuv=$000000)

# Display SkipOver value on Output="over
Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
FlowOver = (Oput==O_OVER||DEBUG) ? Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",
\ args="Disp_EMskip",Local=True) : 0

## Convert masks to desired frame rate
EM = EM.ChangeFPS(NewNum, NewDen)
Sc = Sc.ChangeFPS(NewNum, NewDen)

## the FrameRateConverter magic happens
M = mt_merge(Flow, B, EM, luma=true, chroma="process")
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M


Z_AUTO = (DEBUG||OPut==O_AUTO) ? (FrameDouble ? Interleave(C, SelectOdd(M)) : M) : 0
Z_FLOW = (DEBUG||OPut==O_FLOW) ? Flow : 0
Z_NONE = (DEBUG||OPut==O_NONE) ? B : 0
Z_MASK = (DEBUG||OPut==O_MASK) ? EM : 0
Z_SKIP = (DEBUG||OPut==O_SKIP) ? OutSkip : 0
Z_RAW = (DEBUG||OPut==O_RAW) ? OutRaw : 0
Z_OVER = (DEBUG||Oput==O_OVER)
\ ? mt_merge(
\ FlowOver.Overlay(MergeRGB(BlankClip(EM,color_yuv=$000000),EM,EM),mode="Add",opacity=0.40,pc_range=true),
\ BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
\ : 0

R=(Debug)
\ ? StackHorizontal(
\ StackVertical(FRSub(Z_AUTO,"Auto"),FRSub(Z_FLOW,"Flow"),FRSub(Z_NONE.ChangeFps(NewNum, NewDen),"None")),
\ StackVertical(FRSub(Z_MASK,"Mask"),FRSub(Z_SKIP,"Skip"),FRSub(Z_OVER,"Over")),
\ StackVertical(FRSub(EM,"EM"),FRSub(SC,"SC"),FRSub(M,"M"))
\ )
\ :(Oput==O_AUTO) ? Z_AUTO [** auto: artifact masking *]
\ : (Oput==O_FLOW) ? Z_FLOW [** flow: interpolation only *]
\ : (Oput==O_NONE) ? Z_NONE [** none: ConvertFPS only *]
\ : (Oput==O_MASK) ? Z_MASK [** mask: mask only *]
\ : (Oput==O_SKIP) ? Z_SKIP [** skip: skip mask *]
\ : (Oput==O_RAW) ? Z_RAW [** raw: raw mask *]
\ : (Oput==O_OVER) ? Z_OVER [** over: mask as cyan overlay *]
\ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")
return R
}


My test

Avisource("F:\v\XMen2.avi")
#ShowFrameNumber # EDIT: Only to find flash frames, no longer needed
#return last

FrameRateConverter(Debug=True)
#FrameRateConverter(Output="Auto")
#FrameRateConverter(Output="Flow")
#FrameRateConverter(Output="None")
#FrameRateConverter(Output="Mask")
#FrameRateConverter(Output="Skip")
#FrameRateConverter(Output="Raw")
#FrameRateConverter(Output="Over")

[chainik_svp]

ok, I've testes JM's clip and I must say:
1. don't use "algo=13" (default in Interframe w/o "smooth" preset)!
2. higher mask.area values lead to more blurry image (Interframe uses 150 in "smooth" preset, which is too high)

so, Interframe:
- default mode: algo=13, mask.area=0
- smooth preset: algo=23, mask.area=150

BOTH modes are BAD, at least for this extremely low-quality source
this doesn't mean they're ALWAYS bad, I believe SubJunk made a lot of comparisons when he selected these values

set algo=21/23, mask.area <= 50 and you'll be fine ;)

[kolak]

To get some values which work with specific source is 1 thing, to find good general settings is very different.

[StainlessS]

MX, your times for fast are probably quite a bit off, bug in fast processing for prefilter.

Mod

CalcPrefilter = Defined(Prefilter) || Pset != P_FAST # If !Defined(Prefilter) && P_FAST : Prefilter MUST be Input C
# Prefilter = Default(Prefilter, C.RemoveGrain(22) )
Prefilter = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C)
# EDIT: OR alternative: Prefilter = Default(Prefilter, !P_FAST ? C.RemoveGrain(22) : C)


Am doing a few changes (mainly cosmetic).

EDIT: Post #207 updated as per above.
+ Asserts show error value.


EDIT: In addition to above bug, both SuperFilt and Super would both be from C.RemoveGrain(22), ie quality loss,
output always Prefiltered. Above fix fixes that too.
Extreme care should be taken if making alterations anywhere in vicinity of those settings.

[MysteryX]

Comparison with prefilter and artifact masking disabled.

FrameRateConverter(60, output="auto", prefilter=last, masktrh=0, skipover=0)
https://s1.postimg.org/gk7oin67f/106-frc.png (https://postimg.org/image/gk7oin67f/)

InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, OverrideArea=0, Cores=1)
https://s13.postimg.org/as2thdd77/106-inter.png (https://postimg.org/image/as2thdd77/)

[chainik_svp]

"OverrideArea" IS zero by default... which means "override" is off

===
BTW, here's another sample video (http://www.svp-team.com/files/temp/telnyashka.avi)

It's almost impossible with MVTools to get rid of this:
http://www.svp-team.com/files/temp/telnyashka-321.png

this thing is not static, it jumps all over the striped vest randomly ;)
(well, there's one solution - set search radius to 1, but it's not great)

[MysteryX]

Whole frame blurring is one thing. Artifact masking is another.

I'll do some test later with 1080p source that has severe artifacts problems.

This is the script generated by Interframe, which I isolated earlier; but then didn't know how to convert settings to/from MFlow syntax

function InterFrameProcess(clip Input) {
SuperString = "{scale:{up:0,down:4},gpu:1,rc:false}"
VectorsString = "{block:{w:8,overlap:2},main:{search:{distance:0,coarse:{distance:-10,bad:{sad:2000}}}},refine:[{thsad:250}]}"
SmoothString = "{rate:{num:60,den:1,abs:true},algo:23,mask:{area:150,area_sharp:1.2},scene:{blend:true, mode:0}}"

# Make interpolation vector clip
Super = SVSuper(Input, SuperString)
Vectors = SVAnalyse(Super, VectorsString)

# Put it together
smooth_video = SVSmoothFps(Input, Super, Vectors, SmoothString, url="www.svp-team.com", mt=1)
smooth_video
}

[chainik_svp]

in your message above:
Comparison with prefilter and artifact masking disabled.
artifact masking for Interframe IS NOT disabled
it's still equal to "150"

[MysteryX]

HD sample: Girl's Day - Female President (https://www.youtube.com/watch?v=v0f9ifrDSp8)

FrameRateConverter(60, output="over", blksize=32)
FrameRateConverter(60, output="over", blksize=16)
FrameRateConverter(60, output="over", blksize=16, preset="slow")
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1)
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1, overridearea=1)

Frame 3497
https://s10.postimg.org/51co8xzet/girl3497-frc32.png (https://postimg.org/image/51co8xzet/) https://s10.postimg.org/bp99vjix1/girl3497-frc16.png (https://postimg.org/image/bp99vjix1/) https://s10.postimg.org/z4r70w2o5/girl3497-frc16-dct.png (https://postimg.org/image/z4r70w2o5/) https://s10.postimg.org/iwayrettx/girl3497-inter.png (https://postimg.org/image/iwayrettx/) https://s28.postimg.org/ju9ng0m89/girl3497-inter-1.png (https://postimg.org/image/ju9ng0m89/)

Frame 8900
https://s10.postimg.org/a53xx5ait/girl8900-frc32.png (https://postimg.org/image/a53xx5ait/) https://s10.postimg.org/bhlmz17yd/girl8900-frc16.png (https://postimg.org/image/bhlmz17yd/) https://s10.postimg.org/x5al9h8cl/girl8900-frc16-dct.png (https://postimg.org/image/x5al9h8cl/) https://s10.postimg.org/je6479jet/girl8900-inter.png (https://postimg.org/image/je6479jet/) https://s28.postimg.org/xpcjc8615/girl8900-inter-1.png (https://postimg.org/image/xpcjc8615/)

Note: DCT=1 is prohibitively slow in 1080p!!!

Note: DCT's darker images is because it produces stronger artifact masks and darker means the frames will be skipped -- but those frames actually look better

Note: For frame 8900, most such frames are skipped by my script, but this bad frame does go through -- as an exception

Performance:

blksize=32

FPS (min | max | average): 3.728 | 137555 | 18.62
Memory usage (phys | virt): 1389 | 1395 MiB
Thread count: 29
CPU usage (average): 70%


blksize=16

FPS (min | max | average): 3.314 | 111354 | 15.65
Memory usage (phys | virt): 1483 | 1496 MiB
Thread count: 29
CPU usage (average): 66%


blksize=16, preset="slow"

FPS (min | max | average): 0.040 | 53146 | 0.123
Memory usage (phys | virt): 1465 | 1466 MiB
Thread count: 26
CPU usage (average): 44%



EDIT: I just had an idea!

If frames are "skipped" at blksize=16, I could fallback to blksize=32, and if it fails again, then skip. It would be a bit more processing but only on bad frames, and frame 8900 would then render correctly.

[MysteryX]

Here's the problem with DCT mask strength

DCT=0
https://s13.postimg.org/8zg5r1hf7/mask-dct0.png (https://postimg.org/image/8zg5r1hf7/)

DCT=1
https://s13.postimg.org/6jecd6zcj/mask-dct1.png (https://postimg.org/image/6jecd6zcj/)

Now I have to find a way to adjust to DCT=1 mask strength. Average luminosity is almost twice higher!!

Update: I'm finding it hard to do something decent with DCT=1. The raw image looks nicer, but then the mask is also much stronger, and while it removes some artifacts, it creates others. Sometimes it puts lines in subtitles, and to remove it, the mask treshold must be high, which causes a lot more artifact removal to occur; and thus more frame blending and more double shadows.

Combining the facts that...
- DCT=1 is gruelly slow (see benchmark above)
- DCT=1 generates much stronger artifact masks
- DCT=1 creates some ugly artifacts preventing lowering the mask treshold much

...

I'm considering dropping DCT altogether.

Here's what I managed to achieve.

DCT=0 / DCT=1 with MaskTrh-40
https://s7.postimg.org/i0x9qrptj/dct0.png (https://postimg.org/image/i0x9qrptj/) https://s7.postimg.org/yrd8tudmf/dct1.png (https://postimg.org/image/yrd8tudmf/)

If I lower the treshold by 50, then I get artifacts like this in the text
https://s7.postimg.org/n0upypvg7/dct0-50.png (https://postimg.org/image/n0upypvg7/) https://s7.postimg.org/qzwiva9h3/dct1-50.png (https://postimg.org/image/qzwiva9h3/)

[kolak]

HD sample: Girl's Day - Female President (https://www.youtube.com/watch?v=v0f9ifrDSp8)

FrameRateConverter(60, output="over", blksize=32)
FrameRateConverter(60, output="over", blksize=16)
FrameRateConverter(60, output="over", blksize=16, preset="slow")
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1)
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1, overridearea=1)

Frame 3497
https://s10.postimg.org/51co8xzet/girl3497-frc32.png (https://postimg.org/image/51co8xzet/) https://s10.postimg.org/bp99vjix1/girl3497-frc16.png (https://postimg.org/image/bp99vjix1/) https://s10.postimg.org/z4r70w2o5/girl3497-frc16-dct.png (https://postimg.org/image/z4r70w2o5/) https://s10.postimg.org/iwayrettx/girl3497-inter.png (https://postimg.org/image/iwayrettx/) https://s28.postimg.org/ju9ng0m89/girl3497-inter-1.png (https://postimg.org/image/ju9ng0m89/)

Frame 8900
https://s10.postimg.org/a53xx5ait/girl8900-frc32.png (https://postimg.org/image/a53xx5ait/) https://s10.postimg.org/bhlmz17yd/girl8900-frc16.png (https://postimg.org/image/bhlmz17yd/) https://s10.postimg.org/x5al9h8cl/girl8900-frc16-dct.png (https://postimg.org/image/x5al9h8cl/) https://s10.postimg.org/je6479jet/girl8900-inter.png (https://postimg.org/image/je6479jet/) https://s28.postimg.org/xpcjc8615/girl8900-inter-1.png (https://postimg.org/image/xpcjc8615/)

Note: DCT=1 is prohibitively slow in 1080p!!!

Note: DCT's darker images is because it produces stronger artifact masks and darker means the frames will be skipped -- but those frames actually look better

Note: For frame 8900, most such frames are skipped by my script, but this bad frame does go through -- as an exception


Well, this is exactly my problem with interframe- double edges. Don't have this issue with mvtools.

[MysteryX]

Mask strength discrepancy isn't only an issue with YV12/YV24, and DCT=0 or 1, but also blksize 8, 16 and 32 produce a different mask strength...

Right now it is tweaked for YV12 with blksize=16. A different format or block size won't give as good artifact masking.

[MysteryX]

Does anyone *ever* use non-square blocks? Handling 16x8 and 32x16 would make mask strength adjustments considerably more complex.

Any objection to dropping BlkSizeV?

Instead, what I'm working on is fallback mode, so you can process in 16x16, and fallback to either 8x8 or 32x32 before reverting back to "hard blending" if it fails too. It's just that doing such fallback with 16x8 would make it considerably more complex.

For SD videos, 16x16 with fallback to 8x8 works best, while for HD videos, 16x16 with fallback to 32x32 works best, so both options must be available.

[MysteryX]

OK I made a bunch of changes again.

I removed StainlessS's debug code and added a new debug option that displays AverageLuma details in the corner, in any output mode.

I removed DCT. Preset Slow is still there but it's not currently in use.

I made some adjustments based on the block size. Now it should work fine with any block sizes and any video sizes.

MaskTrh = MaskTrh + (BlkSize == 8 ? 30 : BlkSize == 32 ? -60 : 0)
SkipTrh = SkipTrh + (BlkSize == 8 ? 25 : BlkSize == 32 ? -25 : 0)


I removed BlkSizeV.

I added Fallback so that we try a different block size before skipping. 1 to try larger, -1 to try smaller, 0 to disable. For some videos, I found it to make nearly no difference. For other videos, however, a considerable amount of Fallback frames are coming through. Please test and give your feedback. You can view Fallback frames with Debug=true.

In theory, the fallback mode should only fetch the frames when Skip is triggered, but in practice, it makes the whole script run twice slower, which indicates it is calculating the fallback frame every time. Any way to improve this? Also, Fallback+Debug causes the info to be written twice on top of each other; anyone is welcomed to fix this.


# Frame Rate Converter
# Version: 28-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools (default prefilter), GRunT (output="over")
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize - The block size (default = Width>1600||Height>1200 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only;
## skip=mask used by SkipOver; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ MaskGam - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255.
## 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## 0 to disable. (Default=16)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
## @ Debug - Whether to display AverageLuma values of Skip, Mask and Raw. (Default=false)
##
## @ Fallback - When SkipOver is triggered, -1 to try with smaller block size,
## 1 to try with smaller block size, or 0 to disable. (Default=0)
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize",
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc",
\ int "SkipOver", clip "Prefilter", bool "Debug", int "Fallback")
{
Preset = Default(Preset, "normal")
P_SLOW = 0 P_NORMAL = 1 P_FAST = 2
Pset = Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1
Assert(Pset != -1, "FrameRateConverter: 'Preset' must be slow, normal or fast {'" + Preset + "'}")
Output = Default(Output, "auto")
O_AUTO = 0 O_FLOW = 1 O_NONE = 2 O_MASK = 3 O_SKIP = 4 O_RAW = 5 O_OVER = 6
OPut = Output == "auto" ? O_AUTO : Output == "flow" ? O_FLOW : Output == "none" ? O_NONE : Output == "mask" ? O_MASK :
\ Output == "skip" ? O_SKIP : Output == "raw" ? O_RAW : Output == "over" ? O_OVER : -1
Assert(OPut != -1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over) {'" + Output + "'}")

FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
BlkSize = Default(BlkSize, C.Width>1600||C.Height>1200 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
MaskGam = Default(MaskGam, .5)
MaskTrh = Default(MaskTrh, 120)
MaskOcc = MaskTrh > 0 ? Default(MaskOcc, 150) : 0
SkipTrh = 155
SkipOver = Default(SkipOver, 16)
CalcPrefilter = Defined(Prefilter) || Pset != P_FAST
Prefilter = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C)
Debug = Default(Debug, false)
#Fallback = Default(Fallback, C.Width>934||C.Height>700 ? (BlkSize<32?1:-1) : (BlkSize>8?-1:1))
Fallback = Default(Fallback, 0)
OutFps = OPut!=O_MASK && OPut!=O_SKIP && OPut!=O_RAW # Whether output will have altered frame rate

Assert(BlkSize==8 || BlkSize==16 || BlkSize==32, String(BlkSize, "FrameRateConverter: BlkSize must be 8, 16 or 32 {%.f}"))
Assert(MaskGam > 0.0 && MaskGam <= 1.0, String(MaskGam, "FrameRateConverter: MaskGam must be between 0.0 and 1.0 {%.1f}"))
Assert(MaskTrh >= 0 && MaskTrh <= 255, String(MaskTrh, "FrameRateConverter: MaskTrh must be between 0 and 255 {%.f}"))
Assert(MaskOcc >= 0 && MaskOcc <= 255, String(MaskOcc, "FrameRateConverter: MaskOcc must be between 0 and 255 {%.f}"))
Assert(SkipOver >= 0 && SkipOver <= 255, String(SkipOver, "FrameRateConverter: SkipOver must be between 0 and 255 {%.f}"))
Assert((Fallback >= -1 && Fallback <= 1) || Fallback == 69, String(SkipOver, "FrameRateConverter: Fallback must be -1, 0 or 1 {%.f}"))
Assert(Fallback != -1 || BlkSize != 8, "FrameRateConverter: Fallback cannot be -1 with BlkSize = 8")
Assert(Fallback != 1 || BlkSize != 32, "FrameRateConverter: Fallback cannot be 1 with BlkSize = 32")

# Performance settings: slow, normal, fast
Recalculate = PSET <= P_NORMAL

## "B" - Blending, "BHard" - No blending
BHard = Fallback == 1 || Fallback == -1 ? C.FrameRateConverter(NewNum, NewDen, Preset,
\ Fallback == -1 ? BlkSize/2 : BlkSize*2, FrameDouble, Output, MaskGam, MaskTrh, MaskOcc, SkipOver, Prefilter, Debug, 69)
\ : C.ChangeFPS(NewNum, NewDen)
B = C.ConvertFPS(NewNum, NewDen)

## Adjust parameters for different block sizes, causing stronger or weaker masks
MaskTrh = MaskTrh + (BlkSize == 8 ? 30 : BlkSize == 32 ? -60 : 0)
SkipTrh = SkipTrh + (BlkSize == 8 ? 25 : BlkSize == 32 ? -25 : 0)

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad=16, vpad=16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad=16, vpad=16, levels=1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, overlap = BlkSize>8?4:BlkSize>4?2:0, search=3, dct=0)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, overlap = BlkSize>8?4:BlkSize>4?2:0, search=3, dct=0)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, overlap = BlkSize>8?2:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, overlap = BlkSize>8?2:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num=NewNum, den=NewDen, blend=false, ml=200, mask=2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255).ConvertToY8() :
\ BlankClip(C, pixel_type="Y8", color_yuv=$000000)
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=.6, mode="lighten", pc_range=true) : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
OutRaw = EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSize/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(255-SkipTrh)
EM = EM.mt_binarize(255-MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
OutSkip = EMskip.BicubicResize(C.width, C.Height)
ShowEM = EM

## "Sc" - scene detection / SkipOver
Sc = SkipOver > 0 ? ConditionalFilter(EMskip,BlankClip(EM, color_yuv=$ff0000),BlankClip(EM,color_yuv=$000000),
\ "AverageLuma", ">", string(SkipOver)) : BlankClip(EM, color_yuv=$000000)

## Convert masks to desired frame rate
EM = OutFps ? EM.ChangeFPS(NewNum, NewDen) : EM
Sc = OutFps ? Sc.ChangeFPS(NewNum, NewDen) : Sc

## the FrameRateConverter magic happens
M = mt_merge(Flow, B, EM, luma=true, chroma="process")
M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

R= (Oput==O_AUTO) [** auto: artifact masking *]
\ ? (FrameDouble ? Interleave(C, SelectOdd(M)) : M)
\ : (Oput==O_FLOW) [** flow: interpolation only *]
\ ? Flow
\ : (Oput==O_NONE) [** none: ConvertFPS only *]
\ ? B
\ : (Oput==O_MASK) [** mask: mask only *]
\ ? EM
\ : (Oput==O_SKIP) [** skip: skip mask *]
\ ? OutSkip
\ : (Oput==O_RAW) [** raw: raw mask *]
\ ? OutRaw
\ : (Oput==O_OVER) [** over: mask as cyan overlay *]
\ ? mt_merge(
\ Flow.Overlay(MergeRGB(BlankClip(EM, color_yuv=$000000), EM, EM), mode="Add", opacity=0.40, pc_range=true),
\ BlankClip(Flow, color=$B8860B), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
\ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")

# Display AverageLuma values of Skip, Mask and Raw
ShowSkip = OutFps ? EMskip.ChangeFPS(NewNum, NewDen) : EMskip
ShowRaw = OutFps ? OutRaw.ChangeFPS(NewNum, NewDen) : OutRaw
R = Debug ? R.GScriptClip("""(Fallback == 69 || Fallback == 0 || Sc.AverageLuma < 255) ?
\ Subtitle("Skip: " + string(ShowSkip.AverageLuma) + "\nMask: " + string(EM.AverageLuma) +
\ "\nRaw: " + string(ShowRaw.AverageLuma) +
\ (Fallback == 69 && Sc.AverageLuma == 0 ? "\nFallback" : "") +
\ ((Fallback == 69 || Fallback == 0) && Sc.AverageLuma == 255 ? "\nSkipped" : ""), lsp=0) : last""",
\ args = "ShowSkip,EM,ShowRaw,Sc,Fallback", Local=true) : R
return R
}


EDIT: The debug display bug is fixed.

[Selur]

Hmm,.. tried this version on my sample (see: https://forum.doom9.org/showthread.php?t=174541) with 'framerateconverter()' after frame 32 or so, each frame is simply doubled,...

[MysteryX]

Set SkipOver=0 to interpolate all frames and Debug=true to see what's going on under the hood and find the right SkipOver value. You can also use the new Fallback option to try a different block size before reverting back to doubling frames.

[MysteryX]

Something I could add: FallbackOver, BlendOver and SkipOver. We can do different actions based on the severity of the frame.

[Sharc]

This sounds like a good idea to me :)

[chainik_svp]

BTW, this's almost what SVP does in "adaptive" interpolation mode - calculates scene "quality" and modifies intermediate frames times (switching source frames only in the worst case) accordingly.

Let me quote the docs:

scene - Extended "scene change" controls.

scene.mode - Frames interpolation mode:
0 - uniform interpolation for maximum smoothness. For example for 24->60 conversion output will be: "1mmmm1mmmm...", where "1" stands for original frame and "m" for interpolated one.
1 - "1m" mode that gives "1mm1m1mm1m..." output in the above example => less artifacts at the cost of less smoothness.
2 - "2m" mode: "1m11m11m11..." => much less artifacts and much less smoothness.
3 - adaptive mode that switches between modes 0,1,2 based on overall vector field quality.

scene.blend - Blend frames at scene change like ConvertFps if true, or repeat last frame like ChangeFps if false.

scene.limits - Limits for vector field quality / scene change detection.
For example scene change will be detected if number of blocks with "adjusted SAD" > "limits.scene" will be more than "limits.blocks" percents of all blocks, that has "adjusted SAD" value > "limits.zero", where "adjusted SAD" is "block SAD"/"block average luma".

scene.limits.m1 - Limit for changing uniform mode to "1m".
scene.limits.m2 - Limit for changing "1m" mode to "2m".
scene.limits.scene - Limit for scene change detection.
scene.limits.zero - Vectors with "adjusted SAD" less than this value are excluded from consideration.
scene.limits.blocks - Threshold which sets how many blocks in percents have to change.

[MysteryX]

BTW, this's almost what SVP does in "adaptive" interpolation mode - calculates scene "quality" and modifies intermediate frames times (using source frames only in the worst case) accordingly.
The question remains: why am I achieving much better results than SVP? The logic is the same, but somewhere along the line, quality gets lost.

[chainik_svp]

Because you're not using SVP.

Interframe != SVP


> why am I achieving much better results than SVP?

where?
keep in mind that both 13th "SVP shader" (*) and "area masking" are missing from MVTools
w/o these new features you got almost the same still images as with MVTools

(*) "13th shader" is the one from MBlockFps, but applied per-pixel rather than per-block

[MysteryX]

Because you're not using SVP.

Interframe != SVP
Interframe is a preset to generate the same scripts as SVP. Is there something that Interframe is doing wrong that we could fix?

> why am I achieving much better results than SVP?

where?

https://forum.doom9.org/showthread.php?p=1805304#post1805304
https://forum.doom9.org/showthread.php?p=1805396#post1805396

keep in mind that both 13th "SVP shader" (*) and "area masking" are missing from MVTools
w/o these new features you got almost the same still images as with MVTools

(*) "13th shader" is the one from MBlockFps, but applied per-pixel rather than per-block
We can talk about theory all day long -- and at the end of the day, it's the result that matters. We bring screenshots and compare between various methods to see which is better.

Your comments here haven't been exactly useful. You've only said that "SVP is fine" but we still aren't any closer to getting similar results with SVP (and with GPU acceleration) than we do with FrameRateConverter (slower, unfortunately).

[MysteryX]

The latest script introduced some problems. I'm working on it.

[MysteryX]

OK this version will work much better than the latest script.

Fallback wasn't working. The reason is that although some frames may generate better with blksize 8 or 16, the artifact mask doesn't change enough to know for sure which is better. More often than not, if the artifact mask is too strong, interpolation should be discarded and if the fallback frame passes, it risks being a "false good". If a frame is flagged as bad, better revert to Blending or Skip right away.

There is now BlendOver and SkipOver. Frames that aren't good will revert to blending, and frames that are really bad and scene changes will revert to skip. That's working very well.

The adjustments I made for varying block sizes were wrong. I corrected and re-adjusted it so that 8, 16 and 32 work.

So far it's all for YV12. YV24 has slightly stronger mask so I still need to make up for that.

This version should skip much less than before. Use Debug=true to see what it's doing.


# Frame Rate Converter
# Version: 29-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, GRunT, rgtools (default prefilter)
##
## @ NewNum - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize - The block size: 8, 16 or 32.
## (default = Width>1600||Height>1200 ? 32 : Width>720||C.Height>480 ? 16 : 8)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
## auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only;
## skip=mask used to Skip; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ Debug - Whether to display AverageLuma values of Skip, Mask and Raw. (Default=false)
##
## @ Prefilter - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
## @ MaskTrh - The treshold where a block is considered bad, between 0 and 255. Smaller = stronger.
## 0 to disable artifact masking. (Default=135)
##
## @ MaskOcc - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=105)
##
## @ BlendOver - Try fallback block size when artifacts cover more than specified treshold, or 0 to disable.
## If it fails again, it will revert to frame blending. (default=20)
##
## @ SkipOver - Skip interpolation of frames when artifacts cover more than specified treshold,
## or 0 to disable. (Default=45)
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", bool "FrameDouble",
\ string "Output", bool "Debug", clip "Prefilter", int "MaskTrh", int "MaskOcc", int "BlendOver", int "SkipOver")
{
Preset = Default(Preset, "normal")
P_SLOW = 0 P_NORMAL = 1 P_FAST = 2
Pset = Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1
Assert(Pset != -1, "FrameRateConverter: 'Preset' must be slow, normal or fast {'" + Preset + "'}")
Output = Default(Output, "auto")
O_AUTO = 0 O_FLOW = 1 O_NONE = 2 O_MASK = 3 O_SKIP = 4 O_RAW = 5 O_OVER = 6
OPut = Output == "auto" ? O_AUTO : Output == "flow" ? O_FLOW : Output == "none" ? O_NONE : Output == "mask" ? O_MASK :
\ Output == "skip" ? O_SKIP : Output == "raw" ? O_RAW : Output == "over" ? O_OVER : -1
Assert(OPut != -1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over) {'" + Output + "'}")

FrameDouble = Default(FrameDouble, Defined(NewNum) ? false : true)
NewNum = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
NewDen = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
BlkSize = Default(BlkSize, C.Width>1600||C.Height>1200 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
MaskTrh = Default(MaskTrh, 100)
SkipTrh = 90
MaskOcc = MaskTrh > 0 ? Default(MaskOcc, 105) : 0
BlendOver = Default(BlendOver, 20)
SkipOver = Default(SkipOver, 45)
CalcPrefilter = Defined(Prefilter) || Pset != P_FAST
Prefilter = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C)
Debug = Default(Debug, false)
OutFps = OPut!=O_MASK && OPut!=O_SKIP && OPut!=O_RAW # Whether output will have altered frame rate
Recalculate = PSET <= P_NORMAL

Assert(BlkSize==8 || BlkSize==16 || BlkSize==32, String(BlkSize, "FrameRateConverter: BlkSize must be 8, 16 or 32 {%.f}"))
Assert(MaskTrh >= 0 && MaskTrh <= 255, String(MaskTrh, "FrameRateConverter: MaskTrh must be between 0 and 255 {%.f}"))
Assert(MaskOcc >= 0 && MaskOcc <= 255, String(MaskOcc, "FrameRateConverter: MaskOcc must be between 0 and 255 {%.f}"))
Assert(BlendOver >= 0 && BlendOver <= 255, String(BlendOver, "FrameRateConverter: BlendOver must be between 0 and 255 {%.f}"))
Assert(SkipOver >= 0 && SkipOver <= 255, String(SkipOver, "FrameRateConverter: SkipOver must be between 0 and 255 {%.f}"))
Assert(BlendOver==0 || SkipOver==0 || SkipOver > BlendOver, "FrameRateConverter: SkipOver must be greater than BlendOver")

## "B" - Blending, "BHard" - No blending
B = C.ConvertFps(NewNum, NewDen)
BHard = C.ChangeFps(NewNum, NewDen)

## Adjust parameters for different block sizes, causing stronger or weaker masks
MaskTrh = MaskTrh + (BlkSize == 16 ? 25 : BlkSize == 32 ? 50 : 0)
SkipTrh = SkipTrh + (BlkSize == 16 ? 30 : BlkSize == 32 ? 70 : 0)

## jm_fps interpolation
superfilt = MSuper(prefilter, hpad=16, vpad=16) # all levels for MAnalyse
super = CalcPrefilter ? MSuper(C, hpad=16, vpad=16, levels=1) : superfilt # one level is enough for MRecalculate
bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, overlap = BlkSize>4?BlkSize/4:0, search=3, dct=0)
fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, overlap = BlkSize>4?BlkSize/4:0, search=3, dct=0)
fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, overlap = BlkSize/2>4?BlkSize/4:0, thSAD=100) : fwd
bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, overlap = BlkSize/2>4?BlkSize/4:0, thSAD=100) : bak
Flow = MFlowFps(C, super, bak, fwd, num=NewNum, den=NewDen, blend=false, ml=200, mask=2, thSCD2=255)

## "EM" - error or artifact mask
# Mask: SAD
EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=2, ysc=255, thSCD2=255).ConvertToY8() :
\ BlankClip(C, pixel_type="Y8", color_yuv=$000000)
# Mask: Temporal blending
EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=2, thSCD2=255).ConvertToY8() : EM
EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=.6, mode="lighten", pc_range=true) : EM
# Mask: Occlusion
EMocc = MaskOcc > 0 ? C.MMask(bak, ml=MaskOcc, kind=2, gamma=2, ysc=255, thSCD2=255)
\ .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
OutRaw = EM

## Mask processing
EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSize/4.0)*4)
\ .mt_expand(mode= mt_circle(zero=true, radius=1))
EMskip = EM.mt_binarize(SkipTrh)
EM = EM.mt_binarize(MaskTrh)
\ .Blur(.6)
\ .BicubicResize(C.Width, C.Height)
OutSkip = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")

## "M" - Apply artifact removal
EM = OutFps ? EM.ChangeFPS(NewNum, NewDen) : EM
EMskip = OutFps ? EMskip.ChangeFPS(NewNum, NewDen) : EMskip
M = OutFps ? mt_merge(Flow, B, EM, luma=true, chroma="process") : Flow

## Apply BlendOver and SkipOver
M = M.GScriptClip("Skip = EMskip.AverageLuma()
\ (" + string(SkipOver) + " > 0 && Skip >= " + string(SkipOver) + ") ? BHard :
\ (" + string(BlendOver) + " > 0 && Skip >= " + string(BlendOver) + ") ? B : M",
\ args = "EMskip,M,B,BHard", Local=true)

# Output modes
R= (Oput==O_AUTO) [** auto: artifact masking *]
\ ? (FrameDouble ? Interleave(C, SelectOdd(M)) : M)
\ : (Oput==O_FLOW) [** flow: interpolation only *]
\ ? Flow
\ : (Oput==O_NONE) [** none: ConvertFPS only *]
\ ? B
\ : (Oput==O_MASK) [** mask: mask only *]
\ ? EM
\ : (Oput==O_SKIP) [** skip: skip mask *]
\ ? OutSkip
\ : (Oput==O_RAW) [** raw: raw mask *]
\ ? OutRaw
\ : (Oput==O_OVER) [** over: mask as cyan overlay *]
\ ? Flow.Overlay(MergeRGB(BlankClip(EM, color_yuv=$000000), EM, EM), mode="Add", opacity=0.40, pc_range=true)
\ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")

# Debug: display AverageLuma values of Skip, Mask and Raw
ShowRaw = OutFps ? OutRaw.ChangeFPS(NewNum, NewDen) : OutRaw
R = Debug ? R.GScriptClip("""Skip = EMskip.AverageLuma()
\ SkipSoft = BlendOver > 0 && Skip >= BlendOver && (Skip < SkipOver || SkipOver == 0)
\ Subtitle("Skip: " + string(Skip) + "\nMask: " + string(EM.AverageLuma) +
\ "\nRaw: " + string(ShowRaw.AverageLuma) +
\ "\nBlkSize: " + string(BlkSize) +
\ (SkipSoft ? " - Blend" : "") +
\ (SkipOver > 0 && Skip >= SkipOver ? " - Skip" : ""), lsp=0)""",
\ args = "EMskip,EM,ShowRaw,BlkSize,SkipOver,BlendOver", Local=true) : R
return R
}

[chainik_svp]

MysteryX
Your comments here haven't been exactly useful. You've only said that "SVP is fine" but we still aren't any closer to getting similar results with SVP (and with GPU acceleration) than we do with FrameRateConverter (slower, unfortunately).

I'm sorry my English is so bad you can't understand it :(
I'll try to write as short sentences as I can.

Rule #1. Don't use algo=13. Use algo=21 or 23 instead.
Rule #2. Don't set mask.area to 150. Use 50 at maximum.

> https://forum.doom9.org/showthread.php?p=1805304#post1805304

this's a comparison between Interframe and MVTools, NOT between SVPflow and MVTools
you can get results similar to MVTools by following rules #1 and #2 above

which is clear enough in your 2nd comparison, first scene:
> https://forum.doom9.org/showthread.p...96#post1805396

2nd scene with stripes is a mess, this's the only one where I can agree "yes, SVPflow with default settings has a problem here"
however you prefer to ignore the opposite example - https://forum.doom9.org/showthread.php?p=1805379#post1805379

And if you think that my comment about "adaptive interpolation mode" was not useful - I'm sorry to interrupt you, please continue your re-invention of the wheel...

[MysteryX]

Chainik, this code is what Interframe(tuning="smooth") does, and I changed area from 150 to 1. It uses algo 23.

Do you have a better script to propose for testing?


function InterFrameProcess(clip Input) {
SuperString = "{scale:{up:0,down:4},gpu:1,rc:false}"
VectorsString = "{block:{w:8,overlap:2},main:{search:{distance:0,coarse:{distance:-10,bad:{sad:2000}}}},refine:[{thsad:250}]}"
SmoothString = "{rate:{num:60,den:1,abs:true},algo:23,mask:{area:1,area_sharp:1.2},scene:{blend:true, mode:0}}"
Super = SVSuper(Input, SuperString)
Vectors = SVAnalyse(Super, VectorsString)
smooth_video = SVSmoothFps(Input, Super, Vectors, SmoothString, url="www.svp-team.com", mt=1)
smooth_video
}

[chainik_svp]

VectorsString = "{refine:[{thsad:250}]}"
SmoothString = "{rate:{num:60,den:1,abs:true},algo:23}"

[MysteryX]

VectorsString = "{refine:[{thsad:250}]}"
SmoothString = "{rate:{num:60,den:1,abs:true},algo:23}"

FrameRateConverter / SVP with above settings

https://s9.postimg.org/llf9zsarf/377-frc.png (https://postimg.org/image/llf9zsarf/) https://s9.postimg.org/nre6813ln/377-svp.png (https://postimg.org/image/nre6813ln/)

[kolak]

I agree with you. Whatever setting you try by average mvtools offers better quality.
For me main issue in svp are double edges (even on fairly easy scenes).
Another thing- with much older svp libraries quality was better.

[Groucho2004]

Here (https://www.dropbox.com/s/s63qy9spnfohnrw/str.mkv?dl=0) is a test clip that produces artefacts (on the striped patterns) with almost all methods I tried.
Mystery's latest script works very well on that clip, SVP/Interframe (even with chainik_svp's suggested parameters) is the worst by far.

[Sharc]

@MysteryX
Very nice work.
Just a cosmetic note: The default values in the ##comments and the actual default values should be aligned for SkipOver and BlendOver

[chainik_svp]

> FrameRateConverter / SVP with above settings

ok, assuming your effective MVTools options are

bak = MAnalyse(superfilt, isb=true, blksize=8, overlap = 2, search=3, dct=DCT)
fwd = MAnalyse(superfilt, isb=false, blksize=8, overlap = 2, search=3, dct=DCT)
fwd = MRecalculate(super, fwd, blksize=4, overlap = 0, thSAD=100)
bak = MRecalculate(super, bak, blksize=4, overlap = 0, thSAD=100)
MFlowFps(C, super, bak, fwd, num = 60, den = 1, blend = false, ml = 200, mask = 2, thSCD2=255)

here's the closest SVPflow settings:

VectorsString = "{block:{w:8,overlap:2},main:{search:{distance:2,coarse:{satd:false,distance:2}},penalty:{lambda:20.0}},refine:[{thsad:100}]}"
SmoothString= "{gpuid:11,rate:{num:60,den:1,abs:true},algo:23}"

[MysteryX]

here's the closest SVPflow settings:

VectorsString = "{block:{w:8,overlap:2},main:{search:{distance:2,coarse:{satd:false,distance:2}},penalty:{lambda:20.0}},refine:[{thsad:100}]}"
SmoothString= "{gpuid:11,rate:{num:60,den:1,abs:true},algo:23}"


FrameRateConverter / SVP
https://s8.postimg.org/98pcphimp/377-frc.png (https://postimg.org/image/98pcphimp/) https://s8.postimg.org/7i6bo013l/377-svp.png (https://postimg.org/image/7i6bo013l/)

[MysteryX]

Made minor changes to the script above. Increased overlap with BlkSize==32, and fixed comments default values.

[manolito]

Thanks MysteryX, this script has come a long way...

I agree that using DCT=1 is forbiddingly slow, but for some sources it just makes the difference. Right now the Preset="slow" does not do anything at all, so you might just as well remove it completely.

I modified the script so the slow preset does invoke DCT=1 and also forces Output="flow". For some of my sources this gives the best results since artifact removal using error masks does not really work well with DCT=1.

BTW in the comments for the params you need to replace the output value "inter " by "flow".


Cheers
manolito

[StainlessS]

Small speed increase possible (In several places).


EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=2, ysc=255, thSCD2=255).ConvertToY8() :
\ BlankClip(C, pixel_type="Y8", color_yuv=$000000)


You ConvertToY8 after MMask, so throwing away processed chroma, how bout convertToY8 beforehand instead, avoid chroma processing.
(unless due to MaskTools not supporting Y8). [EDIT: I presume MMAsk processes chroma. Does it ?. EDIT: Looks like it does]

Every little helps.

[bin.n2f]

any chance we get FrameRateConverter script translated into vapoursynth lang.

thanx

[SpoCk0nd0pe]

Thank you very much for your efforts! I read this with a lot of interest.

[MysteryX]

You ConvertToY8 after MMask, so throwing away processed chroma, how bout convertToY8 beforehand instead, avoid chroma processing.

It definitely processes the chroma because the mask strength is different between Y12, Y16 and Y24 because of the chroma plane.

However, the mask only contains junk in the chroma planes.

If I create the mask only on the Luma plane, there could be slight performance gain, and also resolve the problem of mask strength discrepancy between formats, but I'd have to see what kind of impact it has on quality and mask accuracy.

any chance we get FrameRateConverter script translated into vapoursynth lang.

thanx
Perhaps, if someone with knowledge of Vapoursynth takes an interest. I've never touched it.

[StainlessS]

any chance we get FrameRateConverter script translated into vapoursynth lang.

You best ask in VapourSynth forum, where the VS people live, and also a good idea to wait until finalized, aint nobody gonna wanna keep doing a conversion several times a day.

[MysteryX]

Why is it that DCT=1 is so crushingly slow? But only on HD sources. On low-resolution sources, you're barely seeing the difference. Any way to improve this?

[hydra3333]

Hello, a related link just for interest (not my code, saw it whilst googling for OpenCL ICD loader stuff)
https://github.com/dthpham/butterflow

[MysteryX]

Hello, a related link just for interest (not my code, saw it whilst googling for OpenCL ICD loader stuff)
https://github.com/dthpham/butterflow
Can someone try it and post some comparison screenshots? Thanks

ahah! DCT=1 is crushingly slow with BlkSize=32 !!

With BlkSize=8, it is almost as fast. With BlkSize=16, it's useable. With BlkSize=32, forget it. Plus, it's giving me BAD results with BlkSize=32 on HD content. I'll just make it work with BlkSize 8 and 16.

Small speed increase possible (In several places).


EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=2, ysc=255, thSCD2=255).ConvertToY8() :
\ BlankClip(C, pixel_type="Y8", color_yuv=$000000)


You ConvertToY8 after MMask, so throwing away processed chroma, how bout convertToY8 beforehand instead, avoid chroma processing.
(unless due to MaskTools not supporting Y8). [EDIT: I presume MMAsk processes chroma. Does it ?. EDIT: Looks like it does]

Every little helps.
This change doesn't affect the mask. It's working with the vectors data and I don't think it's using C at all except to determine the output format. So I can call ConvertToY8 first but I don't think it will change performance much at all.

[johnmeyer]

Why is it that DCT=1 is so crushingly slow? But only on HD sources. On low-resolution sources, you're barely seeing the difference. Any way to improve this?This would be a great improvement to MVTools2, if it is technically possible. I still haven't had time to play with this new script, but I have used DCT=1 on many projects to reduce flicker. For some videos, it is actually better than Deflicker and other dedicated plugins, but it is unbelievably slow.

The fact that it can reduce flicker perhaps provides a hint of why it might be slow: the only way I know to reduce flicker is to take averages across many, many frames. I have never looked at the MVTools2 source code, but if I did, I'd be looking for some interaction between the DCT setting and the number of frames being evaluated for each block. I suspect that the number may be much larger when DCT is set to something other than zero. If so, improving performance may require the same sort of math genius like what was needed to create the FFT from standard Fourier analysis. If you've ever looked into how that is done, it is completely non-intuitive and requires a math aptitude far beyond anything I posses.