SVP-like frame interpolation? - Page 11

StainlessS · 28th April 2017, 00:56

Modified similar to your last post, Still getting weird flashes, and M jumps from eg frame 10000, to frame 5000 when flashing.

Code:

# Frame Rate Converter
# Version: 27-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools (default prefilter), GRunT (output="over")
##
## @ NewNum      - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen      - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset      - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize     - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
## 
## @ BlkSizeV    - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output      - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
##                 auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only; 
##                 skip=mask used by SkipOver; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ MaskGam     - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh     - The treshold where a block is considered bad, between 0 and 255.
##                 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc     - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver    - Skip interpolation of frames when artifacts cover more than specified treshold, 
##                 0 to disable. (Default=16)
##          
## @ Prefilter   - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##          
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV", 
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", int "SkipOver", clip "Prefilter",Bool "Debug")
{
    Function FRSub(clip c,string tit){ # Auto Convert clips to YV12 if not already (NOP if YV12) 
        c.ConvertToYV12.Subtitle(tit,align=5)
    }
    Preset     = Default(Preset, "normal")
    P_SLOW=0 P_NORMAL=1 P_FAST=2
    Pset=Preset=="slow"?P_SLOW:Preset=="normal"?P_NORMAL:Preset=="fast"?P_FAST:-1
    Assert(Pset!=-1, "FrameRateConverter: Preset must be slow, normal or fast")
    Output     = Default(Output, "auto")
    O_AUTO=0 O_FLOW=1 O_NONE=2 O_MASK=3 O_SKIP=4 O_RAW=5 O_OVER=6
    OPut=Output=="auto"?O_AUTO:Output=="flow"?O_FLOW:Output=="none"?O_NONE:Output=="mask"?O_MASK:Output=="skip"?O_SKIP:Output=="raw"?O_RAW:Output=="over"?O_OVER:-1
    Assert(OPut!=-1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over)")
        
    FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
    NewNum     = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
    NewDen     = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
    BlkSize    = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
    BlkSizeV   = Default(BlkSizeV, BlkSize)
    blkmin     = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
    MaskGam    = Default(MaskGam, .5)
    MaskTrh    = Default(MaskTrh, 120)
    MaskOcc    = Default(MaskOcc, 150)
    MaskOcc    = MaskTrh > 0 ? MaskOcc : 0
    SkipOver   = Default(SkipOver, 16)
    CalcPrefilter = Defined(Prefilter) || Pset != P_FAST
    Prefilter  = Default(Prefilter, C.RemoveGrain(22))
    Debug      = Default(Debug,False)

    Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
    Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
    Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
    Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
    Assert(MaskOcc >= 0 && MaskOcc <= 255, "FrameRateConverter: MaskOcc must be between 0 and 255")
    Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

    # Performance settings: slow, normal, fast
    Recalculate = PSet==P_SLOW || PSET==P_NORMAL
    DCT = PSet==P_SLOW ? 1 : 0

    ## "B" - Blending, "BHard" - No blending
    BHard = C.ChangeFPS(NewNum, NewDen)
    B  = C.ConvertFPS(NewNum, NewDen)
    B  = FrameDouble ? SelectOdd(B) : B

    ## jm_fps interpolation
    superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
    super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
    bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
    fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
    fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
    bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
    Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

    ## "EM" - error or artifact mask
    # Mask: SAD
    EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255).ConvertToY8() : 
	\                  BlankClip(C, pixel_type="Y8", color_yuv=$000000)
    EM = EM.ConvertToY8()
    # Mask: Temporal blending
    EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
    EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
    EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
    # Mask: Occlusion
    EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
    \                     .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
    EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
	OutRaw = EM
    
    ## Mask processing
    EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
    \       .mt_expand(mode= mt_circle(zero=true, radius=1))
    EMskip = EM.mt_binarize(100)
    EM = EM.mt_binarize(255-MaskTrh)
    \       .Blur(.6)
    \       .BicubicResize(C.Width, C.Height)
    OutSkip = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")
    
    ## "Sc" - scene detection / SkipOver
    Sc = SkipOver > 0 ? ConditionalFilter(EMskip, BlankClip(EM, color_yuv=$ff0000), BlankClip(EM, color_yuv=$000000), 
    \    "AverageLuma", ">", string(SkipOver)) : BlankClip(EM, color_yuv=$000000)
   
    # Display SkipOver value on Output="over
    Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
    FlowOver = (Oput==O_OVER||DEBUG) ? Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",args="Disp_EMskip",Local=True) : 0

    ## Convert masks to desired frame rate
    EM = EM.ChangeFPS(NewNum, NewDen)
    Sc = Sc.ChangeFPS(NewNum, NewDen)

    ## the FrameRateConverter magic happens
    M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
    M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

    Z_AUTO = (DEBUG||OPut==O_AUTO) ? (FrameDouble ? Interleave(C, M) : M) : 0
    Z_FLOW = (DEBUG||OPut==O_FLOW) ? Flow    : 0
    Z_NONE = (DEBUG||OPut==O_NONE) ? B       : 0
    Z_MASK = (DEBUG||OPut==O_MASK) ? EM      : 0
    Z_SKIP = (DEBUG||OPut==O_SKIP) ? OutSkip : 0
    Z_RAW  = (DEBUG||OPut==O_RAW)  ? OutRaw  : 0
    Z_OVER = (DEBUG||Oput==O_OVER) 
        \  ? mt_merge(
        \     FlowOver.Overlay(MergeRGB(BlankClip(EM, color_yuv=$000000), EM, EM), mode="Add", opacity=0.40, pc_range=true),
        \     BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
        \ : 0

    R=(Debug)
    \     ? StackHorizontal(
    \          StackVertical(FRSub(Z_AUTO,"Auto"),FRSub(Z_FLOW,"Flow"),FRSub(Z_NONE.ChangeFps(NewNum, NewDen),"None")),
    \          StackVertical(FRSub(Z_MASK,"Mask"),FRSub(Z_SKIP,"Skip"),FRSub(Z_OVER,"Over")),
    \          StackVertical(FRSub(EM,"EM"),FRSub(SC,"SC"),FRSub(M,"M"))
    \       )
    \ :(Oput==O_AUTO)  ? Z_AUTO  [** auto:  artifact masking     *]
    \ : (Oput==O_FLOW) ? Z_FLOW  [** flow: interpolation only    *] 
    \ : (Oput==O_NONE) ? Z_NONE  [** none: ConvertFPS only       *] 
    \ : (Oput==O_MASK) ? Z_MASK  [** mask:  mask only            *] 
    \ : (Oput==O_SKIP) ? Z_SKIP  [** skip:  skip mask            *] 
    \ : (Oput==O_RAW)  ? Z_RAW   [** raw:   raw mask             *] 
    \ : (Oput==O_OVER) ? Z_OVER  [** over:  mask as cyan overlay *] 
    \ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")
    return R
}

Avisource("F:\v\XMen2.avi")
ShowFrameNumber
#return last

FrameRateConverter(Debug=True)
#FrameRateConverter(Output="Auto")
#FrameRateConverter(Output="Flow")
#FrameRateConverter(Output="None")
#FrameRateConverter(Output="Mask")
#FrameRateConverter(Output="Skip")
#FrameRateConverter(Output="Raw")
#FrameRateConverter(Output="Over")

EDIT: is this an error

Code:

    B  = FrameDouble ? SelectOdd(B) : B      # B is original rate when FrameDouble
    ...
    ## Convert masks to desired frame rate
    EM = EM.ChangeFPS(NewNum, NewDen)        # EM is double original rate when FrameDouble
    ...
    ## the FrameRateConverter magic happens
    M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
                 <-original rate when framedouble -->, B Orig, EM is Doublerate

Above in blue, different rates ???

MysteryX · 28th April 2017, 01:04

Just tried FrameDouble again, and everything is fine. No, I'm not getting any weird flash. What ColorSpace are you using? (YV12 here) What Avisynth version are you using?

Btw I'm trying to split long lines so that it displays nicely in the browser.

Your script is also working fine here, no frame jumps. However, "SC" isn't displaying and "Mask" and "EM" are displaying the same. I believe Mask should be the raw mask. I think it's showing too much to be useful. 4 views would be more than enough.

Or perhaps, on top of "over", to have "debug" which shows 4 views and "debug2" which shows 9 views ... but isn't it starting to be overkill when the script is already ready and working?

One advantage of your "debug" mode is to help others understand what this script is doing under the hood. It took me a while to understand the basic YFRC script.

If you're going to display 9 clips, I'd recommend this order, from top-left to bottom-right: Output, MFlow, Blending, Raw, Mask, Skip --- these 6 are enough. The 3 others are redundant and unnecessary.

If we display 4 clips, Output, Over, Raw, Skip -- or something like that.

But first, what would be the intent? With the script working, it's only about tweaking settings by comparing details. If too many clips are showing up the details of each clip are too small to see and compare.

Edit: OK I see some frames not matching the others. That happens because some clips are 15fps while others are 60fps. If you display a 15fps clip next to a 60fps clip, you're going to see such mismatch.

StainlessS · 28th April 2017, 01:17

See edit

EDIT: I'm using AVS standard, YV12. With test script given including ShowFrameNumber.

M clip bottom RHS debug is jumping 5000 -> 10000 -> 5000 etc. (original frame numbers)

Only just updated to latest mvtools and masktools, was getting exact same type thing with older versions
(several versions ago)

EDIT: OverKill, not when your trying to find a bug (and its called debug mode).

EDIT: 20 frames flashing, ~ 200KB:- http://www.mediafire.com/file/cj3kwpaq748bwzm/MX.mp4
With Output=Auto.
Selur has been posting about the flashes in vapoursynth forum.

MysteryX · 28th April 2017, 01:52

I see no problem with AVS 2.6 here... nor with FrameDouble.

I did see a bug though: with FrameDouble, scene changes aren't being applied.

StainlessS · 28th April 2017, 01:57

Code:

    FlowX=FrameDouble ? SelectOdd(Flow) : Flow
    RT_DebugF("Flow_Rate=%f B_Rate=%f EM_Rate=%f",FlowX.FrameRate,B.FrameRate,EM.FrameRate)
    RT_DebugF("BHard_Rate=%f SC_Rate=%f",BHard.FrameRate,SC.FrameRate)

    ## the FrameRateConverter magic happens
    M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
    RT_DebugF("M_1 Rate=%f",M.FrameRate)
    M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M
    RT_DebugF("M_2 Rate=%f",M.FrameRate)

Debug output

Code:

[3888] RT_DebugF: Flow_Rate=25.000000 B_Rate=25.000000 EM_Rate=50.000000
[3888] RT_DebugF: BHard_Rate=50.000000 SC_Rate=50.000000
[3888] RT_DebugF: M_1 Rate=25.000000
[3888] RT_DebugF: M_2 Rate=25.000000

M_2 Rate=25.0 but that is probably just because the first clip framerate FlowX was 25.0,
The Mt_merge stuff obviously aint working proper there.

I think that mt_merge pair is where the flashes originate, result of first one mangles things up, but result is 25.0 fps
and probably no longer than first clip (FlowX).

EDIT: Re-did above debug output, more than one problem there

EDIT: As when DoubleRate you use SelectOdd(Flow), so should not all of the other clips be SelectOdd,
however, B has already had SelectOdd applied.
Also, would that mean that when NOT double rate, then no SelectOdd should be applied to any, including B.
I'm not sure I understand what its supposed to do so I'm not sure how to correct it.

MysteryX · 28th April 2017, 02:53

I think the whole "SelectOdd" and "FrameDelete" in the processing parts of the clip are unnecessary. It's only at the end that you interlace source and processed clips. Processed clip will only be generated for such requested frames.

I'll make those changes tomorrow.

Is 60fps working for you, or are both modes failing?

StainlessS · 28th April 2017, 03:17

I have only been trying it with default doublerate, I'de already reached same conclusion as you about selectodd stuff and already implemented my fix,
seems to be working just fine, except I did not know what to do with this line (so I've left it for you

)

Code:

    EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd

here fix, seems to be working just fine, no flashes at all.

Code:

# Frame Rate Converter
# Version: 27-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools (default prefilter), GRunT (output="over")
##
## @ NewNum      - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen      - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset      - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize     - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
## 
## @ BlkSizeV    - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output      - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
##                 auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only; 
##                 skip=mask used by SkipOver; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ MaskGam     - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh     - The treshold where a block is considered bad, between 0 and 255.
##                 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc     - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver    - Skip interpolation of frames when artifacts cover more than specified treshold, 
##                 0 to disable. (Default=16)
##          
## @ Prefilter   - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##          
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV", 
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", 
\ int "SkipOver", clip "Prefilter",Bool "Debug")
{
    Function FRSub(clip c,string tit){ # Auto Convert clips to YV12 if not already (NOP if YV12) 
        c.ConvertToYV12.Subtitle(tit,align=5)
    }
    Preset = Default(Preset, "normal")
    P_SLOW=0 P_NORMAL=1 P_FAST=2
    Pset=Preset=="slow"?P_SLOW:Preset=="normal"?P_NORMAL:Preset=="fast"?P_FAST:-1
    Assert(Pset!=-1,"FrameRateConverter: 'Preset' must be slow, normal or fast {'"+Preset+"'}")
    Output = Default(Output, "auto")
    O_AUTO=0 O_FLOW=1 O_NONE=2 O_MASK=3 O_SKIP=4 O_RAW=5 O_OVER=6
    OPut=Output=="auto"?O_AUTO:Output=="flow"?O_FLOW:Output=="none"?O_NONE:Output=="mask"?O_MASK:
        \ Output=="skip"?O_SKIP:Output=="raw"?O_RAW:Output=="over"?O_OVER:-1
    Assert(OPut!=-1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over) {'"+Output+"'}")
        
    FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
    NewNum     = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
    NewDen     = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
    BlkSize    = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
    BlkSizeV   = Default(BlkSizeV, BlkSize)
    blkmin     = Min(BlkSize,BlkSizeV)
    MaskGam    = Default(MaskGam, 0.5)
    MaskTrh    = Default(MaskTrh, 120)
    MaskOcc    = MaskTrh > 0 ? Default(MaskOcc, 150) : 0
    SkipOver   = Default(SkipOver, 16)
    CalcPrefilter = Defined(Prefilter) || Pset != P_FAST    # If !Defined(Prefilter) && P_FAST : Prefilter MUST be Input C
    Prefilter  = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C)
    Debug      = Default(Debug,False)

    Assert(BlkSize==8||BlkSize==16||BlkSize==32,String(BlkSize,"FrameRateConverter: BlkSize must be 8, 16 or 32 {%.f}"))
    Assert(BlkSizeV==8||BlkSizeV==16||BlkSizeV==32,String(BlkSizeV,"FrameRateConverter: BlkSizeV must be 8, 16 or 32 {%.f}"))
    Assert(MaskGam>0.0&&MaskGam<=1.0,String(MaskGam,"FrameRateConverter: MaskGam must be between 0.0 and 1.0 {%.1f}"))
    Assert(MaskTrh>=0&& MaskTrh<= 255,String(MaskTrh,"FrameRateConverter: MaskTrh must be between 0 and 255 {%.f}"))
    Assert(MaskOcc>=0&&MaskOcc<=255,String(MaskOcc,"FrameRateConverter: MaskOcc must be between 0 and 255 {%.f}"))
    Assert(SkipOver>=0&&SkipOver<=255,String(SkipOver,"FrameRateConverter: SkipOver must be between 0 and 255 {%.f}"))

    # Performance settings: slow, normal, fast
    Recalculate = PSet==P_SLOW || PSET==P_NORMAL
    DCT = PSet==P_SLOW ? 1 : 0

    ## "B" - Blending, "BHard" - No blending
    BHard = C.ChangeFPS(NewNum, NewDen)
    B  = C.ConvertFPS(NewNum, NewDen)

    ## jm_fps interpolation
    superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
    super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
    bak = MAnalyse(superfilt,isb=true,blksize=BlkSize,blksizeV=BlkSizeV,overlap=blkmin>8?4:blkmin>4?2:0,search=3,dct=DCT)
    fwd = MAnalyse(superfilt,isb=false,blksize=BlkSize,blksizeV=BlkSizeV,overlap=blkmin>8?4:blkmin>4?2:0,search=3,dct=DCT)
    fwd = Recalculate ? MRecalculate(super,fwd,blksize=BlkSize/2,blksizeV=BlkSizeV/2,overlap=blkmin>8?2:0,thSAD=100) : fwd
    bak = Recalculate ? MRecalculate(super,bak,blksize=BlkSize/2,blksizeV=BlkSizeV/2,overlap=blkmin>8?2:0,thSAD=100) : bak
    Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

    ## "EM" - error or artifact mask
    # Mask: SAD
    EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255).ConvertToY8() : 
    \                  BlankClip(C, pixel_type="Y8", color_yuv=$000000)
    EM = EM.ConvertToY8()
    # Mask: Temporal blending
    EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
    EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
    EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
    # Mask: Occlusion
    EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
    \                     .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
    EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
    OutRaw = EM
    
    ## Mask processing
    EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
    \       .mt_expand(mode= mt_circle(zero=true, radius=1))
    EMskip = EM.mt_binarize(100)
    EM = EM.mt_binarize(255-MaskTrh)
    \       .Blur(.6)
    \       .BicubicResize(C.Width, C.Height)
    OutSkip = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")
    
    ## "Sc" - scene detection / SkipOver
    Sc = SkipOver > 0 ? ConditionalFilter(EMskip,BlankClip(EM, color_yuv=$ff0000),BlankClip(EM,color_yuv=$000000), 
    \    "AverageLuma", ">", string(SkipOver)) : BlankClip(EM, color_yuv=$000000)
   
    # Display SkipOver value on Output="over
    Disp_EMskip = EMskip.ChangeFPS(NewNum, NewDen)
    FlowOver = (Oput==O_OVER||DEBUG) ? Flow.GScriptClip("Subtitle(string(Disp_EMskip.AverageLuma))",
        \ args="Disp_EMskip",Local=True) : 0

    ## Convert masks to desired frame rate
    EM = EM.ChangeFPS(NewNum, NewDen)
    Sc = Sc.ChangeFPS(NewNum, NewDen)

    ## the FrameRateConverter magic happens
    M = mt_merge(Flow, B, EM, luma=true, chroma="process")
    M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M
    

    Z_AUTO = (DEBUG||OPut==O_AUTO) ? (FrameDouble ? Interleave(C, SelectOdd(M)) : M) : 0
    Z_FLOW = (DEBUG||OPut==O_FLOW) ? Flow    : 0
    Z_NONE = (DEBUG||OPut==O_NONE) ? B       : 0
    Z_MASK = (DEBUG||OPut==O_MASK) ? EM      : 0
    Z_SKIP = (DEBUG||OPut==O_SKIP) ? OutSkip : 0
    Z_RAW  = (DEBUG||OPut==O_RAW)  ? OutRaw  : 0
    Z_OVER = (DEBUG||Oput==O_OVER) 
        \  ? mt_merge(
        \     FlowOver.Overlay(MergeRGB(BlankClip(EM,color_yuv=$000000),EM,EM),mode="Add",opacity=0.40,pc_range=true),
        \     BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
        \ : 0

    R=(Debug)
    \     ? StackHorizontal(
    \          StackVertical(FRSub(Z_AUTO,"Auto"),FRSub(Z_FLOW,"Flow"),FRSub(Z_NONE.ChangeFps(NewNum, NewDen),"None")),
    \          StackVertical(FRSub(Z_MASK,"Mask"),FRSub(Z_SKIP,"Skip"),FRSub(Z_OVER,"Over")),
    \          StackVertical(FRSub(EM,"EM"),FRSub(SC,"SC"),FRSub(M,"M"))
    \       )
    \ :(Oput==O_AUTO)  ? Z_AUTO  [** auto:  artifact masking     *]
    \ : (Oput==O_FLOW) ? Z_FLOW  [** flow: interpolation only    *] 
    \ : (Oput==O_NONE) ? Z_NONE  [** none: ConvertFPS only       *] 
    \ : (Oput==O_MASK) ? Z_MASK  [** mask:  mask only            *] 
    \ : (Oput==O_SKIP) ? Z_SKIP  [** skip:  skip mask            *] 
    \ : (Oput==O_RAW)  ? Z_RAW   [** raw:   raw mask             *] 
    \ : (Oput==O_OVER) ? Z_OVER  [** over:  mask as cyan overlay *] 
    \ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")
    return R
}

My test

Code:

Avisource("F:\v\XMen2.avi")
#ShowFrameNumber    # EDIT: Only to find flash frames, no longer needed
#return last

FrameRateConverter(Debug=True)
#FrameRateConverter(Output="Auto")
#FrameRateConverter(Output="Flow")
#FrameRateConverter(Output="None")
#FrameRateConverter(Output="Mask")
#FrameRateConverter(Output="Skip")
#FrameRateConverter(Output="Raw")
#FrameRateConverter(Output="Over")

chainik_svp · 28th April 2017, 09:08

ok, I've testes JM's clip and I must say:
1. don't use "algo=13" (default in Interframe w/o "smooth" preset)!
2. higher mask.area values lead to more blurry image (Interframe uses 150 in "smooth" preset, which is too high)

so, Interframe:
- default mode: algo=13, mask.area=0
- smooth preset: algo=23, mask.area=150

BOTH modes are BAD, at least for this extremely low-quality source
this doesn't mean they're ALWAYS bad, I believe SubJunk made a lot of comparisons when he selected these values

set algo=21/23, mask.area <= 50 and you'll be fine

kolak · 28th April 2017, 13:45

To get some values which work with specific source is 1 thing, to find good general settings is very different.

StainlessS · 28th April 2017, 14:57

MX, your times for fast are probably quite a bit off, bug in fast processing for prefilter.

Mod

Code:

    CalcPrefilter = Defined(Prefilter) || Pset != P_FAST    # If !Defined(Prefilter) && P_FAST : Prefilter MUST be Input C
    # Prefilter  = Default(Prefilter, C.RemoveGrain(22) )
    Prefilter  = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C)
    # EDIT: OR alternative: Prefilter  = Default(Prefilter, !P_FAST ? C.RemoveGrain(22) : C)

Am doing a few changes (mainly cosmetic).

EDIT: Post #207 updated as per above.
+ Asserts show error value.

EDIT: In addition to above bug, both SuperFilt and Super would both be from C.RemoveGrain(22), ie quality loss,
output always Prefiltered. Above fix fixes that too.
Extreme care should be taken if making alterations anywhere in vicinity of those settings.

MysteryX · 28th April 2017, 14:57

Comparison with prefilter and artifact masking disabled.

FrameRateConverter(60, output="auto", prefilter=last, masktrh=0, skipover=0)

InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, OverrideArea=0, Cores=1)

chainik_svp · 28th April 2017, 15:53

"OverrideArea" IS zero by default... which means "override" is off

===
BTW, here's another sample video

It's almost impossible with MVTools to get rid of this:

this thing is not static, it jumps all over the striped vest randomly

(well, there's one solution - set search radius to 1, but it's not great)

MysteryX · 28th April 2017, 16:39

Whole frame blurring is one thing. Artifact masking is another.

I'll do some test later with 1080p source that has severe artifacts problems.

This is the script generated by Interframe, which I isolated earlier; but then didn't know how to convert settings to/from MFlow syntax

Code:

function InterFrameProcess(clip Input) {
	SuperString = "{scale:{up:0,down:4},gpu:1,rc:false}"
	VectorsString = "{block:{w:8,overlap:2},main:{search:{distance:0,coarse:{distance:-10,bad:{sad:2000}}}},refine:[{thsad:250}]}"
	SmoothString = "{rate:{num:60,den:1,abs:true},algo:23,mask:{area:150,area_sharp:1.2},scene:{blend:true, mode:0}}"

	# Make interpolation vector clip
	Super   = SVSuper(Input, SuperString)
	Vectors = SVAnalyse(Super, VectorsString)

	# Put it together
	smooth_video = SVSmoothFps(Input, Super, Vectors, SmoothString, url="www.svp-team.com", mt=1)
	smooth_video
}

chainik_svp · 28th April 2017, 17:11

in your message above:

Quote:

Comparison with prefilter and artifact masking disabled.

artifact masking for Interframe IS NOT disabled
it's still equal to "150"

MysteryX · 28th April 2017, 17:59

HD sample: Girl's Day - Female President

FrameRateConverter(60, output="over", blksize=32)
FrameRateConverter(60, output="over", blksize=16)
FrameRateConverter(60, output="over", blksize=16, preset="slow")
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1)
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1, overridearea=1)

Frame 3497

Frame 8900

Note: DCT=1 is prohibitively slow in 1080p!!!

Note: DCT's darker images is because it produces stronger artifact masks and darker means the frames will be skipped -- but those frames actually look better

Note: For frame 8900, most such frames are skipped by my script, but this bad frame does go through -- as an exception

Performance:

blksize=32

Code:

FPS (min | max | average):      3.728 | 137555 | 18.62
Memory usage (phys | virt):     1389 | 1395 MiB
Thread count:                   29
CPU usage (average):            70%

blksize=16

Code:

FPS (min | max | average):      3.314 | 111354 | 15.65
Memory usage (phys | virt):     1483 | 1496 MiB
Thread count:                   29
CPU usage (average):            66%

blksize=16, preset="slow"

Code:

FPS (min | max | average):      0.040 | 53146 | 0.123
Memory usage (phys | virt):     1465 | 1466 MiB
Thread count:                   26
CPU usage (average):            44%

EDIT: I just had an idea!

If frames are "skipped" at blksize=16, I could fallback to blksize=32, and if it fails again, then skip. It would be a bit more processing but only on bad frames, and frame 8900 would then render correctly.

MysteryX · 28th April 2017, 22:45

Here's the problem with DCT mask strength

DCT=0

DCT=1

Now I have to find a way to adjust to DCT=1 mask strength. Average luminosity is almost twice higher!!

Update: I'm finding it hard to do something decent with DCT=1. The raw image looks nicer, but then the mask is also much stronger, and while it removes some artifacts, it creates others. Sometimes it puts lines in subtitles, and to remove it, the mask treshold must be high, which causes a lot more artifact removal to occur; and thus more frame blending and more double shadows.

Combining the facts that...
- DCT=1 is gruelly slow (see benchmark above)
- DCT=1 generates much stronger artifact masks
- DCT=1 creates some ugly artifacts preventing lowering the mask treshold much

...

I'm considering dropping DCT altogether.

Here's what I managed to achieve.

DCT=0 / DCT=1 with MaskTrh-40

If I lower the treshold by 50, then I get artifacts like this in the text

kolak · 29th April 2017, 00:43

Quote:

Originally Posted by MysteryX

HD sample: Girl's Day - Female President

FrameRateConverter(60, output="over", blksize=32)
FrameRateConverter(60, output="over", blksize=16)
FrameRateConverter(60, output="over", blksize=16, preset="slow")
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1)
InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1, overridearea=1)

Frame 3497

Frame 8900

Note: DCT=1 is prohibitively slow in 1080p!!!

Note: DCT's darker images is because it produces stronger artifact masks and darker means the frames will be skipped -- but those frames actually look better

Note: For frame 8900, most such frames are skipped by my script, but this bad frame does go through -- as an exception

Well, this is exactly my problem with interframe- double edges. Don't have this issue with mvtools.

MysteryX · 29th April 2017, 01:04

Mask strength discrepancy isn't only an issue with YV12/YV24, and DCT=0 or 1, but also blksize 8, 16 and 32 produce a different mask strength...

Right now it is tweaked for YV12 with blksize=16. A different format or block size won't give as good artifact masking.

MysteryX · 29th April 2017, 04:43

Does anyone *ever* use non-square blocks? Handling 16x8 and 32x16 would make mask strength adjustments considerably more complex.

Any objection to dropping BlkSizeV?

Instead, what I'm working on is fallback mode, so you can process in 16x16, and fallback to either 8x8 or 32x32 before reverting back to "hard blending" if it fails too. It's just that doing such fallback with 16x8 would make it considerably more complex.

For SD videos, 16x16 with fallback to 8x8 works best, while for HD videos, 16x16 with fallback to 32x32 works best, so both options must be available.

MysteryX · 29th April 2017, 05:48

OK I made a bunch of changes again.

I removed StainlessS's debug code and added a new debug option that displays AverageLuma details in the corner, in any output mode.

I removed DCT. Preset Slow is still there but it's not currently in use.

I made some adjustments based on the block size. Now it should work fine with any block sizes and any video sizes.

Code:

MaskTrh = MaskTrh + (BlkSize == 8 ? 30 : BlkSize == 32 ? -60 : 0)
SkipTrh = SkipTrh + (BlkSize == 8 ? 25 : BlkSize == 32 ? -25 : 0)

I removed BlkSizeV.

I added Fallback so that we try a different block size before skipping. 1 to try larger, -1 to try smaller, 0 to disable. For some videos, I found it to make nearly no difference. For other videos, however, a considerable amount of Fallback frames are coming through. Please test and give your feedback. You can view Fallback frames with Debug=true.

In theory, the fallback mode should only fetch the frames when Skip is triggered, but in practice, it makes the whole script run twice slower, which indicates it is calculating the fallback frame every time. Any way to improve this? Also, Fallback+Debug causes the info to be written twice on top of each other; anyone is welcomed to fix this.

Code:

# Frame Rate Converter
# Version: 28-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools (default prefilter), GRunT (output="over")
##
## @ NewNum      - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen      - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset      - The speed/quality preset [slow|normal|fast]. (default=normal)
##
## @ BlkSize     - The block size (default = Width>1600||Height>1200 ? 32 : Width>720||C.Height>480 ? 16 : 8)
## 
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output      - Output mode [auto|inter|none|mask|skip|raw|over] (default=auto)
##                 auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; mask=mask only; 
##                 skip=mask used by SkipOver; raw=raw mask; over=mask as cyan overlay for debugging
##
## @ MaskGam     - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh     - The treshold where a block is considered bad, between 0 and 255.
##                 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc     - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver    - Skip interpolation of frames when artifacts cover more than specified treshold, 
##                 0 to disable. (Default=16)
##
## @ Prefilter   - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##
## @ Debug       - Whether to display AverageLuma values of Skip, Mask and Raw. (Default=false)
##
## @ Fallback    - When SkipOver is triggered, -1 to try with smaller block size, 
##                 1 to try with smaller block size, or 0 to disable. (Default=0)
##
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", 
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", 
\ int "SkipOver", clip "Prefilter", bool "Debug", int "Fallback")
{
    Preset = Default(Preset, "normal")
    P_SLOW = 0  P_NORMAL = 1  P_FAST = 2
    Pset = Preset == "slow" ? P_SLOW : Preset == "normal" ? P_NORMAL : Preset == "fast" ? P_FAST : -1
    Assert(Pset != -1, "FrameRateConverter: 'Preset' must be slow, normal or fast {'" + Preset + "'}")
    Output = Default(Output, "auto")
    O_AUTO = 0  O_FLOW = 1  O_NONE = 2  O_MASK = 3  O_SKIP = 4  O_RAW = 5  O_OVER = 6
    OPut = Output == "auto" ? O_AUTO : Output == "flow" ? O_FLOW : Output == "none" ? O_NONE : Output == "mask" ? O_MASK :
        \ Output == "skip" ? O_SKIP : Output == "raw" ? O_RAW : Output == "over" ? O_OVER : -1
    Assert(OPut != -1, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|raw|over) {'" + Output + "'}")
    
    FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
    NewNum     = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
    NewDen     = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
    BlkSize    = Default(BlkSize, C.Width>1600||C.Height>1200 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
    MaskGam    = Default(MaskGam, .5)
    MaskTrh    = Default(MaskTrh, 120)
    MaskOcc    = MaskTrh > 0 ? Default(MaskOcc, 150) : 0
    SkipTrh    = 155
    SkipOver   = Default(SkipOver, 16)
    CalcPrefilter = Defined(Prefilter) || Pset != P_FAST
    Prefilter  = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C)
    Debug      = Default(Debug, false)
    #Fallback   = Default(Fallback, C.Width>934||C.Height>700 ? (BlkSize<32?1:-1) : (BlkSize>8?-1:1))
    Fallback   = Default(Fallback, 0)
    OutFps     = OPut!=O_MASK && OPut!=O_SKIP && OPut!=O_RAW  # Whether output will have altered frame rate

    Assert(BlkSize==8 || BlkSize==16 || BlkSize==32, String(BlkSize, "FrameRateConverter: BlkSize must be 8, 16 or 32 {%.f}"))
    Assert(MaskGam > 0.0 && MaskGam <= 1.0, String(MaskGam, "FrameRateConverter: MaskGam must be between 0.0 and 1.0 {%.1f}"))
    Assert(MaskTrh >= 0 && MaskTrh <= 255, String(MaskTrh, "FrameRateConverter: MaskTrh must be between 0 and 255 {%.f}"))
    Assert(MaskOcc >= 0 && MaskOcc <= 255, String(MaskOcc, "FrameRateConverter: MaskOcc must be between 0 and 255 {%.f}"))
    Assert(SkipOver >= 0 && SkipOver <= 255, String(SkipOver, "FrameRateConverter: SkipOver must be between 0 and 255 {%.f}"))
    Assert((Fallback >= -1 && Fallback <= 1) || Fallback == 69, String(SkipOver, "FrameRateConverter: Fallback must be -1, 0 or 1 {%.f}"))
    Assert(Fallback != -1 || BlkSize != 8, "FrameRateConverter: Fallback cannot be -1 with BlkSize = 8")
    Assert(Fallback != 1 || BlkSize != 32, "FrameRateConverter: Fallback cannot be 1 with BlkSize = 32")

    # Performance settings: slow, normal, fast
    Recalculate = PSET <= P_NORMAL

    ## "B" - Blending, "BHard" - No blending
    BHard = Fallback == 1 || Fallback == -1 ? C.FrameRateConverter(NewNum, NewDen, Preset, 
    \  Fallback == -1 ? BlkSize/2 : BlkSize*2, FrameDouble, Output, MaskGam, MaskTrh, MaskOcc, SkipOver, Prefilter, Debug, 69) 
    \  : C.ChangeFPS(NewNum, NewDen)
    B = C.ConvertFPS(NewNum, NewDen)

    ## Adjust parameters for different block sizes, causing stronger or weaker masks
    MaskTrh = MaskTrh + (BlkSize == 8 ? 30 : BlkSize == 32 ? -60 : 0)
    SkipTrh = SkipTrh + (BlkSize == 8 ? 25 : BlkSize == 32 ? -25 : 0)

    ## jm_fps interpolation
    superfilt = MSuper(prefilter, hpad=16, vpad=16) # all levels for MAnalyse
    super = CalcPrefilter ? MSuper(C, hpad=16, vpad=16, levels=1) : superfilt # one level is enough for MRecalculate
    bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, overlap = BlkSize>8?4:BlkSize>4?2:0, search=3, dct=0)
    fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, overlap = BlkSize>8?4:BlkSize>4?2:0, search=3, dct=0)
    fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, overlap = BlkSize>8?2:0, thSAD=100) : fwd
    bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, overlap = BlkSize>8?2:0, thSAD=100) : bak
    Flow = MFlowFps(C, super, bak, fwd, num=NewNum, den=NewDen, blend=false, ml=200, mask=2, thSCD2=255)

    ## "EM" - error or artifact mask
    # Mask: SAD
    EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255).ConvertToY8() : 
    \                  BlankClip(C, pixel_type="Y8", color_yuv=$000000)
    # Mask: Temporal blending
    EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
    EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=.6, mode="lighten", pc_range=true) : EM
    # Mask: Occlusion
    EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
    \                     .ConvertToY8().mt_inpand() : BlankClip(C, pixel_type="Y8", color_yuv=$000000)
    EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
    OutRaw = EM
    
    ## Mask processing
    EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSize/4.0)*4)
    \       .mt_expand(mode= mt_circle(zero=true, radius=1))
    EMskip = EM.mt_binarize(255-SkipTrh)
    EM = EM.mt_binarize(255-MaskTrh)
    \       .Blur(.6)
    \       .BicubicResize(C.Width, C.Height)
    OutSkip = EMskip.BicubicResize(C.width, C.Height)
    ShowEM = EM
    
    ## "Sc" - scene detection / SkipOver
    Sc = SkipOver > 0 ? ConditionalFilter(EMskip,BlankClip(EM, color_yuv=$ff0000),BlankClip(EM,color_yuv=$000000), 
    \    "AverageLuma", ">", string(SkipOver)) : BlankClip(EM, color_yuv=$000000)

    ## Convert masks to desired frame rate
    EM = OutFps ? EM.ChangeFPS(NewNum, NewDen) : EM
    Sc = OutFps ? Sc.ChangeFPS(NewNum, NewDen) : Sc
   
    ## the FrameRateConverter magic happens
    M = mt_merge(Flow, B, EM, luma=true, chroma="process")
    M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M
    
    R= (Oput==O_AUTO)                               [** auto: artifact masking     *]
    \     ? (FrameDouble ? Interleave(C, SelectOdd(M)) : M)
    \ : (Oput==O_FLOW)                              [** flow: interpolation only   *]
    \     ? Flow
    \ : (Oput==O_NONE)                              [** none: ConvertFPS only      *]
    \     ? B
    \ : (Oput==O_MASK)                              [** mask: mask only            *]
    \     ? EM
    \ : (Oput==O_SKIP)                              [** skip: skip mask            *]
    \     ? OutSkip
    \ : (Oput==O_RAW)                               [** raw:  raw mask             *]
    \     ? OutRaw
    \ : (Oput==O_OVER)                              [** over: mask as cyan overlay *]
    \     ? mt_merge(
    \         Flow.Overlay(MergeRGB(BlankClip(EM, color_yuv=$000000), EM, EM), mode="Add", opacity=0.40, pc_range=true),
    \         BlankClip(Flow, color=$B8860B), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
    \ : Assert(false, "FrameRateConverter: 'Output' INTERNAL ERROR")

    # Display AverageLuma values of Skip, Mask and Raw
    ShowSkip = OutFps ? EMskip.ChangeFPS(NewNum, NewDen) : EMskip
    ShowRaw = OutFps ? OutRaw.ChangeFPS(NewNum, NewDen) : OutRaw
    R = Debug ? R.GScriptClip("""(Fallback == 69 || Fallback == 0 || Sc.AverageLuma < 255) ? 
    \           Subtitle("Skip:  " + string(ShowSkip.AverageLuma) + "\nMask: " + string(EM.AverageLuma) + 
    \           "\nRaw:  " + string(ShowRaw.AverageLuma) + 
    \           (Fallback == 69 && Sc.AverageLuma == 0 ? "\nFallback" : "") +
    \           ((Fallback == 69 || Fallback == 0) && Sc.AverageLuma == 255 ? "\nSkipped" : ""), lsp=0) : last""",
    \           args = "ShowSkip,EM,ShowRaw,Sc,Fallback", Local=true) : R
    return R
}

EDIT: The debug display bug is fixed.

28th April 2017, 01:04	#202 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	Just tried FrameDouble again, and everything is fine. No, I'm not getting any weird flash. What ColorSpace are you using? (YV12 here) What Avisynth version are you using? Btw I'm trying to split long lines so that it displays nicely in the browser. Your script is also working fine here, no frame jumps. However, "SC" isn't displaying and "Mask" and "EM" are displaying the same. I believe Mask should be the raw mask. I think it's showing too much to be useful. 4 views would be more than enough. Or perhaps, on top of "over", to have "debug" which shows 4 views and "debug2" which shows 9 views ... but isn't it starting to be overkill when the script is already ready and working? One advantage of your "debug" mode is to help others understand what this script is doing under the hood. It took me a while to understand the basic YFRC script. If you're going to display 9 clips, I'd recommend this order, from top-left to bottom-right: Output, MFlow, Blending, Raw, Mask, Skip --- these 6 are enough. The 3 others are redundant and unnecessary. If we display 4 clips, Output, Over, Raw, Skip -- or something like that. But first, what would be the intent? With the script working, it's only about tweaking settings by comparing details. If too many clips are showing up the details of each clip are too small to see and compare. Edit: OK I see some frames not matching the others. That happens because some clips are 15fps while others are 60fps. If you display a 15fps clip next to a 60fps clip, you're going to see such mismatch. __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 28th April 2017 at 01:37.

28th April 2017, 01:17	#203 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	See edit EDIT: I'm using AVS standard, YV12. With test script given including ShowFrameNumber. M clip bottom RHS debug is jumping 5000 -> 10000 -> 5000 etc. (original frame numbers) Only just updated to latest mvtools and masktools, was getting exact same type thing with older versions (several versions ago) EDIT: OverKill, not when your trying to find a bug (and its called debug mode). EDIT: 20 frames flashing, ~ 200KB:- http://www.mediafire.com/file/cj3kwpaq748bwzm/MX.mp4 With Output=Auto. Selur has been posting about the flashes in vapoursynth forum. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 28th April 2017 at 01:42.

28th April 2017, 01:52	#204 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	I see no problem with AVS 2.6 here... nor with FrameDouble. I did see a bug though: with FrameDouble, scene changes aren't being applied. __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 28th April 2017 at 01:55.

28th April 2017, 01:57	#205 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	Code: FlowX=FrameDouble ? SelectOdd(Flow) : Flow RT_DebugF("Flow_Rate=%f B_Rate=%f EM_Rate=%f",FlowX.FrameRate,B.FrameRate,EM.FrameRate) RT_DebugF("BHard_Rate=%f SC_Rate=%f",BHard.FrameRate,SC.FrameRate) ## the FrameRateConverter magic happens M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process") RT_DebugF("M_1 Rate=%f",M.FrameRate) M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M RT_DebugF("M_2 Rate=%f",M.FrameRate) Debug output Code: [3888] RT_DebugF: Flow_Rate=25.000000 B_Rate=25.000000 EM_Rate=50.000000 [3888] RT_DebugF: BHard_Rate=50.000000 SC_Rate=50.000000 [3888] RT_DebugF: M_1 Rate=25.000000 [3888] RT_DebugF: M_2 Rate=25.000000 M_2 Rate=25.0 but that is probably just because the first clip framerate FlowX was 25.0, The Mt_merge stuff obviously aint working proper there. I think that mt_merge pair is where the flashes originate, result of first one mangles things up, but result is 25.0 fps and probably no longer than first clip (FlowX). EDIT: Re-did above debug output, more than one problem there EDIT: As when DoubleRate you use SelectOdd(Flow), so should not all of the other clips be SelectOdd, however, B has already had SelectOdd applied. Also, would that mean that when NOT double rate, then no SelectOdd should be applied to any, including B. I'm not sure I understand what its supposed to do so I'm not sure how to correct it. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 28th April 2017 at 02:39.

28th April 2017, 02:53	#206 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	I think the whole "SelectOdd" and "FrameDelete" in the processing parts of the clip are unnecessary. It's only at the end that you interlace source and processed clips. Processed clip will only be generated for such requested frames. I'll make those changes tomorrow. Is 60fps working for you, or are both modes failing? __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer

28th April 2017, 09:08	#208 \| Link
chainik_svp Registered User Join Date: Mar 2012 Location: Saint-Petersburg Posts: 239	ok, I've testes JM's clip and I must say: 1. don't use "algo=13" (default in Interframe w/o "smooth" preset)! 2. higher mask.area values lead to more blurry image (Interframe uses 150 in "smooth" preset, which is too high) so, Interframe: - default mode: algo=13, mask.area=0 - smooth preset: algo=23, mask.area=150 BOTH modes are BAD, at least for this extremely low-quality source this doesn't mean they're ALWAYS bad, I believe SubJunk made a lot of comparisons when he selected these values set algo=21/23, mask.area <= 50 and you'll be fine __________________ SVPflow motion interpolation

28th April 2017, 13:45	#209 \| Link
kolak Registered User Join Date: Nov 2004 Location: Poland Posts: 2,843	To get some values which work with specific source is 1 thing, to find good general settings is very different.

28th April 2017, 14:57	#210 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	MX, your times for fast are probably quite a bit off, bug in fast processing for prefilter. Mod Code: CalcPrefilter = Defined(Prefilter) \|\| Pset != P_FAST # If !Defined(Prefilter) && P_FAST : Prefilter MUST be Input C # Prefilter = Default(Prefilter, C.RemoveGrain(22) ) Prefilter = Default(Prefilter, CalcPrefilter ? C.RemoveGrain(22) : C) # EDIT: OR alternative: Prefilter = Default(Prefilter, !P_FAST ? C.RemoveGrain(22) : C) Am doing a few changes (mainly cosmetic). EDIT: Post #207 updated as per above. + Asserts show error value. EDIT: In addition to above bug, both SuperFilt and Super would both be from C.RemoveGrain(22), ie quality loss, output always Prefiltered. Above fix fixes that too. Extreme care should be taken if making alterations anywhere in vicinity of those settings. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 28th April 2017 at 15:47.

28th April 2017, 14:57	#211 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	Comparison with prefilter and artifact masking disabled. FrameRateConverter(60, output="auto", prefilter=last, masktrh=0, skipover=0) InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, OverrideArea=0, Cores=1) __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer

28th April 2017, 15:53	#212 \| Link
chainik_svp Registered User Join Date: Mar 2012 Location: Saint-Petersburg Posts: 239	"OverrideArea" IS zero by default... which means "override" is off === BTW, here's another sample video It's almost impossible with MVTools to get rid of this: this thing is not static, it jumps all over the striped vest randomly (well, there's one solution - set search radius to 1, but it's not great) __________________ SVPflow motion interpolation Last edited by chainik_svp; 28th April 2017 at 16:16.

28th April 2017, 17:59	#215 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	HD sample: Girl's Day - Female President FrameRateConverter(60, output="over", blksize=32) FrameRateConverter(60, output="over", blksize=16) FrameRateConverter(60, output="over", blksize=16, preset="slow") InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1) InterFrame(Tuning="Smooth", NewNum=60, NewDen=1, GPU=true, Cores=1, overridearea=1) Frame 3497 Frame 8900 Note: DCT=1 is prohibitively slow in 1080p!!! Note: DCT's darker images is because it produces stronger artifact masks and darker means the frames will be skipped -- but those frames actually look better Note: For frame 8900, most such frames are skipped by my script, but this bad frame does go through -- as an exception Performance: blksize=32 Code: FPS (min \| max \| average): 3.728 \| 137555 \| 18.62 Memory usage (phys \| virt): 1389 \| 1395 MiB Thread count: 29 CPU usage (average): 70% blksize=16 Code: FPS (min \| max \| average): 3.314 \| 111354 \| 15.65 Memory usage (phys \| virt): 1483 \| 1496 MiB Thread count: 29 CPU usage (average): 66% blksize=16, preset="slow" Code: FPS (min \| max \| average): 0.040 \| 53146 \| 0.123 Memory usage (phys \| virt): 1465 \| 1466 MiB Thread count: 26 CPU usage (average): 44% EDIT: I just had an idea! If frames are "skipped" at blksize=16, I could fallback to blksize=32, and if it fails again, then skip. It would be a bit more processing but only on bad frames, and frame 8900 would then render correctly. __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 28th April 2017 at 18:09.

28th April 2017, 22:45	#216 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	Here's the problem with DCT mask strength DCT=0 DCT=1 Now I have to find a way to adjust to DCT=1 mask strength. Average luminosity is almost twice higher!! Update: I'm finding it hard to do something decent with DCT=1. The raw image looks nicer, but then the mask is also much stronger, and while it removes some artifacts, it creates others. Sometimes it puts lines in subtitles, and to remove it, the mask treshold must be high, which causes a lot more artifact removal to occur; and thus more frame blending and more double shadows. Combining the facts that... - DCT=1 is gruelly slow (see benchmark above) - DCT=1 generates much stronger artifact masks - DCT=1 creates some ugly artifacts preventing lowering the mask treshold much ... I'm considering dropping DCT altogether. Here's what I managed to achieve. DCT=0 / DCT=1 with MaskTrh-40 If I lower the treshold by 50, then I get artifacts like this in the text __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 28th April 2017 at 23:38.

29th April 2017, 01:04	#218 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	Mask strength discrepancy isn't only an issue with YV12/YV24, and DCT=0 or 1, but also blksize 8, 16 and 32 produce a different mask strength... Right now it is tweaked for YV12 with blksize=16. A different format or block size won't give as good artifact masking. __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer

29th April 2017, 04:43	#219 \| Link
MysteryX Soul Architect Join Date: Apr 2014 Posts: 2,559	Does anyone ever use non-square blocks? Handling 16x8 and 32x16 would make mask strength adjustments considerably more complex. Any objection to dropping BlkSizeV? Instead, what I'm working on is fallback mode, so you can process in 16x16, and fallback to either 8x8 or 32x32 before reverting back to "hard blending" if it fails too. It's just that doing such fallback with 16x8 would make it considerably more complex. For SD videos, 16x16 with fallback to 8x8 works best, while for HD videos, 16x16 with fallback to 32x32 works best, so both options must be available. __________________ FrameRateConverter \| AvisynthShader \| AvsFilterNet \| Natural Grounding Player with Yin Media Encoder, 432hz Player, Powerliminals Player and Audio Video Muxer Last edited by MysteryX; 29th April 2017 at 04:46.