Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th April 2017, 16:40   #1  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Frame rate interpolation artifact problem

I got a sample forInterpolation.mp4 and when trying to get it from 29,97 fps to 60fps, I get a lot of artifacts in the fence that is visible.

I tried Interframe/SVP for Vapoursynth and I also stumbled over:
Code:
clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)
sup = core.mv.Super(clip, pel=2, hpad=0, vpad=0)
bvec = core.mv.Analyse(sup, blksize=16, isb=True, chroma=True, search=3, searchparam=1)
fvec = core.mv.Analyse(sup, blksize=16, isb=False, chroma=True, search=3, searchparam=1)
clip = core.mv.FlowFPS(clip, sup, bvec, fvec, num=60, den=1, mask=2)
#clip = core.mv.BlockFPS(clip, sup, bvec, fvec, num=60, den=1, mask=2)
problem is all those methods have lots of problems with fences.

-> Is there a solution for this?

Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 20th April 2017, 16:49   #2  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,234
There is this active thread about a script with basically the same purpose:
https://forum.doom9.org/showthread.php?t=174410

The script MysteryX is working on still needs tweaking I think. I played around with it myself an came up with the script in this post (same thread as above):
https://forum.doom9.org/showthread.p...37#post1804437

It's a lot more full on than your script, and a little more full on that MysteyX's combined script, but I found it handled some things better.
burfadel is offline   Reply With Quote
Old 20th April 2017, 16:52   #3  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Thanks will try those scripts, I somehow overlooked that thread.
Ah, okay I mainly looked for a way using Vapoursynth.
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..

Last edited by Selur; 20th April 2017 at 16:55.
Selur is online now   Reply With Quote
Old 20th April 2017, 17:10   #4  |  Link
burfadel
Registered User
 
Join Date: Aug 2006
Posts: 2,234
I forgot to mention in the post that modplus plugin is required for the modded script:
http://www.avisynth.nl/users/vcmohan...s/modPlus.html
burfadel is offline   Reply With Quote
Old 20th April 2017, 17:19   #5  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Thanks for the info
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 20th April 2017, 17:35   #6  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Just tried it with the default values and it produced some even worse artifacts than any of the others.


Cu Selur
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 25th April 2017, 16:59   #7  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 462
Hi Selur. That script is a Avisynth script right? Were you able to use it in Vapoursynth? I vaguely remember that Vapoursynth is able to load avs scripts, but my memory is rusty regarding this and Google is failing me.
aegisofrime is offline   Reply With Quote
Old 25th April 2017, 17:02   #8  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Only tried the script from MysterX only using Avisynth MT and not Vapoursynth.
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 27th April 2017, 09:20   #9  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,592
Quote:
Originally Posted by Selur View Post
Only tried the script from MysterX only using Avisynth MT and not Vapoursynth.
It's much better now with MysteryX latest version and SkipOver=20, but the fence is still a challenge
Sharc is offline   Reply With Quote
Old 27th April 2017, 17:37   #10  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Thanks for the info, really busy in real life atm. but will check it out at the weekend.
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 27th April 2017, 18:56   #11  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
argh, did a fast check with 'FrameRateConverter(SkipOver=20)' and the output is jumping between the next frame and a frame multiple frames in the past,...
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 27th April 2017, 19:31   #12  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,592
Hmmm...I observed the jerkiness as well, before I resized to 1280x720 or set output="flow", I think.
Here my script which I used for comparison:
Code:
clip=DGSource("C:\Temp\Selur\forInterpolation.dgi")
clip=clip.bilinearresize(1280,720)
conv1=clip.changefps(60000,1001).subtitle("ChangeFPS",x=20,y=20,size=32)
conv2=clip.framerateconverter(NewNum=60000,NewDen=1001,preset="normal",BlkSize=32,output="auto",SkipOver=25).subtitle("FrameRateConverter",x=20,y=20,size=32)
conv3=clip.interframe(cores=4, gpu=false, newnum=60000, newden=1001, tuning="film").subtitle("Interframe",x=20,y=20,size=32)

return stackhorizontal(conv2,conv3)
Sharc is offline   Reply With Quote
Old 27th April 2017, 19:37   #13  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Code:
framerateconverter(NewNum=60000,NewDen=1001,preset="normal",BlkSize=32,output="auto",SkipOver=25)
is crap to, because it simply repeats each frame which could be done a lot faster
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 27th April 2017, 19:43   #14  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,592
No way.
conv1 duplicates, conv2 and conv3 interpolate.

Note that SkipOver=25 replaces the worst pictures only by duplicates. Set it to 0 in order to interpolate all frames.

Last edited by Sharc; 27th April 2017 at 19:51.
Sharc is offline   Reply With Quote
Old 27th April 2017, 19:59   #15  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,993
Note that I use it on the full resolution version and:
Code:
# Frame Rate Converter
# Version: 26-Apr-2017
# By Etienne Charland
# Based on Oleg Yushko's YFRC artifact masking,
# johnmeyer's frame interpolation code, and
# raffriff42's "weak mask" and output options.
# Special thanks to Pinterf for adding 16-bit support to MvTools2 and MaskTools2
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA, or visit
# http:#www.gnu.org/copyleft/gpl.html.

#######################################################################################
### Frame Rate Converter
### Increases the frame rate with interpolation and strong artifact removal.
##
## YV12/YV24/Y8/YUY2
## Requires: MaskTools2, MvTools2, rgtools
##
## @ NewNum      - The new framerate numerator (if FrameDouble = false, default = 60)
##
## @ NewDen      - The new framerate denominator (if FrameDouble = false, default = 1)
##
## @ Preset      - The speed/quality preset [slow|normal|fast|faster]. (default=normal)
##
## @ BlkSize     - The horizontal block size (default = Width>1200||Height>900 ? 32 : Width>720||C.Height>480 ? 16 : 8)
## 
## @ BlkSizeV    - The vertical block size (default = BlkSize)
##
## @ FrameDouble - Whether to double the frame rate and preserve original frames (default = true)
##
## @ Output      - Output mode [auto|inter|none|mask|skip|over] (default=auto)
##                 auto=normal artifact masking; flow=interpolation only; none=ConvertFPS only; 
##                 mask=mask only; skip=mask used by SkipOver; over=mask as cyan overlay for debugging
##
## @ MaskGam     - A gamma to be applied to the raw mask, between 0 and 1. (Default=.5)
##
## @ MaskTrh     - The treshold where a block is considered bad, between 0 and 255.
##                 0 to disable artifact masking. (Default=120)
##
## @ MaskOcc     - Occlusion mask treshold, between 0 and 255. 0 to disable occlusion masking. (Default=150)
##
## @ SkipOver    - Skip interpolation of frames when artifacts cover more than specified treshold, 
##                 0 to disable. (Default=16)
##          
## @ Prefilter   - Specified a custom prefiltered clip. (Default=RemoveGrain(22))
##          
function FrameRateConverter(clip C, int "NewNum", int "NewDen", string "Preset", int "BlkSize", int "BlkSizeV", 
\ bool "FrameDouble", string "Output", float "MaskGam", int "MaskTrh", int "MaskOcc", int "SkipOver", clip "Prefilter")
{
    Output     = Default(Output, "auto")
    FrameDouble= Default(FrameDouble, Defined(NewNum) ? false : true)
    NewNum     = FrameDouble ? C.FrameRateNumerator * 2 : Default(NewNum, 60)
    NewDen     = FrameDouble ? C.FrameRateDenominator : Default(NewDen, 1)
    Preset     = Default(Preset, "normal")
    BlkSize    = Default(BlkSize, C.Width>1200||C.Height>900 ? 32 : C.Width>720||C.Height>480 ? 16 : 8)
    BlkSizeV   = Default(BlkSizeV, BlkSize)
    blkmin     = BlkSize > BlkSizeV ? BlkSizeV : BlkSize
    MaskGam    = Default(MaskGam, .5)
    MaskTrh    = Default(MaskTrh, 120)
    MaskOcc    = Default(MaskOcc, 150)
    MaskOcc    = MaskTrh > 0 ? MaskOcc : 0
    SkipOver   = Default(SkipOver, 16)
    CalcPrefilter = Defined(Prefilter) || Preset != "faster"
    Prefilter  = Default(Prefilter, Preset != "faster" ? C.RemoveGrain(22) : C)

    Assert(Preset == "slow" || Preset == "normal" || Preset == "fast" || Preset == "faster", 
    \      "FrameRateConverter: Preset must be slow, normal, fast or faster")
    Assert(BlkSize == 8 || BlkSize == 16 || BlkSize == 32, "FrameRateConverter: BlkSize must be 8, 16 or 32")
    Assert(BlkSizeV == 8 || BlkSizeV == 16 || BlkSizeV == 32, "FrameRateConverter: BlkSizeV must be 8, 16 or 32")
    Assert(MaskGam > 0 && MaskGam <= 1, "FrameRateConverter: MaskGam must be between 0 and 1")
    Assert(MaskTrh >= 0 && MaskTrh <= 255, "FrameRateConverter: MaskTrh must be between 0 and 255")
    Assert(MaskOcc >= 0 && MaskOcc <= 255, "FrameRateConverter: MaskOcc must be between 0 and 255")
    Assert(SkipOver >= 0 && SkipOver <= 255, "FrameRateConverter: SkipOver must be between 0 and 255")

    # Performance settings: slow, normal, fast, faster
    Recalculate = preset == "slow" || preset == "normal"
    DCT = preset == "slow" ? 1 : 0

    ## "B" - Blending, "BHard" - No blending
    BHard = C.ChangeFPS(NewNum, NewDen)
    B  = C.ConvertFPS(NewNum, NewDen)
    B  = FrameDouble ? SelectOdd(B) : B

    ## jm_fps interpolation
    superfilt = MSuper(prefilter, hpad = 16, vpad = 16) # all levels for MAnalyse
    super = CalcPrefilter ? MSuper(C, hpad = 16, vpad = 16, levels = 1) : superfilt # one level is enough for MRecalculate
    bak = MAnalyse(superfilt, isb=true, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
    fwd = MAnalyse(superfilt, isb=false, blksize=BlkSize, blksizeV=BlkSizeV, overlap = blkmin>8?4:blkmin>4?2:0, search=3, dct=DCT)
    fwd = Recalculate ? MRecalculate(super, fwd, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : fwd
    bak = Recalculate ? MRecalculate(super, bak, blksize=BlkSize/2, blksizeV=BlkSizeV/2, overlap = blkmin>8?2:0, thSAD=100) : bak
    Flow = MFlowFps(C, super, bak, fwd, num = NewNum, den = NewDen, blend = false, ml = 200, mask = 2, thSCD2=255)

    ## "EM" - error or artifact mask
    # Mask: SAD
    EM = MaskTrh > 0 ? C.MMask(bak, ml=255, kind=1, gamma=1.0/MaskGam, ysc=255, thSCD2=255) : BlankClip(C)
    EM = EM.ConvertToY8()
    # Mask: Temporal blending
    EMfwd = MaskTrh > 0 ? C.MMask(fwd, ml=255, kind=1, gamma=1.0/MaskGam, thSCD2=255).ConvertToY8() : EM
    EMfwd = FrameDouble ? EMfwd.DeleteFrame(0) : EMfwd
    EM = MaskTrh > 0 ? EM.Overlay(EMfwd, opacity=0.5, mode="lighten") : EM
    # Mask: Occlusion
    EMocc = MaskOcc > 0 ? C.MMask(bak, ml=255-MaskOcc, kind=2, gamma=1.0/MaskGam, ysc=255, thSCD2=255)
    \                     .ConvertToY8().mt_inpand() : BlankClip(C)
    EM = MaskOcc > 0 ? EM.Overlay(EMocc, opacity=.4, mode="lighten", pc_range=true) : EM
    
    ## Mask processing
    EM = EM.BicubicResize(Round(C.Width/BlkSize/4.0)*4, Round(C.Height/BlkSizeV/4.0)*4)
    \       .mt_expand(mode= mt_circle(zero=true, radius=1))
    EMskip = EM.mt_binarize(100)
    EM = EM.mt_binarize(255-MaskTrh)
    \       .Blur(.6)
    \       .BicubicResize(C.Width, C.Height)
    EMskipOut = EMskip.BicubicResize(C.width, C.Height).ScriptClip("Subtitle(string(AverageLuma()))")
    
    ## "Sc" - scene detection / SkipOver
    Sc = SkipOver > 0 ? ConditionalFilter(EMskip, BlankClip(EM, color=$FFFFFF), BlankClip(EM), 
    \    "AverageLuma", ">", string(SkipOver)) : BlankClip(EM)
    Sc = Sc.mt_binarize(128)

    # Display SkipOver value on Output="over
    Global GEMskip = EMskip.ChangeFPS(NewNum, NewDen)
    FlowOver = Flow.ScriptClip("Subtitle(string(GEMskip.AverageLuma()))")

    ## Convert masks to desired frame rate
    EM = EM.ChangeFPS(NewNum, NewDen)
    Sc = Sc.ChangeFPS(NewNum, NewDen)

    ## the FrameRateConverter magic happens
    M = mt_merge(FrameDouble ? SelectOdd(Flow) : Flow, B, EM, luma=true, chroma="process")
    M = SkipOver > 0 ? mt_merge(M, BHard, Sc, luma=true, chroma="process") : M

    R = (StrCmpi(Output, "auto")==0)        [** auto:  artifact masking     *] 
    \     ? (FrameDouble ? Interleave(C, M) : M)
    \ : (StrCmpi(Output, "flow")==0)        [** flow: interpolation only   *] 
    \     ? Flow
    \ : (StrCmpi(Output, "none")==0)        [** none: ConvertFPS only       *] 
    \     ? B
    \ : (StrCmpi(Output, "mask")==0)        [** mask:  mask only            *] 
    \     ? EM
    \ : (StrCmpi(Output, "skip")==0)        [** skip:  skip mask          *] 
    \     ? EMskipOut
    \ : (StrCmpi(Output, "over")==0)        [** over:  mask as cyan overlay *] 
    \     ? mt_merge(
    \         FlowOver.Overlay(MergeRGB(BlankClip(EM), EM, EM), mode="Add", opacity=0.40, pc_range=true),
    \         BlankClip(Flow, color=color_darkgoldenrod), Sc.mt_lut("x 2 / "), luma=true, chroma="process")
    \ : Assert(false, "FrameRateConverter: 'Output' not one of (auto|flow|none|mask|skip|over)")
    return R
}
LoadPlugin("G:\Hybrid\AVISYN~1\LSMASHSource.dll")
LoadPlugin("G:\Hybrid\AVISYN~1\masktools2.dll")
LoadPlugin("G:\Hybrid\AVISYN~1\mvtools2.dll")
LoadPlugin("G:\Hybrid\AVISYN~1\rgtools.dll")
# loading source: C:\Users\Selur\Desktop\forInterpolation.mp4
#  input luminance scale tv
LWLibavVideoSource("C:\Users\Selur\Desktop\forInterpolation.mp4",cache=false,stacked=true,format="YUV420P8",repeat=true)
# current resolution: 3840x2160
framerateconverter(NewNum=60000,NewDen=1001,preset="normal",BlkSize=32,output="auto")
return last
still leaves me with every (aside from second frame a duplicate aside from the first)
Will do some testing over the weekend,..
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is online now   Reply With Quote
Old 28th April 2017, 10:07   #16  |  Link
Sharc
Registered User
 
Join Date: May 2006
Posts: 3,592
I couldn't reproduce the frame jumping anymore, but apparently Stainless experienced it as well (and fixed it in his version).
For comparison I encoded your clip with x264 and streamed it to my 50" TV. My subjective ranking:
1. (best): ConvertFPS (blending)
2. FrameRateConverter (MysteryX/Stainless version, default settings except SkipOver=0)
3. ChangeFPS (frame duplication). Probably the TV applied its own interpolation here, introducing its own artefacts. I didn't tweak the TV options, maybe later ...
4. (worst): Interframe/SVPflow (tune="film")

Edit:
I have to revise my rating. Unless one focuses on the fence but rather on the girl on the bike, the viewing impression is:
1. (winner): FrameRateConverter (MysteryX/Stainless version, default settings except SkipOver=0)
2. ConvertFPS (blending)
3. Change FPS (with interpolation by the TV)
4. Interframe/SVPflow (the fence artefacts are dominant)

Last edited by Sharc; 28th April 2017 at 10:59.
Sharc is offline   Reply With Quote
Old 29th April 2017, 09:29   #17  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,176
Quote:
Originally Posted by Sharc View Post
1. (winner): FrameRateConverter (MysteryX/Stainless version, default settings except SkipOver=0)


Now try with the new fallback option -- which I designed specifically for a fence-like problem I was having. It didn't actually help much there, but it's helping in other places.
MysteryX is offline   Reply With Quote
Old 29th April 2017, 23:49   #18  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 462
Quote:
Originally Posted by Sharc View Post
I couldn't reproduce the frame jumping anymore, but apparently Stainless experienced it as well (and fixed it in his version).
For comparison I encoded your clip with x264 and streamed it to my 50" TV. My subjective ranking:
1. (best): ConvertFPS (blending)
2. FrameRateConverter (MysteryX/Stainless version, default settings except SkipOver=0)
3. ChangeFPS (frame duplication). Probably the TV applied its own interpolation here, introducing its own artefacts. I didn't tweak the TV options, maybe later ...
4. (worst): Interframe/SVPflow (tune="film")

Edit:
I have to revise my rating. Unless one focuses on the fence but rather on the girl on the bike, the viewing impression is:
1. (winner): FrameRateConverter (MysteryX/Stainless version, default settings except SkipOver=0)
2. ConvertFPS (blending)
3. Change FPS (with interpolation by the TV)
4. Interframe/SVPflow (the fence artefacts are dominant)
Did you manage to run it in Vapoursynth? How's the speed like?
aegisofrime is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:33.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.