SVPflow motion interpolation (Avisynth + Vapoursynth, Linux, Mac?)

[markfilipak]

SVPflow was recommended to me by a fellow on the ffmpeg-user list.
I reckon that it's a library, not an application. If so, what is the prerequisite?

I've been developing ffmpeg transcodes based on this:
ffmpeg -i INPUT -filter_complex "minterpolate ..." ... OUTPUT

A typical transcode of a 2 hour movie takes 3 days on a 3.6GHz, 8 thread CPU with both INPUT & OUTPUT on an SSD-RAID0. Why? Because minterpolate is single-threaded and there's no GPU support -- I have an NVIDIA GTX M960M GPU with 1356 CUDA cores.

Related (OT): The ffmpeg telecine filter has a memory leak that kills lengthy transcodes. When combined with minterpolate, transcodes of 5 minute videos (which take ~3 hours) exhaust swap and don't complete -- my RAM is 32GB and swap is 128GB. Any alternative suggestion would be greatly appreciated.

Thanks!

[johnmeyer]

I suggest you look at the original SVPFlow thread:

SVPflow motion interpolation

Post #1 includes links to the various downloads. I haven't clicked on that in years, so I don't know if all the prerequisites (other DLLs, etc.) are included, but hopefully they are, and you can then get everything in one click.

[markfilipak]

(Hmmm... I'm still not getting notifications...)

Thanks John,

Originally Posted by johnmeyer

I suggest you look at the original SVPFlow thread:

SVPflow motion interpolation

Post #1 includes links to the various downloads. I haven't clicked on that in years, so I don't know if all the prerequisites (other DLLs, etc.) are included, but hopefully they are, and you can then get everything in one click.

Nope. There's no links to applications that support SVPflow. I think they include Avisynth and Avisynth+ and Vapoursynth (and maybe more). I once downloaded Vaporsynth -- I don't remember why I chose it -- but there's no installer, the start menu just contains fluff. I think there's some controversy regarding Avisynth versus Avisynth+ versus Vapoursynth but I can't remember what they are. It all looks too disorganized and I've already lost too much of my remaining lifetime with ffmpeg.

Are there any wise souls here who can advise me? I'm just trying to get the best quality transcodes possible. I'm not streaming. I've run Handbrake many times but it makes fairly low quality transcodes and doesn't provide for ffmpeg functions to be pasted into the workflow. MKVToolNix GUI is wonderful, but remuxes solely.

Holy Smoke! I just downloaded 'http://www.spirton.com/uploads/InterFrame/20130218-Sample-InterFrame.mkv' & 'http://www.spirton.com/uploads/InterFrame/20130218-Sample-Original.mkv'. This is what I've been trying to do for almost a year to various reactions: "Why do you want to do that?" "60Hz TVs are inherently bad" "You don't know what you're doing" "That's just how movies look on TV" "What judder?" etc. ffmpeg -vf minterpolate is too slow (a 2 hour video to 60FPS in 3 days!). How do I replicate what I downloaded?

[manolito]

Hi Mark,

I think that johnmeyer is the main guy to answer your questions, but he is probably too shy to advertise his own frame interpolation scripts...

Whatever, I'll do my best to give you a short overview about what is currently available for AviSynth:

1. MotionProtectedFPS
Based on a plugin by Clouded, very fast even for real time playback. Causes way more artifacts than the MVTools2 based solutions. The source needs to be mod8.

2. Interframe based solutions (SVPFlow)
Optimized for speed to make it work for real time playback. More artifacts than the slower MVTools2 based scripts by johnmeyer and MysteryX.

3. jm_fps by johnmeyer
Derived from an MVTools2 example script, but using optimized parameters. Delivers very nice results. The following scripts are based on it.

4. mx_fps based on Frame Rate Converter by MysteryX
This is a simplified and stripped down version of Frame Rate Converter. It also works under classic AVS 2.6, no support for Hi Bit and Hi Color (only 8 bits and 4:2:0 chroma supported). Works under WinXP and does not require a SSE2 capable CPU.

5. Frame Rate Converter by MysteryX
Adds artifact masking and a few other improvements to the johnmeyer script. Sometimes useful, sometimes not. Very tweakable.


I have uploaded a couple of test conversions using the source you have linked to. This source is not very demanding for frame interpolation, so even the simpler and faster scripts do a pretty good job on it. FWIW I also included AIO packages for MotionProtectedFPS, jm_fps and mx_fps. If you want to play with them just make sure that you don't have duplicate DLLs in your AviSynth\Plugins folder which contain identical function names. If you already have the RGTools plugin in your autoload folder then do not add RemoveGrain.dll to your audoload folder...



Have fun
manolito

[markfilipak]

Originally Posted by manolito

Hi Mark,

Hi Manalito,

Thanks for the overview.

I'm fair-use transcoding movies I own for my media server. I seek the best possible transcodes to show on a 60Hz TV -- I hate judder! My laptop has an NVIDIA GTX 980M GPU, so any solution should use it.

What I've been doing -- I'm leaving some things out for simplicity:
This:
ffmpeg -i INPUT -filter_complex "minterpolate=fps=60/1.001" OUTPUT
goes straight to 60fps but takes 1-1/2 days per hour of running time. It also makes some minor errors.
This:
ffmpeg - INPUT -filter_complex "minterpolate=fps=48/1.001, telecine=pattern=3322, pp=linblenddeint" OUTPUT
goes to 48fps (so fewer errors), then telecines, then blends the combed frame -- good enough but still takes 1 day per hour of running time.

minterpolate is single-threaded, no GPU use. I think SVPflow would be better, faster.

I've done a lot of reading. I can't figure out the relationship between Avisynth, Avisynth+, Vapoursynth, and Python.

My case is probably pretty strange since I'm starting with a desired filter (SVPflow) and then trying to figure out how to use it. It's a bit like walking into "The Matrix" half way through and asking, "Who's the guy with the gun? And how can he fly?"

Got any advice (or links) to help figure out the path with heart?

[markfilipak]

Originally Posted by manolito

https://www.sendspace.com/file/823xkp

I download that stuff. Thanks, but...

'20130218-Sample-Original.mkv'
- 24pps@24/1.001fps
- h264 (High), yuv420p(progressive), 1280x534 [SAR 1:1 DAR 640:267], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
- 1697 frames

'20130218-Sample-InterFrame.mkv'
- 60pps(interpolated)@60/1.001fps
- Motion compensated target that is incredibly beautiful.
- h264 (High), yuv420p(progressive), 1280x534 [SAR 1:1 DAR 640:267], 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)
- 4241 frames

'jm_fps.mkv'
- 24pps(3322telecined)@60/1.001fps
- Extreme judder
- h264 (High), yuv420p(progressive), 1280x532, SAR 1:1 DAR 320:133, 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)
- 4242 frames

'mx_fps.mkv'
- 24pps(3322telecined)@60/1.001fps
- Extreme judder
- h264 (High), yuv420p(progressive), 1280x532, SAR 1:1 DAR 320:133, 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)
- 4242 frames

'motionprotectedfps.mkv'
- 24pps(3322telecined)@60/1.001fps
- Extreme judder
- h264 (High), yuv420p(progressive), 1280x528, SAR 1:1 DAR 80:33, 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)
- 4241 frames

Is there something you wanted me to see because what I did see made me ill.

[manolito]

Sorry, I made these tests using StaxRip, and I must have inserted the FPS conversion command in the wrong place. Please disregard these clips and use the ones from here instead:
https://www.sendspace.com/file/e74ism

[markfilipak]

Originally Posted by manolito

Sorry, I made these tests using StaxRip, and I must have inserted the FPS conversion command in the wrong place. Please disregard these clips and use the ones from here instead:
https://www.sendspace.com/file/e74ism

That's okay, you probably have as many balls in the air as I do. However...

20130218-Sample-Original.mkv - 1697 frames
20130218-Sample-Original.mkv - h264 (High), yuv420p(progressive), 1280x534 [SAR 1;1 DAR 640;267],
20130218-Sample-Original.mkv - 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)

20130218-Sample-InterFrame.mkv - 4241 frames
20130218-Sample-InterFrame.mkv - h264 (High), yuv420p(progressive), 1280x534 [SAR 1;1 DAR 640;267],
20130218-Sample-InterFrame.mkv - 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)

jm_fps.mkv - 4238 frames
jm_fps.mkv - h264 (High), yuv420p(progressive), 1280x528 [SAR 3439;3478 DAR 137560;57387],
jm_fps.mkv - SAR 88;89 DAR 640;267, 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)
** Frames have been lost
** Frame height has been truncated to 33 slices
** SAR is wrong

motionprotectedfps.mkv - 4238 frames
motionprotectedfps.mkv - h264 (High), yuv420p(progressive), 1280x528 [SAR 3439;3478 DAR 137560;57387],
motionprotectedfps.mkv - SAR 88;89 DAR 640;267, 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)
** Frames have been lost
** Frame height has been truncated to 33 slices
** SAR is wrong

mx_fps.mkv - 4240 frames
mx_fps.mkv - h264 (High), yuv420p(progressive), 1280x528 [SAR 3439;3478 DAR 137560;57387],
mx_fps.mkv - SAR 88;89 DAR 640;267, 59.94 fps, 59.94 tbr, 1k tbn, 119.88 tbc (default)
** Frames have been lost
** Frame height has been truncated to 33 slices
** SAR is wrong

[manolito]

These were quick and dirty conversions via ffmpeg command line. I made the source mod8 first because motion.dll requires it. This should explain the slight SAR offset. The source filter was DSS2Mod which is known to skip a frame at the end quite often.

The important thing to look for is the smoothness of the motion, and you need to look for the typical interpolation artifacts. As I said before, this source is not very demanding, so even the MotionProtectedFPS result looks pretty good to me. If you feed the interpolators with a demanding source like Anime or with moving vertical edges (like driving along a fence) you certainly will see lots of artifacts.



//EDIT//
I have to correct myself. When I said that this source is not too demanding for frame interpolation I was not looking close enough...
The problems start in the scene where she goes into the house. When she takes off her coat have a look at her arms. And it gets worse when she walks to the stairs. The camera rolls with her so her face stays pretty much in the center of the frames. But the background moves all the time, and the background is not in focus. A typical problem for frame interpolation. All the scripts I tested show heavy artifacts, they all revert to blending. And still they all exhibit warped lines. It is a tradeoff between motion judder or artifacts, pick your poison.

[markfilipak]

Originally Posted by manolito

These were quick and dirty conversions via ffmpeg command line.

Howdy. That's what I'm trying to do but I don't know how to form the ffmpeg command. Specifically, I don't know how to incorporate svpflow (either one). If I knew how, I'd simply buy them from SVP but so far the SVP folks have not responded.

Do you care to share your ffmpeg command line, Sir?
And is there a special build to make it happen? The nightly git builds don't seem to incorporate SVP.

I have a fairly recent NVIDIA GPU and the latest driver.

Thanks,
Mark.

PS: I have a terrific, 20 second clip.

[manolito]

Originally Posted by markfilipak

PS: I have a terrific, 20 second clip.

Please upload it, I like to play with such stuff...


For the ffmpeg command line, I think this is a misunderstanding. I used ffmpeg just to invoke its built-in X264 encoder. All the frame interpolation stuff is handled by AviSynth.

As you already found out, the ffmpeg "minterpolate" filter is not really usable. All the frame interpolate filters I was talking about are based on AviSynth, and the same goes for SVPFlow.

For my conversion I abused an older tool which can change the speed of video clips. It creates an AVS script which is then fed as the source to an ffmpeg command line.

ChangeSpeed.avs:

new_fps = 23.976 * 100 / 100
video = DSS2("F:\Download\20130218-Sample-Original.mkv", fps=23.976, preroll=15)
audio = DirectShowSource("F:\Download\20130218-Sample-Original.mkv", video=false)
AudioDub(video, audio)
Crop(0,0,-Width % 4,-Height % 4)
ConvertToYV12()
AssumeFPS(new_fps)
jm_fps(59.94)

ffmpeg command:

ffmpeg.exe -i changespeed.avs -vcodec libx264 -aspect 2.397 -crf 19 -preset medium -tune film -acodec libfdk_aac -ac 6 "F:\Download\20130218-Sample-Original_speed=100%.mkv"

Bottom line is that for all the interpolation scripts I talked about you will need to use AviSynth. Do you have Avisynth installed at all? If so, do you use AVS+ (which can do multitasking)?

[Ceppo]

Is there YV24 support?

[manolito]

Not for jm_fps and my stripped down version of the original FrameRateConverter mx_fps (because I nneded it to run on ancient machines without SSE2 capable CPUs running classic AVS 2.6 under WinXP). For SVPFlow see here:
https://forum.doom9.org/showthread.p...97#post1811797

But the original FrameRateConverter by MysteryX running under AVS+ and using the latest plugin versions which support high bitdepth and Hi Color should support YV24 (did not test it myself).

[StainlessS]

From Manolito post 311

new_fps = 23.976 * 100 / 100
video = DSS2("F:\Download\20130218-Sample-Original.mkv", fps=23.976, preroll=15)
audio = DirectShowSource("F:\Download\20130218-Sample-Original.mkv", video=false)
AudioDub(video, audio)
Crop(0,0,-Width % 4,-Height % 4)
ConvertToYV12()
AssumeFPS(new_fps)
jm_fps(59.94)

Just thought I'de point out that
new_fps = 23.976 * 100 / 100 = 23.976002 #with a bit of precision rounding error --- 100 not a power of 2 [should be same ie 23.976].
# not sure what is intended.

blankclip(width=320,height=240)
old_fps=23.976
new_fps = old_fps * 100 / 100
Subtitle( string(old_fps) + " : " +  string(new_fps),size=32)

result


EDIT: Should it be 24000 / 1001.0, ~ 23.97602398 [or 23.9760239760239760239760 ... ie 239760 Recurring]

EDIT:

Originally Posted by markfilipak

Specifically, I don't know how to incorporate svpflow (either one).

I have the same question
Can you just use the old script with current dll's ?
EDIT: this one, InterFrame2.avsi, mine seem to be from 2015, current dlls' from 2019.
Latest script by subjunk appears to be this one:- https://forum.doom9.org/showthread.php?p=1486831

[Selur]

I stumbled over what chainik_svp wrote (https://forum.doom9.org/showthread.p...#post1574161):

sup3 = last.msuper(pel=2,sharp=2,hpad=0,vpad=0)

# SVAnalyse accepts MSuper with zero padding!
v31=sup3.SVAnalyse("{"+search_params+"}")
v32=sup3.SVAnalyse("{"+search_params+",special:{delta:2}}")
v33=sup3.SVAnalyse("{"+search_params+",special:{delta:3}}")

and tried using it with:

ClearAutoloadDirs()
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\LSMASHSource.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\svpflow1_64.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\mvtools2.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\masktools2.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\DePanEstimate.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\DePan.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\RgTools.dll")
LoadPlugin("I:\Hybrid\64bit\Avisynth\AVISYN~1\MedianBlur2.dll")
Import("I:\Hybrid\64bit\Avisynth\avisynthPlugins\mtmodes.avsi")

# loading source: G:\TestClips&Co\files\test.avi
# color sampling YV12@8, matrix: bt601, scantyp: progressive, luminance scale: limited
LWLibavVideoSource("G:\TESTCL~1\files\test.avi",cache=false,dr=true,format="YUV420P8", prefer_hw=0)

FixRipsP2SVP()
# Converting from 8 to 10bit for encoder
ConvertBits(10)
# setting output fps to 25.000fps
AssumeFPS(25,1)
PreFetch(16)
#  output: color sampling YV12@10, matrix: bt601, scantyp: progressive, luminance scale: limited
return last

function FixRipsP2SVP(clip a)
{
    a
    clense(reduceflicker=false).merge(last,0.5).clense(reduceflicker=false)
    mot=removegrain(11,0).removegrain(20,0).DepanEstimate(range=2)
    take2=a.depaninterleave(mot,prev=2,next=2,subpixel=2)
    clean1=take2.TMedian2().selectevery(5,2)
    search_params="gpu: 1, block:{w:4,overlap:2}" # can't use blocksize 4, w:4 <> blocksize 8

    sup1 = clean1.minblur(1).removegrain(11,0).removegrain(11,0)
     \           .mt_lutxy(clean1,"x 1 + y < x 2 + x 1 - y > x 2 - y ? ?",U=2,V=2)
    sup1 = sup1.SVSuper("{ pel: 2, gpu: 1, scale: { up: 0 } }")
    v21=sup1.SVAnalyse("{"+search_params+"}")
    fv21=v21.SVConvert(isb=false)
    bv21=v21.SVConvert(isb=true)

    sup2 = a.msuper(pel=2,levels=1,sharp=2)
    v22=sup1.SVAnalyse("{"+search_params+", special: { delta: 2 } }")
    bv22=v22.SVConvert(isb=true)
    fv22=v22.SVConvert(isb=false)

    interleave(a.mcompensate(sup2,fv22),a.mcompensate(sup2,fv21),a,a.mcompensate(sup2,bv21),a.mcompensate(sup2,bv22))
    TMedian2().selectevery(5,2)

    sup3 = last.msuper(pel=2,sharp=2,hpad=0,vpad=0)
    # SVAnalyse accepts MSuper with zero padding!
    v31=sup3.SVAnalyse("{"+search_params+"}")
    bv31=v31.SVConvert(isb=true)
    fv31=v31.SVConvert(isb=false)
    
    v32=sup3.SVAnalyse("{"+search_params+",special:{delta:2}}")
    bv32=v32.SVConvert(isb=true)
    fv32=v32.SVConvert(isb=false)
    
    v33=sup3.SVAnalyse("{"+search_params+",special:{delta:3}}")
    bv33=v33.SVConvert(isb=true)
    fv33=v33.SVConvert(isb=false)

    last.mdegrain3(sup3,bv31,fv31,bv32,fv32,bv33,fv33,thSAD=499)

    interleave()
    return last
}


function MinBlur(clip clp, int r, int "uv")
{
    uv   = default(uv,3)
    uv2  = (uv==2) ? 1 : uv
    rg4  = (uv==3) ? 4 : -1
    rg11 = (uv==3) ? 11 : -1
    rg20 = (uv==3) ? 20 : -1
    medf = (uv==3) ? 1 : -200

    RG11D = (r==0) ? mt_makediff(clp,clp.sbr(),U=uv2,V=uv2)
     \    : (r==1) ? mt_makediff(clp,clp.removegrain(11,rg11),U=uv2,V=uv2)
     \    : (r==2) ? mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20),U=uv2,V=uv2)
     \    :          mt_makediff(clp,clp.removegrain(11,rg11).removegrain(20,rg20).removegrain(20,rg20),U=uv2,V=uv2)
    RG4D  = (r<=1) ? mt_makediff(clp,clp.removegrain(4,rg4),U=uv2,V=uv2)
     \    : (r==2) ? mt_makediff(clp,clp.medianblur(2,2*medf,2*medf),U=uv2,V=uv2)
     \    :          mt_makediff(clp,clp.medianblur(3,3*medf,3*medf),U=uv2,V=uv2)
    DD    = mt_lutxy(RG11D,RG4D,"x 128 - y 128 - * 0 < 128 x 128 - abs y 128 - abs < x y ? ?",U=uv2,V=uv2)
    clp.mt_makediff(DD,U=uv,V=uv)
    return(last)
}

# median of 5 clips from helpers.avs by G-force
function Median2(clip "input_1", clip "input_2", clip "input_3", clip "input_4", clip "input_5", string "chroma")
{
    chroma = default(chroma,"process") #default is "process". Alternates: "copy first" or "copy second"
    #MEDIAN(i1,i3,i5)
    Interleave(input_1,input_3,input_5)
    chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
    m1 = selectevery(3,1)
    #MAX(MIN(i1,i3,i5),i2)
    m2  = input_1.MT_Logic(input_3,"min",chroma=chroma).MT_Logic(input_5,"min",chroma=chroma).MT_Logic(input_2,"max",chroma=chroma)
    #MIN(MAX(i1,i3,i5),i4)
    m3  = input_1.MT_Logic(input_3,"max",chroma=chroma).MT_Logic(input_5,"max",chroma=chroma).MT_Logic(input_4,"min",chroma=chroma)
    Interleave(m1,m2,m3)
    chroma == "process" ? Clense(reduceflicker=false) : Clense(reduceflicker=false,grey=true)
    selectevery(3,1)
    chroma == "copy first" ? last.MergeChroma(input_1) : chroma == "copy second" ? last.MergeChroma(input_2) : last
    Return(last)
}

function TMedian2(clip c) {
    Median2(c.selectevery(1,-2), c.selectevery(1,-1), c, c.selectevery(1,1), c.selectevery(1,2) )
}

But my problem is that

sup3 = last.msuper(pel=2,sharp=2,hpad=0,vpad=0)
    # SVAnalyse accepts MSuper with zero padding!
    v31=sup3.SVAnalyse("{"+search_params+"}")

aborts with "SVAnalyse, non-valid number of levels".


-> Does anyone know how to fix this?

Cu Selur

[StainlessS]

You usin' DTL mod of Mvtools [I think he made some test changes which might possibly be cause] ?

Have you tried something like add Levels=0 or Levels=5 to mSuper().
Think 0 is default which might be internally converted to 5, maybe, possibly, perhaps.

[Selur]

Have you tried something like add Levels=0 or Levels=5 to mSuper().

yes, tried 0-6.

You usin' DTL mod of Mvtools [I think he made some test changes which might possibly be cause] ?

Nope, I tried with https://github.com/pinterf/mvtools.
(Now, I also tried with mvtools2_asb1_PT1std_msvc.dll, but that didn't change anything).