SVPflow motion interpolation (Avisynth + Vapoursynth, Linux, Mac?)

[Keiyakusha]

Originally Posted by Reel.Deel

To be fair, Chainik_svp did post a disclaimer.

Why you telling this to me? Tell it to those who want to "use SVP in complex scripts". Personally I'm not interested in this project that much.

[kolak]

Originally Posted by Reel.Deel

@ chainik_svp

Enjoy your summer!!

+1

[johnmeyer]

[Edit]See later post for more up-to-date attempt (which still doesn't work ...).

Trying to replace MFlow in a motion-compensated dirt removal script, but not succeeding. Here's the original function:

function RemoveDirtMC(clip,int "limit", bool "_grey")
{
  _grey=default(_grey, false)
  limit = default(limit,6)
  i=MSuper(clip,pel=2)
  bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
  fvec = MAnalyse(i,isb=true,  blksize=8, delta=1, truemotion=true)
  backw = MFlow(clip,i,bvec)
  forw  = MFlow(clip,i,fvec)
  clp=interleave(forw,clip,backw)
  clp=clp.RemoveDirt(limit,_grey)
  clp=clp.SelectEvery(3,1)
  return clp
}

And this is my attempt to re-write:

function RemoveDirtMCSVP(clip,int "limit", bool "_grey")
{
  svp_flow_lib="C:\Program Files\AviSynth 2.5\plugins\MVTools\SVP Version\libflowgpu.dll"
  _grey=default(_grey, false)
  limit = default(limit,6)
  i=MSuper(clip,pel=2)
  bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
  fvec = MAnalyse(i,isb=true,  blksize=8, delta=1, truemotion=true)
  backw = MSmoothFps (clip,i,fvec, bvec, algo=1)  #These two lines appear to be the problem
  forw  = MSmoothFps (clip,i,fvec, bvec, algo=1)
  clp=interleave(forw,clip,backw)
  clp=clp.RemoveDirt(limit,_grey)
  clp=clp.SelectEvery(3,1)
  return clp
}

I think the problem is that, despite what the SVP doc says, MSmoothFPS doesn't really encapsulate ALL of the functionality of the various MVTools2 functions.

My purpose in doing this is initially to improve performance, but also perhaps also (eventually) improve quality by adding some of the refinements in the updated MVTools2 (which I installed) and SVP itself.

Any help would be appreciated. Thanks!

[Reel.Deel]

Hi John, to succesfully use SVP with MVTools2 you have to use SVAnalyze and SVConvert.

From the documentation:

SVConvert(vectors, isb: bool) Convert SVAnalyse output to the format of MAnalyse compatible with "client" MVTools 2.5 functions. 

vectors - motion vectors data produced by SVAnalyse function, 
isb - which vectors to extracts: forward if isb=false, backward if isb=true. 

Example:
   super = SVSuper(super_params)
   vectors = SVAnalyse(super, analyse_params)
   
   forward_mv = SVConvert(vectors, false)
   backward_mv = SVConvert(vectors, true)
   
   super_mv = MSuper(pel=1, hpad=0, vpad=0) #padding should be zero here!
   MFlowFps(super_mv, backward_mv, forward_mv, num=60, den=1)

* Edit *

Also, for SVSuper, SVAnalyze, and SVConvert you only need svpflow1.dll.

[johnmeyer]

Originally Posted by Reel.Deel

Hi John, to succesfully use SVP with MVTools you have to use SVAnalyze and SVConvert

I have used that code in other projects, but I was lead to believe that the special MVTools2 build available at the SVP project site lets you use MVTools2 vectors. Here's the sample code that is included with that download:

SetMemoryMax(1024)

pel=2
recalc=true

svp_flow_lib="..\libflowgpu.dll"
LoadPlugin ("..\mvtools2.dll")

SetMTMode(3,10)
DirectShowSource("path\to\video.avi")
ConvertToYV12()
SetMTMode(2)

super=MSuper(pel=pel)
finest=pel==1 ? super : MFinest(super)

backward_vec=MAnalyse(super, isb=true, blksize=16, overlap=4)
forward_vec=MAnalyse(super, isb=false, blksize=16, overlap=4)

backward_vec = recalc==0 ? backward_vec : MRecalculate(super, backward_vec, blksize=8, overlap=2)
forward_vec  = recalc==0 ? forward_vec  : MRecalculate(super, forward_vec, blksize=8, overlap=2)

MSmoothFps(super, backward_vec, forward_vec, finest=finest, num=60, den=1, algo=13, sadml=200, blend=false)

The implication from the last line in this code is that MSmoothFPS can use the vectors from this version of MVTools2.

While I'm awaiting further replies, I'll try your suggestions. Also, this is a later version of my attempt to get it to work (but still not working):

function RemoveDirtMCSVP(clip,int limit, bool "_grey")
{
  svp_flow_lib="C:\Program Files\AviSynth 2.5\plugins\MVTools\SVP Version\libflowgpu.dll"
  _grey=default(_grey, false)
  limit = default(limit,6)
  i=MSuper(clip,pel=2)
  bvec = MAnalyse(i,isb=false, blksize=8, delta=1, truemotion=true)
  fvec = MAnalyse(i,isb=true,  blksize=8, delta=1, truemotion=true)
  backw = MSmoothFps (clip,i,fvec, algo=1,num=0, den=1,block=false)
  forw  = MSmoothFps (clip,i,bvec, algo=1,num=0, den=1,block=false)
  clp=interleave(forw,clip,backw)
  clp=clp.RemoveDirt(limit,_grey)
  clp=clp.SelectEvery(3,1)
  return clp
}

[Reel.Deel]

Ahh I see, you're trying to use the SVP MVTools and not the SVPFlow plugin, correct?

[johnmeyer]

Originally Posted by Reel.Deel

Ahh I see, you're trying to use the SVP MVTools and not the SVPFlow plugin, correct?

Yes, but my goal, as I stated above, is to improve performance and, possibly, quality. Therefore, I am not "stuck" on doing it this way. The reason I decided on this path is that it looked like I might not only be able to improve the performance in this part of the script, but also another part that uses MDegrain. I know there is a way to use SVP with MDegrain, even though it is a barely-supported afterthought to the main goal of the project (frame interpolation), but using this special MVtools2 build seemed like it might be more direct and have fewer problems.

Still working on it ...

[johnmeyer]

Getting closer, but still no joy. The video is 16 fps progressive. This looks like it should work, but I don't think algo=2 does anything different than algo=1. Neither line interpolates anything, but instead just gives me the original. I can change num to 32 and then get interpolation. I suppose I could do this for this frame, do the same thing for the previous frame (trim(last,0)) but that seems wrong somehow.

In the code below, the

interleave(backw,clip,forw)

line gives me three identical frames, i.e., I'm not getting an intermediate estimated frame as I would get with MFlow.

function RemoveDirtMCSVP(clip,int limit, bool "_grey")
{
  svp_flow_lib="C:\Program Files\AviSynth 2.5\plugins\MVTools\SVP Version\libflowgpu.dll"
  _grey=default(_grey, false)
  limit = default(limit,6)
  i=MSuper(clip,pel=2)
  bvec = MAnalyse(i,isb=true, blksize=8, delta=1, truemotion=true)
  fvec = MAnalyse(i,isb=false,  blksize=8, delta=1, truemotion=true)
  backw = MSmoothFps (clip,i,bvec,fvec, algo=2,num=16, den=1,block=false)
  forw  = MSmoothFps (clip,i,bvec,fvec, algo=1,num=16, den=1,block=false)
  clp=interleave(forw,clip,backw)
  clp=clp.RemoveDirt(limit,_grey)
  clp=clp.SelectEvery(3,1)
  return clp
}

[Didée]

The behavior is correct. MSmoothFPS is a framerate changer. And the wrong filter for what you want to do.

You have a 16fps clip. If you use num=32,den=1, then MSmoothFPS converts 16fps to 32fps by inserting 1 interpolated frame between each two original frames.
If you set num=16,den=1, then MSSmoothFPS converts 16fps to 16fps, by doing nothing. That's perfectly fine.

What you need to use is MFlow, or (rather) MCompensate. But not MSmoothFPS / MFlowFPS.

[johnmeyer]

Originally Posted by Didée

What you need to use is MFlow, or (rather) MCompensate. But not MSmoothFPS / MFlowFPS.

I have come to the same conclusion. However, the reason I thought this would work was the following statement on the SmoothVideo Project MVTools 2 Page:

MSmoothFps [is] a completely new function that encapsulates MFlow, MFlowFps and MBlockFps functionality and with GPU support enabled is almost indefinitely faster than MFlowFps.

So, wanting to get better performance, and being a reasonable person, I took this to mean that I could get this "indefinitely faster" performance when estimating forward and backwards prior to using an external denoiser, in this case RemoveDirt.

I still think there might be a way to do what I want if I first decimate every other frame, then use MSmoothFPS to double the framerate from this decimated version and, following that step, use what is now an interpolated frame. This will certainly work, but I don't know how to get the estimation to work both in the backward and forward direction. Maybe I can get the second estimated frame that I need by working on a clip that is played backwards ...

So, if I get some time later today I'm going to try this route.

[edit] Oh, upon further thinking, decimation followed by interpolation won't work at all because it's like using a delta of 2 and the estimated frame won't be very accurate, even though it will be in the correct temporal position.

[chainik_svp]

johnmeyer
Sorry I'm too late...

By "encapsulates MFlow" I meant "it can replace MFlow in 'low-end' motion interpolation scripts", like this:
===
f50=src.MFlow()
interleave(src,f50)
===

Thanks for pointing me to the other use cases of MFlow.
May be it'll be implemented in some future version cause this's rather simple to add

Still you can use SVAnalyse's vectors with MFlow...

=======
BTW that one-way compensation (with MFlow) can't take any real improvements from GPU acceleration cause it's just trivial.

[johnmeyer]

Thanks for the reply!

[kolak]

chainik_svp any progress in SVP development?

[chainik_svp]

Originally Posted by kolak

chainik_svp -any news about improvements to scene change detection?

could you remind me what's wrong with scene detection?

[kolak]

Well- not reliable- you (or SubJunk) also said this
Can you borrow code from eg x264? It's just not working very well- when set to low it misses scene changes, when set to high than there are false detection in the middle of one the scene and motion is broken.

[SubJunk]

kolak: After thinking about it a lot, I think 100% accurate scene-change detection is impossible. The reason is just that in high-action scenes, two frames in the same scene can be as different as a scene change even though they aren't a scene change. So it means any program will always struggle.
However, this works out to be a good thing in my opinion because if we try to interpolate two frames in the same scene that are very different, the results will be even worse than a simple frame-blend, because there is nothing to estimate from, so every estimate will be incorrect and look funny with warps, tearing, etc.
Those are my thoughts anyway

[Nick [D]vB]

Hi all, I've only just discovered SVP, I have not had time to test anything yet but it looks very interesting.

Sorry if this has been asked before but I was just wondering if you could use DGDecNV to fully accelerate all the non-avisynth stuff?

http://neuron2.net/dgdecnv/dgdecnv.html

Would it help reduce CPU usage much, has anyone tried it?

thanks

[SubJunk]

I used to use it all the time, it does increase the speed of loading/reading the input file.

[Nick [D]vB]

But not the CPU usage so much?

[SubJunk]

Yeah it would lower the CPU usage with this