How to double framerate?

[smsmasters]

I'm playing some h264 hd movies in the mkv container format @ 23.975fps.

I've tried ffdshow's framerate doubler but it adds "double images" and doesn't look too good.

Can somone recommend some good avisynth plugins for doubling framerate to make motion smoother?

Thank you

[mgh]

mvflowfps and mvflowfps2 from mvtools

[Dark Shikari]

"Fast":

backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1, divide=2)
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1, divide=2)
source.MVFlowFps(backward_vec,forward_vec,num=48000,den=1001,idx=1)

Absurdly Slow (but best quality):

backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=4, idx=1, divide=2,overlap=4,search=3)
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=4, idx=1, divide=2,overlap=4,search=3)
cropped = source.crop(4,4,-4,-4)
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=4, idx=2, divide=2,overlap=4,search=3)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=4, idx=2, divide=2,overlap=4,search=3)
source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=48000,den=1001,idx=1,idx2=2)

[2Bdecided]

The "absurdly slow" version crashes for me after 1 frame output. VirtualDub closes, TMPGEnc gives a black screen.

What am I doing wrong? The Fast version works fine.

All I'm doing is...
mpeg2source("HVD.2007-10-05 09-24-40 25p pan.d2v")
assumeframebased()

#ConvertToYUY2(interlaced=false)

trim(118,277)

# Get motion vectors (slow version)...
source=last
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=4, idx=1, divide=2,overlap=4,search=3)
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=4, idx=1, divide=2,overlap=4,search=3)
cropped = source.crop(4,4,-4,-4)
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=4, idx=2, divide=2,overlap=4,search=3)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=4, idx=2, divide=2,overlap=4,search=3)

# change frame rate (slow verssion)
source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=50,den=1,idx=1,idx2=2)


I've tried it with and without the colour space conversion, and with a target frame rate of 75 and 50 fps. No luck.

Any ideas?

Cheers,
David.

[foxyshadis]

Not enough memory, perhaps? Check task manager, see if it crosses 2G allocated for the process, or whatever your total physical+swap is, whichever's less.

[Didée]

Try reducing pel=4 to pel=2. Of all the problems motion interpolation is having, subpel precision is the most unimportant. Halfpel is fully sufficient IMO.

Still, in case it is so that MVFlowFPS/2 works with pel=2, but not with pel=4, it could be a bug.

[2Bdecided]

Thanks for your help guys.

It now seems my whole AVIsynth installation had become corrupt. Other scripts were working yesterday, but nothing is working today. Re-install didn't fix it. Never mind - it's an excuse to re-install XP SP1; SP2 hasn't been my friend!

Cheers,
David.

[Fizick]

By the way, recently Moscow university released frame rate conversion (doubler) plugin for Avisynth.
http://www.compression.ru/video/frame_rate_conversion/index_en_msu.html

It is a litle buggy.
They advertise it as a best frame interpolator.
Of course, you know WHAT is really best ;)
(I make some tests)

[ficofico]

I've tried also this plugins, not bad but not very good.
In my test the avisynth "best" framerate doubler it's motionprotectedfps, Very fast and with very good output. I've made a program that improve video recorded with the last nokia smartphone, frame rate it's from 4-5 to 40 fps,..... motionprotectedfps it's the only plugins that give me what i want, no jerky videos and poor cpu usage., better than mpfpsscd or in this post script or plugins

[2Bdecided]

Do you have a direct link to motionprotectedfps? A search has found 46 discussions, but... ?

Cheers,
David.

[ficofico]

sure... MOTION PROTECTED FPS (http://ficofico2.interfree.it/motion.rar)

[2Bdecided]

Try reducing pel=4 to pel=2. Of all the problems motion interpolation is having, subpel precision is the most unimportant. Halfpel is fully sufficient IMO.

Still, in case it is so that MVFlowFPS/2 works with pel=2, but not with pel=4, it could be a bug.

I re-installed windows. I still had the same problem.

So I tried pel=2 and it solves it - now it works just fine. The slow version is certainly worth trying: with the "fast" vesion, I had artefacts when an object moved behind another thing object - with the "fast" version these problems have gone.

However, pel=4 does work OK if I re-size to SD (704x576) first. It's only HD (1440x1080) which crashes with pel=4: the memory and CPU usage go through the roof (though supposedly the process doesn't run out of memory - not sure this is true).

So thanks (again) for your help Didée, and everyone else.

Cheers,
David.

[Fizick]

Pel=4 is memory consumed.
Do not use overlap with MVFlowFps2!
Try "slow" version with pel=2, overlap=4 and search=3 :)

backward_vec = source.MVAnalyse(isb = true, overlap=4, pel=2, idx=1, divide=2,search=3)
forward_vec = source.MVAnalyse(isb = false, overlap=4, pel=2, idx=1, divide=2, search=3)
source.MVFlowFps(backward_vec,forward_vec,num=48000,den=1001,idx=1)

[Dark Shikari]

Pel=4 is memory consumed.
Do not use overlap with MVFlowFps2!
Try "slow" version with pel=2, overlap=4 and search=3 :)

backward_vec = source.MVAnalyse(isb = true, overlap=4, pel=2, idx=1, divide=2,search=3)
forward_vec = source.MVAnalyse(isb = false, overlap=4, pel=2, idx=1, divide=2, search=3)
source.MVFlowFps(backward_vec,forward_vec,num=48000,den=1001,idx=1)
Why is overlap bad with MVFlowFPS2?

[2Bdecided]

backward_vec = source.MVAnalyse(isb = true, overlap=4, pel=2, idx=1, divide=2,search=3)
forward_vec = source.MVAnalyse(isb = false, overlap=4, pel=2, idx=1, divide=2, search=3)
source.MVFlowFps(backward_vec,forward_vec,num=48000,den=1001,idx=1)

Ouch - that is slow! On the clip I'm trying, it seems to be worse than the mvflowfps2 with pel=2, and much slower. It doesn't crash though! ;)

Cheers,
David.

[Terka]

is it really 50fps output when input is 25fps?
i got 47.952fps when applying it.

[2Bdecided]

In the source.MVFlowFps line, the output rate is set by numerator divided by denominator.

So for 50fps, you just need num=50,den=1

Cheers,
David.

[Terka]

thank you!

[WorBry]

For 25p to 50p, I've obtained pretty good results with the un-named ("whatever") rate conversion function Didee posted in the 'Closest AVS script to Alchemist results' thread:

http://forum.doom9.org/showthread.php?p=853714#post853714

I wonder if the motion interpolation concept discussed later in the same thread was ever developed ?

http://forum.doom9.org/showthread.php?p=913136#post913136

[Adub]

Uh, what's with the necromancy?

[WorBry]

Uh, whats with the needless sarcasm? Perhaps you are suggesting my post is 'dead-boring" or has Didee suddenly passed away? :D

Here's how it goes. I'm interested in finding information on a particular subject and do a search. It pulls up a series of threads which I read. If the search provides the information I need, all well and good. If not, I may make further inquiry.

So what if it revives a thread that is 5 months old. The subject is still valid.

So, does anyone have any constructive input?

[2Bdecided]

I recall Didée said that the method suggested in the current thread could do with some kind of error detection to handle the nasties that can appear when motion can't be predicted properly. I think he mentioned the kind of error detection he developed in that alchemist thread as being a possible candidate.

That fps changer which only blends the "difficult" parts of a frame can look very strange on single frames, but so can the other options. On whatever I tried, it was the break-point between motion compensated and blending that was weird, but I can't remember where I got that code from - somewhere in the alchemist thread, I think. It probably wasn't "finished" ;)

If anyone has a "finished" version, please post/link.

Cheers,
David.

[WorBry]

This is the function posted by Didee in the Alchemist thread that I have been using:

function WhatEver(clip clp, int "fps_num", int "fps_den", string "edgemode", int "sensitivity")
{
fps_num = default(fps_num, 50) # Numerator & Denominator for
fps_den = default(fps_den, 1) # target framerate, as usual
edgemode = default(edgemode, "prewitt") # or "min/max"
sensitivity = default(sensitivity, 9) # the lower, the more blending

ox = clp.width()
oy = clp.height()
SNS = string(sensitivity)

#blend = interleave(clp,merge(clp,clp.deleteframe(0),0.5))
blend = clp.converttoyuy2().convertfps(float(fps_num)/fps_den).converttoyv12()
edge_orig = clp.mt_edge(mode=edgemode,thY1=0,thY2=255)

bw1_vec1 = clp.MVAnalyse(isb = true, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
fw1_vec1 = clp.MVAnalyse(isb = false, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
cropped = clp.crop(4,4,-4,-4,true)
bw1_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)
fw1_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)

FPSconverted = clp.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=1,idx2=2)
FPSconverted_edge = FPSconverted.mt_edge(mode=edgemode,thY1=0,thY2=255)
edge_FPSconverted = edge_orig.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=3,idx2=4)

edgediff = mt_lutxy(edge_FPSconverted,FPSconverted_edge,yexpr="y x - 1 - x 1 + / 300 * x 2 ^ x 2 ^ "+SNS+" + / *",U=1,V=1)
repmask = edgediff.mt_expand().mt_expand().bicubicresize(ox/3/4*4,oy/3/4*4)
\ .mt_inflate().blur(1).bicubicresize(ox,oy,1,0)

return mt_merge(FPSconverted,blend,repmask,U=3,V=3,luma=true)
}

In the light of the foregoing discussion, I've tried playing around with it myself eg knocking off the 'overlap' and increasing search=3, but this just seems to upset the fine tuning and introduce aberrations.

Any chance Didee could have a look at it again.....please :):) i.e. are the original parameters still valid for the latest mvtools version (v1.9.2)or is there scope for further tuning?

PS - Tried the MSU FRC plugin - not too impressed - lots of blocking, warps. Motion Perfect also, somewhat imperfect.

[Didée]

Well, from my POV, in the area of motion interpolation the possible "improvements" through script-fiddling are pretty much at their end. For more worthwile improvements, you'd have to dig deep into the guts of MVTools themselves.


Oh, I remember something ... quoting myself:
just a quick case example...


Reference frame
http://img201.imageshack.us/img201/5760/1sourceat6.th.png (http://img201.imageshack.us/my.php?image=1sourceat6.png)

Frame to compensate
http://img180.imageshack.us/img180/9779/1sourcedelta2vc9.th.png (http://img180.imageshack.us/my.php?image=1sourcedelta2vc9.png)

f2vec1 = o.MVAnalyse(isb=false, truemotion=true, delta=2, pel=2, blksize=8, overlap=4, searchparam=2, sharp=2, idx=1)
f2vec2 = o.MVAnalyse(isb=false, truemotion=false, delta=2, pel=2, blksize=8, overlap=4, searchparam=2, sharp=2, idx=2)

f2cmp2 = o.MVCompensate(f2vec2,idx=1) .subtitle("delta=2 : MVCompensate / truemotion= NO")
f2cmp3 = o.MVFlow(f2vec1,idx=2) .subtitle("delta=2 : MVFlow / truemotion=YES")

Results:

MVFlow (truemotion = true) : http://img179.imageshack.us/img179/3068/2amvflowtruemotionez7.th.png (http://img179.imageshack.us/my.php?image=2amvflowtruemotionez7.png)

MVCompensate(truemotion=false): http://img139.imageshack.us/img139/7297/3bmvcompnottruemotionzg9.th.png (http://img139.imageshack.us/my.php?image=3bmvcompnottruemotionzg9.png)

Curing symptoms is one thing. Eliminating their causes is another. The former is common practice, but basically pointless. The latter is preferable, but also more difficult.

[WorBry]

If I understand the example correctly, it illustrates that motion interpolation based on pixel motion vectors (i.e. MVFlow), although deemed to be more representative of 'true motion', may be more prone to structural artifacts than purely block-based vectors (i.e. MVCompensate). Correct?

Actually, in digging up some of the rate conversion routines that I used in the past for applying a motion blur to bob-deinterlaced 50i > 25p footage, I've noted that those based on the older (pre-true motion) MVTools, such as Scharfis_Brain's MVFps, do a pretty decent job as straight frame-rate doublers with less artifacts. Downside is, they were very slow.

Hope I'm not committing sacrilege by saying that....but at least Merlin 7777 can now justly accuse me of necromancing :D

[Adub]

Fair enough. ;)

[2Bdecided]

Curing symptoms is one thing. Eliminating their causes is another. The former is common practice, but basically pointless. The latter is preferable, but also more difficult.I know, but when something has to be converted this month, I'll take whatever is on offer. The difficulty I've found is that the choice of which method minimises artefacts is both content and conversion ratio dependent.

Thankfully, most people I convert things for are just happy to have the content - but it's nice if I can mix different frame rate and resolution sources on a DVD, and people don't really notice the join. Sometimes I think it would be easier to trash the "good" (native) footage rather than improve the converted footage!

Cheers,
David.

[espy2]

Given a long-form video with scene changes in PAL 25p format, what's the best way to re-interlace it to 50i (which is still 25 fps) for standard TV. Would the following code be reasonable?
#Noobie attempt at a framerate-preserving reinterlace (using “fast” version of frame interpolation)
ConvertToYV12 #Or YUY2 – works with either
source=last
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1, divide=2)
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1, divide=2)
source.MVFlowFps(backward_vec, forward_vec, num=50, den=1, idx=1) #doubles the fps
SeparateFields.SelectEvery(4,1,2).Weave #halves it again

Or is there a better way? Any best way of minimising issues with scene-changes (cuts, dissolves) in a low-motion video? On the other hand would a different approach be recommended for more dynamic (flashy zoomy cutty) vids?

As an experiment, I generated a (progressive) text-zoom animation in AviSynth, reinterlaced it it using the above script and deinterlaced again (using TDeint). The result looked ok, unlike when I had (naively) tried reinterlacing via DoubleWeave (with no MVFlowFPS). I guess that might be because DoubleWeave re-uses temporal snapshots from each original frame, whereas normal interlaced footage involves only fresh snapshots in time.