Create an Interpolated Frame every second of frames. Possible?

[Undead Sega]

Hi, i am wanting to create a script or if peolpe can help me with one where a newly interpolated frames gets created every second; so, in a 24p video, every 24 frames, the first new frame will be made between frame 24 and 26 and the seond will be between 48 and 50, and so on.

i have done similar examples where i take a 50p video and double the framerate using YadifMod and MCBob:

(YadifMod example):
LoadPlugin("C:\Downloads\VirtualDubMod_1_5_10_2_b2542\mvtools-v1.8.6\mvtools.dll")
LoadPlugin("C:\Downloads\VirtualDubMod_1_5_10_2_b2542\yadifmod\yadifmod.dll")
LoadPlugin("C:\Downloads\VirtualDubMod_1_5_10_2_b2542\decomb522\Decomb.dll")
LoadPlugin("C:\Downloads\VirtualDubMod_1_5_10_2_b2542\nnedi_v1.3\nnedi.dll")
AviSource("C:\X.avi")
ConvertToYV12()
yadifmod(order=0,field=0,mode=0,edeint=nnedi,opt=0) # assuming TFF
smoothfps(100)
assumefps(100)
function smoothfps(clip source, float fps) {
fp=fps*100
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = source.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=int(fp),den=100, idx=1,idx2=2)
}

with the YadifMod line command replaced by MCBob, i go get slightly better results, but neither is perfect, and i get alot of frames that are like all bubblely or screwy, due to the fact (probably) the MC filter needs to look at past & future frames to help determine the missing spaces, and it seems either can do the job properly.

i was wondering first, if such MC filter would exist to give clean results like commercial products? and also if a better frame interpolator script is known?

sounds crazy yes, but i am hoping to acheive something here and i would appreciate it if anyone here can help me on this.

thanks.

[mikeytown2]

You need to get a common framerate... so 24*25=600. Then do a select every x to bring the 600fps down to 25fps.

Another option would be a large statement using MVFlowInter's time variable.

[Undead Sega]

Another option would be a large statement using MVFlowInter's time variable.

and what would this be precisely may i ask? i dont believe i have heard of it before.

[mikeytown2]

MVFlowInter
http://avisynth.org.ru/mvtools/mvtools.html#functions

24/25=0.96

So we redo frame 1 and set the time variable to 0.96. frame 2 would be 0.92, ect... until we use the last frame in the sequence, frame 24 as frame 25. 96% of the video will be made from MVTools; only one frame out of your 25 will be untouched. Using about 25 SelectEvery() (http://avisynth.org/mediawiki/Select) statements would be needed, or a smart function.

[Undead Sega]

oh okay then, im trying to understand wat u are saying, but remember, i want to create a new frame every second, and i am refering to 24fps, so 1 frame made every 24 frames or 1 second.

by the way, can u help me answer my question about the whole bubblely or screwy frames (ones that are not interpolated properly) when using MCBob or YadifMod, or is there a better solution to the script i posted when doubling the framerate?

[Fizick]

pseudocode:

assumefps(24)
vf=mvanalyse(isb=false)
vb=mvanalyse(isb=true)
mvflowfps(vb,vf, num=48) # double framerate
selectevery(48,0,2,4,6,8,...,42,44,46,47) # select all original (even) frames plus one (n=47) interpolated, in result 25

you must tune the script to find proper place for interpolated frame.
Note: Really 24p is converted to 25p by other way.

[Undead Sega]

Cool, does this script do exactly wat i am asking for?

[Fizick]

test it

[Undead Sega]

Okay then, what filters do i need? Also, which process will it do that im asking? double framerating perfectly? or Interpolating a frame every second?

[Fizick]

test it

[Undead Sega]

Filters?

[thetoof]

mvtools and it will interpolate a frame every second (which you would've found out by testing it as Fizick said) I guess he assumed that you already had mvtools since you posted a script using it...

[Undead Sega]

tried it, and there seems to be a problem with the last line aprantly. Syntax error?

[45tripp]

so you seriously just did a copy-paste...?

the dotted lines mean fill in the blanks...

here:

assumefps(24)
vf=mvanalyse(isb=false)
vb=mvanalyse(isb=true)
mvflowfps(last,vb,vf, num=48) # double framerate
selectevery(48,0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,47)

[Undead Sega]

Corrected, but now a new error has came in, stating (for the seond to last line) "Vector Clip must be specified"

[Alex_ander]

Try this way (mvflowfps doesn't understand 'last' for some reason):


a=assumefps(24)
vf=a.mvanalyse(isb=false)
vb=a.mvanalyse(isb=true)
a.mvflowfps(vb,vf, num=48)
#+the last line, of course

[Undead Sega]

Cool, i got it working finally!

i have been looking at every second of frames, and i do see some flaws as i have described in my original post, can someone explain to me why this happens?

one thing i think i notice is that the picture is slightly brighter than the original, correct me if i am wrong on this?

also i would like to ask out of curiousity, is that has this kind of script ever been made before? Because i intention of trying to create a script or if someone can help me on trying to make this technique is because of its flawless interpolation of 24p to 25p :D kinda speaks for itself doesnt it? :D

[Alex_ander]

It's not flawless in terms of preserving smoothest possible motion (each 25th frame some slowdown takes place), but it keeps the 24 original frames intact. Closest to this method is pulldown with its repeating a field each 12th frame and consequent field-shifting in half of frames. Produced e.g. by [ChangeFPS(50) + interlacing] or with DGPulldown 24->25 for MPEG.

[Undead Sega]

Closest to this method is pulldown with its repeating a field each 12th frame and consequent field-shifting in half of frames. Produced e.g. by [ChangeFPS(50) + interlacing] or with DGPulldown 24->25 for MPEG.

ohh i am very familiar of this method, cause i go through it all the time, thus why i am looking for a somewhat flawless interpolator, which is what im asking for, and only for the process to be applied to every second, so the calculator would only focs on that and not every frame. Is there any way to get a flawless frame interpolation using any method in AviSynth?

[Fizick]

flawless frame interpolation is impossible.

[Undead Sega]

With commercial applications like Philips' TrimensionDNM and Motion Morphing MultiSampling, could they still be flawless or are just comparable with the AviSynth filters?

also, what needs to be odne as well to create flawless interpolation?

in addition, does anyone know of hardware that does Frame upconverting using Motion Compensation?

[Fizick]

it is impossible "flawless" restore notexistant information.
But people try to do it better and better.

It was discussed here and there. Use search.

chao.

[Gavino]

mvflowfps doesn't understand 'last' for some reason
This is because the vector clips are defined as optional arguments, so in mvflowfps(vb, vf, ...), Avisynth takes vb as the source clip and vf as the backward vector - hence the error message "Vector Clip must be specified", because MVTools sees this as missing the forward vector.

@Fizick: Is this a bug? Why are the vector arguments 'optional'?

[Undead Sega]

dont worry, its okay, it didnt work because i didnt have the word 'last' in the script, so its all cool :D

hardware wise, i know it is possible to use hardware such as ur graphics card to do Motion Compensation, is there any way to take advantage of it for any kind of interpolation whatsoever?

[Gavino]

dont worry, its okay, it didnt work because i didnt have the word 'last' in the script, so its all cool :D
It may be okay for you, but I'm the sort of guy that worries about things like this. :)

The point is you shouldn't have to put the word 'last' in yourself, Avisynth should do it for you behind the scenes.

[Fizick]

Gavino,
mvfw is named agrument. Avisynth syntax have not named optional arguments.
it is not bug but feature. :)
Whos? MVtools or Avisynth.
I could made mvbw, mvfw as unnamed, but it will brake compatibility.
If you are the sort of guy that worries about things like this, and you know solution, please suggest ;)

[45tripp]

The point is you shouldn't have to put the word 'last' in yourself, Avisynth should do it for you behind the scenes.

exactly

[Fizick]

without last:

avisource("test.avi")
vf=mvanalyse(isb=false,idx=1)
vb=mvanalyse(isb=true,idx=1)
mvflowfps(mvbw=vb,mvfw=vf, num=50,idx=1)

but last exist in avisynth documentation, so it may be used

[Undead Sega]

would there even be a difference in the way how it interpolated with and without the word 'last' ?

hardware as well people?

[Gavino]

If you are the sort of guy that worries about things like this, and you know solution, please suggest ;)
A possible solution would be to add a second function definition starting with with exactly two mandatory clip arguments. Then for MVTools to treat this as a wrapper to the real function, itself supplying GetVar("last") as the source clip. So internally it treats mvflowFPS(vb, vf) as mvflowFPS(last, vb, vf).

It's not very satisfactory as plugins should not have to know about 'last' (a language feature), but it does at least solve the problem.
I'll leave you to decide whether you think it's worth it. ;)

[Fizick]

i considered some trick with IsDefined(args[1]), but it not very simple (and nice)

[Gavino]

I can't see any clean solution so any cure is likely to be worse than the disease (as we say in Spain).

As you have shown, the workaround is to always name the arguments in the calling script. Perhaps you could mention this in the documentation.

[Undead Sega]

If flawless Frame Interpolation is impossible, then how is VidFIRE possible then?: http://en.wikipedia.org/wiki/Vidfire

also, i think wat i am looking for is Motion Compensated Frame Interpolation as the correct term.

[Fizick]

I am sure it is not flowless :)

[Undead Sega]

well, Motion Estimation seems to be a way forward due to the Motion Vectors used to anayse previous and future frames, to which it help determines what was inbetween.

i was wondering if it is possible to take a Motion Estimation filter (like MSU's) and help intergrate that into a script where it uses it to help anaylse and interpolate frames, is that possible?

[Fizick]

1. not determine, but suggest, estimate (what was inbetween).
2. it is what MVanalyse and MVFlowPps function tryes to do (but there are billion ways to do it differently). A am sure you read MVTools documentation (or not?)
3. did you say about msu_frc ? http://compression.ru/video/frame_rate_conversion/index_en_msu.html
4. do you have any VidFIRE - processed DVD fragment to compare results?

[Undead Sega]

2. it is what MVanalyse and MVFlowPps function tryes to do (but there are billion ways to do it differently). A am sure you read MVTools documentation (or not?)

i have indeed, but not all, i take it the more complex the way to do things the better the result maybe? if so, what part of MVTool does this? and is there a better ME method?

3. did you say about msu_frc

i have tried that ages ago, but it really did not give results that are satisfactionary, even the script made here did a better job.

4. do you have any VidFIRE - processed DVD fragment to compare results?

not yet! funny enough i was thinking of getting a Doctor Who episode or any video that was processed with this method inorder to see the results.

[Fizick]

better ME? better for what and for who ?
there are a lot :)
http://www.freepatentsonline.com/result.html?query_txt=%22motion%20estimation%22

[thetoof]

Btw, to help reduce the artifact you're talking about, you can try making more accurate motion vectors calculation by tweaking mvanalyse
a=assumefps(24)
vf=a.mvanalyse(isb=false, overlap=4, search=3, pel=4)
vb=a.mvanalyse(isb=true, overlap=4, search=3, pel=4)
a.mvflowfps(last,vb,vf, num=48) # double framerate
selectevery(48,0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,47)
These are a bit crazy and further improvement is possible... read the docs to see if you can find something.

[Gavino]

a.mvflowfps(last,vb,vf, num=48) # double framerate
This should bemvflowfps(a,vb,vf, num=48) # double framerate
(see posts earlier in this thread)

[Alex_ander]

or the similar a.mvflowfps(vb,vf, num=48)
would also work, while undefined 'last' in arguments causes an error.

[Gavino]

Yes, since a.filter(b, c) is just another way of saying filter(a, b, c).

To clarify your second point, if you want to use last as the source clip to MVFlowFPS, you have to write the word 'last' explicitly (instead of the normal practice of leaving it implicit), or else use named notation for the vector clip arguments, so:
mvflowfps(vb, vf, ... ) # fails
last.mvflowfps(vb, vf, ...) # OK
mvflowfps(last, vb, vf, ...) # OK
mvflowfps(mvbw=vb,mvfw=vf, ...) # OK

See earlier, especially posts #23 and #28.

I expect the same issue also applies to other MVTools functions.

[Undead Sega]

Btw, to help reduce the artifact you're talking about, you can try making more accurate motion vectors calculation by tweaking mvanalyse

a=assumefps(24)
vf=a.mvanalyse(isb=false, overlap=4, search=3, pel=4)
vb=a.mvanalyse(isb=true, overlap=4, search=3, pel=4)
a.mvflowfps(last,vb,vf, num=48) # double framerate
selectevery(48,0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,47)

These are a bit crazy and further improvement is possible... read the docs to see if you can find something.

i realy do hope so, because i tried it out, and in very few places there was small improvement but in many others, it mayb looked a tad more noticable.

dont worry people, still trying to get my hands on a VidFire DVD, that will probably show that Motion Interpolation can be done flawlessly.

[Fizick]

BTW, recently I found that pel=4 mode is broken and always was broken :(
Next version will produce correct result (but not flawless) :)

[Undead Sega]

ohhh, what fault did pel=4 give?

and when can i expect the new version?

[Fizick]

Sorry. probably i posted in wrong tread.
When? may be after you post frames from VidFire DVD :)
(joke)

[Undead Sega]

the VidFire DVD is an old Doctor Who episode, which was telerecorded, and involves using a film camera to record from a large TV at 25p instead of the usual 24p. because of this, only half of the motion is captured (because PAL video is 50i), sooo...what VidFire does is, it interpolates frames thus creating 50p and on those interpolated frames, they are interlaced, thus giving 50i, which then is now restored on how it was suppose to be.

[Undead Sega]

It just came to my attention, that if this script (or the new one i am awaiting) tends to give less artifacts than the previous (and MCBob), does that mean it would make a good Deinterlacer perhaps?

[Undead Sega]

Hello everyone, i know this maybe slightly off but still within topic, but i have found something quite interesting:

http://video.golem.de/desktop-applikationen/1590/aus-standbildern-videos-machen.html

it apprantly is Motion Interpolation using CUDA in Nvidia's GPU. does anyone else know about this?

[neuron2]

Rule 13: The official language is English.

[Undead Sega]

Would a translated version be an acception?:

http://translate.google.co.uk/translate?hl=en&sl=de&u=http://video.golem.de/desktop-applikationen/1590/aus-standbildern-videos-machen.html&sa=X&oi=translate&resnum=1&ct=result&prev=/search%3Fq%3Dhttp://video.golem.de/desktop-applikationen/1590/aus-standbildern-videos-machen.html%26hl%3Den%26safe%3Doff

[Gavino]

Rule 13: The official language is English.
Does this prohibit someone posting a message in English drawing attention to some information that might be of interest, that is only available in some other language?

Some of the Doom9 guides do exactly that.

[neuron2]

Please stay on topic, Gavino. You can send me a PM about it if you wish. Undead posted a translation. That is a perfect solution. Guides are not forum postings.

[Maccara]

dont worry people, still trying to get my hands on a VidFire DVD, that will probably show that Motion Interpolation can be done flawlessly.

Just note, that you can't compare these directly, as the drwho restoration team manually fixes the artifacts VidFIRE produces. Therefore the end product isn't comparable with a "raw" produced by e.g. mvflowfps (unless you can conjure a "raw" VidFIRE clip too from somewhere).

VidFIRE is by no means 'flawless'. (to date, no flawless process exists and information theory suggests there never will)

See f.e. thread http://www.rtforum.co.uk/read.php?id=244490 which discusses a bit about the VidFIRE processing and some typical artifacts it produces.

[Comatose]

I'm guessing the fact each resultant frame is turned into a field makes artifacts much less apparent (because the vertical resolution is halved)

[Maccara]

I'm guessing the fact each resultant frame is turned into a field makes artifacts much less apparent (because the vertical resolution is halved)

And especially by that it interpolates every frame, i.e. there are no original frames left after processing (or so I understood by quickly browsing).

This alleviates the common issue where you get "unstable" results (every other frame blurry/artifacted/etc and other is the original crisp/artifact free original frame, in case of double framerate) which you can notice quite easily on a moving picture.

PS. I haven't seen VidFIRE in action myself, so can't judge exactly how good/bad it is compared to anything else - just that I know it's not flawless. ;)