Closest AVS scripts to Alchemist results

[Clown shoes]

I am wondering if anyone has an opinion currently as to whether there are any avs scripts that allow a standards conversion close to the results produced by Snell & Willcox's Alchemist. I ask this for two reasons;

1. Snell & Willcox are about to release a software based converter called Helios http://www.snellwilcox.com/news_events/press_releases/197 which according to the salesman at Broadcast Live will give results comparable to the Alchemist (obviously not realtime though)

2. We get quite a few standards conversion jobs at work, where time is not an issue. It would be nice to have the confidence to tell a client our software solution was comparable to hiring an Alchemist, but at a much lower cost.


Over the past few years I have used many of the scripts posted on the Doom9 forums by people like Xesdeeni (who I have not seen around for a while) and Scharfis_brain. I am currently using:

for pure video

import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
interp=separatefields().eedi2(field=-2)
tdeint(mode=1,full=false,edeint=interp)
assumefps(50.05)
motionprotectedfps(60)
assumefps(60000,1001)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()


for hybrid content

MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
interp=separatefields().eedi2(field=-2)
tdeint(mode=1,full=false,edeint=interp)
converttoyuy2()
convertfps(60000,1001)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()



I would be grateful for anyones input on this matter, great or small.

Thanks in advance

ClownShoes

[Clown shoes]

(sorry didn't mean to double post, but this second post was in response to a comment made by Pookie. For some reason he deleted it though!)



:p

The software hasn't actually been released yet. And as I understand it, it will be in the region of £3,000.00

My query is really as to whether existing filters are able to emulate or get close to the output of the Alchemist hardware converter. If Snell & Willcox's claims are to be believed, they have managed it with Hellios.

My knowledge of AVIsynth and it's scripting is not in the same league as many of the videophiles here and I am also hard pushed to afford the hiring of an Alchemist on a regular basis for testing and comparisons. So I guess I'm hoping that any one with any standards converting experience can give thier opinions. I am currently encoding with an adaption of the first script I posted, but using mvbob:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob~.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
assumefps(50.05)
motionprotectedfps(60)
assumefps(60000,1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()


I'm pleasantly suprised to be getting 3fps with this script and am curious to see what it looks like.

[foxyshadis]

scharfis prefers mvfps & mvfps2, and of course fizick because he wrote it :p, because they can be tweaked to offer higher quality with less artifacts. Personally, my experience has been with motionprotected (base and personally modified versions - it makes fast excellent motion noise masks) and it's always given me good, dependable quality in movies. It animation, Twixtor for AE was better (but it ran out and AE is torture anyway) but I couldn't really see it in normal movies.

One thing you might want to think about is that motion compensation works significantly better with a denoised source. That's a strong argument in favor of mvfps2, or modding motionprotectedfps for two clips. (Attached.) Alternately you can just denoise, compensate, and then add your own grain back in.


function MotionProtectedFPS(
\ clip input, float FPS, clip "denoised",
\ int "mode", int "protection",
\ int "iterate", int "reset", int "initialise")
{
denoised = default(denoised, input)
mode = default(mode, warpfast)
protection = default(protection, 80)
iterate = default(iterate, 2)
reset = default(reset, 50)
initialise = default(initialise, 6)

originalFPS = input.framerate()
input

mp = denoised.FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise)
p = MotionFPS(FPS,mode,Motion = mp)
maskp = mp.SumStretchFPSMask(FPS, protection)

mn = denoised.FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise, from = next)
n = MotionFPS(FPS,mode,Motion = mn, source = next)
maskn = mn.SumStretchFPSMask(FPS, protection, source = next)

maskp
assumeFPS(originalFPS)
CombineFPS(FPS, maskp, maskn.Invert())
bilinearresize(width*8, height*8)
crop(0, 0, input.width(), input.height())
fity2uv()

mt_merge(p, n, last, chroma="process")
}

[Pookie]

Sorry about deleting my post. I read it back and I realized it didn't add any constructive advice to the thread. Not that this one does, either.

[Trixter]

My query is really as to whether existing filters are able to emulate or get close to the output of the Alchemist hardware converter. If Snell & Willcox's claims are to be believed, they have managed it with Hellios.


Alchemist, as well as Folsom's top of the line standards converter, indeed use motion synthesis along with several quality checks for the best results. Occaisonally they make mistakes (usually with film sources that aren't telecined properly), but most of the time the results look 99% close as if the material was shot in the other standard.

Luckily, MVTools comes pretty close. If the material you're converting is interlaced (ie. actually does have 50 or 60 different images per second) and doesn't contain any surprises, MVTools works damn near perfectly. Best of all, it's free :-)

I am currently encoding with an adaption of the first script I posted, but using mvbob:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob~.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
assumefps(50.05)
motionprotectedfps(60)
assumefps(60000,1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

I'm pleasantly suprised to be getting 3fps with this script and am curious to see what it looks like.

Not sure why you're doing three different FPS conversions, but if you're using mvbob then mvconv is in the same package and it converts 50<-->59.9401 (60000/1001) like so:


mvbob
mvconv # <-- automatically figures out which way you want to go
#...resize to 480 lines, re-weave...


If mvconv isn't in whatever package you're using, I used to do it manually this way:


function MOCOMP_50_to_5994(clip c)
{
backward_vec = c.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = c.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = c.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
c.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=60000,den=1001,idx=1,idx2=2) # the 60000 / 1001 = 59.9401
}


The problem with mvbob is speed, obviously -- it goes overboard going for the best possible bob (and succeeds). For much faster processing, this is acceptable as long as you don't have single-pixel horizontal lines (like fine detail in the titles) in the source:


TDeint(order=1,mode=1,type=0,sharp=false,AP=50,APtype=0)


Let us know what you come up with. Standards conversion has been a hobby of mine for more than half a decade, as I've had to take very goofy source material (66p, 70p, and 72p computer sources) and convert it to NTSC without it looking like total ass... after using a gazillion products like ReTimer, Twixtor, MotionPerfect with less than optimal workflows, MVTools finally offers me a great workflow at about 99% of the quality. By 99%, I mean about 50 frames out of an hour's worth of footage might need to be touched up, which is par for the course for any converter, really.

[Revgen]

Whomever does that Doom9 firefox plugin should add this page to the default bookmarks. Good info throughout.

Thanks guys.

[Clown shoes]

Ok now we're cooking :)

I am currently testing MVBob with MVConv. That is happily encoding away. I am however having a problem finding a current script for MVFps or MVFps2 as mentioned by Foxyshadis though!

I can find much mention of it when I use the search function and have ended up spending most of the day reading pages of posts but to ne avail. I can find no link or scripting except for a post by Scharfis in february of 2005 http://forum.doom9.org/showthread.php?p=615027#post615027 but that script needed an earlier version of MVTools which I cannot find. I did try changing the version in the script but that does not work. A small pointer in the right direction would be appreciated right now as I am quite sure that this a case of not being able to see the wood for the trees.

@ Trixter

I have also cut out the unnecesary FPS conversions. I was under the impression (having borrowed this script from another recent SC post) that MotionProtectedFPS was unable to accept fractional framerates. This is clearly not the case.

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("E:\SC tests\AmcQ.d2v")
mvbob()
motionprotectedfps(59.94)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()

[Boulder]

Maybe foxyshadis meant MVFlowFPS and MVFlowFPS2 which are a part of MVTools?

[Clown shoes]

I thought that might be the case. But there was a function called MVFps used for the same purpose so it has me a little confused. Also that link in my previous post does link to some scripting by Scharfi called MVFps just over a year ago.

Edit; OK scratch that. I think your right Boulder. I will try it now

Edit2; Hmmm, having problems using MVFlowFPS2. I am clearly calling it in the wrong way. I'm afraid my scripting knowledge is very limited. I have tried to read through the MVTools documentation but it is a little beyond me. I presume this is an incorrect way to call it:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
MVFlowFps2(59.94)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

I would be greatful is someone could briefly explain how to use this.

[Boulder]

I've used MVFlowFPS2 like this:
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=100,den=1)

That would convert the framerate to 100fps, so you probably need num=60 in MVFlowFPS2 and AssumeFPS(59.94) afterwards.

[Clown shoes]

Thanks Boulder, I've got it working now. This is how my script currently reads:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=60,den=1)
assumefps(59.94)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

I'm getting approximately 1.5fps right now using CCE 2.70 on a 3ghz xeon with 2gb of ram. This seems quite reasonable to me.

[foxyshadis]

The Motion (http://forum.doom9.org/showthread.php?t=101859) thread, or the one I posted above has support for compensating off a different source than the one analysed.

Sorry about mistaking mvfps and mvflowfps ^^;

You can also use num=5994,den=1000 (or more precisely, num=6000,den=1001) to eliminate any minor desync assumefps would cause.

[Chainmax]

I used a custom function suggested by scharfis_brain in some extremely shaky digicam clips. It gave me really good results.

[Clown shoes]

Ok I've just encoded some footage using the last script I posted, but I am getting some weird motion effects on some frames during fast motion sections.

Any idea what's causing this? and more importantly, is there a workaround for it?

[Chainmax]

That looks like interlacing, are you sure you're reinterlacing correctly? By the way, this is what I'm using:

Source loading, including audio
prefiltering up to (and including) final resizing
input=last

vectors=depanestimate(input)
input_1_of_x = input
input_2_of_x = input.depan(vectors,offset=1/x-1,mirror=15,blur=35)
input_3_of_x = input.depan(vectors,offset=2/x-1,mirror=15,blur=35)
...
...
input_x-1_of_x = input.depan(vectors,offset=x-2/x-1,mirror=15,blur=35)
input_x_of_x = input.depan(vectors,offset=x-1/x-1,mirror=15,blur=35)
interleave(input_1_of_x,input_2_of_x,......,input_x-1_of_x,input_x_of_x)
AssumeFPS(whatever,true)
ChangeFPS(whatever)

sharpen (if necessary)
addgrain (if necessary)
add borders (if necessary)
levels (if necessary)

x is the number of steps and should be twice the minimum common multiple of the source and destination FPS divided by the source fps. AssumeFPS should be set to the nearest multiple of 23.976 of the extrapolated fps to make the job easier for ChangeFPS. You'll then have to resample the audio by just a tiny bit, I recommend to use SRCDrop v0.5 (http://rarewares.org/others.html) for that, as SSRC will most likely not work in such a case.

For example, if the source is 30fps and the destination is 24fps, then:
- x = [2 x MCM(24,30)] / 30 = 2 x 120 / 30 = 240 / 30 = 8

- Since the extrapolation leaves the video at a little more than 240fps due to rounding errors when calculating the fractions, you have to use AssumeFPS(239.76,true) in this case.

[Clown shoes]

Sorry Chainmax, I didn't mean the interlacing. Im going from 25i to 29.97i so you should be able to see interlacing on a still. What I'm refering to is the warping in the high motion areas. I have also encoded the same clip using mvbob and motionprotectedfps. It is also having some issues with that frame but not nearly as bad. When played back it is not noticable but I am trying to work out what the cause is.

Here is the same frame from the original source and then the motionprotectedfps frame.

[Didée]

To make a long story short: there's very little you can do. Currently, at least.

a) motion estimation & compensation/interpolation will *always* make errors

b) for compansation (target: denoising), the faithful scripter has *tons of possibilities* to check if a given compensation is "good" and can be used, or if it is bad and should not be used / replaced by a fallback solution.

c) for interpolation (target: rate changing), the faithful scripter can do *nothing*, because
c1) the plugins hardly offer any grips to hook in
c2) there's no reference frame one could compare the compensation with.

So, for the case of interpolation, you are fully at the mercy of the plugin coders. From the scripting side there is nothing that could be done, because the crucial points are out of reach.

Sidenote: if you need this stuff "at work", then IMHO the investment into a professional solution would be advisable, anyway. ;)

[Clown shoes]

Excuse my ignorance Chainmax but some of this scripting is beyond me. I must confess that with the mvflowfps2 script that Boulder posted, I have just dropped it straight into my script without a true understanding of the parameters. I have spent some time going through the documentation, but right now it is a little like learning a new language! This could well be the reason for my warped frames in the previous script.

I am going to post the sample I have been working with. If anyone gets a chance to take a look I would be greatful.

My aim is still the same; to find a software solution for 25i to 29.97i or vice versa standards conversions, that can be used as an alternative to the hardware Alchemist route, when time is not an issue

Edit;


Sorry Didee, I didn't notice you had just posted. In response; The reason for my quest was the discovery of the imminent release of Snell & Willcox's software encoder, that they claim is an equal to thier Alchemist hardware converter but without the real time returns. That made me wonder whether the current possible permutations of AVISynth filters might be put to the same use.

The results I have received so far have been amazing! Combinations of MVbob with both MVflowfps2 and motionprotectedfps have given results that when transfered back to DVD or a tape format look almost identical to the original (at least to the untrained eye) and at acceptable speeds.

I appreciate what you are saying with regard to the existing restrictions and limitations, but surely they will always be there. I am just trying to find out what the most effective current solutions are and bring them all together. That fact that thier are multiple video standards and probably always will be, means that standards conversion is something that will always be of interest to people.

With regard to using proffesional standards conversion hardware for work, the problem is that is has always been prohibitively expensive. I work for a small post production company, so spending in excess of £100,000.00 is just not an option, hiring is also overly expensive. Currently the only real software options available are Procoder 2, Motion Perfect and NLE plugins such as Twixtor and Magic Bullet. NONE of these is even a patch on what can be done with AVISynth filters. Therefore the most sensible thing, for me at least, is to find an effective solution within this community.

So in conclusion, I must thank all you coders and scripters because the hard work you are doing really is appreciated.

[Boulder]

What I posted is almost directly from the MVTools' documentation ;)

[Chainmax]

The complete script for 25i to 29.97i should be something like this:

Import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DePan.dll")



input=MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v").MVBob()

vectors=depanestimate(input)
input_1_of_12 = input
input_2_of_12 = input.depan(vectors,offset=1/11,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset=2/11,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset=3/11,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset=4/11,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset=5/11,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset=6/11,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset=7/11,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset=8/11,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset=9/11,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10/11,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11/11,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)

AssumeFPS(599.4,true)
ChangeFPS(59.94)

ConvertToYUY2()
Lanczos4Resize(width,480)
Assumetff().SeparateFields().SelectEvery(4,0,3).Weave()

Remember to resample the audio (as it will be downsampled by a few Hz) with SRCDrop after this.

[Clown shoes]

lol oh! Thanks Boulder. This is taken directly from my mvtools 1.30 documentation;


Description

MVFlowFps2 (clip, clip "mvbw", clip "mvfw", clip "mvbw2", clip "mvfw2", int "num", int "den", float "ml", int "idx", int "idx2")

Will change the framerate (fps) of the clip (and number of frames) like MVFlowFps, but with a little better quality (and slower processing). In addition to backward "mvbw" and forward "mvfw" motion vectors of original source clip, the function uses backward "mvbw2" and forward "mvfw2" motion vectors of second (modified) source clip. Second clip must be produced from original source clip by cropping (i.e. diagonal shift) by half block size. It must be done with command Crop(a,a,-b,-b), where a=b=4 must be used for blksize=8, a=b=8 for blksize=16, and a=2, b=6 for blksize=4 (see example). Blocks boundaries will be at different parts of objects. MVFlowFps2 reverses the shift internally and averages motion vectors from these two sources to decrease motion estimation errors. The function uses pixel-based motion compensation (as MVFlow, MVFlowInter). Internal forward and backward occlusion masks (MVMask kind=2 method) and time weighted factors are used to produce the output image with minimal artefactes. True motion estimation is strongly recommended for this function.

num: fps numerator (default=25)

den: fps denominator (default=1). Resulted fps = num/den.

ml: mask scale parameter. The greater values are corresponded to more weak occlusion mask (as in MVMask function, use it to tune and debug). Default=100.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

idx2 is MVAnalyse index of second (shifted) clip (must not coincide with first idx).


Usage

To change fps with MVFlowFps2:

# Assume progressive PAL 25 fps source. Lets try convert it to 50.
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = source.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=50,idx=1,idx2=2)



I clearly have a lot of reading and learning to do, because I have no idea how you made the correlation between what's in the documentation and what you posted. I feel a bit stupid, much to learn... :o


@ Chainmax,

Thank you. I will try this now. Can I ask though, why assumefps(599.4,true) followed by changefps(59.94) Secondly, why does the audio need to be downsampled by a few hz? is there a part of this script that is causing the running time to change?

[Chainmax]

Because of rounding errors when calculation the fractions, the resulting framerate after the Interleave line will be something like 600.4fps. The AssumeFPS line brings it down to 599.4fps so that ChangeFPS can do a clean 10-to-1 decimation instead of a ~10.0167-to-1 one.
Now, AssumeFPS changes framerate by changing the speed of the video, and while a 0.15% slowdown will never be noticeable, audio and video will become very slightly asynch. The true switch in the AssumeFPS line corrects that by altering the samplerate of the audio to avoid that asynch, which is why you'll need to resample afterwards.
SSRC can't do such minor (~ 0.0070% or less) adjustments, which is why I recommended SRCDrop.


P.S:mmm...I wonder if ChangeFPS can be substituted by TDecimate...

[Didée]

Chainmax, that script seems incorrect to me. The offsets you're specifying are all integers: 1/11=[zero], 2/11=[zero], etc. When I try that script, it results in just one duplicate field every 6th field, practically the same as going the good old Bob.ChangeFps.SelectEvery.Weave road. For getting that result, DePan is not even needed...
When changing the offsets to floats, as expected: 1./11. , 2./11. , etc. then: there is indeed "true" motion throughout all fields of the output, but also the "oddities" from interpolation are there again ...

Alas, currently I have to try with an older version of DePan. On Fizick's site I see no download link to the current version ... and since Fizick currently is in vacation (supposely), I can't ask him, though being a "supporter" ...

[Boulder]

Do you mean this one : http://avisynth.org.ru/depan/depan17.zip

[Didée]

Oh, it's indeed there. Thanks for the hint. :)

Going the official road, you end up in this depan page (http://avisynth.org.ru/depan/depan.html), which doesn't contain a download link.

[Boulder]

That's where I got the link :) It's just down at the bottom of the page, second line from the end. It's easy to miss because there is just the latest version available whereas many other filters of his have all the versions for downloading.

[Didée]

Eh, now we have it: my browser stopped before loading the complete document, probably because of a timeout. Hence I had no link. Sorry for the noise ;)

BTW, to make your above example script for 25p --> 29.97i work correctly, needed changes were:

- replace "1/11" with "1./12." and so on for the rest
- replace "input_1_of_12 = input" with "input_1_of_12 = input.duplicateframe(0)"

Result is free or artefacts, of course. But really good it is only on clear pans. On "globally still" frames, all in-frame motion still is stuttering ... and on some difficult scenes (global shift to one side, but big objects moving to the other side), the stutter is, well, funny.
No surprise, though ... depan is doing exactly what it's designed to do: shifting the frame as a whole.

[Clown shoes]

Are you saying that script is for 25p to 29.97i ? Would that be no good for my 25i footage then?

[Trixter]

Where did you get MotionprotectedFPS? I don't think I've seen that, and am curious to know what it does...

Scratch that, I found it.

...and after testing it for about 30 minutes, I have to say that I like MVTools' MVFlowFPS2 much better -- the decrease in speed is well worth the estimation accuracy. I'll stick with them.

[Didée]

Arrrr...

With the noted changes, you can use it. I did miss that at the end of the loooooong mpeg2source-line, Chainmax had put a small ".MVBob()". :D
(I didn't even look at the end of that line, because I'm not used to put any filters in that place -- that's not exactly "structured scripting" ...)

So, for what the "interlacing impaired" person I am can see, the following should be correct for you:
Import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DePan.dll")

input = MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
input = input.MVBob()

vectors = depanestimate(input)
input_1_of_12 = input.duplicateframe(0)
input_2_of_12 = input.depan(vectors,offset= 1./12.,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset= 2./12.,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset= 3./12.,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset= 4./12.,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset= 5./12.,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset= 6./12.,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset= 7./12.,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset= 8./12.,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset= 9./12.,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10./12.,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11./12.,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,
\ input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)

AssumeFPS(599.4,true)
ChangeFPS(59.94)

ConvertToYUY2()
Lanczos4Resize(width,480)
Assumetff().SeparateFields().SelectEvery(4,0,3).Weave()

But again: on scenes without "global motion", (only some objects within the frame are moving), the result basically will still just be one duplicate field for every 6th field. It's only the panning scenes that benefit.

[Trixter]

vectors=depanestimate(input)
input_1_of_12 = input
input_2_of_12 = input.depan(vectors,offset=1/11,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset=2/11,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset=3/11,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset=4/11,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset=5/11,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset=6/11,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset=7/11,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset=8/11,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset=9/11,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10/11,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11/11,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)

AssumeFPS(599.4,true)
ChangeFPS(59.94)


Sweet Lincoln's Mullet, what is this doing? Why 12 different motion vector calcs, and why the up-to-600-then-back-to-60? How is this better than MVFlowFPS(2)?

[Trixter]

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("E:\SC tests\AmcQ.d2v")
mvbob()
motionprotectedfps(59.94)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()

If you want Helios-like results without attempting your own threshold masking, that's pretty much it. For slightly better estimation at the cost of processing speed, you can use MVFlowFPS2, already a part of mvconv() in the mvbob.avs you import. Give this a shot and see if it works better with your difficult footage:


import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("E:\SC tests\AmcQ.d2v")
mvbob()
mvconv()
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()


(BTW if you have an older version you might get a script error on "convertttoyuy2" not being found... edit mvbob.avs and fix the typo to "converttoyuy2".)

[Clown shoes]

Trixter, can you see anything wrong with my implementation of mvflowfps2.

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\JUMP_BRITAIN\Jump Britain.d2v")
mvbob()
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=6000,den=1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

It is giving me amazing results except for some warping on fast moving scenes. I have posted a still on the previous page. This is not something that is apparent when I use motionprotectedfps though. Is there a fundemental difference between the two that may explain this. Or is it just my use of the filter?

[Didée]

Well, it's the typical "choose-your-poison" situation:
The depan-method is virtually free of artefacts, but will leave motion of moving objects stuttering. In some special situations, slight "backward-motion" phenomenae may occur.
The MVFlowFps/2 method will interpolate all motion, big or small one, but is prone to produce artefacts on not-compensateable motion (e.g. un/hidening objects).

There is no inbetween.

[foxyshadis]

Well, there is, but most of the research in that area is locked up with the brainiacs of Snell & Wilcox. MVFlow is about the best available with the currently available research; there should be ways to obtain a better confidence estimate for covered/uncovered objects but Clouded once told me that it's a lot harder than it sounds. Snell & Wilcox use algorithms that attempt to identify and track individual objects through an occulsion (how, I couldn't say, but I can guess the basics), but mvtools is nowhere near that strong/slow.

[Clown shoes]

Sorry Didee, I was actually comparing mvflowfps2 and motionprotectedfps which uses mg262s motion filter.

[Didée]

@ foxyshadis

Yeah. What I meant was "there is no inbetween with our current filters for Avisynth."
Ways to deal better with the problem surely do exist. I'd even have a bunch of 'em in front of my inner eye ... but who cares for those flatbrained scripters that can't even code a hello-world proggy in some modern C mutation. :)


@ Clown shoes

Yep, I know. My last post was directed at Trixter ... too much simultaneous posting back then. ;)

Regarding MVFlowFPS/2 -vs- MotionProtectedFPS:
While these two use different methods to do the vector search, technically they fight the very same problems, and in fact even their results are very similar. What makes them appear so different at first glance (you noticed that MotionFPS comes out with less artefacts) is that, with default(!) settings, MotionFPS is more cautious about avoiding artefacts than MVFlowFPS. The key is the "ml" value in MVFlowFPS, which defaults to "80". Try setting "ml" to, say, "1000" instead, and then compare both again. You'll see that MVFlowFPS2 now has much less artefacts. Probably it now even has a slight edge over MotionFPS. Still, MotionFPS comes very close ... and it comes in at much faster speed. ;)

[Chainmax]

I was using integers instead of floats? No wonder those "." were there in the readme :o.

Didée: it's funny that you mentioned the stuttering, I was showing portions of the video to my GF and at some point I jumped to a very stuttery scene. It's really disappointing since the script was doing so well on some of the most difficult scenes :(.
You give me, however, a spark of hope: you say artifacts decrease as ml increases, which setting would you say results in zero artifacts? Would such setting have negative effects besides speed (which is of no concern to me)?

[Didée]

Well, zero artefacts is out of reach ... except for not doing any framerate conversion ;)
Also, I've little clue what the safe zone for "ml" might be ... it's just today that I fiddled as much with framerate conversion as in the whole past year :D ... ml=1000 was just a random shot at some random high value. Seems to work not too bad, however ...

edit - speed should be about the same, since the protection work is done anyway. Somewhat similar like in MaskedMerge, where it won't matter if the mask covers almost all pixels, or just a few.

[Chainmax]

I'll try to see what setting for ml reduces artifacting the most in my clips and report back. In any case, how about this: a new parameter that would be linked to the artifacting (no idea how) and if it exceeds or falls below a user-defined threshold then the frame in question would be replaced by an interpolation of the previous and next frame?

[Clown shoes]

...and after testing it for about 30 minutes, I have to say that I like MVTools' MVFlowFPS2 much better -- the decrease in speed is well worth the estimation accuracy. I'll stick with them.

What is it you prefer about MVFlowFPS2? I have been comparing high motion sections frame by frame and although I can find hardly any difference between them, I would say I've found fractionaly more motion artifacts with MVFlowFPS2. I'm curious to know what advantage there is to the MVTools route in your opinion.

@ Didee

I tried increasing ml but I didn't see any difference at all! Very strange.

[Trixter]

Trixter, can you see anything wrong with my implementation of mvflowfps2.

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\JUMP_BRITAIN\Jump Britain.d2v")
mvbob()
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=6000,den=1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

It is giving me amazing results except for some warping on fast moving scenes. I have posted a still on the previous page. This is not something that is apparent when I use motionprotectedfps though. Is there a fundemental difference between the two that may explain this. Or is it just my use of the filter?

num should be 60000, not 6000. (as typed above, your script converts video to about 6fps; typo?) Otherwise, you're doing it right, and the still you posted looks like it has so much of a difference between frames that no amount of motion estimation could compensate. In cases like that, hardware converters fall back to a typical line interpolation mode for that field. In full motion, you hardly notice it. The results are very similar to something like ConvertFPS(59.94, zone=80). This leaves a "gradiated band" in one field that gradually switches from the previous image to the new image, but in full motion on a regular interlaced television it is extremely difficult to detect (of course it's obvious on a full field freeze-frame on computer).

Now for the news you don't want to hear: For a DVD I'm doing with source material in PAL with a LOT of full motion smooth movements (think dance videos), I am using MVFlowFPS2 for everything and I put that on track B; I then do a conversion using ConvertFPS(59.94, zone=80) and put that on track A. Anywhere MVFlowFPS2 makes a mistake, I cut that frame out and use the frame from the ConvertFPS conversion. Tedious, but it's pretty much a flawless conversion when I'm done.

[Trixter]

What is it you prefer about MVFlowFPS2? I have been comparing high motion sections frame by frame and although I can find hardly any difference between them, I would say I've found fractionaly more motion artifacts with MVFlowFPS2. I'm curious to know what advantage there is to the MVTools route in your opinion.


It depends on the source material, of course. My source material is very synthetic (it was all computer generated; for examples of the type of motion, think title rolls and crawls) and for me, MVTools results in slightly less artifacts (and the artifacts that are there are less severe). Of course you should use what works best for your footage.

Here is a particularly fast panning head using MVTools' MVFlowFPS2:

http://img220.imageshack.us/img220/3555/mvfps2example9ei.th.png (http://img220.imageshack.us/my.php?image=mvfps2example9ei.png)

...and here is the same interpolation as done by MotionProtectedFPS:

http://img139.imageshack.us/img139/3795/motionprotectedfpsexample2cm.th.png (http://img139.imageshack.us/my.php?image=motionprotectedfpsexample2cm.png)

As you can see, the MotionProtectedFPS example has more distortion, as well as chroma flying all over the place :-) This isn't a massive slam on MotionProtectedFPS, just an example of why I use MVFlowFPS2. If I'm doing a quickie for viewing an MPEG-1 on television or something, MotionProtectedFPS's speed is fantastic. If I'm transcoding footage for a DVD for commercial sale, I use MVTools for the most quality. Just my $0.02.

[videoFred]

Result is free or artefacts, of course. But really good it is only on clear pans. On "globally still" frames, all in-frame motion still is stuttering ... and on some difficult scenes (global shift to one side, but big objects moving to the other side), the stutter is, well, funny.
No surprise, though ... depan is doing exactly what it's designed to do: shifting the frame as a whole.

Because all my old 8mm films are all original made at 15-16-18fps, and without the use of a tripod, I need a good frame rate conversion/stabilising script.

I confirm the depan, method is perfect for clear pans...
But not for moving objects and/or zoom scenes. Stuttering!
I have the best results with mvflowfps() followed by depan for stabilising.

Motionprotectedfps() gives very good results too, and it is much faster.

Actualy, both are giving me near perfect results in 90% of the cases... The artefacts are almost invisible on TV.

Very fast moving people/objects, original recorded at low speeds like 15FPS are giving me the most problems. Maybe I am going to use convertfps() for these scenes instead.

*EDIT* MvFlowFPS2 works a lot better for this.



Fred.

[Chainmax]

...
@ Didee

I tried increasing ml but I didn't see any difference at all! Very strange.

Neither did I on my problem video :(.

...
Now for the news you don't want to hear: For a DVD I'm doing with source material in PAL with a LOT of full motion smooth movements (think dance videos), I am using MVFlowFPS2 for everything and I put that on track B; I then do a conversion using ConvertFPS(59.94, zone=80) and put that on track A. Anywhere MVFlowFPS2 makes a mistake, I cut that frame out and use the frame from the ConvertFPS conversion. Tedious, but it's pretty much a flawless conversion when I'm done.

That's exactly what I was trying to do a while back but didn't know what to use as an alternate framerate converter. I'll try that, thanks for the suggestion :). I have a question though, I use the following MVFlowFPS lines:

vf=last.mvanalyse(isb=false,blksize=4,pel=2,search=3,truemotion=true)
vb=last.mvanalyse(isb=true,blksize=4,pel=2,search=3,truemotion=true)
MVFlowFPS(last,vb,vf,num=30000,den=1001)

Would that give worse results than MVFlowFPS2?

[Clown shoes]

num should be 60000, not 6000.

lol, yes that's most definately a typo!

I then do a conversion using ConvertFPS(59.94, zone=80) and put that on track A. Anywhere MVFlowFPS2 makes a mistake, I cut that frame out and use the frame from the ConvertFPS conversion. Tedious, but it's pretty much a flawless conversion when I'm done.

Sounds like a good idea, but wow that is really gonna add to the overall time it takes to do a standards conversion. That is exactly what I asked for though, the best quality disregarding time as an issue. So thank you trixter and it's good to hear you using it in a commercial enviroment. If a client comes to me with either a short peice of work or a longer peice where time is not an issue. I am now confident I can produce a professional looking standards conversion on a par with a hardware solution.

[Trixter]

That's exactly what I was trying to do a while back but didn't know what to use as an alternate framerate converter. I'll try that, thanks for the suggestion :). I have a question though, I use the following MVFlowFPS lines:

vf=last.mvanalyse(isb=false,blksize=4,pel=2,search=3,truemotion=true)
vb=last.mvanalyse(isb=true,blksize=4,pel=2,search=3,truemotion=true)
MVFlowFPS(last,vb,vf,num=30000,den=1001)

Would that give worse results than MVFlowFPS2?

That completely depends on your source material, so I can't answer that directly.

I see that you shrunk blocksize from 8 to 4... from the docs, larger blocks are less sensitive to noise, are faster, but also less accurate. A blocksize of 4 might exacerbate the noise already present in your source. I see you increased the search algorithm to exhaustive... with small blocks, you're going to be waiting a very long time :-)

MVFlowFPS2 uses two different motion analyses: A regular one and another one with the entire frame shifted half the block size diagonally. It then averages the results of both so that there are less estimation errors. So MVFlowFPS2 is always going to provide better motion estimation than just MVFlowFPS.

I would only use small block sizes like 4 if your source is less than 480 lines (like 320x240 640x360). If you're working with 720x480 or source, 8 is sufficient.

Finally, doing some quick math off the top of my head, MVFlowFPS2 with the default (subdividing) search algo and 8x8 blocksize should run faster than MVFlowFPS with an exhaustive search and 4x4 blocksize.

I shudder to think how long 4x4, exhaustive, FPS2 would take ;-)

[Chainmax]

...
I see that you shrunk blocksize from 8 to 4... from the docs, larger blocks are less sensitive to noise, are faster, but also less accurate. A blocksize of 4 might exacerbate the noise already present in your source.
...
I assumed that lower blocksizes meant better motion end results.
By the way, once I have the MVFlowFPS/2 and ChangeFPS encodes, is there a faster way to replace individual frames or frame ranges of the former with the corresponding ones from the latter than cutting and pasting (i.é: can I do it directly from Avisynth)?

[Clown shoes]

I would imagine the quickest way would be using an NLE like Avid or Premiere. Especially one with multicam mode which Avid definately has.

[Trixter]

I would imagine the quickest way would be using an NLE like Avid or Premiere. Especially one with multicam mode which Avid definately has.

I'm using Premiere Pro 1.5 currently (but as soon as this project is done you can be sure I'm going to upgrade). Multicam switching is for "live" switches (you literally press buttons while the video is running, like a real broadcast switcher), but I'm switching to a single frame, maybe two, so manually cutting the bad frames out is pretty much the only way to do it.

As for an AVISYNTH only solution, I would imagine you can use trim() but I've never done it... I would love to hear from the seasoned scriptwriters how something like this would be done.

[Trixter]

I assumed that lower blocksizes meant better motion end results.

Last night I did a quick test using smaller blocksizes and it actually made the bad frames even more worse. I'll post pictures when I get home (I'm at work now).

[Chainmax]

...
I am using MVFlowFPS2 for everything and I put that on track B; I then do a conversion using ConvertFPS(59.94, zone=80) and put that on track A. Anywhere MVFlowFPS2 makes a mistake, I cut that frame out and use the frame from the ConvertFPS conversion. Tedious, but it's pretty much a flawless conversion when I'm done.

I am trying ConvertFPS(29.97, zone=80) on a 15-to-29.97 conversion and I have to say that ConvertFPS does an awful job. When stepping through the ConvertFPS script, it sometimes a given frame is exactly like the previous one but with the bottom half shifted to the left or right. It's very nasty :(.

I really wish MVTools could be altered so that a new MVFlowFPS parameter would be created. This new parameter would be linked to the artifacting (no idea how) and if in a given frame it exceeds or falls below a user-defined threshold then the frame in question would be replaced by an interpolation of the previous and next frame. How feasible would that be?

[actionman133]

Last night I did a quick test using smaller blocksizes and it actually made the bad frames even more worse. I'll post pictures when I get home (I'm at work now).

I agree with Trixter on this. I've found that lower blksize = 4 produces poorer results than blksize=8. I attributed it to the fact that a 4x4 block doesn't have much detail in it to provide reasonable vectors, whereas an 8x8 block can show better edges, and thus get a better vector. I also think the radius defined by searchparam is directly related to the block size. So blksize=4 has half the search radius of blksize=8....

I might be wrong on that last point, so feel free to correct me if I am.

[Revgen]

I wish there was a way to adaptively use multiple block sizes kinda like what X264 does.

[actionman133]

Perhaps an algorithm could be written where it initially tests 16x16 blocks. If the amount of detail (determined by some algorithm that searches for texture) in a block exceeds one threshold, it reduces that block to 4 blocks of 8x8. If it exceeds a higher threshold, it could reduce it to 16 blocks of 4x4.

Would that be possible?

Actually, should this go to the MVTools thread?

[Trixter]

I am trying ConvertFPS(29.97, zone=80) on a 15-to-29.97 conversion and I have to say that ConvertFPS does an awful job. When stepping through the ConvertFPS script, it sometimes a given frame is exactly like the previous one but with the bottom half shifted to the left or right. It's very nasty :(.

Well, you have three options without motion compensation:


ChangeFPS: Add or drop frames to meet target. Good: No blended frames. Bad: Jerky motion on pans.
ConvertFPS: Blend frames across time to meet target. Good: Softer (not smoother) motion. Bad: Blended frames.
ConvertFPS(zone): Blend frames, but only at the precise moment in a CRT's scanrate. Good: Precise moment-in-time switches. Bad: Individual "half" frames.


Yes, with the third option you can see a frame blend across 80 lines into the new image... but when you're watching on a television that scans top to bottom 60 times a second, 99% of the population won't notice it. It hurts horizontal panning/crawls, but little else. And remember, most people aren't going to be pausing to stare at a bad image that is 1/60th of a second long :-)

Since my DVD is mastered specifically for television, the zone=80 method is what I'm using. In motion, it's less jerky than without zone=80. Freeze-framed, it has those "inbetween" frames. So you have to pick what you like best.


I really wish MVTools could be altered so that a new MVFlowFPS parameter would be created. This new parameter would be linked to the artifacting (no idea how) and if in a given frame it exceeds or falls below a user-defined threshold then the frame in question would be replaced by an interpolation of the previous and next frame. How feasible would that be?

Not at all, since what metric would you have that the estimation was wrong? How could you tell? Remember, you're creating frames where none exist. Our eyes can tell us when something doesn't look right, but to the mocomp algorithm, it's making it's best guess.

That being said, it might be possible to "fall through" to a ChangeFPS/ConvertFPS frame if the certainty of the mocomp vector analysis falls below a certain threshhold, but I have no idea how to script that. Scharfis might have already done something like that somewhere in mvbob() but I'm not brave enough to comment every line to find out.

[Trixter]

I agree with Trixter on this. I've found that lower blksize = 4 produces poorer results than blksize=8.

Thanks, you saved me from making new shots ;-)

[actionman133]

Your welcome, Trixter... always glad to help where I can. :D

But do you reckon the possibility of variable block size analysis would be feasible for application? They do it for motion compensation in video compression... Would it work with MVFlowFPS?

[foxyshadis]

x264 has some of the most advanced ME algorithms available in the open source world, if they could be cut-and-pasted into mvtools I'm sure it would improve output (and cut speed), though it'd take quite a bit of work to port their interfaces. It's certainly possible. On the other hand, x264 gets to rely on texture bits to make up for any deficiencies, and it's designed for qpel (although it doesn't necessarily have to use it).

It'd certainly be a cool summer project by someone interested. ;)

[actionman133]

Should this recommendation be forwarded over to the MVTools thread?

[Mug Funky]

FWIW, here's a _severely_ bodge-hacked version of motionprotectedfps that provides an extra (important) layer of protection.

i'm currently using it for NTSC to PAL conversions that include pure interlaced, progressive, telecine and blendy video.

basically it blends when it can't compensate, and uses a very conservative mask with soft edges.

i'm sure it can be greatly improved, at the very least by code cleanup...

function salFPS(
\ clip input, float FPS,
\ int "mode", int "protection", int "protection2",
\ int "iterate", int "reset", int "initialise")
{
mode = default(mode, warpfast)
protection = default(protection, 80)
protection2 = default(protection2, 32)
iterate = default(iterate, 2)
reset = default(reset, 50)
initialise = default(initialise, 6)

originalFPS = input.framerate()
input

mp = FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise)
p = MotionFPS(FPS,mode,Motion = mp)
maskp = mp.SumStretchFPSMask(FPS, protection)

mn = FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise, from = next)
n = MotionFPS(FPS,mode,Motion = mn, source = next)
maskn = mn.SumStretchFPSMask(FPS, protection, source = next)

maskp
assumeFPS(originalFPS)

tehmask=last.CombineFPS(FPS, maskp, maskn)
last.CombineFPS(FPS, maskp, maskn.Invert())
bilinearresize(width*8, height*8)
crop(0, 0, input.width(), input.height())
fity2uv()
mpro=last

tehmask
mt_binarize(protection2,false)
mt_expand()
mt_inflate()
bicubicresize(width*8, height*8,1,0)
crop(0, 0, input.width(), input.height())
fity2uv()
tehmask=last
mt_merge(p, n, mpro, u=3, v=3)

mt_merge(last,input.blendfps(FPS),tehmask, u=3,v=3)
#merge(last,tehmask.greyscale(),.5)
}

if you uncomment the last line it'll show you which parts are being blended (white) and which compensated (black).

"protection2" is the one to vary - it's a crude threshold.

enjoy :)

[edit]

did i mention that this produces almost no visible artefacts with default settings on "generic" video?

[Chainmax]

I can't wait to try this once it's optimized. Thanks for a new toy to play with :).

[Mug Funky]

you can try it now... speed isn't much different from the original motionprotectedfps.

the code is just a little uglier because i was in a hurry.

i think i might need to think a little more about how i go about choosing which parts to blend and which to compensate, as there's a bit of "noise around the edges" so to speak, where parts of the same object get different treatments when motion gets higher. it looks better than regular blendfps, but could be better still.

also, it sucks at making slow-motion. use the original script for that...

[Didée]

Small suggestion to Clouded's part of the script: Make the following mod:
crop(0,0,width+8,height+8,true)
bicubicresize(width*8, height*8,1,0)
crop(0, 0, input.width(), input.height(),true)
It's not a big deal ... but upsizing a original-resolution clip to 800% * 800% of the original resolution, just to immediately crop & keep an 1.5% area of the huge 6400% frame, is not exactly "efficient". ;)


About getting a more on-spot error correction ... well, it would be doable, but rather complicated, and very slow. For sure, if you try to do it with an Avisynth script.
The current methods of MVFlowFPS and Motion(Protected)FPS are solely based on vector probabilities and vector SADs. And these measures, albeit giving usable indications, are simply not significant enough to make reliable decisions. They didn't do for full compensations, and the same way they'll not do for intermediate interpolations. For full compensation, meanwhile I've a method that works out clearly better ... but it's almost impossible (at least extremely hard & cumbersome) to translate that method to the interpolation domain. As already said, the plugins don't offer enough grips to ply with the vectors.
For the limited case of interpolating to 2*fps, something could be hacked together, probably. If someone could supply me with an 8th day per week, that is.

[Backwoods]

For the limited case of interpolating to 2*fps, something could be hacked together, probably. If someone could supply me with an 8th day per week, that is.

I have 3 days off every other week, I could give you one of those once in a while.

[Didée]

Yippieh, thank you. Send'em by postal, please :D

---

Okay, here is something to try. It's not at all what I had in mind, but another approach. A trick, better said.

The most annoying parts are those where the warping creates those "funny structures", aren't they. By try-and-error, I found that compairing

edges(source)-->MVConvertFPS

with

edges(source-->MVConvertFPS)

seems to give a halfway usable indication of those typical artefacts. The following script builds a mask out of this comparison, and as a replacement copies a frameblend onto the affected areas.
It's far from perfect, and surely some spots can be found where it looks worse than before. But overall, it looks like an improvement.

(Needs mt_masktools.dll) fps_num = 50 # Numerator & Denominator for
fps_den = 1 # target framerate, as usual
edgemode = "prewitt" # or "min/max"
sensitivity = 9 # the lower, the more blending

clp = last
ox = clp.width()
oy = clp.height()
SNS = string(sensitivity)

#blend = interleave(clp,merge(clp,clp.deleteframe(0),0.5))
blend = o.converttoyuy2().convertfps(float(fps_num)/fps_den).converttoyv12()
edge_orig = clp.mt_edge(mode=edgemode,thY1=0,thY2=255)

bw1_vec1 = clp.MVAnalyse(isb = true, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
fw1_vec1 = clp.MVAnalyse(isb = false, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
cropped = clp.crop(4,4,-4,-4,true)
bw1_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)
fw1_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)

FPSconverted = clp.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=1,idx2=2)
FPSconverted_edge = FPSconverted.mt_edge(mode=edgemode,thY1=0,thY2=255)
edge_FPSconverted = edge_orig.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=3,idx2=4)

edgediff = mt_lutxy(edge_FPSconverted,FPSconverted_edge,yexpr="y x - 1 - x 1 + / 300 * x 2 ^ x 2 ^ "+SNS+" + / *",U=1,V=1)
repmask = edgediff.mt_expand().mt_expand().bicubicresize(ox/3/4*4,oy/3/4*4)
\ .mt_inflate().blur(1).bicubicresize(ox,oy,1,0)

mt_merge(FPSconverted,blend,repmask,U=3,V=3,luma=true)

Result on a short sequence:

http://img256.imageshack.us/img256/4751/motiontestzb7.th.jpg (http://img256.imageshack.us/my.php?image=motiontestzb7.jpg)


But now, please, somebody help out the clueless n00b that I am ...
I've tried it only on 25-->50 conversion. In the script, you can see that I used a manual blending for the replacement. This is okay for integral FPS conversion, but obviously not for free ones.
Now, just a minute ago, I discovered that the "ConvertFPS" blending looks quite different than my manual one ... it doesn't even look like an even merge. Why is this? Shouldn't they be identical in the case of 25->50 fps? What is ConvertFPS actually doing?

[foxyshadis]

Perhaps instead of plain blending, replacing with a blend of back & forward MVDepan (you already have the analysis, after all), but I don't think MVDepan can actually give you that sort of interframe interpolation. (Perhaps for v1.5.0.) Object-based depan would be even cooler as a fallback, but I'm sure it's an area fraught with dragons.

Any "trick function" of yours usually becomes a widely recommended , cpu-abusive filter, so make it a good one. ;) btw, it might've been nice to add a pure convertfps row on your example, to find out if it looks much better than both or leans too heavily on the convertfps.

[Didée]

Yeah, right ... of above 5 frames, here are the middle 3 frames for comparison, with a plain blend conversion:

http://img69.imageshack.us/img69/3094/motiontest2ako7.th.jpg (http://img69.imageshack.us/my.php?image=motiontest2ako7.jpg)

In comparison with the previous 5-frame shot, one can see that the correctly interpolated areas are hardly affected ... from another point of view:

These are the masks that come up from the edgemask-comparison -- the raw errormask on top, on bottom the broadened mask for covering the suspicious areas:

http://img107.imageshack.us/img107/7148/motiontest2bmx0.th.jpg (http://img107.imageshack.us/my.php?image=motiontest2bmx0.jpg)


However, I fear there isn't too much headroom to further improve this thing. The sensitivity can be adjusted, and perhaps the raw errormask could get a little more hotpixel removed, before broadening. But in general, I don't see how to make it more intelligent ... I'm surprised enough that it works as nice as it actually does. ;)

***

Object-based depan, eh? Well, spelled it is easily. :D

[foxyshadis]

Ah, yes, that really does seem to be the best of both worlds (as far as a quick and dirty function goes). I'll point fizick toward it, if he hasn't seen it already, and perhaps a variation on it can be included in the next revision as a better mask.

I realized just now that "object-based depan" could be approximated simply with a big block size. 48 or 64 or more. I wonder if playing around with ml would force it to be more like a block-based compensation that would presumably look better for fallback, given that it's flow's artifacts we're trying to avoid here. ^^;

[Chainmax]

http://img204.imageshack.us/img204/5669/lovethreadxs3.jpg (http://imageshack.us)

:)

[Fizick]

Didйe,
Yes, ConvertFPS has a bug.

int nsrc = int( n * fa / fb );
int frac = int( (((n*fa) % fb) << resolution) / fb );

if( zone < 0 ) {

// Mode 1: Blend full frames
int mix_ratio = one - min( int( (fb * (one - frac)) / fa ), one);

// Don't bother if the blend ratio is small
if( mix_ratio < threshold )
return child->GetFrame(nsrc, env);
....

a_data[x] += ((b_data[x] - a_data[x]) * mix_ratio + half) >> resolution;



IMHO, must be simple :


int mix_ratio = frac;

I can change v2.5.7 source in CVS, while IanB is busy :)

Thanks for interesting discusion.
Some comments to rate conversion.
MVFlowFPS currently produces sharp output.
Probably I will add some (motion ?) blur to "funny" regions in next version (and some parameter to control it).
May be by weigthed mix of original, shifted, and half-way shifted pixels.

[Chainmax]

...
(Needs mt_masktools.dll) fps_num = 50 # Numerator & Denominator for
fps_den = 1 # target framerate, as usual
edgemode = "prewitt" # or "min/max"
sensitivity = 9 # the lower, the more blending

clp = last
ox = clp.width()
oy = clp.height()
SNS = string(sensitivity)

#blend = interleave(clp,merge(clp,clp.deleteframe(0),0.5))
blend = o.converttoyuy2().convertfps(float(fps_num)/fps_den).converttoyv12()
edge_orig = clp.mt_edge(mode=edgemode,thY1=0,thY2=255)

bw1_vec1 = clp.MVAnalyse(isb = true, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
fw1_vec1 = clp.MVAnalyse(isb = false, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
cropped = clp.crop(4,4,-4,-4,true)
bw1_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)
fw1_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)

FPSconverted = clp.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=1,idx2=2)
FPSconverted_edge = FPSconverted.mt_edge(mode=edgemode,thY1=0,thY2=255)
edge_FPSconverted = edge_orig.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=3,idx2=4)

edgediff = mt_lutxy(edge_FPSconverted,FPSconverted_edge,yexpr="y x - 1 - x 1 + / 300 * x 2 ^ x 2 ^ "+SNS+" + / *",U=1,V=1)
repmask = edgediff.mt_expand().mt_expand().bicubicresize(ox/3/4*4,oy/3/4*4)
\ .mt_inflate().blur(1).bicubicresize(ox,oy,1,0)

mt_merge(FPSconverted,blend,repmask,U=3,V=3,luma=true)
...


So, this needs MT_MaskTools and MVTools, right? Can someone transform it into a standalone function?

[Boulder]

Would it simply be
function WhatEver(clip clp, int "fps_num", int "fps_den", string "edgemode", int "sensitivity")
{
fps_num = default(fps_num, 50) # Numerator & Denominator for
fps_den = default(fps_den, 1) # target framerate, as usual
edgemode = default(edgemode, "prewitt") # or "min/max"
sensitivity = default(sensitivity, 9) # the lower, the more blending

ox = clp.width()
oy = clp.height()
SNS = string(sensitivity)

#blend = interleave(clp,merge(clp,clp.deleteframe(0),0.5))
blend = o.converttoyuy2().convertfps(float(fps_num)/fps_den).converttoyv12()
edge_orig = clp.mt_edge(mode=edgemode,thY1=0,thY2=255)

bw1_vec1 = clp.MVAnalyse(isb = true, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
fw1_vec1 = clp.MVAnalyse(isb = false, truemotion=true, lambda=200, searchparam=8, pel=2, idx=1)
cropped = clp.crop(4,4,-4,-4,true)
bw1_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)
fw1_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, lambda=4000, searchparam=8, pel=2, idx=2)

FPSconverted = clp.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=1,idx2=2)
FPSconverted_edge = FPSconverted.mt_edge(mode=edgemode,thY1=0,thY2=255)
edge_FPSconverted = edge_orig.MVFlowFps2(bw1_vec1,fw1_vec1,bw1_vec2,fw1_vec2,num=fps_num,den=fps_den,idx=3,idx2=4)

edgediff = mt_lutxy(edge_FPSconverted,FPSconverted_edge,yexpr="y x - 1 - x 1 + / 300 * x 2 ^ x 2 ^ "+SNS+" + / *",U=1,V=1)
repmask = edgediff.mt_expand().mt_expand().bicubicresize(ox/3/4*4,oy/3/4*4)
\ .mt_inflate().blur(1).bicubicresize(ox,oy,1,0)

return mt_merge(FPSconverted,blend,repmask,U=3,V=3,luma=true)
}

Probably not :D

[Didée]

Yep, that's all. Adding a function header line, one { on top, one } at bottom, and do the "default()" thingy for the parameters. Easy, isn't it? ;)

Next exercise: porting for motion.dll filters. :)

[Chainmax]

Thanks, Boulder :). So, the dependencies are MVTools and MT_Masktools only? Also, it doesn't have anything to do with salFPS (what Mug Funky posted), right?

[Pookie]

Boulder - Shouldn't the line

blend = o.converttoyuy2()

be modified to

blend = clp.converttoyuy2()

Maybe I missed the "o" variable declaration earlier in the thread.

[Didée]

Yes, it should be like that.

Oh-oh, my fault. I missed that one during replacing all o's with clp's, in order to place a linear script in front of Chainmax, so that he'll feel the urgent need to make a function out of it.
... The important thing is, the fish has bitten. :D

[Chainmax]

At the risk of quoting my custom title: huh?

[Backwoods]

German Didée English to American English translation: Something slipped through while changing calls over, whoops! Well at least people are interested and caught the mistake.

I haven't tried it yet (nothing on my machine to play with), have you Chainmax?

[Chainmax]

Will do ASAP, I have a 15fps video that shows very noticeable artifacts on rapid camera movements.

[Chainmax]

Brom a very brief gander at one trouble scene, there's still quite a bit of artifacting. IIRC, it's not as messy as plain MVFlowFPS/2, but it's still there.

I just had an idea: remember how I was talking about trying to come up with a metric that would be tied to the artifacting and would be used as a threshold? Well, MVAnalyse can estimate motion, right? What if Didée's function was altered so that when motion exceeds a (user definable) threshold then the extra frames would be created via duplication or interpolation of the source's frames?

[PeppeontheNet]

Hello,
taking pieces of code here and there....

PAL to NTSC conversion

loadplugin("C:\Programmi\AviSynth 2.5\plugins\dgdecode.dll")
loadplugin("C:\Programmi\AviSynth 2.5\plugins\tdeint.dll")
loadplugin("C:\Programmi\AviSynth 2.5\plugins\tivtc.dll")
loadplugin("C:\Programmi\AviSynth 2.5\plugins\depan.dll")
video=MPEG2Source("C:\...\bbb.d2v").LanczosResize(352,576)
audio=WAVsource("C:\...\bbb T01 2_0ch 256Kbps 48KHz.wav")
audiodub(video,audio)
delayaudio(0.0)
blindpp(cpu=4, iPP=true)
assumetff()
TDeint(mode=1, tryweave=true, full=false)
Bicubicresize(width, 480)
pal2ntsc()
separatefields().selectevery(4,0,3).weave()

NTSC to PAL conversion

loadplugin("C:\Programmi\AviSynth 2.5\plugins\dgdecode.dll")
loadplugin("C:\Programmi\AviSynth 2.5\plugins\tdeint.dll")
loadplugin("C:\Programmi\AviSynth 2.5\plugins\tivtc.dll")
loadplugin("C:\Programmi\AviSynth 2.5\plugins\depan.dll")
video=MPEG2Source("C:\...\aaa.d2v").Bicubicresize(352, 480)
audio=WAVsource("C:\...\aaa T01 2_0ch 256Kbps 48KHz.wav")
audiodub(video,audio)
delayaudio(0.0)
blindpp(cpu=4, iPP=true)
assumetff()
TDeint(mode=1, tryweave=true, full=false)
ntsc2pal()
LanczosResize(352,576)
separatefields().selectevery(4,0,3).weave()

where pal2ntsc() and ntsc2pal() are functions written in an .avsi file placed in the avisynth plugins' directory.
Here's that .avsi file:

function PAL2NTSC(clip video)
{
backward_vec = video.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
forward_vec = video.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = video.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return video.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=60000,den=1001,idx=1,idx2=2)
}

function NTSC2PAL(clip video)
{
backward_vec = video.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
forward_vec = video.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = video.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return video.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=50,den=1,idx=1,idx2=2)
}

These are the outcomes:
PAL to NTSC sample (http://www.savefile.com/files/206253)
NTSC to PAL sample (http://www.savefile.com/files/206377)

I cannot judge the quality of the PAL to NTSC conversion, because I don't have an NTSC tv screen.... but the outcome of the NTSC to PAL conversion is, by far, the best I have ever experienced.

Suggestions to improve?

[Mug Funky]

here's my latest attempt:

function salFPS3(
\ clip input, float FPS,
\ int "mode", int "protection", int "protection2",
\ int "iterate", int "reset", int "initialise")
{
mode = default(mode, warp)#fast)
protection = default(protection, 80)
protection2= default(protection2,20)
iterate = default(iterate, 2)
reset = default(reset, 50)
initialise = default(initialise, 6)

originalFPS = input.framerate()
input

mp = FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise)
p = MotionFPS(FPS,mode,Motion = mp)
p_edge = mt_edge(mode="prewitt",thy1=protection2,thy2=0).MotionFPS(FPS,mode,Motion = mp)
maskp = mp.SumStretchFPSMask(FPS, protection)

mn = FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise, from = next)
n = MotionFPS(FPS,mode,Motion = mn, source = next)
n_edge = mt_edge(mode="prewitt",thy1=protection2,thy2=0).MotionFPS(FPS,mode,Motion = mn, source=next)
maskn = mn.SumStretchFPSMask(FPS, protection, source = next)

maskp
assumeFPS(originalFPS)
CombineFPS(FPS, maskp, maskn.Invert())
bilinearresize(width*8, height*8)
crop(0, 0, input.width(), input.height())
fity2uv()
mpro=last

mt_merge(p_edge, n_edge, last, u=3, v=3)

mt_lut(expr="128 x 128 - abs - 2 *")
mt_inpand().greyscale().converttorgb32()
bicubicresize(width/8,height/8,1/3.,1/3.)
mod16().converttoyv12(matrix="pc.601")
mt_binarize(protection2,false)
temporalsoften(1,255,255,255)
mt_binarize(80,false)
mt_expand(mode=mt_circle(3))
mt_inflate()
bicubicresize(width*8, height*8,1,0)
crop(0, 0, 720, 480)
fity2uv()
tehmask=last

mt_merge(p, n, mpro, u=3, v=3)

mt_merge(last,input.blendfps(FPS,aperture=FPS/float(input.framerate()))
\,tehmask, u=3,v=3)
}


basically, it builds on didee's edgemasking idea.

the same motion data is used to motion compensate both the original clip and a binary edgemask of it.

my assumption is that when the vectors are poor, the converted binary mask will be less binary :) run a LUT on it that makes the middle tones white and the darks and lights both black, and you have your mask.

this appears to work pretty well on some 30p animation i'm playing with (intro to Clone High), but i haven't tried it on natural video yet. i'll do that on monday when i'm at work and have some video to play with...

observations so far:

it seems to work the way i thought it would, however that doesn't play out visually - the edges of objects go very wobbly sometimes.

i think we need some more tools in our mask-making endeavors. namely something that can detect objects, or at least follow a likely edge around objects. this way we can decide on an object-by-object basis whether blending or compensation is the way to go. however, with that said this kind of data can also very much improve motion compensation, and possibly remove much of the need for blending - if we can detect objects we can define their motion and interpolate it, possibly taking occlusion into account. i think we're a fair way off that.

so anyway, have a look at this script and see if it gives anyone some ideas...

[krieger2005]

What is "mod16()"?

[Mug Funky]

ah crap, sry i forgot about that. it's a little function to force a clip to mod16 size, so it can be worked with more easily. it's got a counterpart "unmod16()", but that's not used in there.

function Mod16 (clip c, bool "mirror")
{
mirror = default(mirror,false)

modwidth = ceil(c.width/16 + 0.5)*16
modheight = ceil(c.height/16 + 0.5)*16

out = c.addborders(0,0,modwidth - c.width,modheight - c.height)
out2 = stackvertical(stackhorizontal(c,c.fliphorizontal()),stackhorizontal(c.flipvertical(),c.turn180())).crop(0,0,modwidth,modheight,align=true)
global gl_diff_width = int(out.width - c.width)
global gl_diff_height = int(out.height - c.height)
mirror==false? out : out2
}

function unMod16 (clip c)
{
c.crop(0,0,-gl_diff_width,-gl_diff_height)
}

just copypasta that somewhere and it should work.

[Chainmax]

Mug Funky, do you want me to upload a short clip of the trouble scene I used to test some of the early methods posted in here?

[Chainmax]

By the way, do you think it would be possible to include some sort of motion metric (DeShaker uses one on the first pass IIRC) and on scenes with motion higher than a threshold the frames that have to be created would be a blend of its two neighbors?

[Mug Funky]

there probably wouldn't be much to be gained from that, speed wise.

consider that motion.dll tools by default need 50 frames previous to the current frame to initialise... this means a conditional filter that's decided 3 frames in a talking head video have not enough motion to need compensation will actually be performing a motion search on an extra 50 frames once the low motion frames are done with, and 47 of these frames will have been searched before already... that's a fair bit of double-handling.

i know because the current incarnation of "autoPAL" does similar to this, and lemme tell you it's a big and unnecessary speed hit. of course you can lower the "reset" parm in the appropriate functions, but this will have a (very slightly) negative effect on smooth video.

btw, when the above function is working the way i'd like, it could probably be posted in this thread as an alternative to the alchemist - already it's kicking quite a bit of arse, it's just got a couple of sticking points (like 3:2 pattern changes getting blended unnecessarily, and very subtle but visible interlaced motion being picked up as film because the metrics say yes while the heart says no).

[foxyshadis]

That sounds like a very interesting alternative to the hclip I suggested for TDecimate. It's rather more widely applicable too. Based on your experience with it, would you say it'd be fairly easy to take any such mess and generate a single progressive stream in a somewhat arbitrary framerate? For instance, if this was to be used in NTSC hybrid->cfr conversions, or pc-based applications with integral framerates.

[Mug Funky]

with a good enough framerate converter (or an interlaced output), it's perfectly possible to go to an arbitrary frame rate. i've implemented a very simple function that takes the "data" clip (8x4 red, green or blue clip that's the output of the analysis function) and outputs whatever framerate you ask for. it's not optimal yet because i've been focusing on the NTSC-to-PAL part of things for the last couple of months.

certainly if tfm could output a similar data clip, it'd be very useful - it'd be faster and support more than the vanilla 3:2 with no edits which my scripted version detects. there'd also be less double-handling as it could be used as hints for the IVTC that follows (right now there's tons of double-handling in autopal mainly due to my own laziness - expanding the format of the data clip from more than just 3 colours to something that includes the position in the 3:2 cycle and stuff like that would be good).

[Chainmax]

there probably wouldn't be much to be gained from that, speed wise.

consider that motion.dll tools by default need 50 frames previous to the current frame to initialise... this means a conditional filter that's decided 3 frames in a talking head video have not enough motion to need compensation will actually be performing a motion search on an extra 50 frames once the low motion frames are done with, and 47 of these frames will have been searched before already... that's a fair bit of double-handling.

i know because the current incarnation of "autoPAL" does similar to this, and lemme tell you it's a big and unnecessary speed hit. of course you can lower the "reset" parm in the appropriate functions, but this will have a (very slightly) negative effect on smooth video.
...

It can always be made optional.

[Mug Funky]

it could, but to what end? blending frames with low enough motion would give similar-but-probably-worse results than motion-compensation, and take longer unless the video being processed is very still indeed. i don't see a reason to use it to be honest, though i think there's room for the mocomped converting function to take low motion into account and work faster to save time - perhaps regular motionprotectedfps (very fast) gets used if the motion-masks used are below a certain peak luma or something? it'd have to be something that gives a net gain in speed though, so metrics have to be very simple and fast to calculate.

[Chainmax]

Actually, what I was referring to was to use blending in scenes with high motion, which is usually when warping occurs. In any case, do you want me to upload the clip I talked about for testing purposes?

[Mug Funky]

aah. okay.

uploading wont be necessary - i haven't really got the time to check out samples :( (there's enough stuff here to deal with already :)).

one possible problem is that pans can be very high motion and yet still be easily compensated. what's needed is a good way to detect warping. didee's edge-detection idea is the strongest contender so far (i had a shot at one, but it's severely lacking in some cases, like sudden luma changes where no motion is detected. whole frames can get very odd looking). perhaps if edge-based warp detection could be combined with the block-warping masks made in motionprotectedfps something nice could be done. i'm not sure how fast such a solution would be though.

i've read a little on what akupenguin's wanting to do with snow - mesh-based warping falling back on OBMC on edges. perhaps the switching decision does something we'd be interested in seeing here?

[Didée]

Some time ago I had the following thought about (perhaps) improving on the warping-artefacts problem of motion interpolation. However it's impossible to do by a script - the needed steps can only be done by the motion engine itself.

Basic problem:
A motion vector in itself might appear reasonably good (low enough SAD-or-whatever) when looking its result at the 'discrete' (frame->frame) target point. But then, although the result of the "full" vector seems good, its result is visually bad when only a *part* of the vector is used. ("part" of a vector -> e.g. the "time" parameter of interpolating commands). This is because in fact the vector is not aligned with the "true" motion.
Now, the machine won't really reckognize what the "true" motion really is, because the machine is dumb. Also, there is no reference frame available that we could check the created intermediate frame against. (Frame->frame compensations, as e.g. in denoising, can be checked nicely that way ... but not fractional interpolations.)

Okay, here the idea:
A filter like MVFlowFPS creates a new frame, at some certain "time" between to existing frames. To create this inbetween frame, of each motion vector's length only a certain fraction is used.

Now, do the following:
Take the created inbetween frame (which is possibly containing artefacts), and do "the remaining" compensation. Means: for each block/pixel, make an interpolation by using those parts of the vectors that have not been used in the former interpolation step. Then, compare this 2nd-step interpolation to the destination frame. If the difference (TwoStepInterpolation - Destination) is noticeably bigger than the difference (OneStepCompensation - Destination), then the related vectors are marked as bad. Finally, the one-step interpolation is recreated, now with the additional information of which vectors are bad.

Visually (kind of):
We have frames A and B. Needed is creation of an intermediate i(A,B), to get A i(A,B) B.
Currently, vectors v(A,B) are searched. Then by means of v(A,B), full compensations A' and B' are created. (Ignoring the checks of vector disrupture etc.), it is checked A against A' and B against B' to decide if vectors are good or not (SAD measure). If yes, from v(A,B) the needed part v'(A,B) is used to create i(A,B).
My suggestion is to not only check the result of A -> A' against B, but to do
A -> i(A,B) -> A'', and to check A'' against B. The artefacts appearing in i(A,B) will also distort A'', so that checking A'' against B should give a reasonable measure to better judge the vectors.


Overall, this sounds more dreadful than it is. Basically there's only one point which I can't judge if it's easy, difficult or impossible to do: for each pixel, it must be memorized and/or figured which parts of which vectors have been used to create the "normal" inbetween interpolation, in order to later on use the not-used "remaining parts" of the vectors.
That's the cruicial point. Can this be done, or can it not? The obvious foottrap is the interpolation-of-vectors that's used by the warping technique.

IF this is possible, then I suspect that the warping artefacts appearing with the normal method should map "nicely" in the 2nd-step interpolation, and could be reckognized thereby.

[Blue_MiSfit]

My brain hurts, but I think I grasp the idea. Now if only I could write C code! :)

[Chainmax]

Mug Funky: what does the switching decision when using warping w/fallback on OBMC on edges do?

Didée: that sounds like an excellent way to reduce warping. Do you expect it to reduce filtering speed by much?

[Fizick]

Didйe,
I remember some motion compensated deinterlace method (patent ?) description, with very long and complex pixels checking and replacing (temporal, spatial, linear, etc), with final part :"if the resulted pixel is considered as bad, use other method". Of course, other method description is omitted. :)

IMO, the main problem is not how to "judge" bad vector, but what to do if we found that it is bad.

we can easy create Ai as partially forward compensated A (at intermediate position).
we can easy create Bi as partially backward compensated B (at the same intermediate position).
We can compare Ai with Bi and declare all non-almost-equal pixels (or block and corresponded vectors) as "BAD".
(we will got very many such pixels).
O.K. What we will use as a result pixels at these places? Fallback to blended? .
(Similar method uses MVBob, and it often loses most of useful motion compensated information).

I do not understand your proposal :)
but IMO it is similar to part of idea of bidirectional vector interpolation - it may be interesting becourse of possibility to improve the of motion vectors (I considered to implement it some time ago, but ... it is not so easy and vector disrupture is not a small problem anyway.

but if only I could write C code of all my rejected projects!
:)
I know one man who try wrote phase based motion interloalation (but for deinterlacing).
But we need more .

[Didée]

I remember some motion compensated deinterlace method (patent ?) description, with very long and complex pixels checking and replacing (temporal, spatial, linear, etc), with final part :"if the resulted pixel is considered as bad, use other method". Of course, other method description is omitted. :)
Ya, I like that solution. Encourages me to start writing patents. :D


IMO, the main problem is not how to "judge" bad vector, but what to do if we found that it is bad.
"What to do else" is a problem too, yes ... but step by step. Firstly, it must be made the distinction between
- vectors that are really bad
- vectors that seem bad but are usable
- vectors that seem good but in fact are bad

Once there is a better certainty of which are the bad vectors, then it is time to think about better handling of the problematic areas. ;)


I do not understand your proposal :)
but IMO it is similar to part of idea of bidirectional vector interpolation - .
Quite possible. I don't claim that all things I think of are worldbreaking news ... so much things have already been thought of, and so much has been written. If I would try to read everything that already has been written, there would be no more time to do thinking-by-myself, and to have own ideas. But I like having "my own" ideas, even if they should be not new. :)


- it may be interesting becourse of possibility to improve the of motion vectors (I considered to implement it some time ago, but ... it is not so easy and vector disrupture is not a small problem anyway
Parallelization of forward & backward motion search? I.e. when dealing with different candidates for a vector, not just take that candidate with little better SAD / a little better predictor / whatever ... but also look at the vector field of the other direction's interpolation, and try using those of the possible candidates that have the same (just reverse) direction?

Sounds easy, but I bet that's a pain to implement. (?) :D

But it ~could~ solve this problem, at least partly:
we can easy create Ai as partially forward compensated A (at intermediate position).
we can easy create Bi as partially backward compensated B (at the same intermediate position).
We can compare Ai with Bi and declare all non-almost-equal pixels (or block and corresponded vectors) as "BAD".
(we will got very many such pixels).
Since doing ^so^, each vector field (forward/backward) has been computed completely independant of the other one, and some vectors might have been "drifted" in an unlucky direction, because SAD & Co. just happend to make it so. A combined motion search might have delivered better results ... choosing slightly worse candidates instead of the better ones, because the slightly worse resulted in better forward/backward correlation.


O.K. What we will use as a result pixels at these places? Fallback to blended? .
(Similar method uses MVBob, and it often loses most of useful motion compensated information).
Blending wouldn't be all that bad as a preliminary solution. :)

A possible way: do as described with bigger (16) blocks; in areas being problematic, try again with smaller (8) blocks ... and once more with (4).


Okay ... just gimme a :angry::angry: if I'm doing too much rambling here ... it's just that if I have some ideas that cannot be scripted, I tend to post them & ask about feasability. :)


or ... would it be preferable if I made some serious-looking PDF documents out of such ideas, starting with "In this document we propose a superior method to do blablabla...", blown up with overcompressed before-after pictures that look both the same, throw in lots of greek letters and integral signs, plus putting a long list of references at the end, which noone ever reads?
If that is the problem, it can be solved. :D

[Chainmax]

Mug Funky: I intend to use salFPS3 on a DVD I'm creating. The source is a bad NTSC-to-PAL conversion which I'll bob then filter. Would this:

Bobbing
filtering
salfps3(fps=59.94)
separatefields().selectevery(4,0,3).weave()

be a good call or would a better (albeit slower) call be possible? Also, what are salFPS3's dependencies?

[Mug Funky]

you should un-blend it as well as bobbing if you want motion compensation to be useful at all. otherwise it stays jerky...

also, salfps3 is experimental and doesn't like scenechanges much. it relies on warped edges in a similar way to one of didee's scripts earlier on in the thread, but scenechanges don't necessarily cause warping on the entire frame - the frame will have sizeable chunks of either side of the scenechange left.

feel free to combine it with the masking used in an earlier version of the script though - combining it with the tension masks that motionprotectedfps makes should improve it, but it's not ideal.

[wonkey_monkey]

Does anyone know where I can get the dll to go with motionprotectedfps? From Googling it looks like the author's site no longer exists, although the script is still in Google's cache.

David

[buzzqw]

i have this two link http://www.64k.it/andres/data/avisynth/MotionProtectedFPS.avsi
and http://www.64k.it/andres/data/avisynth/Motion_12Dec05.zip

BHH

[wonkey_monkey]

Thanks! I'd found that mirrored copy of the script, but going to the parent folder showed a list of plugins, and no link to Motion...

David

[wonkey_monkey]

On my computer, mcbob runs at 1fps. Salfps3 runs at about 4fps. But when I combine them, I get (and this is a very approximate figure) 0.008fps - that's 3 minutes for the first frame! When I stepped to the next frame in VirtualDub, it took a more respectable 15 seconds or so, but going to the third frame caused vdub to crash.

Does anyone know why this is happening?

Thanks,
David

[foxyshadis]

Ran out of memory, probably. Normally, you only get 2GB per process, so you might want to heavily restrict your framebuffer (SetMemoryMax) or just try 64-bit avisynth and whatever you can get working there.

[squid_80]

or just try 64-bit avisynth and whatever you can get working there.
Pretty much limited to convertfps at the moment (and it's significantly slower for unknown reasons) but I am working on getting motion.dll ported (inline assembly in header files = ouch).

[wonkey_monkey]

Oops, I must have edited that part out of my post - task manager showed vdub as hovering around the 300mb mark, which about where it gets with mcbob alone. I'm now having some trouble with salfps3 without mcbob, so I'll go back to the drawing board for a while...

David

[krieger2005]

here's my latest attempt:

function salFPS3(...)
...
mt_expand(mode=mt_circle(3))
mt_inflate()
bicubicresize(width*8, height*8,1,0)
crop(0, 0, 720, 480)
...
}

I think the line in red is a failure in your script. I don't find that someone earlier said something about that before. So maybe you could edit your script.

[Mug Funky]

you're probably right. i certainly can't remember what that line is doing there, but it effectively restricts it to NTSC sizes which is not a good thing...

try comment it out and see what happens :)

[krieger2005]

the script need this line. In general it's the same as in "motionprotectedfps" where the mask must be stratched back to the size of the clip (since you resized it before to the original-size/8). If you don't won't to look closer at the problem here is what i think it should be replaced with:
crop(0, 0, input.width(), input.height())

[Didée]

Glancing over the script, I saw that salFPS3 is a bit laborious in some places, and couldn't leave my fingers off ;)

Simplifications:
- crop/resize in one step, instead resizing to 8x8 plus cropping of 1/8x1/8 (as mentioned [p]ages earlier)
- don't use FitY2UV of old MaskTools. New MaskTools has "luma" parameter for mt_merge.
- chroma thresholds in TemporalSoften zero'ed, since the mask has no valid chroma anyway.
- eleminated two colorspace conversions by ditching "MOD16()" & making the resize inherently mod16 by m16().

No biggies, only nitpicking. Perhaps it gains one fps. ;)

What makes me frown a bit is the hard "black/white" thresholding (mt_binarize(80)) in relation to error detection. Conceptually, I'd prefer some softer gradual thresholding, to prevent "effects" in case that the mask should happen to oscillate around the threshold. But that's only me, you know...;)

edit: one resizer was wrong, corrected.
edit2: compatible with AS < v2.57.
function salFPS3(
\ clip input, float FPS,
\ int "mode", int "protection", int "protection2",
\ int "iterate", int "reset", int "initialise")
{
mode = default(mode, warp)#fast)
protection = default(protection, 80)
protection2= default(protection2,20)
iterate = default(iterate, 2)
reset = default(reset, 50)
initialise = default(initialise, 6)

originalFPS = input.framerate()
input

mp = FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise)
p = MotionFPS(FPS,mode,Motion = mp)
p_edge = mt_edge(mode="prewitt",thy1=protection2,thy2=0).MotionFPS(FPS,mode,Motion = mp)
maskp = mp.SumStretchFPSMask(FPS, protection)

mn = FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise, from = next)
n = MotionFPS(FPS,mode,Motion = mn, source = next)
n_edge = mt_edge(mode="prewitt",thy1=protection2,thy2=0).MotionFPS(FPS,mode,Motion = mn, source=next)
maskn = mn.SumStretchFPSMask(FPS, protection, source = next)

eps = 0.0001
maskp
assumeFPS(originalFPS)
CombineFPS(FPS, maskp, maskn.Invert())
bilinearresize(input.width(), input.height(), 0,0, eps+width/8.,eps+height/8. )
mpro=last

mt_merge(p_edge, n_edge, last, luma=true, u=3, v=3)

mt_lut(yexpr="128 x 128 - abs 112 / 128 * - 2 *") # 112/128* added to roughly compensate for the inherent
# clamping formerly done by the (now missing) ConvertToRGB.
mt_inpand()
bicubicresize(m16(width/8.),m16(height/8.),1/3.,1/3.)
mt_binarize(protection2,false)
temporalsoften(1,255,0,255)
mt_binarize(80,false)
mt_expand(mode=mt_circle(3))
mt_inflate()
bicubicresize(input.width(), input.height(), 1,0 )
tehmask=last

mt_merge(p, n, mpro, luma=true, u=3, v=3)

mt_merge(last,input.blendfps(FPS,aperture=FPS/float(input.framerate()))
\,tehmask, luma=true, u=3,v=3)
}

function m16(float x) {x<16?16:int(round(x/16.0)*16)}

[Pookie]

Perhaps it gains one fps

That might mean a 100% speedup in performance in some cases:D

[Mug Funky]

thanks Didée :)

good to know about the luma merge thing. i'll have to check that out, as i use that a lot.

and i'm not surprised it was a bit laborious in parts, considering it's mostly a semi-blind copy of clouded's original script.

i was just about to post an updated version of the script that supported other-than NTSC sizes, but looks like you've beaten me to it :)

[edit]

btw, the hard threshold probably should be ditched in favour of some sort of soft one, but i suspect then the metrics would have to be better :) certainly motion can get a bit jarring on local fast/complex motion, but a soft threshold would have it's own problems (like triple-edged moving objects... i suppose this happens with a hard threshold as well, but it would happen more otherwise).

[zambelli]

Another great thread, guys. This is the kind of stuff we all keep coming back to Doom9 for. :)

Does using MCBob have any advantage for standards conversion over TDeint that most people seem to be using?

[Mug Funky]

it has a marginal benefit in vertical sharpness, but unless you're going from say 50i to 60p there will be almost no visual difference, but a big speed hit :) certainly if you're going from interlaced to interlaced, the only noticable differences will occur on sharp moving graphics, and this difference will be very minor (slight up and down motion by about half a pixel, a few times a second). even this is completely invisible on a CRT.

[Chainmax]

How feasible would it be to add a motion analysis tool (similar to DeShaker's 1st pass) to SalFPS3 so that in scenes with motion above a certain threshold the extra frame(s) would be created via blending of the nearest ones? That should reduce artifacting a great deal, right?

[Cunhambebe]

That might be a stupid question (I'm a newbie), but after reading all of this thread, how would I write a script to convert *.tga (TARGA) sequences redered as NTSC at 30 fps to NTSC 24 film, keeping the sequence smooth and not slow. Thanks in advance.
Mark

[Mug Funky]

imagesource("x:\dir\to\imagesequence_%04d.tga",0,147)

converttoyv12()

whicheverscriptyouchoose(fps=24,args)


... that assumes you've got files "imagesequence_0000.tga" up to "imagesequence_0147.tga" in a directory somewhere.

also make sure the images are mod 8 or you'll have problems.

depending on the script you use, you'll get results comparable to optical flow in AE, except not as shit (but the sacrifice is having to step down to 4:2:0 8 bit)

[Cunhambebe]

Thanks so much Mug Funky; I'll give it a try and post the results back as soon as possible. Thanks, really!
Mark

PS: whicheverscriptyouchoose(fps=24,args)

-Which one? Convert to 24, for instance? cvan you give me an example? :) Thanks again.

[Cunhambebe]

This one is not working, definitely:
imagesource("D:\Nova pasta\FightersFinal_FFN0000.tga",0,0300)
ConvertToYv12()
ConvertFPS(24)

Neither this one - says there's an error; it says: Convert FPS requires YUY2 input.

LoadPlugin("C:\avs_plugins\ImageSequence.dll")
CoronaSequence("D:\Nova pasta\FFN.tga",sort=1)
ConvertToYv12()
ConvertFPS(24)

Any ideas?

[wonkey_monkey]

converttoyuy2() instead of converttoyv12().

David

[Cunhambebe]

Thank you very much David. Now the scrit looks like this:
LoadPlugin("C:\avs_plugins\ImageSequence.dll")
CoronaSequence("C:\FightersFinal\FFN0000.tga",sort=1)
ConvertToyuy2()
ConvertFps(24)

There's a warning: Corona Sequence: first picture found cannot be decoded.

So what? Is there anything wrong with my Targa 32 files?
One more thing: I see a lot of people adding ConvertToyuy2(). In may case (not the case above that is a targa sequence) , when I'm converting from MPEG2 to Xvid, it seems the conversion is made through the Chroma. So, I guess, according to what I've read around here, it is not necessary adding ConvertToyuy2(). Is that OK?

Thanks in advance.

[cwk]

Hello all,

I am experimenting with the using MVFps from MVTools within the salFPS scripts in this thread. I cannot get past the call to SumStretchFPSMask. Where does this function come from? I cannot find reference to it in documentation for either MaskTools or Motion.

Thanks,
cwk

[Mug Funky]

it comes from motion.dll, but i'm not sure if it's documented. it was designed specially for the script function "motionprotectedfps", which salfps scripts are derived (hacked) from :)

it makes a mask based on possible discontinuities in the vector field, but i'm not entirely sure how it works.

[Clown shoes]

Can anyone suggest the best way to call salFPS3

My current script is;

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
import("C:\Program Files\AviSynth 2.5\plugins\salFPS3.avs")
QTInput("I:\Crossfire\track 1.mov")
converttoyv12()
mvbob()
salfps3(50)
lanczos4resize(width,576)
assumetff().separatefields().selectevery(4,0,3).weave()


I have some particularly difficult 29.97i footage I'm working with,
that involves lots of very fast motion and quick flashes. Any suggestions as to a better script would be greatly appreciated. :)

[cwk]

Thanks Mug Funky,

I'll take a look.

cwk

[Chainmax]

I am getting a "I don't know what WARP means", pointing to line 6 of salFPS3. Also, I am not sure I'm getting its dependencies right, can someone tell me which are they?

[Mug Funky]

that "warp" part will be in motion.dll. you could hunt it down and remove it from the script - it'll switch to the default mode "warpfast" which will be slightly chunkier but still good when moving.