Closest AVS scripts to Alchemist results

[Clown shoes]

I am wondering if anyone has an opinion currently as to whether there are any avs scripts that allow a standards conversion close to the results produced by Snell & Willcox's Alchemist. I ask this for two reasons;

1. Snell & Willcox are about to release a software based converter called Helios http://www.snellwilcox.com/news_events/press_releases/197 which according to the salesman at Broadcast Live will give results comparable to the Alchemist (obviously not realtime though)

2. We get quite a few standards conversion jobs at work, where time is not an issue. It would be nice to have the confidence to tell a client our software solution was comparable to hiring an Alchemist, but at a much lower cost.


Over the past few years I have used many of the scripts posted on the Doom9 forums by people like Xesdeeni (who I have not seen around for a while) and Scharfis_brain. I am currently using:

for pure video

import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
interp=separatefields().eedi2(field=-2)
tdeint(mode=1,full=false,edeint=interp)
assumefps(50.05)
motionprotectedfps(60)
assumefps(60000,1001)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()


for hybrid content

MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
interp=separatefields().eedi2(field=-2)
tdeint(mode=1,full=false,edeint=interp)
converttoyuy2()
convertfps(60000,1001)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()



I would be grateful for anyones input on this matter, great or small.

Thanks in advance

ClownShoes

[Clown shoes]

(sorry didn't mean to double post, but this second post was in response to a comment made by Pookie. For some reason he deleted it though!)



:p

The software hasn't actually been released yet. And as I understand it, it will be in the region of £3,000.00

My query is really as to whether existing filters are able to emulate or get close to the output of the Alchemist hardware converter. If Snell & Willcox's claims are to be believed, they have managed it with Hellios.

My knowledge of AVIsynth and it's scripting is not in the same league as many of the videophiles here and I am also hard pushed to afford the hiring of an Alchemist on a regular basis for testing and comparisons. So I guess I'm hoping that any one with any standards converting experience can give thier opinions. I am currently encoding with an adaption of the first script I posted, but using mvbob:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob~.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
assumefps(50.05)
motionprotectedfps(60)
assumefps(60000,1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()


I'm pleasantly suprised to be getting 3fps with this script and am curious to see what it looks like.

[foxyshadis]

scharfis prefers mvfps & mvfps2, and of course fizick because he wrote it :p, because they can be tweaked to offer higher quality with less artifacts. Personally, my experience has been with motionprotected (base and personally modified versions - it makes fast excellent motion noise masks) and it's always given me good, dependable quality in movies. It animation, Twixtor for AE was better (but it ran out and AE is torture anyway) but I couldn't really see it in normal movies.

One thing you might want to think about is that motion compensation works significantly better with a denoised source. That's a strong argument in favor of mvfps2, or modding motionprotectedfps for two clips. (Attached.) Alternately you can just denoise, compensate, and then add your own grain back in.


function MotionProtectedFPS(
\ clip input, float FPS, clip "denoised",
\ int "mode", int "protection",
\ int "iterate", int "reset", int "initialise")
{
denoised = default(denoised, input)
mode = default(mode, warpfast)
protection = default(protection, 80)
iterate = default(iterate, 2)
reset = default(reset, 50)
initialise = default(initialise, 6)

originalFPS = input.framerate()
input

mp = denoised.FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise)
p = MotionFPS(FPS,mode,Motion = mp)
maskp = mp.SumStretchFPSMask(FPS, protection)

mn = denoised.FindMotion(iterate = iterate, reset = reset,
\ initialise = initialise, from = next)
n = MotionFPS(FPS,mode,Motion = mn, source = next)
maskn = mn.SumStretchFPSMask(FPS, protection, source = next)

maskp
assumeFPS(originalFPS)
CombineFPS(FPS, maskp, maskn.Invert())
bilinearresize(width*8, height*8)
crop(0, 0, input.width(), input.height())
fity2uv()

mt_merge(p, n, last, chroma="process")
}

[Pookie]

Sorry about deleting my post. I read it back and I realized it didn't add any constructive advice to the thread. Not that this one does, either.

[Trixter]

My query is really as to whether existing filters are able to emulate or get close to the output of the Alchemist hardware converter. If Snell & Willcox's claims are to be believed, they have managed it with Hellios.


Alchemist, as well as Folsom's top of the line standards converter, indeed use motion synthesis along with several quality checks for the best results. Occaisonally they make mistakes (usually with film sources that aren't telecined properly), but most of the time the results look 99% close as if the material was shot in the other standard.

Luckily, MVTools comes pretty close. If the material you're converting is interlaced (ie. actually does have 50 or 60 different images per second) and doesn't contain any surprises, MVTools works damn near perfectly. Best of all, it's free :-)

I am currently encoding with an adaption of the first script I posted, but using mvbob:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob~.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
assumefps(50.05)
motionprotectedfps(60)
assumefps(60000,1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

I'm pleasantly suprised to be getting 3fps with this script and am curious to see what it looks like.

Not sure why you're doing three different FPS conversions, but if you're using mvbob then mvconv is in the same package and it converts 50<-->59.9401 (60000/1001) like so:


mvbob
mvconv # <-- automatically figures out which way you want to go
#...resize to 480 lines, re-weave...


If mvconv isn't in whatever package you're using, I used to do it manually this way:


function MOCOMP_50_to_5994(clip c)
{
backward_vec = c.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = c.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = c.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
c.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=60000,den=1001,idx=1,idx2=2) # the 60000 / 1001 = 59.9401
}


The problem with mvbob is speed, obviously -- it goes overboard going for the best possible bob (and succeeds). For much faster processing, this is acceptable as long as you don't have single-pixel horizontal lines (like fine detail in the titles) in the source:


TDeint(order=1,mode=1,type=0,sharp=false,AP=50,APtype=0)


Let us know what you come up with. Standards conversion has been a hobby of mine for more than half a decade, as I've had to take very goofy source material (66p, 70p, and 72p computer sources) and convert it to NTSC without it looking like total ass... after using a gazillion products like ReTimer, Twixtor, MotionPerfect with less than optimal workflows, MVTools finally offers me a great workflow at about 99% of the quality. By 99%, I mean about 50 frames out of an hour's worth of footage might need to be touched up, which is par for the course for any converter, really.

[Revgen]

Whomever does that Doom9 firefox plugin should add this page to the default bookmarks. Good info throughout.

Thanks guys.

[Clown shoes]

Ok now we're cooking :)

I am currently testing MVBob with MVConv. That is happily encoding away. I am however having a problem finding a current script for MVFps or MVFps2 as mentioned by Foxyshadis though!

I can find much mention of it when I use the search function and have ended up spending most of the day reading pages of posts but to ne avail. I can find no link or scripting except for a post by Scharfis in february of 2005 http://forum.doom9.org/showthread.php?p=615027#post615027 but that script needed an earlier version of MVTools which I cannot find. I did try changing the version in the script but that does not work. A small pointer in the right direction would be appreciated right now as I am quite sure that this a case of not being able to see the wood for the trees.

@ Trixter

I have also cut out the unnecesary FPS conversions. I was under the impression (having borrowed this script from another recent SC post) that MotionProtectedFPS was unable to accept fractional framerates. This is clearly not the case.

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("E:\SC tests\AmcQ.d2v")
mvbob()
motionprotectedfps(59.94)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()

[Boulder]

Maybe foxyshadis meant MVFlowFPS and MVFlowFPS2 which are a part of MVTools?

[Clown shoes]

I thought that might be the case. But there was a function called MVFps used for the same purpose so it has me a little confused. Also that link in my previous post does link to some scripting by Scharfi called MVFps just over a year ago.

Edit; OK scratch that. I think your right Boulder. I will try it now

Edit2; Hmmm, having problems using MVFlowFPS2. I am clearly calling it in the wrong way. I'm afraid my scripting knowledge is very limited. I have tried to read through the MVTools documentation but it is a little beyond me. I presume this is an incorrect way to call it:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
MVFlowFps2(59.94)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

I would be greatful is someone could briefly explain how to use this.

[Boulder]

I've used MVFlowFPS2 like this:
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=100,den=1)

That would convert the framerate to 100fps, so you probably need num=60 in MVFlowFPS2 and AssumeFPS(59.94) afterwards.

[Clown shoes]

Thanks Boulder, I've got it working now. This is how my script currently reads:

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
mvbob()
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=60,den=1)
assumefps(59.94)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

I'm getting approximately 1.5fps right now using CCE 2.70 on a 3ghz xeon with 2gb of ram. This seems quite reasonable to me.

[foxyshadis]

The Motion (http://forum.doom9.org/showthread.php?t=101859) thread, or the one I posted above has support for compensating off a different source than the one analysed.

Sorry about mistaking mvfps and mvflowfps ^^;

You can also use num=5994,den=1000 (or more precisely, num=6000,den=1001) to eliminate any minor desync assumefps would cause.

[Chainmax]

I used a custom function suggested by scharfis_brain in some extremely shaky digicam clips. It gave me really good results.

[Clown shoes]

Ok I've just encoded some footage using the last script I posted, but I am getting some weird motion effects on some frames during fast motion sections.

Any idea what's causing this? and more importantly, is there a workaround for it?

[Chainmax]

That looks like interlacing, are you sure you're reinterlacing correctly? By the way, this is what I'm using:

Source loading, including audio
prefiltering up to (and including) final resizing
input=last

vectors=depanestimate(input)
input_1_of_x = input
input_2_of_x = input.depan(vectors,offset=1/x-1,mirror=15,blur=35)
input_3_of_x = input.depan(vectors,offset=2/x-1,mirror=15,blur=35)
...
...
input_x-1_of_x = input.depan(vectors,offset=x-2/x-1,mirror=15,blur=35)
input_x_of_x = input.depan(vectors,offset=x-1/x-1,mirror=15,blur=35)
interleave(input_1_of_x,input_2_of_x,......,input_x-1_of_x,input_x_of_x)
AssumeFPS(whatever,true)
ChangeFPS(whatever)

sharpen (if necessary)
addgrain (if necessary)
add borders (if necessary)
levels (if necessary)

x is the number of steps and should be twice the minimum common multiple of the source and destination FPS divided by the source fps. AssumeFPS should be set to the nearest multiple of 23.976 of the extrapolated fps to make the job easier for ChangeFPS. You'll then have to resample the audio by just a tiny bit, I recommend to use SRCDrop v0.5 (http://rarewares.org/others.html) for that, as SSRC will most likely not work in such a case.

For example, if the source is 30fps and the destination is 24fps, then:
- x = [2 x MCM(24,30)] / 30 = 2 x 120 / 30 = 240 / 30 = 8

- Since the extrapolation leaves the video at a little more than 240fps due to rounding errors when calculating the fractions, you have to use AssumeFPS(239.76,true) in this case.

[Clown shoes]

Sorry Chainmax, I didn't mean the interlacing. Im going from 25i to 29.97i so you should be able to see interlacing on a still. What I'm refering to is the warping in the high motion areas. I have also encoded the same clip using mvbob and motionprotectedfps. It is also having some issues with that frame but not nearly as bad. When played back it is not noticable but I am trying to work out what the cause is.

Here is the same frame from the original source and then the motionprotectedfps frame.

[Didée]

To make a long story short: there's very little you can do. Currently, at least.

a) motion estimation & compensation/interpolation will *always* make errors

b) for compansation (target: denoising), the faithful scripter has *tons of possibilities* to check if a given compensation is "good" and can be used, or if it is bad and should not be used / replaced by a fallback solution.

c) for interpolation (target: rate changing), the faithful scripter can do *nothing*, because
c1) the plugins hardly offer any grips to hook in
c2) there's no reference frame one could compare the compensation with.

So, for the case of interpolation, you are fully at the mercy of the plugin coders. From the scripting side there is nothing that could be done, because the crucial points are out of reach.

Sidenote: if you need this stuff "at work", then IMHO the investment into a professional solution would be advisable, anyway. ;)

[Clown shoes]

Excuse my ignorance Chainmax but some of this scripting is beyond me. I must confess that with the mvflowfps2 script that Boulder posted, I have just dropped it straight into my script without a true understanding of the parameters. I have spent some time going through the documentation, but right now it is a little like learning a new language! This could well be the reason for my warped frames in the previous script.

I am going to post the sample I have been working with. If anyone gets a chance to take a look I would be greatful.

My aim is still the same; to find a software solution for 25i to 29.97i or vice versa standards conversions, that can be used as an alternative to the hardware Alchemist route, when time is not an issue

Edit;


Sorry Didee, I didn't notice you had just posted. In response; The reason for my quest was the discovery of the imminent release of Snell & Willcox's software encoder, that they claim is an equal to thier Alchemist hardware converter but without the real time returns. That made me wonder whether the current possible permutations of AVISynth filters might be put to the same use.

The results I have received so far have been amazing! Combinations of MVbob with both MVflowfps2 and motionprotectedfps have given results that when transfered back to DVD or a tape format look almost identical to the original (at least to the untrained eye) and at acceptable speeds.

I appreciate what you are saying with regard to the existing restrictions and limitations, but surely they will always be there. I am just trying to find out what the most effective current solutions are and bring them all together. That fact that thier are multiple video standards and probably always will be, means that standards conversion is something that will always be of interest to people.

With regard to using proffesional standards conversion hardware for work, the problem is that is has always been prohibitively expensive. I work for a small post production company, so spending in excess of £100,000.00 is just not an option, hiring is also overly expensive. Currently the only real software options available are Procoder 2, Motion Perfect and NLE plugins such as Twixtor and Magic Bullet. NONE of these is even a patch on what can be done with AVISynth filters. Therefore the most sensible thing, for me at least, is to find an effective solution within this community.

So in conclusion, I must thank all you coders and scripters because the hard work you are doing really is appreciated.

[Boulder]

What I posted is almost directly from the MVTools' documentation ;)

[Chainmax]

The complete script for 25i to 29.97i should be something like this:

Import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DePan.dll")



input=MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v").MVBob()

vectors=depanestimate(input)
input_1_of_12 = input
input_2_of_12 = input.depan(vectors,offset=1/11,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset=2/11,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset=3/11,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset=4/11,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset=5/11,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset=6/11,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset=7/11,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset=8/11,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset=9/11,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10/11,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11/11,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)

AssumeFPS(599.4,true)
ChangeFPS(59.94)

ConvertToYUY2()
Lanczos4Resize(width,480)
Assumetff().SeparateFields().SelectEvery(4,0,3).Weave()

Remember to resample the audio (as it will be downsampled by a few Hz) with SRCDrop after this.

[Clown shoes]

lol oh! Thanks Boulder. This is taken directly from my mvtools 1.30 documentation;


Description

MVFlowFps2 (clip, clip "mvbw", clip "mvfw", clip "mvbw2", clip "mvfw2", int "num", int "den", float "ml", int "idx", int "idx2")

Will change the framerate (fps) of the clip (and number of frames) like MVFlowFps, but with a little better quality (and slower processing). In addition to backward "mvbw" and forward "mvfw" motion vectors of original source clip, the function uses backward "mvbw2" and forward "mvfw2" motion vectors of second (modified) source clip. Second clip must be produced from original source clip by cropping (i.e. diagonal shift) by half block size. It must be done with command Crop(a,a,-b,-b), where a=b=4 must be used for blksize=8, a=b=8 for blksize=16, and a=2, b=6 for blksize=4 (see example). Blocks boundaries will be at different parts of objects. MVFlowFps2 reverses the shift internally and averages motion vectors from these two sources to decrease motion estimation errors. The function uses pixel-based motion compensation (as MVFlow, MVFlowInter). Internal forward and backward occlusion masks (MVMask kind=2 method) and time weighted factors are used to produce the output image with minimal artefactes. True motion estimation is strongly recommended for this function.

num: fps numerator (default=25)

den: fps denominator (default=1). Resulted fps = num/den.

ml: mask scale parameter. The greater values are corresponded to more weak occlusion mask (as in MVMask function, use it to tune and debug). Default=100.

idx (may be) works the same way as idx in MVAnalyse for speed increasing.

idx2 is MVAnalyse index of second (shifted) clip (must not coincide with first idx).


Usage

To change fps with MVFlowFps2:

# Assume progressive PAL 25 fps source. Lets try convert it to 50.
backward_vec = source.MVAnalyse(isb = true, truemotion=true, pel=2, idx=1)
# we use explicit idx for more fast processing
forward_vec = source.MVAnalyse(isb = false, truemotion=true, pel=2, idx=1)
cropped = source.crop(4,4,-4,-4) # by half of block size 8
backward_vec2 = cropped.MVAnalyse(isb = true, truemotion=true, pel=2, idx=2)
forward_vec2 = cropped.MVAnalyse(isb = false, truemotion=true, pel=2, idx=2)
return source.MVFlowFps2(backward_vec,forward_vec,backward_vec2,forward_vec2,num=50,idx=1,idx2=2)



I clearly have a lot of reading and learning to do, because I have no idea how you made the correlation between what's in the documentation and what you posted. I feel a bit stupid, much to learn... :o


@ Chainmax,

Thank you. I will try this now. Can I ask though, why assumefps(599.4,true) followed by changefps(59.94) Secondly, why does the audio need to be downsampled by a few hz? is there a part of this script that is causing the running time to change?

[Chainmax]

Because of rounding errors when calculation the fractions, the resulting framerate after the Interleave line will be something like 600.4fps. The AssumeFPS line brings it down to 599.4fps so that ChangeFPS can do a clean 10-to-1 decimation instead of a ~10.0167-to-1 one.
Now, AssumeFPS changes framerate by changing the speed of the video, and while a 0.15% slowdown will never be noticeable, audio and video will become very slightly asynch. The true switch in the AssumeFPS line corrects that by altering the samplerate of the audio to avoid that asynch, which is why you'll need to resample afterwards.
SSRC can't do such minor (~ 0.0070% or less) adjustments, which is why I recommended SRCDrop.


P.S:mmm...I wonder if ChangeFPS can be substituted by TDecimate...

[Didée]

Chainmax, that script seems incorrect to me. The offsets you're specifying are all integers: 1/11=[zero], 2/11=[zero], etc. When I try that script, it results in just one duplicate field every 6th field, practically the same as going the good old Bob.ChangeFps.SelectEvery.Weave road. For getting that result, DePan is not even needed...
When changing the offsets to floats, as expected: 1./11. , 2./11. , etc. then: there is indeed "true" motion throughout all fields of the output, but also the "oddities" from interpolation are there again ...

Alas, currently I have to try with an older version of DePan. On Fizick's site I see no download link to the current version ... and since Fizick currently is in vacation (supposely), I can't ask him, though being a "supporter" ...

[Boulder]

Do you mean this one : http://avisynth.org.ru/depan/depan17.zip

[Didée]

Oh, it's indeed there. Thanks for the hint. :)

Going the official road, you end up in this depan page (http://avisynth.org.ru/depan/depan.html), which doesn't contain a download link.

[Boulder]

That's where I got the link :) It's just down at the bottom of the page, second line from the end. It's easy to miss because there is just the latest version available whereas many other filters of his have all the versions for downloading.

[Didée]

Eh, now we have it: my browser stopped before loading the complete document, probably because of a timeout. Hence I had no link. Sorry for the noise ;)

BTW, to make your above example script for 25p --> 29.97i work correctly, needed changes were:

- replace "1/11" with "1./12." and so on for the rest
- replace "input_1_of_12 = input" with "input_1_of_12 = input.duplicateframe(0)"

Result is free or artefacts, of course. But really good it is only on clear pans. On "globally still" frames, all in-frame motion still is stuttering ... and on some difficult scenes (global shift to one side, but big objects moving to the other side), the stutter is, well, funny.
No surprise, though ... depan is doing exactly what it's designed to do: shifting the frame as a whole.

[Clown shoes]

Are you saying that script is for 25p to 29.97i ? Would that be no good for my 25i footage then?

[Trixter]

Where did you get MotionprotectedFPS? I don't think I've seen that, and am curious to know what it does...

Scratch that, I found it.

...and after testing it for about 30 minutes, I have to say that I like MVTools' MVFlowFPS2 much better -- the decrease in speed is well worth the estimation accuracy. I'll stick with them.

[Didée]

Arrrr...

With the noted changes, you can use it. I did miss that at the end of the loooooong mpeg2source-line, Chainmax had put a small ".MVBob()". :D
(I didn't even look at the end of that line, because I'm not used to put any filters in that place -- that's not exactly "structured scripting" ...)

So, for what the "interlacing impaired" person I am can see, the following should be correct for you:
Import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DePan.dll")

input = MPEG2Source("I:\Gucci NTSC M2Vs\Alexander McQueen Women.d2v")
input = input.MVBob()

vectors = depanestimate(input)
input_1_of_12 = input.duplicateframe(0)
input_2_of_12 = input.depan(vectors,offset= 1./12.,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset= 2./12.,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset= 3./12.,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset= 4./12.,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset= 5./12.,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset= 6./12.,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset= 7./12.,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset= 8./12.,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset= 9./12.,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10./12.,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11./12.,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,
\ input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)

AssumeFPS(599.4,true)
ChangeFPS(59.94)

ConvertToYUY2()
Lanczos4Resize(width,480)
Assumetff().SeparateFields().SelectEvery(4,0,3).Weave()

But again: on scenes without "global motion", (only some objects within the frame are moving), the result basically will still just be one duplicate field for every 6th field. It's only the panning scenes that benefit.

[Trixter]

vectors=depanestimate(input)
input_1_of_12 = input
input_2_of_12 = input.depan(vectors,offset=1/11,mirror=15,blur=35)
input_3_of_12 = input.depan(vectors,offset=2/11,mirror=15,blur=35)
input_4_of_12 = input.depan(vectors,offset=3/11,mirror=15,blur=35)
input_5_of_12 = input.depan(vectors,offset=4/11,mirror=15,blur=35)
input_6_of_12 = input.depan(vectors,offset=5/11,mirror=15,blur=35)
input_7_of_12 = input.depan(vectors,offset=6/11,mirror=15,blur=35)
input_8_of_12 = input.depan(vectors,offset=7/11,mirror=15,blur=35)
input_9_of_12 = input.depan(vectors,offset=8/11,mirror=15,blur=35)
input_10_of_12 = input.depan(vectors,offset=9/11,mirror=15,blur=35)
input_11_of_12 = input.depan(vectors,offset=10/11,mirror=15,blur=35)
input_12_of_12 = input.depan(vectors,offset=11/11,mirror=15,blur=35)
interleave(input_1_of_12,input_2_of_12,input_3_of_12,input_4_of_12,input_5_of_12,input_6_of_12,input_7_of_12,input_8_of_12,input_9_of_12,input_10_of_12,input_11_of_12,input_12_of_12)

AssumeFPS(599.4,true)
ChangeFPS(59.94)


Sweet Lincoln's Mullet, what is this doing? Why 12 different motion vector calcs, and why the up-to-600-then-back-to-60? How is this better than MVFlowFPS(2)?

[Trixter]

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
import("C:\Program Files\AviSynth 2.5\plugins\MotionProtectedFPS.avsi")
MPEG2Source("E:\SC tests\AmcQ.d2v")
mvbob()
motionprotectedfps(59.94)
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()

If you want Helios-like results without attempting your own threshold masking, that's pretty much it. For slightly better estimation at the cost of processing speed, you can use MVFlowFPS2, already a part of mvconv() in the mvbob.avs you import. Give this a shot and see if it works better with your difficult footage:


import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("E:\SC tests\AmcQ.d2v")
mvbob()
mvconv()
lanczos4resize(width,480)
converttoyuy2()
assumetff().separatefields().selectevery(4,0,3).weave()


(BTW if you have an older version you might get a script error on "convertttoyuy2" not being found... edit mvbob.avs and fix the typo to "converttoyuy2".)

[Clown shoes]

Trixter, can you see anything wrong with my implementation of mvflowfps2.

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\JUMP_BRITAIN\Jump Britain.d2v")
mvbob()
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=6000,den=1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

It is giving me amazing results except for some warping on fast moving scenes. I have posted a still on the previous page. This is not something that is apparent when I use motionprotectedfps though. Is there a fundemental difference between the two that may explain this. Or is it just my use of the filter?

[Didée]

Well, it's the typical "choose-your-poison" situation:
The depan-method is virtually free of artefacts, but will leave motion of moving objects stuttering. In some special situations, slight "backward-motion" phenomenae may occur.
The MVFlowFps/2 method will interpolate all motion, big or small one, but is prone to produce artefacts on not-compensateable motion (e.g. un/hidening objects).

There is no inbetween.

[foxyshadis]

Well, there is, but most of the research in that area is locked up with the brainiacs of Snell & Wilcox. MVFlow is about the best available with the currently available research; there should be ways to obtain a better confidence estimate for covered/uncovered objects but Clouded once told me that it's a lot harder than it sounds. Snell & Wilcox use algorithms that attempt to identify and track individual objects through an occulsion (how, I couldn't say, but I can guess the basics), but mvtools is nowhere near that strong/slow.

[Clown shoes]

Sorry Didee, I was actually comparing mvflowfps2 and motionprotectedfps which uses mg262s motion filter.

[Didée]

@ foxyshadis

Yeah. What I meant was "there is no inbetween with our current filters for Avisynth."
Ways to deal better with the problem surely do exist. I'd even have a bunch of 'em in front of my inner eye ... but who cares for those flatbrained scripters that can't even code a hello-world proggy in some modern C mutation. :)


@ Clown shoes

Yep, I know. My last post was directed at Trixter ... too much simultaneous posting back then. ;)

Regarding MVFlowFPS/2 -vs- MotionProtectedFPS:
While these two use different methods to do the vector search, technically they fight the very same problems, and in fact even their results are very similar. What makes them appear so different at first glance (you noticed that MotionFPS comes out with less artefacts) is that, with default(!) settings, MotionFPS is more cautious about avoiding artefacts than MVFlowFPS. The key is the "ml" value in MVFlowFPS, which defaults to "80". Try setting "ml" to, say, "1000" instead, and then compare both again. You'll see that MVFlowFPS2 now has much less artefacts. Probably it now even has a slight edge over MotionFPS. Still, MotionFPS comes very close ... and it comes in at much faster speed. ;)

[Chainmax]

I was using integers instead of floats? No wonder those "." were there in the readme :o.

Didée: it's funny that you mentioned the stuttering, I was showing portions of the video to my GF and at some point I jumped to a very stuttery scene. It's really disappointing since the script was doing so well on some of the most difficult scenes :(.
You give me, however, a spark of hope: you say artifacts decrease as ml increases, which setting would you say results in zero artifacts? Would such setting have negative effects besides speed (which is of no concern to me)?

[Didée]

Well, zero artefacts is out of reach ... except for not doing any framerate conversion ;)
Also, I've little clue what the safe zone for "ml" might be ... it's just today that I fiddled as much with framerate conversion as in the whole past year :D ... ml=1000 was just a random shot at some random high value. Seems to work not too bad, however ...

edit - speed should be about the same, since the protection work is done anyway. Somewhat similar like in MaskedMerge, where it won't matter if the mask covers almost all pixels, or just a few.

[Chainmax]

I'll try to see what setting for ml reduces artifacting the most in my clips and report back. In any case, how about this: a new parameter that would be linked to the artifacting (no idea how) and if it exceeds or falls below a user-defined threshold then the frame in question would be replaced by an interpolation of the previous and next frame?

[Clown shoes]

...and after testing it for about 30 minutes, I have to say that I like MVTools' MVFlowFPS2 much better -- the decrease in speed is well worth the estimation accuracy. I'll stick with them.

What is it you prefer about MVFlowFPS2? I have been comparing high motion sections frame by frame and although I can find hardly any difference between them, I would say I've found fractionaly more motion artifacts with MVFlowFPS2. I'm curious to know what advantage there is to the MVTools route in your opinion.

@ Didee

I tried increasing ml but I didn't see any difference at all! Very strange.

[Trixter]

Trixter, can you see anything wrong with my implementation of mvflowfps2.

import("C:\Program Files\AviSynth 2.5\plugins\mvbob\mvbob.avs")
MPEG2Source("I:\JUMP_BRITAIN\Jump Britain.d2v")
mvbob()
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mvtools.dll")
vvbw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=3,chroma=true,sharp=1)
vvfw=MVAnalyse(truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=3,chroma=true,sharp=1)
vvbw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=true,idx=4,chroma=true,sharp=1)
vvfw2=MVAnalyse(last.Crop(4,4,-4,-4,true),truemotion=true,blksize=8,delta=1,pel=2,isb=false,idx=4,chroma=true,sharp=1)
MVFlowFPS2(last,vvbw,vvfw,vvbw2,vvfw2,idx=3,idx2=4,num=6000,den=1001)
converttoyuy2()
lanczos4resize(width,480)
assumetff().separatefields().selectevery(4,0,3).weave()

It is giving me amazing results except for some warping on fast moving scenes. I have posted a still on the previous page. This is not something that is apparent when I use motionprotectedfps though. Is there a fundemental difference between the two that may explain this. Or is it just my use of the filter?

num should be 60000, not 6000. (as typed above, your script converts video to about 6fps; typo?) Otherwise, you're doing it right, and the still you posted looks like it has so much of a difference between frames that no amount of motion estimation could compensate. In cases like that, hardware converters fall back to a typical line interpolation mode for that field. In full motion, you hardly notice it. The results are very similar to something like ConvertFPS(59.94, zone=80). This leaves a "gradiated band" in one field that gradually switches from the previous image to the new image, but in full motion on a regular interlaced television it is extremely difficult to detect (of course it's obvious on a full field freeze-frame on computer).

Now for the news you don't want to hear: For a DVD I'm doing with source material in PAL with a LOT of full motion smooth movements (think dance videos), I am using MVFlowFPS2 for everything and I put that on track B; I then do a conversion using ConvertFPS(59.94, zone=80) and put that on track A. Anywhere MVFlowFPS2 makes a mistake, I cut that frame out and use the frame from the ConvertFPS conversion. Tedious, but it's pretty much a flawless conversion when I'm done.

[Trixter]

What is it you prefer about MVFlowFPS2? I have been comparing high motion sections frame by frame and although I can find hardly any difference between them, I would say I've found fractionaly more motion artifacts with MVFlowFPS2. I'm curious to know what advantage there is to the MVTools route in your opinion.


It depends on the source material, of course. My source material is very synthetic (it was all computer generated; for examples of the type of motion, think title rolls and crawls) and for me, MVTools results in slightly less artifacts (and the artifacts that are there are less severe). Of course you should use what works best for your footage.

Here is a particularly fast panning head using MVTools' MVFlowFPS2:

http://img220.imageshack.us/img220/3555/mvfps2example9ei.th.png (http://img220.imageshack.us/my.php?image=mvfps2example9ei.png)

...and here is the same interpolation as done by MotionProtectedFPS:

http://img139.imageshack.us/img139/3795/motionprotectedfpsexample2cm.th.png (http://img139.imageshack.us/my.php?image=motionprotectedfpsexample2cm.png)

As you can see, the MotionProtectedFPS example has more distortion, as well as chroma flying all over the place :-) This isn't a massive slam on MotionProtectedFPS, just an example of why I use MVFlowFPS2. If I'm doing a quickie for viewing an MPEG-1 on television or something, MotionProtectedFPS's speed is fantastic. If I'm transcoding footage for a DVD for commercial sale, I use MVTools for the most quality. Just my $0.02.

[videoFred]

Result is free or artefacts, of course. But really good it is only on clear pans. On "globally still" frames, all in-frame motion still is stuttering ... and on some difficult scenes (global shift to one side, but big objects moving to the other side), the stutter is, well, funny.
No surprise, though ... depan is doing exactly what it's designed to do: shifting the frame as a whole.

Because all my old 8mm films are all original made at 15-16-18fps, and without the use of a tripod, I need a good frame rate conversion/stabilising script.

I confirm the depan, method is perfect for clear pans...
But not for moving objects and/or zoom scenes. Stuttering!
I have the best results with mvflowfps() followed by depan for stabilising.

Motionprotectedfps() gives very good results too, and it is much faster.

Actualy, both are giving me near perfect results in 90% of the cases... The artefacts are almost invisible on TV.

Very fast moving people/objects, original recorded at low speeds like 15FPS are giving me the most problems. Maybe I am going to use convertfps() for these scenes instead.

*EDIT* MvFlowFPS2 works a lot better for this.



Fred.

[Chainmax]

...
@ Didee

I tried increasing ml but I didn't see any difference at all! Very strange.

Neither did I on my problem video :(.

...
Now for the news you don't want to hear: For a DVD I'm doing with source material in PAL with a LOT of full motion smooth movements (think dance videos), I am using MVFlowFPS2 for everything and I put that on track B; I then do a conversion using ConvertFPS(59.94, zone=80) and put that on track A. Anywhere MVFlowFPS2 makes a mistake, I cut that frame out and use the frame from the ConvertFPS conversion. Tedious, but it's pretty much a flawless conversion when I'm done.

That's exactly what I was trying to do a while back but didn't know what to use as an alternate framerate converter. I'll try that, thanks for the suggestion :). I have a question though, I use the following MVFlowFPS lines:

vf=last.mvanalyse(isb=false,blksize=4,pel=2,search=3,truemotion=true)
vb=last.mvanalyse(isb=true,blksize=4,pel=2,search=3,truemotion=true)
MVFlowFPS(last,vb,vf,num=30000,den=1001)

Would that give worse results than MVFlowFPS2?

[Clown shoes]

num should be 60000, not 6000.

lol, yes that's most definately a typo!

I then do a conversion using ConvertFPS(59.94, zone=80) and put that on track A. Anywhere MVFlowFPS2 makes a mistake, I cut that frame out and use the frame from the ConvertFPS conversion. Tedious, but it's pretty much a flawless conversion when I'm done.

Sounds like a good idea, but wow that is really gonna add to the overall time it takes to do a standards conversion. That is exactly what I asked for though, the best quality disregarding time as an issue. So thank you trixter and it's good to hear you using it in a commercial enviroment. If a client comes to me with either a short peice of work or a longer peice where time is not an issue. I am now confident I can produce a professional looking standards conversion on a par with a hardware solution.

[Trixter]

That's exactly what I was trying to do a while back but didn't know what to use as an alternate framerate converter. I'll try that, thanks for the suggestion :). I have a question though, I use the following MVFlowFPS lines:

vf=last.mvanalyse(isb=false,blksize=4,pel=2,search=3,truemotion=true)
vb=last.mvanalyse(isb=true,blksize=4,pel=2,search=3,truemotion=true)
MVFlowFPS(last,vb,vf,num=30000,den=1001)

Would that give worse results than MVFlowFPS2?

That completely depends on your source material, so I can't answer that directly.

I see that you shrunk blocksize from 8 to 4... from the docs, larger blocks are less sensitive to noise, are faster, but also less accurate. A blocksize of 4 might exacerbate the noise already present in your source. I see you increased the search algorithm to exhaustive... with small blocks, you're going to be waiting a very long time :-)

MVFlowFPS2 uses two different motion analyses: A regular one and another one with the entire frame shifted half the block size diagonally. It then averages the results of both so that there are less estimation errors. So MVFlowFPS2 is always going to provide better motion estimation than just MVFlowFPS.

I would only use small block sizes like 4 if your source is less than 480 lines (like 320x240 640x360). If you're working with 720x480 or source, 8 is sufficient.

Finally, doing some quick math off the top of my head, MVFlowFPS2 with the default (subdividing) search algo and 8x8 blocksize should run faster than MVFlowFPS with an exhaustive search and 4x4 blocksize.

I shudder to think how long 4x4, exhaustive, FPS2 would take ;-)

[Chainmax]

...
I see that you shrunk blocksize from 8 to 4... from the docs, larger blocks are less sensitive to noise, are faster, but also less accurate. A blocksize of 4 might exacerbate the noise already present in your source.
...
I assumed that lower blocksizes meant better motion end results.
By the way, once I have the MVFlowFPS/2 and ChangeFPS encodes, is there a faster way to replace individual frames or frame ranges of the former with the corresponding ones from the latter than cutting and pasting (i.é: can I do it directly from Avisynth)?

[Clown shoes]

I would imagine the quickest way would be using an NLE like Avid or Premiere. Especially one with multicam mode which Avid definately has.

[Trixter]

I would imagine the quickest way would be using an NLE like Avid or Premiere. Especially one with multicam mode which Avid definately has.

I'm using Premiere Pro 1.5 currently (but as soon as this project is done you can be sure I'm going to upgrade). Multicam switching is for "live" switches (you literally press buttons while the video is running, like a real broadcast switcher), but I'm switching to a single frame, maybe two, so manually cutting the bad frames out is pretty much the only way to do it.

As for an AVISYNTH only solution, I would imagine you can use trim() but I've never done it... I would love to hear from the seasoned scriptwriters how something like this would be done.