InterFrame

[SubJunk]

This is the old InterFrame thread, the new one is here (http://forum.doom9.org/showthread.php?p=1486831) so please use that one from now on.
Thanks

[Didée]

Some might enjoy using the "HighQualitySharpen" plugin as a post-processor for this. HighQualitySharpen has been successfully tested, over more than a decade, by thousands of people on billions of different videos.

function HighQualitySharpen(clip c)
{
c.sharpen(0.6).mergechroma(c)
}


I mean, c'mon! It's not a bad function, but did it really take 3 years and thousands of videos to NOT include "blend=false", to NOT expose useful MAnalyse parameters, to instead hard-code EXHAUSTIVE search even for the fast mode, and to use non-effective search ranges for the MRecalculate stages?

It's not bad, but you are ringing a pretty big bell for something that looks like just having left the proof-of-concept stage.

[jmac698]

There's a lot of blending on that, and seems to be no scene change detection. If that's your intent, great, but people are usually looking to *fix* blended sources.
I don't see any actual interpolation. Look at the missiles firing, all I get is blends of two of the original frames.
Can you explain what it's supposed to do better?
See frame 687, horrible artifacts. A lot of the motion switches between interpolated and blended at random. Double legs appear every other frame. Overall very blurry and flickery. I can even see that at full speed.

[SubJunk]

If you're interested in seeing the discussion we've had up to this point it's at AVS Forum here (http://www.avsforum.com/avs-vb/showthread.php?t=1025800).
I've added blend=false for the next release since that seems like a good idea.

jmac698, please feel free to convert the original video with a different function like blendfps or convertfps to see the difference.
Here is a comparison I made comparing BlendFPS with InterFrame:
InterFrame (http://www.avsforum.com/avs-vb/attachment.php?attachmentid=205105&d=1299967341) - BlendFPS (http://www.avsforum.com/avs-vb/attachment.php?attachmentid=205106&d=1299967419)

I should also update those sample videos since they are from an earlier version of the script. I will post back when they are updated.

[aegisofrime]

I have been looking for a good way to convert my 29.97 fps content to 59.94. Despite the negativity so far in the thread, I will give it a go once my current encoding job is done. Thanks for the hard work :)

Edit: I think I will wait for the blend=false version first...

[SubJunk]

I have been looking for a good way to convert my 29.97 fps content to 59.94. Despite the negativity so far in the thread, I will give it a go once my current encoding job is done. Thanks for the hard work :)

Edit: I think I will wait for the blend=false version first...Thanks for trying it :)
I have updated to version 1.0.1 now which includes the blend=false change

[Robert Martens]

I can't speak to the motion estimation/compensation side of this, but from a scripting point of view you could simplify things a bit (line breaks added to avoid breaking tables):

function InterFrame(clip Input, string "Preset", int "NewNum", int "NewDen") {

# Defaults
Preset = default(Preset, "Quality")

# Determine new framerate
NewNum = default(NewNum, Input.Framerate == 25 ? 50 : Input.Framerate == 30 ? 60 : 60000)
NewDen = default(NewDen, Input.Framerate == 25 || Input.FrameRate == 30 ? 1 : 1001)

super = MSuper(Input, hpad=16, vpad=16, rfilter=4)

dct = Preset == "Quality" ? 5 : 0
chroma = Preset == "Quality" ? true : false

# Make interpolation vector clip
backward_1 = MAnalyse( super, blksize=16, searchparam=3, search=3, dct=dct,
\ chroma=chroma, plevel=0, badrange=(-24), isb=true )
forward_1 = MAnalyse( super, blksize=16, searchparam=3, search=3, dct=dct,
\ chroma=chroma, plevel=0, badrange=(-24), isb=false)
backward_2 = MRecalculate( super, backward_1, blksize=8,
\ searchparam=1, search=3, chroma=chroma)
forward_2 = MRecalculate( super, forward_1, blksize=8,
\ searchparam=1, search=3, chroma=chroma)
backward_3 = MRecalculate( super, backward_2, blksize=4,
\ searchparam=0, search=3, chroma=chroma)
forward_3 = MRecalculate( super, forward_2, blksize=4,
\ searchparam=0, search=3, chroma=chroma)

# Put it together
Preset == "Quality" ?
\ MFlowFps( Input, super, backward_3, forward_3, num=NewNum, den=NewDen,
\ ml=10000, thSCD1=300, blend=false) :
\ MBlockFps( Input, super, backward_3, forward_3, num=NewNum, den=NewDen,
\ mode=0, thSCD1=300, blend=false)

}


I'm a big fan of Evaluate, myself, but it's not really beneficial unless you're dynamically generating strings; in this script you can achieve the same outcome with another approach.

First things first, you can use expressions within the Default() filter call, just like any other. Since the selection of values for NewNum and NewDen don't depend on anything but the input framerate, you can eliminate your original "Determine new framerate" section and do the same work up top.

Second, looking up the preset values for 'dct' and 'chroma' in MVTools 2 lets you get rid of the other Eval block. Create a pair of new variables that fall back to the appropriate values, and you can now collapse the two sections of vector clip assignments into one. Two potential drawbacks are that if you ever decide to introduce other differences between the "Quality" and "Speed" presets, you'll need to either add more variables or go back to an Eval() based design, and that if the defaults for 'dct' and 'chroma' change in future releases of MVTools, you'll need to update accordingly. I still think doing it the way I've shown makes the script easier to read and maintain, but you're certainly free to make up your own mind about that.

One final note, don't forget the NOP() filter if you ever need to skip processing at some level of a conditional statement. Eval(""" """) works, too, but in this case a simple no-op would have accomplished the same thing.

[SubJunk]

Thanks for the tips, Robert :) Very much appreciated, and on a personal note thanks to you and aegisofrime for helping to create a more positive feeling here.
The way the thread started was a bit disconcerting for me and I'm really pleased that it seems to be turning around :)

[Gavino]

InterFrame 1.0.1 - Framedoubling/60FPS conversion plugin
On downloading this, I was disappointed to find it wasn't a plugin at all, but 'just' a script function wrapper for the MVTools functions I already use for this purpose. Nothing wrong with that, but the way it's presented is a bit misleading.

I'm a big fan of Evaluate, myself, but it's not really beneficial unless you're dynamically generating strings; in this script you can achieve the same outcome with another approach.

First things first, you can use expressions within the Default() filter call, just like any other. Since the selection of values for NewNum and NewDen don't depend on anything but the input framerate, you can eliminate your original "Determine new framerate" section and do the same work up top.
NewNum = default(NewNum, Input.Framerate == 25 ? 50 : Input.Framerate == 30 ? 60 : 60000)
NewDen = default(NewDen, Input.Framerate == 25 || Input.FrameRate == 30 ? 1 : 1001)
A disadvantage of this style is that the combined NewNum/NewDen result for a given input framerate is less obvious, being split into two places.
Also there is a subtle difference from the original: an explicit value of 0 can no longer be used to indicate 'use input frame rate'.

I do agree that the use of Eval for this purpose is ugly (and cannot be easily extended to nested conditionals).
GScript provides a more usable block-if construct. When used in a separate file from the main script, you don't even need to put the code inside GScript(""" ... """), as you can just use GImport() instead of Import() to load the file. (Unfortunately, this does not extend to auto-loading a .avsi, as Import is always implicitly used there.)

[Robert Martens]

GScript is of course the smart way to do things like this, but if you'll indulge me for a moment:

For the problem of splitting the values from one another, unfolding the OR and using careful spacing should make it a little nicer:

NewNum = default(NewNum, Input.Framerate == 25 ? 50 : Input.Framerate == 30 ? 60 : 60000)
NewDen = default(NewDen, Input.Framerate == 25 ? 1 : Input.FrameRate == 30 ? 1 : 1001)

But that still leaves the other problem, which I'm not completely sure I understand. I'm going to blame this question on lack of sleep, since that sounds like the most plausible excuse, but could you please elaborate on this "explicit zero" issue?

Looking at version 1.01:

# Determine new framerate
NewNum != 0 ? Eval("""
# Use user values
""") : Input.Framerate == 25 ? Eval("""
NewNum = 50
NewDen = 1
""") : Input.Framerate == 30 ? Eval("""
NewNum = 60
NewDen = 1
""") : Eval("""
NewNum = 60000
NewDen = 1001
""")

Won't a user-defined 0 for NewNum be overridden by this conditional?

[Gavino]

In the original, a user-supplied 0 will cause the output frame rate to be determined by the input rate. (NewNum being 0 will give false for the first condition, so move on to test the input rate.)

In your version, a user-supplied 0 stays as 0, giving an output rate of 0/0.

You could fix this this by rewriting it as:
NewNum = Default(NewNum, 0)
NewNum = NewNum != 0 ? NewNum : Input.Framerate == 25 ? 50 : Input.Framerate == 30 ? 60 : 60000
NewDen = NewNum != 0 ? NewDen : Input.Framerate == 25 ? 1 : Input.FrameRate == 30 ? 1 : 1001
BTW I think it would be better to change the default value of NewDen to 1, allowing it to be omitted for integer output rates.

[Robert Martens]

Got it, thank you! I can't believe I let that slip by me. I need to think through my suggestions more thoroughly, that one's particularly embarrassing.

[aegisofrime]

Thanks for the tips, Robert :) Very much appreciated, and on a personal note thanks to you and aegisofrime for helping to create a more positive feeling here.
The way the thread started was a bit disconcerting for me and I'm really pleased that it seems to be turning around :)

No problem! I took a look at the Avatar sample you posted and honestly it looks better than the script that MVTools documentation suggested would have provided. One of the most disconcerting artifacts with the default MVTools script was that fast hand motion will quickly become ghostly hands, which I didn't see in your sample. If there's any artifacts, perhaps the action was too fast for me to tell :p

[Didée]

I welcome any suggestions for improvements [...]
I'm sure you have some great contributions to make to the quality of this process.

Alas, no, I have not. The holy grail is way out of reach. It is not even in sight.

Frame interpolation has a huge pile of hardcore technical problems. For lots of these problems, MVTools does *not* have *any* solution. You can only toy around with the various parameters. But whatever you do, it'll only be "maybe better here, but worse there".

Let "frame interpolation" be a "very big table". Then, MAnalyse+MFlowFPS is a "too small tablecloth".

You can shift the tablecloth around until you get blue in the face, but you can never cover the whole table.


And worse, there is no possibility to make up a "very very slow script with better solutions", because the problematic issues are not accessable from the script level. MVTools has no idea what "motion" actually is. There is no reference for the interpolated frames to check against. And it doesn't even seem possible to transpose a MMask accordingly (would require usage of inverse vector orientation, which feature is not offered by MVTools.)

Dead road, there's hardly anything to do for the script writers.

All in all, you can build your sandcastle however you like it. The only thing guaranteed is that it'll be swept away by the next wave floating along.

[frustum]

Didée used this metaphor:

Let "frame interpolation" be a "very big table". Then, MAnalyse+MFlowFPS is a "too small tablecloth".

I used to work with a guy who had an apt metaphor for these situations. He call it "squeezing a balloon." You can squeeze the problems out of one area but they just show up as problems somewhere else.

[PhrostByte]

I was under the impression that the "smooth motion" in modern TVs was just them a) preserving all frames from deinterlacing, and b) having Hz be a multiple of the source FPS (so 24fps and 30fps content both display with perfect timing).

I wasn't aware that they actually did any motion interpolation.

[SubJunk]

No problem! I took a look at the Avatar sample you posted and honestly it looks better than the script that MVTools documentation suggested would have provided. One of the most disconcerting artifacts with the default MVTools script was that fast hand motion will quickly become ghostly hands, which I didn't see in your sample. If there's any artifacts, perhaps the action was too fast for me to tell :pAwesome! Yeah in this script there are 3 passes that narrow down details to remove some of that wide warping/ghosting :)

Alas, no, I have not. The holy grail is way out of reach. It is not even in sight.I've definitely noticed that tinkering with most of the values, especially the ones governed by truemotion, is fruitless for the reason you said; it makes some scenes better and other scenes worse. In this script we have a good general setting which produces what I think are the best results in the most scenes.
I disagree with your general impression of the process, though; it's very limited and like you said it will never be the holy grail, but I have been using it on all my videos for nearly a year - sometimes in realtime with MPC-HC and sometimes encoded - and have been happy with the results.
It's far from perfect but I think it's good already, and having tried every other script I've come across I can say that I think this one is the best that exists so far. Sometimes I have found a better one so I just incorporate those changes into this one, so this one stays the best :)

I was under the impression that the "smooth motion" in modern TVs was just them a) preserving all frames from deinterlacing, and b) having Hz be a multiple of the source FPS (so 24fps and 30fps content both display with perfect timing).

I wasn't aware that they actually did any motion interpolation.Some might do that but from what I understand Sony TVs at least do some very soft interpolation. It's not as strong as this one (because this one tries harder to tighten up the image where theirs blurs more) but I think they do it.

On downloading this, I was disappointed to find it wasn't a plugin at all, but 'just' a script function wrapper for the MVTools functions I already use for this purpose. Nothing wrong with that, but the way it's presented is a bit misleading.Maybe you're right, I can change the title to say "plugin script" if you think that would be better.

[Gavino]

I can change the title to say "plugin script" if you think that would be better.
I think "... conversion function" would be more accurate.
Plugin implies code loaded as a DLL.

[SubJunk]

I had a look at similar things and the word Script seems to work well, I changed it to that.
Now I'm just waiting for a 3 day encoding job to be done so I can make new samples

[SubJunk]

Just updated to 1.0.2 which just increases speed of both presets.
The Speed preset may be slightly lower quality now, but much faster, and the Quality preset (default) is a bit faster with no quality loss.

[SubJunk]

8:13, I'm not sure this is the right thread to use to post your scripts because you have different goals to me and your scripts are different.
If you could create your own thread instead that would be good :)

[Jeremy Duncan]

Ok, no problem. I deleted my reply and will start my own thread in the avisynth Usage forum.

[SubJunk]

Great, thanks! BTW I'm a Firefly fan, too :)

[Dogway]

SubJunk, I also made my own function based on that same webpage, and some Didée's posts, with lots of tweaking and test and error. DoubleFPS: It performs sometimes better than yours, sometimes worse, have a try for comparison (signature), it might be a help.

[SubJunk]

SubJunk, I also made my own function based on that same webpage, and some Didée's posts, with lots of tweaking and test and error. DoubleFPS: It performs sometimes better than yours, sometimes worse, have a try for comparison (signature), it might be a help.Ah very nice, I didn't know about it. I'll check it out for sure and probably borrow some things ;) Thanks!

[SubJunk]

I just updated to 1.1 and changed the sample videos accordingly.

The changes in 1.1 from 1.0.2 are:
- Renamed Quality preset to Medium (default)
- Renamed Speed preset to Fast
- Increased quality of Medium (default) preset
- Added Placebo preset (little quality gain with big performance hit, not recommended)
- RemoveGrain is now required for Medium (default) preset
- EEDI2 is required for Placebo preset

I also updated the first post to give a more detailed description.

[Didée]

Preset != "Fast" ? Eval("""
DeGrainedSource = Input.RemoveGrain()

Not good. What if a user wants to use "medium" or "placebo", but does not want RemoveGrain( [mode=2] ) to remove the end points of thin lines, to kill the highlights in people's eyes, etc.?



Preset == "Placebo" ? Eval("""
CustomPelClip = EEDI2(Input, field=1).Spline36Resize(2*width(Input), 2*height(Input), src_left=0.25)


Not good. src_left=0.25 is wrong, you are causing a registration error. What you need there is src_top=-0.5.


Though, I'd rather suggest to use NNED3_rpow2. EEDI2 often does look artificial on natural sources. Moreover, NNEDI3 is also faster on MultiCore CPUs.

[Gavino]

Not good. src_left=0.25 is wrong, you are causing a registration error. What you need there is src_top=-0.5.
No, I believe SubJunk is correct, since the pelclip must have original pixels aligned at the top left. The required vertical alignment is produced by EEDI2, and an offset of 0.25 is required for the horizontal resizing of Spline36Resize.

[Didée]

:eek: Ayyy! :eek:

Yes, you're right. I didn't think of the shifted alignment that MVTools is using.

[Fizick]

I had a look at similar things and the word Script seems to work well, I changed it to that.


If it is not a plugin (or core), then this thread is more appropriate to "Avisynth Usage" subforum.


Here is my very fast frame doubling script :)

source=last
super=source.msuper(pel=1)
vec=manalyse(super,isb=true)
interleave(source, source.mFlow(super,vec, time=50))

[SubJunk]

Not good. What if a user wants to use "medium" or "placebo", but does not want RemoveGrain( [mode=2] ) to remove the end points of thin lines, to kill the highlights in people's eyes, etc.?That line is only used for motion analysis; the final video retains the grain :)

Though, I'd rather suggest to use NNED3_rpow2. EEDI2 often does look artificial on natural sources. Moreover, NNEDI3 is also faster on MultiCore CPUs.I had tested NNEDI3 with and without ABS and found EEDI2 produces the best results. Not to the human eye, but for MVTools; remember we are only passing the EEDI2 clip to MVTools as a pelclip which is used to make predictions, it is still mostly using the detail from the main clip in the output :)

If it is not a plugin (or core), then this thread is more appropriate to "Avisynth Usage" subforum.OK, I'll make a new thread there and redirect from here to there
P.S. Welcome to the thread, Fizick! My hero :)

Here is my very fast frame doubling script :)

source=last
super=source.msuper(pel=1)
vec=manalyse(super,isb=true)
interleave(source, source.mFlow(super,vec, time=50))I guess there's some error in there because MeGUI gives me "fatal error" when I run it

[SubJunk]

OK the new thread is here (http://forum.doom9.org/showthread.php?p=1486831) so please move any discussion to that one, thanks :)