Log in

View Full Version : Improve MFlowInter result


ajon
9th July 2012, 12:42
I have a very simple clip that I thought MAnalyze and consequently MFlowInter should easily be able to motion detect and compensate, but my results were less than spectacular when not enough image data surrounded the object to be motion compensated.

Thus I have two questions:
1. What parameters of MAnalyze and / or MFlowInter can be adjusted to improve the following scenario?
2. What parameters should be adjusted to get delta=2 to work at all (not just blend)?

scenario delta 1
The clip is a dot moving from left to right at a constant speed and I'm trying to interpolate an intermediate frame at 50% between two frames (time=50, ml=70 and delta=1)

http://s9.postimage.org/v1xynlkpr/dot_delta1.gif

scenario delta 2
With delta two and the parameters otherwise the same, the result gets a lot worse. It seems pure blending is used without any sort of attempt at motion compensation. I'm assuming the distance between the dot's position in frames N-2 and N+2 is too great for MAnalyze to find it and that's why MFlowInter simply falls back to blurring instead..

http://s17.postimage.org/fwrdnqf6n/dot_delta2.gif

When I blow the pictures up by adding a lot of padding ( AddBorders(64, 48, 48, 48) ), the estimation gets a lot better for delta=1, almost perfect, so I'm guessing it has to do with MAnalyze not getting the data it needs to properly detect all vectors. I'm however currently stuck at trying to figure out which parameter would affect the results in a positive way, so I'm turning to the professionals :)

FYI. Alternative views with only the compensated clip showing and a rectangle around the clip dimensions in effect when MVTools were operating on the clip ("bounding boxes"):

http://s9.postimage.org/v61vy32u7/dot_delta1_bounding_box_nopad.gif http://s7.postimage.org/e217jwk8r/dot_delta1_bounding_box_padded.gif

References:
* The script statements used:

function getVector(super, isb, delta){
v = MAnalyse(super, overlap=4, isb=isb, search=3, blksize=16, delta=delta)
return MRecalculate(super, v, overlap=4, search=3, blksize=8, thSAD=400)
}

avisource("dot.avi")

super = MSuper()
bv = getVector(super,true,1)
fv = getVector(super,false,1)
MFlowInter(super, bv, fv, time=50, ml=70)

* An archive with the source clip and the entire script: dot_sample.zip (http://www.mediafire.com/?p232r82blynfj60)

Didée
9th July 2012, 13:02
Extend searchrange to cover the length of the searching blocksize?

function getVector(super, isb, delta){
v = MAnalyse(super, overlap=4, isb=isb, search=3, searchparam=16, blksize=16, delta=delta)
return MRecalculate(super, v, overlap=4, search=3, blksize=8, thSAD=400)
}

ajon
9th July 2012, 13:44
Extend searchrange to cover the length of the searching blocksize?

:thanks: for the lightning quick reply Didée and for getting me on the right track !



blksize also had to be increased for the new searchparam value to take effect.

Do you also know the relationship between searchparam and blocksize?
Apparently they are related, though I could not discern that from the MVTools2 documentation..


I re-read the mvtools manual for the n-th + 1 time and realized that blocksize likely has something to do with this, since the effect of searchparam seems to depend on it (judging by browsing the source code). With blksize at 32 and searchparam at 16, the result is much better (though still far from perfect).
Since 32 is the largest blocksize, I'm a bit concerned if there were to be even more motion in a clip (rather likely, esp. with HD-clips). From what I've read, on this forum, trickery such as resizing and calculating vectors on other clips and applying them on the intended source seem to be some ways to workaround the limit..

Is it right to assume that the horizontal distance MAnalyze will be able to find related blocks between frames is "(2*radius+1) " pixels?

If so, then I'm wondering why the result with blksize at 16 was so poor, while doubling it yielded great effect.
As I understand the manual it basically states that the horizontal or vertical distance MAnalyze will search for related blocks between frames is 2*searchparam+1 = 2*16+1 = 33 pixels in our case. Somehow that can't be true, since the blob travels 24 pixels each frame and with delta=2, that creates a distance of 4x24 = 96 pixels between the blobs in frame N-2 and N+2. Still the blocksize seemed to affect the search a lot while without touching that parameter, the searchparam property had no effect of note..

I'm still :confused: about how blocksize affects the search range though.

http://s11.postimage.org/tps5y0e37/dot_delta1_padding_searchparam_16_blksize_32.gif

search param = 16, blksize = 32 and delta=2. Great result !

Jenyok
22nd March 2013, 13:42
Any suggestions, changes and improvments for this function, see below.
.
Thanks...
.

#
# InterpolateFrames2() function...
#
# FrameNumber is number of the 1-st frame in Source that needs replacing.
# FrameCount is total number of frames to replace.
# InterpolateFrames2(101, 5) would replace 101, 102, 103, 104, 105,
# by using MRecalculate() and MFlowInter() interpolation.
#
function InterpolateFrames2(clip clp, int FrameNumber, int FrameCount)
{
# Constants
# MSuper()
#
hpad = 8 # Maybe 4, 8, 16, 32, !!! Change this constant for best tuning !!!
vpad = 8 # Maybe 4, 8, 16, 32, !!! Change this constant for best tuning !!!
pel = 4 # default=2
sharp = 2 # default=2 !!!
rfilter = 4 # default=2
chroma = true # default=true
isse = true # default=true
planar = false # default=false
levels = 0 # default=0

# Constants
# MAnalyse()
#
blkh = 32 # Maybe 4, 8, 16, 32, !!! Change this constant for best tuning !!!
blkv = 32 # Maybe 4, 8, 16, 32, !!! Change this constant for best tuning !!!
overlap = 4 # Maybe 2, 4, 8, 16, !!! maybe blkh/2 or litle !!! Change this constant for best tuning !!!
overlapV = 4 # Maybe 2, 4, 8, 16, !!! maybe blkv/2 or litle !!! Change this constant for best tuning !!!
search = 3 # default=4 !!! DO NOT CHANGE =3
searchparam = 16 # default=2 !!! DO NOT CHANGE =16
dct = 5 # default=0
plevel = 2 # default=0
badrange = (-24) # default=24
badsad = 10000 # default=10000
divide = 0 # default=0
sadx264 = 0 # default=0
truemotion = true # default=true

lambda = (truemotion == false) ? 0 : (1000 * blkh * blkv / 64) # default=0 truemotion=false, default=1000*blksize*blksizeV/64 truemotion=true
lsad = (truemotion == false) ? 400 : 1200 # default=400 truemotion=false, default=1200 truemotion=true
pnew = (truemotion == false) ? 0 : 50 # default=0 truemotion=false, default=50 truemotion=true
pzero = pnew # default pzero=pnew

# Constants
# MRecalculate()
#
blkhR = 8 # Maybe 4, 8, 16, 32, !!! Change this constant for best tuning !!!
blkvR = 8 # Maybe 4, 8, 16, 32, !!! Change this constant for best tuning !!!
overlapR = 4 # Maybe 2, 4, 8, 16, !!! maybe blkhR/2 or litle !!! Change this constant for best tuning !!!
overlapVR = 4 # Maybe 2, 4, 8, 16, !!! maybe blkvR/2 or litle !!! Change this constant for best tuning !!!
searchR = 3 # default=4
searchparamR = 1 # default=2
thSAD = 400 # default=200 !!! DO NOT CHANGE =400

lambdaR = (truemotion == false) ? 0 : (1000 * blkhR * blkvR / 64) # default=0 truemotion=false, default=1000*blksize*blksizeV/64 truemotion=true

# Constants
# MFlowInter()
#
thSCD1 = 800 # default=400 !!! DO NOT CHANGE =800
thSCD2 = 130 # default=130
iml = 70 # default=100
blend = true # default=true


# Here is code of function...
#
# Restore bad frames with interpolation with MFlowInter
# Prepare functions
#
# function MSuper(clip, int "hpad", int "vpad", int "pel", int "levels", bool "chroma", \
# int "sharp", int "rfilter", clip "pelclip", bool "isse", bool "planar")
#
super = MSuper(clp, \
hpad = hpad, \
vpad = vpad, \
pel = pel, \
chroma = chroma, \
sharp = sharp, \
rfilter = rfilter, \
isse = isse, \
planar = planar, \
levels = levels)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
backward1 = MAnalyse(super, \
delta = (FrameCount + 1), \
isb = true, \
blksize = blkh, \
blksizeV = blkv, \
overlap = overlap, \
overlapV = overlapV, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = lambda, \
chroma = chroma, \
truemotion = truemotion, \
lsad = lsad, \
plevel = plevel, \
pnew = pnew, \
pzero = pzero, \
dct = dct, \
divide = divide, \
sadx264 = sadx264, \
badSAD = badsad, \
badrange = badrange, \
isse = isse, \
levels = levels, \
global = true, \
pglobal = 0, \
meander = true, \
temporal = false, \
trymany = false)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
forward1 = MAnalyse(super, \
delta = (FrameCount + 1), \
isb = false, \
blksize = blkh, \
blksizeV = blkv, \
overlap = overlap, \
overlapV = overlapV, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = lambda, \
chroma = chroma, \
truemotion = truemotion, \
lsad = lsad, \
plevel = plevel, \
pnew = pnew, \
pzero = pzero, \
dct = dct, \
divide = divide, \
sadx264 = sadx264, \
badSAD = badsad, \
badrange = badrange, \
isse = isse, \
levels = levels, \
global = true, \
pglobal = 0, \
meander = true, \
temporal = false, \
trymany = false)

# MRecalculate(clip super, clip vectors, int "thSAD", int "smooth", int "blksize", int "blksizeV", \
# int "search", int "searchparam", int "lambda", bool "chroma", bool "truemotion", \
# int "pnew", int "overlap", int "overlapV", string "outfile", int "dct", int "divide", \
# int "sadx264", bool "isse")
#
backward2 = MRecalculate(super, \
backward1, \
blksize = blkhR, \
blksizeV = blkvR, \
overlap = overlapR, \
overlapV = overlapVR, \
search = searchR, \
searchparam = searchparamR, \
lambda = lambdaR, \
dct = dct, \
truemotion = truemotion, \
thSAD = thSAD, \
isse = isse, \
chroma = chroma, \
pnew = pnew, \
divide = divide, \
sadx264 = sadx264, \
smooth = 1)

# MRecalculate(clip super, clip vectors, int "thSAD", int "smooth", int "blksize", int "blksizeV", \
# int "search", int "searchparam", int "lambda", bool "chroma", bool "truemotion", \
# int "pnew", int "overlap", int "overlapV", string "outfile", int "dct", int "divide", \
# int "sadx264", bool "isse")
#
forward2 = MRecalculate(super, \
forward1, \
blksize = blkhR, \
blksizeV = blkvR, \
overlap = overlapR, \
overlapV = overlapVR, \
search = searchR, \
searchparam = searchparamR, \
lambda = lambdaR, \
dct = dct, \
truemotion = truemotion, \
thSAD = thSAD, \
isse = isse, \
chroma = chroma, \
pnew = pnew, \
divide = divide, \
sadx264 = sadx264, \
smooth = 1)

GScript("""
if (FrameCount == 1) {

# MFlowInter(clip source, clip super, clip mvbw, clip mvfw, float "time", float "mL", \
# bool "blend", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
inter2 = MFlowInter(clp, \
super, \
backward2, \
forward2, \
time = 50, \
mL = iml, \
blend = blend, \
thSCD1 = thSCD1, \
thSCD2 = thSCD2, \
isse = isse, \
planar = planar)

inter = inter2.Trim(FrameNumber, -1)
} # End of If-Then
else {
# Fill an Inter variable with the MFlowInter() clips with different times
# Times are calculated in each step
#
for (i=0, FrameCount-1) {
itime = (100.0 / Float(FrameCount + 1)) * Float(i + 1)

# MFlowInter(clip source, clip super, clip mvbw, clip mvfw, float "time", float "mL", \
# bool "blend", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
inter2 = MFlowInter(clp, \
super, \
backward2, \
forward2, \
time = itime, \
mL = iml, \
blend = blend, \
thSCD1 = thSCD1, \
thSCD2 = thSCD2, \
isse = isse, \
planar = planar)

if (i == 0) {
inter = inter2.Trim(FrameNumber, -1)
} # End of If-Then
else {
inter = inter ++ inter2.Trim(FrameNumber, -1)
} # End of If-Else
} # End of For ()
} # End of If-Else
""") # End of GScript ()

return (clp.trim(0, FrameNumber - 1) ++ inter ++ clp.trim(FrameNumber + FrameCount, 0))
}