interpolated frames have jagged edges

[4evrplan]

I'm trying to repair dropped frames in a capture (telecined source) by using MVFowInter (of the MVTools plugin) to recreate them from the frames immediately before and after them. The example in the docs is a great resource but doesn't work as is because I have multiple bad frames in a row, not just one at a time. I've created a function to make this easier, but the generated frames have jagged egdes, even in areas where there is no motion. The image below is from frame 19230 and 19231, where the former is a good frame, and the latter is an interpolated frame.

http://farm4.static.flickr.com/3228/2824901267_95c2dc0d3c_o.jpg

The top left is part of the good frame. The top right is from the generated frame immediately after the good one. On the bottom are 4x zooms of a part of the other two.

Also, notice that the interpolated frame is shifted down slightly from the original. The other reference frame used to create the interpolated frame (the one immediately after the range of bad frames) is in the same position as the first reference frame. Does anyone know what may be causing these two problems? My script is below.

AVISource("Capture.avi")
AssumeTFF()
SeparateFields()
even=SelectEven()
even=even.InterFrames(19231,19236)
even=even.InterFrames(45354,45355)
even=even.InterFrames(119262,119266)
even=even.InterFrames(159668,159672)
odd=SelectOdd()
odd=odd.InterFrames(19231,19236)
odd=odd.InterFrames(45354,45355)
odd=odd.InterFrames(119262,119266)
odd=odd.InterFrames(159668,159672)
Interleave(even,odd)
Weave()

function InterFramesSub(clip c, float interTime, float interDelta, backward_vectors, forward_vectors)
{
return (interTime >= 99.99)
\ ? BlankClip(c, 0)
\ : c.MVFlowInter(backward_vectors, forward_vectors, time=interTime, ml=255, idx=1).Trim(0,-1)
\ ++ InterFramesSub(c, interTime+interDelta, interDelta, backward_vectors, forward_vectors)
}

function InterFrames(clip c, int begFrame, int endFrame)
{
c2 = c.Trim(begFrame-1,-1) ++ c.Trim(endFrame+1,-1)
backward_vectors = c2.MVAnalyse(isb=true, truemotion=true, pel=4, delta=1, idx=1)
forward_vectors = c2.MVAnalyse(isb=false, truemotion=true, pel=4, delta=1, idx=1)
interDelta = 100.0 / (endFrame - begFrame + 2.0)
interTime = interDelta
return (begFrame-1 == 0) ? c.Trim(0,-1) : c.Trim(0,begFrame-1)
\ ++ InterFramesSub(c2, interTime, interDelta, backward_vectors, forward_vectors)
\ ++ c.Trim(endFrame+1,0)
}

[mikeytown2]

I forced YUY2 and changed MVAnalyse()

You may want to bob this instead of using SeparateFields()


AVISource("Capture.avi",pixel_type="YUY2")
AssumeTFF()
SeparateFields()
even=SelectEven()
even=even.InterFrames(19231,19236)
even=even.InterFrames(45354,45355)
even=even.InterFrames(119262,119266)
even=even.InterFrames(159668,159672)
odd=SelectOdd()
odd=odd.InterFrames(19231,19236)
odd=odd.InterFrames(45354,45355)
odd=odd.InterFrames(119262,119266)
odd=odd.InterFrames(159668,159672)
Interleave(even,odd)
Weave()

function InterFramesSub(clip c, float interTime, float interDelta, backward_vectors, forward_vectors)
{
return (interTime >= 99.99)
\ ? BlankClip(c, 0)
\ : c.MVFlowInter(backward_vectors, forward_vectors, time=interTime, ml=255, idx=1).Trim(0,-1)
\ ++ InterFramesSub(c, interTime+interDelta, interDelta, backward_vectors, forward_vectors)
}

function InterFrames(clip c, int begFrame, int endFrame)
{
c2 = c.Trim(begFrame-1,-1) ++ c.Trim(endFrame+1,-1)
backward_vectors = c2.MVAnalyse(isb=true, delta=1, blksize=16, blksizeV=16, overlap=8, overlapV=8, divide=2, idx=1, dct=4, truemotion=true, search=0, searchparam=0, lambda=2700, pnew=9)
forward_vectors = c2.MVAnalyse(isb=false, delta=1, blksize=16, blksizeV=16, overlap=8, overlapV=8, divide=2, idx=1, dct=4, truemotion=true, search=0, searchparam=0, lambda=2700, pnew=9)
interDelta = 100.0 / (endFrame - begFrame + 2.0)
interTime = interDelta
return (begFrame-1 == 0) ? c.Trim(0,-1) : c.Trim(0,begFrame-1)
\ ++ InterFramesSub(c2, interTime, interDelta, backward_vectors, forward_vectors)
\ ++ c.Trim(endFrame+1,0)
}

[4evrplan]

I just tried that and got the same results. I'd like to avoid color space conversions if possible, but If I have to convert just the bad portions to YUY2 and then back to YV12 before splicing them back into the rest of the video, so be it. I'd like to find a solution that gets rid of the jagged edges first though.

EDIT: I'd like to play this video on a standalone when I'm done. Doesn't bobbing double the frame rate?

[mikeytown2]

Just reinterlace it after your done processing it. Frame rate does double so you might have to *2 your frame #'s.

AVISource("interlaced.avi")
AssumeTFF() #set order
MCbob()

#Process frames

#Process frames

separatefields()
selectevery(4,0,3)
weave()

[Fizick]

The problem is with idx. You cant use same idx with different source clips.
In your script, try remove idx from InterFramesSub function.

[4evrplan]

Yes, that worked nicely. Thanks Fizick.

I'm thinking about changing my approach on this video and converting it to 24fps progressive, but the telecine pattern is messed up by the dropped frames. Also, there are cuts where the pattern changes. Do IVTC plugins handle these situations intelligently? Also, I'd still like to replace dropped frames, but am at a bit of a loss as to how to do this without messing up the cadence of the pattern. I could simply cut out the bad spots, IVTC, generate new progressive frames where the bad frames were, but then the timing won't be accurate and may even mess up the audio. Take for example a spot where there's six bad frames in a row. Since IVTC converts five frames to three, I'd have to regenerate frames before IVTC for the plugin to have those frames to work with, but then the frames won't be interlaced or progressive in the right spots. I'm really just thinking out loud here; if I decide to pursue this seriously, I'll probably just start a new thread.

EDIT: That should have been "five frames to four".

[Fizick]

anyway, your script above is very interesting :)