How can I interpolate two frames?

[Chainmax]

I'm currently trying to restore a video. In many scenes, the whole frame becomes like a negative with a blue shade on it. In most of these scenes, the bad frame was indentical to the one before or after, so duplicating a frame proved to be a good solution.
There is one frame, however, that can't be solved that way because the stutter will probably become quite evident, so what I need to do is to replace that frame with an interpolation of the previous and next ones. How do I create such an interpolation?

[Richard Berg]

Psuedocode:

AviSource
inter = Overlay(Trim(prev,1), Trim(next, 1))
Trim(0, prev) ++ inter ++ Trim(next, 0)

[Fizick]

Some time ago I released BadFrames plugin.

Replaces given bad frames by neighbors or blend (interpolation). Useful for frames with very large defects.

[Chainmax]

Richard Berg, I tried this:

Interp=Overlay(Trim(95723,1),Trim(95725,1))
Trim(0,95723)+Interp+Trim(95725,0)

And the frame in question (95724) is now a duplicate of frame 95725.


Fizick, can your plugin be used to substitute frame ranges with a single frame and/or accept external images? How would it compare speedwise to

x = last.Trim(n,n)
ApplyRange(n-1,n-1,"Overlay",x)

,which is what I've been using?

[foxyshadis]

Try

inter=mt_average(deleteframe(0),duplicateframe(0),chroma="process")
jdl_replacerange(inter,95724,95724)

For the manual way.

[Fizick]

Only this syntax:

BadFrames (clip, int frame1, int frame2[, ...], bool "blend")

frame1, frame2, ... is the given bad frames list to replace. Negative number designates that this bad frame is at scene change, frame number is absolute value of parameter (to prevent interpolation across the scene).

blend = false : frames are replaced by nearest good neighbors (mainly previous).

blend = true : frames are replaced by interpolation of nearest good neighbors (previous and next).

Default blend = true.


Example

AviSource("test.avi")
# Assume bad frames 5, 30, 51, 52. Frame 51 is at scene change.
BadFrames(5, 30, -51, 52, blend=true)

[scharfis_brain]

somewhere on my othere computer I have a function that does exactly this by motion interpoaltion.

I'll post the function replaceframe() tomorrow.

[Chainmax]

foxyshadis: Interp=mt_average(Trim(95723,95723),Trim(95725,95725),chroma="process").ApplyRange(95724,95724,"Overlay",Interp) seems to have done the trick, thanks :). After seeing it it doesn't seem different from blending though, what would be the visual difference between a 50-50 blend of two frames and an interpolation of them?

scharfis_brain: please do, I'd like to play with it as well.

[foxyshadis]

What I proposed was a 50/50 blend, sorry.

[Richard Berg]

Mind was a blend too. Not sure what you mean by interpolation. If you want to use context from more than 1 frame on each side, wait for scharfi's script.

[Chainmax]

By interpolation I meant something along the lines of frames "x-1" and "x+1" being analized, motion between the two would be calculated and then frame "x" being the middle point of said calculated motion.


By the way, I am discovering more and more of these "blue frames", is there a way to discover them automatically without having to step through the source frame by frame?

[foxyshadis]

The basic idea is something like:

mvf=mvanalyse(...,isb=false)
mvb=mvanalyse(...,isb=true)
global fixed=mvcompensate(mvf,mvb,...)
yourreplaceframefunction(frame#,fixed)

If you wanted to automate it, knowing how slow that could get, you could try a conditional.

Scriptclip(mt_lutxy(last,fixed,"x y - 2 ^"), """ AverageLuma() > 20 ? fixed : last """)

Where 20 is whatever would be a sensible threshold for the SSD.

This is pseudocode and needs fleshing out, of course, and I'm sure scharfis has already done a better job of that than I ever could.

[Trixter]

Scarfis to the rescue: http://forum.doom9.org/showthread.php?t=115469

This saved one of my projects, where I had some bad frames but couldn't re-capture it.

[Chainmax]

Is that the function you were talking about, scharfis_brain?

[Mug Funky]

this is what i use for glitches... all 3 are useful.


function morph (clip c, int in, int "out", int "blksize")
{

Function fill_loop(string s, int stop, int count)
{
return (stop == 0) ? s : string("tofill.mvflowinter(fill_vb,fill_vf,time=" + string(100*(count - stop)/float(count))) + ",thscd1=255,thscd2=255).selectevery(tofill.framecount(),0)," + fill_loop(s,stop-1,count)
}

out=default(out, 0)
blksize=default(blksize,16)

d=c.trim(in,out)# in-1?

numframes=d.framecount-2
tofill=d.selectevery(c.framecount(),0,c.framecount()-1)

fill_vf=tofill.mvanalyse(truemotion=true,blksize=blksize,pel=2,isb=false,overlap=blksize/2)
fill_vb=tofill.mvanalyse(truemotion=true,blksize=blksize,pel=2,isb=true,overlap=blksize/2)
filled=eval("interleave(" + fill_loop("" ,d.framecount()-1,d.framecount()-1) + "tofill.selectevery(tofill.framecount(),tofill.framecount())).assumefps(c.framerate())")
c.trim(0,in-1)++filled++c.trim(out+1,0)
}

function filldrops (clip c, float "threshold")
{
threshold=default(threshold,0.)

global filldrops_th = threshold

vf=c.mvanalyse(truemotion=true,pel=2,isb=false,delta=1,idx=1,blksize=16,overlap=8)
vb=c.mvanalyse(truemotion=true,pel=2,isb=true,delta=1,idx=1,blksize=16,overlap=8)
global filldrops_d = c.mvflowinter(vb,vf,time=50,idx=1)
global filldrops_c = c
c.scriptclip("""ydifferencefromprevious()<=filldrops_th? filldrops_d : filldrops_c""")

}

function imagesplice (clip c, string "filename", int "frameno")
{
d=c.killaudio()
frame=imagesource(filename)
frame=frame.selectevery(frame.framecount,0).assumefps(c.framerate).converttoyv12()
d.trim(0,frameno-1)++frame++d.trim(frameno+1,0)

c.hasaudio()==true ? audiodub(last,c) : last
}


typically, morph works for short runs of badness (up to 3 frames is pretty safe so long as there's not much motion). if it's giving you uglies, you'll have to photoshop a new frame in the middle somewhere, then morph around it... somtimes you'll be able to morph, export the frames, touch them up, then re-import them.

filldrops just mocomps any frame that's identical to the last - good for messy captures, but doesn't work on interlaced footage.

some examples:

say there's a big splat on frames 6,7 and 8...

morph(5,9) # 5 and 9 being the nearest good frames

now that'll often give you ugly artefacts, so you might want to take out frame 7 and photoshop it manually. then you can do this:

imagesplice("f07_fixed.bmp",7)
morph(5,7)
morph(7,9)

that should clear a big glitch pretty well.

for interlaced footage it's probably best to bob, fix, weave. some glitches are so bad that you'll just need to suck it up and photoshop every frame... morph isn't all that stable in new avisynth (most likely my fault), so if it's called too many times or you're interpolating too many frames it'll just crash.

[scharfis_brain]

sorry, Chainmax, I was busy the last days. I totally forgot this :(
But mug Funkys functions are somewhat equal and the function morph() is something I searched for myself, too!

[Chainmax]

It's all good. Now I think I have all the pieces to make a good request in the Avisynth Development forum, keep your eyes peeled for that one :).

Mug Funky: thanks for the function. YGPM, by the way.

[Cela]

Question moved to its own thread.

[Floatingshed]

some glitches are so bad that you'll just need to suck it up and photoshop every frame...

I've been meaning to ask for ages, is there an easy way to get frames from a script into photoshop or whatever and back again.
I've been doing it by exporting image sequences in Vdub and then reading them back in with imagereader. Slow and messy. Am I missing an easier solution?

[Emulgator]

is there an easy way to get frames from a script into photoshop or whatever and back again.
What I have been doing many times:
Export:

<your sourcefilter>
ConvertToRGB24()

In VirtualDub: Export the whole movie as Image sequence of .jpg, Quality 95, 6 digits
Watch the image sequence using IrfanView.
Then manual Photoshop fixing.
Save as .jpg, Quality 10, rename with suffix r.

Reimport ? Take my Image Splicer, refined from doom9 sources...

#[*#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Start of Emulgator's ImageSplicer 1.3 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
## Replaces damaged frames in-place by previously exported and manually repaired frames.
## ImageSplicer 1.3 from 03.November 2010 : Introducing variable for pathname. Faster working with varying paths, less typing. Autoscaling overlay.
## ImageSplicer 1.2 from 25.September 2010 : Fixed insertion of frame 0, frame 1, and last frame, this was buggy since original version.
## ImageSplicer 1.1 from xx.September 2010 : Added FrameReplacerOverlay within function imagesplice.
## ImageSplicer 1.0 from xx.August 2010 : Taken from doom9 forum source,
## This script imports previously exported and manually repaired frames and splices these in while trimming the damaged frame(s) off,
## thus replacing the damaged frame(s). Only path to repaired frame and framenumber are needed.
## Before exporting image sequences from VirtualDub (which will demand RGB24) for manual repair using the whole All-in-All script :
## Comment all following processing after this point out, especially resizers !
## Otherwise the following processing will be applied twice on the repaired frames and so these will not match anymore.
#Import("LoadRepairedPics.avs") ## Not in use at the moment...
#[*#_______Start of Function-Internal Overlay Controls. Bypass these if overlays shall be controlled externally by calling script !_______
ResizerWide= false # Controls horizontal pre-squeezing of overlay. Set true if DAR = 16:9 or more
ResizerHD= false # Control font size of overlay. Set true if width >720
OverlayFontWidth= ResizerWide==true ? 3.75 : 4.0
OverlayFontSize= ResizerHD==true ? 12 : 10
FrameReplacerOverlay=3 ## 0: FrameReplacer Overlay OFF | 1: Marker Only | 2: Marker + Framenumber | 3: Marker+Framenumber+File Path
#*]#_________End of Function-Internal Overlay Controls, Bypass these if overlays shall be controlled externally by calling script !________
1: Marker Only | 2: Marker + Framenumber | 3: Marker+Framenumber+File Path
framepath="I:\folder\"
## Type the path to the repaired frames here.
## Only "DRIVE LETTER:\folder\subfolder\" without the filename. Don't forget the closing backslash "\" !
framecountoffset=0 # This specifies how many frames offset from the original filename the repaired frame shall be inserted. Not yet...
pixtyp=IsYV12 ? "YV12" : IsYUY2 ? "YUY2" : IsRGB24 ? "RGB24" : IsRGB32 ? "RGB32" : "Unspported"
audch=AudioChannels()
audbit=AudioBits()
audsamp=AudioRate()
framect=framecount
bcnull=blankclip(length=0, width=width, height=height, pixel_type=pixtyp, fps=framerate, stereo=true, sixteen_bit=true, audio_rate=audsamp)
#-----------------------------------------------------Start of the list of frames to be replaced----------------------------------------------------------------
#imagesplice(bcnull, framect, string(framepath)+"000000r.jpg", 000000, FrameReplacerOverlay, OverlayFontWidth, OverlayFontSize)
#imagesplice(bcnull, framect, string(framepath)+"000001r.jpg", 000001, FrameReplacerOverlay, OverlayFontWidth, OverlayFontSize)
#imagesplice(bcnull, framect, string(framepath)+"000002r.jpg", 000002, FrameReplacerOverlay, OverlayFontWidth, OverlayFontSize)
#imagesplice(bcnull, framect, string(framepath)+"000003r.jpg", 000003, FrameReplacerOverlay, OverlayFontWidth, OverlayFontSize)
#-------------------------------------------------------End of the list of frames to be replaced----------------------------------------------------------------
function imagesplice (clip c, clip bcnul, int framct, string "filename", int "frameno", int "framereploverlay", float "overlyfontwidth", float "overlyfontsize")
{
still=imagesource(filename)
frame=still.SelectEvery(still.framecount, 0).Assumefps(c.framerate).ConverttoYV12(interlaced=false)
frameno==0 ? frame++Trim(c, frameno+1, 0) : frameno==framct-1 ? Trim(c, 0, frameno-1)++frame : \
frameno==1 ? Trim(c, 0, -1)++frame++Trim(c, frameno+1, 0) : Trim(c, 0, frameno-1)++frame++Trim(c, frameno+1, 0)
framereploverlay>= 1 ? Eval("""
Subtitle("+++ Manually Repaired Frame +++", first_frame=frameno, last_frame=frameno, align=5, x=width/2, y=(height*3/4)+50, font="Arial", size=overlyfontsize+10.0, font_width=overlyfontwidth, text_color=$00AAAAEE, halo_color=$00FF0000, interlaced=false)
framereploverlay>=2 ? Subtitle("Frame "+string(frameno)+" ", first_frame=frameno, last_frame=frameno, align=6, x=(width/2)-40, y=(height*3/4)+70, font="Arial", size=overlyfontsize+4.0, font_width=overlyfontwidth, text_color=$00FFEE00, halo_color=$00FF0000, interlaced=false) : NOP
framereploverlay>=3 ? Subtitle("Replaced by "+string(filename), first_frame=frameno, last_frame=frameno, align=4, x=(width/2)-40, y=(height*3/4)+70, font="Arial", size=overlyfontsize+4.0, font_width=overlyfontwidth, text_color=$00FFEE00, halo_color=$00FF0000, interlaced=false) : NOP
framereploverlay>=3 ? Subtitle("DVD-MF ImageSplicer 1.3", first_frame=frameno, last_frame=frameno, align=5, x=(width/2), y=(height*3/4)+90, font="Arial", size=overlyfontsize+4.0, font_width=overlyfontwidth, text_color=$00AAAAEE, halo_color=$00FF0000, interlaced=false) : NOP
""") : NOP
return(last)
}
#*]#!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! End of Emulgator's ImageSplicer 1.3 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[Floatingshed]

Wow, this looks very interesting, thanks. I will have a play with it. I may have questions...

Please can you explain what is happening here and how it differs from or improves upon simply exporting the entire movie as images, manually repairing the bad ones and reading the whole lot back in using imagereader()

[Emulgator]

It does not differ much, I guess.
Just the little luxury of border situation fixes, easier path handling,
overlays to present to the customer (or yourself) what had been done on which frames and so on...
I felt like having an advantage with a conditional splicer.

Reimporting of frames can be done in any NLE as well...

But this lot becomes much more helpful when you attempt to interpolate the repair results
in a larger script to actually recreate lost frames on bad splices, burned frames and so on,
which is where I took this snippet out...
In such situation one needs to know where the interpolated frames stem from.
The morphing overlays show this beautifully.