Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th September 2022, 10:09   #1  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,389
AI EXTRApolation? (adding frames to the end of a video)

I've been looking at the likes of DAINapp and FlowFrames that do pretty amazing things when it comes to interpolating new frames in a video, but is there anything that will extrapolate new frames?

I have a short, fast-moving, fairly motion-blurred clip (a camera being pulled back through a forest) and I want to add just two frames to the end of it that continue the motion. There's a lot of parallax because of all the trees at different distances to the camera.

Any suggestions?
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 25th September 2022, 22:22   #2  |  Link
AVIL
Registered User
 
Join Date: Nov 2004
Location: Spain
Posts: 404
Hi:

A mad idea. With mvtools, get the motion vector of the previous to the last frame of clip and apply to mcompensate the last frame (e.g. by using a fake clip.trim(clip,1,0).duplicateframe(last)). Repeating the procedure with the newly created frame obtain a second "interpolated" frame (hopely). But no AI involved, Probably no Natural Intelligence involved too.

Last edited by AVIL; 25th September 2022 at 22:54. Reason: Completion of the idea
AVIL is offline   Reply With Quote
Old 26th September 2022, 00:18   #3  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,282
This of any use at all ? [only for single frame prediction]

Quote:
Originally Posted by StainlessS View Post
You beat me to it (was busy with an edit).

Might be able to use these (manually),

Code:
Function PredictFromPrevious(clip Last)
{ # Predict i frame from Previous
	yuy2=(isYUY2())?true:false
	PreFilt=Last.DeGrainMedian() 	# prefiltered for better motion analysis
	Last= (yuy2)?Interleaved2planar():Last
	PreFilt= (yuy2)?PreFilt.Interleaved2planar():PreFilt
	super=Last.MSuper(planar=yuy2)
	superPreFilt=PreFilt.MSuper(planar=yuy2)
	fv = superPreFilt.MAnalyse(isb = false,  truemotion=true,delta = 1)
	Last.MFlow(super,fv,planar=yuy2)
	Last=(yuy2)?Planar2Interleaved():Last	
	Return Last
}


Function PredictFromNext(clip Last)
{
	yuy2=(isYUY2())?true:false
	PreFilt=Last.DeGrainMedian() 	# prefiltered for better motion analysis
	Last= (yuy2)?Interleaved2planar():Last
	PreFilt= (yuy2)?PreFilt.Interleaved2planar():PreFilt
	super=Last.MSuper(planar=yuy2)
	superPreFilt=PreFilt.MSuper(planar=yuy2)
	bv = superPreFilt.MAnalyse(isb = true,  truemotion=true,delta = 1)
	Last.MFlow(super,bv,planar=yuy2)
	Last=(yuy2)?Planar2Interleaved():Last
	return Last
}
And use ClipClop() plug to select replacement frames from generated predicted clips.
EDIT: Returns clip where all frames predicted from previous/next frame.

EDIT: or bit more from elsewhere,

Quote:
Originally Posted by StainlessS View Post
Here a modification of a script function posted by Gavino (Thankyou Maestro)
http://forum.videohelp.com/threads/3...=1#post2089696
EDIT: Update version of below DoctorFrames included in zip, with frame interpolations up to 22 frames.
Code:
Function DoctorFrames(clip c, String "Scmd",String "Cmd",bool "Show",int "dv") {
# Replace damaged single frames using commands in a either command string or file.

    Scmd = Default(Scmd,"")     # User supplied list of newline or semicolon separated commands.
    Cmd  = Default(Cmd,"")      # User supplied Filename containing newline separated commands.
    Show = Default(Show,false)  # Show Info on frame
    dv   = Default(dv,0)        # ClipClop DebugView level (Need DebugView utility)

    sup         =   c.MSuper()
    PreFilt     =   c.DeGrainMedian()       # prefiltered for better motion analysis
    supPreFilt  =   PreFilt.MSuper()
    
    ci_bv   = sup.MAnalyse(isb=true, delta=2)
    ci_fv   = sup.MAnalyse(isb=false, delta=2)
    ci  = c.MFlowInter(sup,ci_bv, ci_fv, time=50.0, ml=100).DuplicateFrame(0).Trim(0,c.Framecount()-1)
    
    cp = c.SelectEvery(1,-1).Trim(0,c.Framecount()-1)           # chop off extra frame
    cn = c.SelectEvery(1,1).DuplicateFrame(c.Framecount()-1)    # Make same length as source

    pp_fv = supPreFilt.MAnalyse(isb = false,  truemotion=true,delta = 1)
    pp=c.MFlow(sup,pp_fv)

    pn_bv = supPreFilt.MAnalyse(isb = true,  truemotion=true,delta = 1)
    pn=c.MFlow(sup,pn_bv)

    NickName="  # Define Command mnemonics allowed in command string and file.
        CI=1    # CopyFromInterpolated
        CP=2    # CopyFromPrevious (frame n replaced with frame n-1)
        CN=3    # CopyFromNext (frame n replaced with frame n+1)
        PP=4    # PredictFromPrevious  (frame n replaced with frame predicted from frame n-1)
        PN=5    # PredictFromNext  (frame n replaced with frame predicted from frame n+1)
    "
    Return c.ClipClop(ci,cp,cn,pp,pn,scmd=Scmd,cmd=Cmd,nickname=Nickname,show=show,dv=dv)  
}

Avisource("D:\avs\avi\TEST.AVI")
Trim(1000,-10) ++ trim(2000,-10) ++ trim(3000,-10) # Make two scene changes

SCMD="
 CI 5           # CopyFromInterpolated clip frame 5
 CP 9           # CopyFromPrevious at last frame before scene change
 CN 10          # CopyFromNext at first frame of scene change
 CI 15          # CopyFromInterpolated clip frame 15
 PP 19          # PredictFromPrevious at last frame before scene change
 PN 20          # PredictFromNext at first frame of scene change
 CI 25          # CopyFromInterpolated clip frame 25
"
    
DoctorFrames(Scmd,show=true)
Function Could easily be extended.

Thread on fixing broken frames here:-
http://forum.doom9.org/showthread.php?t=152758

EDIT:- For single frame repair only.
EDIT:- Easy to mod for eg frame range denoising.
EDIT:- @IanB & Gavino, Thanx for impelling the array of clips, would never have gotten here otherwise.

EDIT: SCMD using mnemonics is equivalent to this (the mnemonic just an alternative means of giving clip index number)
Code:
SCMD="
 1 5           # CopyFromInterpolated clip frame 5
 2 9           # CopyFromPrevious at last frame before scene change
 3 10          # CopyFromNext at first frame of scene change
 1 15          # CopyFromInterpolated clip frame 15
 4 19          # PredictFromPrevious at last frame before scene change
 5 20          # PredictFromNext at first frame of scene change
 1 25          # CopyFromInterpolated clip frame 25
"
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 26th September 2022 at 01:32.
StainlessS is offline   Reply With Quote
Old 26th September 2022, 00:31   #4  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,992
deblurring the motion blur beforehand (using various "AI" methods) will probably help improve the quality of fwd vector interpolation. You can add back blur after to match with original frames
poisondeathray is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 12:44.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.