Possibility of adding keyframe reading to avisynth?

[Zarxrax]

Virtualdub's ability to seek to keyframes is one of it's most useful features. It allows you to very quickly seek through an entire file in a matter of seconds, to search for a specific scene. When you load a file through avisynth though, this functionality is lost, because avisynth outputs every frame as a keyframe. Because of this, it becomes very difficult to search through a file for a specific scene. You can still sort of search through the file by dragging the slider around, but without knowing where real keyframes are located, the video is very jerky and jumpy, requiring you to go through it much more slowly as you wait for each frame to be rendered. The longer the file is, the more difficult it is to seek through even with the slider. Even minor accidental movement could seek far away from the area you meant to be in, and you can easily completely skip over the scene that you are trying to find.

Basically what I was wondering is, would it be possible to write a filter or something for avisynth that would let it report frames the same as the input file? In other words, frames that were originally keyframes will be reported as keyframes, and frames that were delta frames would be reported as delta frames?

[Bidoche]

avs outputs raw video, and in raw all frames are keyframes. That's all.

[gircobain]

Just use ALT-left/right to advance 50 frames back/forward or use CTRL-G to go to a specific frame/timecode

[Mug Funky]

actually, as i understand it, in VfW a frame is a keyframe if it's called a keyframe, regardless of weather it really is. i think an early version of theora (or was it rududu) labelled all frames as keyframes, and the file would play (it would just seek slowly).

perhaps avisource could be modified to simply find out which frames are keyframes and tell that to whatever loads it. so it would still return all frames as uncompressed (of course), but you could still seek to keyframes in virtualdub and get the expected behaviour.

trim() and others would need to be modified to export which frames are really keyframes

[Wilbert]

actually, as i understand it, in VfW a frame is a keyframe if it's called a keyframe, regardless of weather it really is. i think an early version of theora (or was it rududu) labelled all frames as keyframes, and the file would play (it would just seek slowly).
I don't see the point in calling frames keyframes if they are not keyframes. (I assume you know the difference between keyframes and deltaframes.)

perhaps avisource could be modified to simply find out which frames are keyframes and tell that to whatever loads it. so it would still return all frames as uncompressed (of course), but you could still seek to keyframes in virtualdub and get the expected behaviour.
You mean which frames *were* keyframes. I think it's impossible to pass that info to VDub. But why is this necessary? This is just a VDub problem (how to seek if all frames are keyframes). As gircobain says, just skip 50 frames at the time.

[mg262]

Maybe for your purposes a better method would be to generate a list of scenes and then create a clip displaying a few frames (evenly spaced, including end and beginning) from each scene? I've done this in a filterset I'm part way through writing and it's proved quite useful. The first screenshot in this:

http://forum.doom9.org/showthread.php?t=95115

shows the kind of thing I mean (though it's slightly different) . Obviously though it does require you to have spare computer time.

[Zarxrax]

As gircobain says, just skip 50 frames at the time.

But as I said in my original post, the problem with this is that you aren't seeking on real keyframes, and thus its slow as hell. Seeking on the original keyframes is very fast. I routinely have to search through a large number of files (typically 20-50) looking for a particular clip, so this difference in speed means the difference between a 30 minute job and a 4 hour job.


mg262: I'll take a look at your function. It doesn't seem like an optimal solution, but it may be usefull.

[stickboy]

I don't see the point in calling frames keyframes if they are not keyframes. (I assume you know the difference between keyframes and deltaframes.)He meant the other way: reporting uncompressed keyframes as delta frames.

[mg262]

@zarxrax

Sorry, that was a bit unclear - the LineUpScenes filter/screenshot (which displays scenes from 2 clips) I liinked to isn't the same as the 'FramesFromScene' in the filterset. I can create/extract a 1-clip version if you want. You can either pick a scene detection method or I can set it to input scene breaks in a text file (like the filter I linked to).

Do you mind if I I ask what the context is? i.e. what's the broad reason for trying to find particular scenes?

[Zarxrax]

Do you mind if I I ask what the context is? i.e. what's the broad reason for trying to find particular scenes?

Video editing. Anime music videos, particularly.

[adamhj]

But as I said in my original post, the problem with this is that you aren't seeking on real keyframes, and thus its slow as hell. Seeking on the original keyframes is very fast. I routinely have to search through a large number of files (typically 20-50) looking for a particular clip, so this difference in speed means the difference between a 30 minute job and a 4 hour job.


mg262: I'll take a look at your function. It doesn't seem like an optimal solution, but it may be usefull.
eh..i think you can seek with those original files, find the clip you want, remenber the frame number of begining and ending frame of the clip, then use ctrl+g in vdm to go to that point.

[Zarxrax]

eh..i think you can seek with those original files, find the clip you want, remenber the frame number of begining and ending frame of the clip, then use ctrl+g in vdm to go to that point.

That works fine and dandy with avi files, but what about stuff that can't be read by virtualdub? WMV files for instance. Also OGM or MKV files (although vdubmod supposedly reads them, intensive seeking with any version will definately cause it to crash every time for me.)

[IanB]

@Zarxrax,

I understand the problem, and yes it is annoyingly slow to scrub the timeline when processing a "keyframe challanged" source. I had a peek at what was required to reflect the source keyframe information into the output avi stream and it is as David tersely said "avs outputs raw video, and in raw all frames are keyframes. That's all."

Maybe somone cunning can spy a lateral thinking way to represent keyframe info up to the caller.

IanB

[Richard Berg]

What if Avisynth provided another filter like AviKeyframeSource() that only returned keyframes? That would seek super fast no matter which client app you used.

Hmm, wouldn't help much, because you'd also need to return the "real" offset of each keyframe in order to apply edits to the original source. I think the best we can do is a clever extension of Trim().

[tsp]

Couldn't AviKeyframeSource() just create a variable called RealFrameNumber that was equal to the original framenumber or something like that so the user could see the real number by using subtitle(string(RealFrameNumber))?

[stickboy]

Ick. That sounds complicated, and you'd probably need to use ScriptClip.

If we went down the AviKeyframeSource route, why not just add a parameter to it that invokes the text renderer automatically (e.g. showFrameNumber=true).

[mg262]

What if Avisynth provided another filter like AviKeyframeSource() that only returned keyframes? That would seek super fast no matter which client app you used.

Hmm, wouldn't help much, because you'd also need to return the "real" offset of each keyframe in order to apply edits to the original source.
How about duplicating keyframes to replace non-keyframes? So e.g.

K1 2 3 4 5 K6 7 8 ...

becomes

K1 K1 K1 K1 K1 K6 K6 ...

I still don't think this is a very clean solution, but it seems better than changing the frame count.

[stickboy]

That no longer meets the original goal of making it easy to skip from scene to scene.

[tsp]

Ick. That sounds complicated, and you'd probably need to use ScriptClip.

If we went down the AviKeyframeSource route, why not just add a parameter to it that invokes the text renderer automatically (e.g. showFrameNumber=true).

because the filters after AviKeyframeSource could destroy the text (ie crop). But what about a function ShowRealFrameNumber that is just an alias for scriptclip("subtitle(string(RealFrameNumber))",after_frame=true)

[stickboy]

I still think that's overkill. Why would you waste time applying filters? The original goal was to find scene changes more quickly.

[mg262]

That no longer meets the original goal of making it easy to skip from scene to scene.

I think we're talking slightly at cross-purposes... duplicating keyframes was meant to deal with the 'jerky and jumpy' problem in the original post so you could seek more fluidly. have I got the wrong end of the stick?

[stickboy]

Well, if Zarxrax got what he wanted (AviSynth passing the keyframe/deltaframe information through to VirtualDub), then I assume he'd be using the arrow keys, not the slider...

[Zarxrax]

Some interesting suggestions so far. It sounds like having it as was requested would be quite difficult or even impossible with the current avisynth.
Having the original Keyframe/Delta frame references for vdub would also be helpfull for lots of other applications, such as fansubbing, where seeking to keyframes is important for doing timing and all. But alas, perhaps it not meant to be.

Something like AVIKeyframesource would probably be helpfull for just searching through stuff, though it would really need to be "DirectShowKeyframeSource" for non-avi stuff, which is what poses the most difficulties.

[mg262]

I still think that's overkill. Why would you waste time applying filters?
That makes me wonder... what purpose is avisynth serving in this process? Is it just there for the convenience of DGIndex, etc.? (Sorry if that's a dense q. -- its been a v. long day and :stupid: a.t.m.!)

[IanB]

Nice one Richard :D

How about this for a functionality spec for AVIKeyframesource and DirectShowKeyframeSource.

Let N be the currently requested frame, let Last be the last frame requested and let K (default=1) be a user chunkyness parameter.If ((N < Last) || (N > Last+K))
F=PreviousKeyFrame(N)
Last=FrameNumber(F)
return F.SomeOptionalText()
else
F=Frame(N)
Last=FrameNumber(F)
return F /* no text */So if you jump around the timeline you get the previous keyframe displayed with some optional text telling what is happening. When you step forward less than K frames you take the delta frame performance hit but get to see the real frame. This allows you to play thru a scene but still seek around very fast.

One small problem will be the cache once a frame is rendered the cache will continue to return that instance of the frame until it is overwritten in the cache by newer frames (play with Normalize(show=true) to see the problem).

Thorts
IanB