Reliable scene change detection?

[Mentar]

Is there any filter, codec or other perceivable way which would make for a relatively reliable scene change detection? And which would output these scene changes in any form of logfile which could be parsed?

Decimate metrics are one approach. Those tend to be more or less okay, but once there's major movement in the frame, they seem to be not too reliable.

I-Frames from a codec pass might be another approach. But here, most codecs seem to insert many i-frames in fades, so they alone won't be reliable too.

Any idea? Or maybe a weighed combination of several methods? I have no problem if this detection takes tons of time, as long as it's scriptable and produces some form of parsable logfile.

[Fizick]

There are many methods but no universal.
For example, my global motion estimation plugin DePan use inter-frame correlation (with FFT). It may output parameter "trust" to screen.
Also it may generate log file with scene change marked as null (0.00)horizontal shift. Probably, it can be used for your experiments.

[Mentar]

Hmm .. I was thinking about motion vectors being a good additional indicator for scene changes (since a real scene change should have NO perceivable vectors) - but no motion vectors alone might not be enough, see anime-typical still scenes with only small moving mouths. It would have to be "no motion vectors and these two frames don't look alike at all". Can depan do that?

Anyway, thanks for the feedback, I'll try it out tonight :)

[scharfis_brain]

maybe ANDed motionmasking could do the trick.


pseudocode:


x=last.motionmask()
y=logic(x,x.trim(1,0),"AND")


then build a frameevaluate environment, that checks the avgluma of y relative to a given number of prev. and next. frames of y.

here I include 2 prev and 2 next frames in that decision:

if y>y_-1 && y>y_-2 && y>y_+1 && y>y_+2 then we have a scenechange else not

[WarpEnterprises]

[QUOTE]Originally posted by Mentar
[B]since a real scene change should have NO perceivable vectors/QUOTE]

Then you have only sudden scenechanges in mind? Aren't there many SCs made at least with fast fades over few frames?

[Mentar]

I would define "scene change" as "something new is shown in the picture". Fades would not qualify. The background of this question is that I want to be able to cut a video into sections (the different scenes) and then make certain operations on the seperate sections.

[Fizick]

Similar to HandySaw DS for Adobe Premier?
http://www.davisr.com/cgi-bin/content/products/handysaw/description.htm

[Mentar]

Something like that, yeah (based on the description) .. I's only need the frame numbers though.

[Fizick]

script which outputs any kind of information into a logfile which can be parsed afterwards in any way to identify the scene changes.


The simplest code for progressive:


# some your source
converttoyv12()
depanestimate(last,trust=4,log="scenes.log")


load script in virtualdub(mod) and play from start to end. Exit.

Look scenes.log file. If number in second colunm (dx) is 0.00, therefore for this frame number (first colunm) we have scenechange.

If you add parameter info=true, you may see frames in virtualdub(mod) and tune the trust parameter and others.

[Mentar]

I've tried it, but frankly, I don't think it's configured right for anime.

To start with it - anime sources have lots of duplicates, so finding no motion vectors alone won't do the trick. What I need is

1) No motion vectors AND
2) Different picture

At the settings you gave I get a gazillion "scene changes" in a row, while 90% of them are none. Unless DePan can somehow establish 2) aswell, I'm afraid it won't be able to help much.