Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th December 2012, 06:55   #1  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Scene change detection

scenechange-0.2.0-2.7z

This is a scene change detection plugin for VapourSynth.
This plugin detects the scene changes of a clip and attaches _SceneChangePrev/_SceneChangeNext property (0 or 1) to all the frames.

I think that this will make it a bit easy to devellop temporal filters.

usage:
Code:
>>> import vapoursynth as vs
>>> core = vs.Core()
>>> core.std.LoadPlugin('/path/to/scenechange.dll')
>>> clip = something

>>> clip = core.scd.Detect(clip clip[, int thresh, int interval_h, int interval_v, data log])
Detect scene changes and attach _SceneChange properies to the clip.

thresh: The threshold of average of differences of the luma value between previous/next frames.
When the average of luma differences exceeds this, the frame is judged with a scene change.
(range is 1 to 254*2^(bitdepth - 8), default(or out of range) is assumed 15*2^(bitdepth-8))

interval_h: The horizontal interval of the pixel used for measurement of difference.
(1 to width, default(or out of range) is auto adjust)

interval_v: The vertical interval of the pixel used for measurement of difference.
(1 to height, default(or out of range) is auto adjust)

log: The name of the log file to output.
If this is set, the frame numbers applicable to a scene change are outputted as a text.
Describing full path is recommended.
(default is unset)

Code:
>>> clip = core.scd.ApplyLog(clip clip, data log)
Apply _SceneChangePrev/_SceneChangeNext properties to the clip based on the log outputted by scd.Detect.

note:
suppouted color families are GRAY(8/16bits) and YUV(8/9/10/16bits).
__________________
my repositories

Last edited by Chikuzen; 30th December 2012 at 07:15.
Chikuzen is offline   Reply With Quote
Old 24th December 2012, 14:25   #2  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,390
Still no better idea than the simple old threshold thing? - Looking at only the difference the difference between N-1 and N1 is moot (high motion etc.). Go second order. Build diff between N-1 and N, then look at how that diff is changing. The method has been there since years in SCSelect(), part of RemoveDirt.dll. It's not perfect either, of course, but much more robust than simple thresholding.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline   Reply With Quote
Old 24th December 2012, 19:13   #3  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
My main purpose is to consider practical use of the frame properties of VapourSynth instead of a better scene change detection algorithm.
yes, the algorithm which this plugin is using is old and simple. If someone writes a better thing, I will use it.

BTW, I think that it is very difficult to port kassandro's plugins to VapourSynth.
His softwares are exactry Open Binary.
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 24th December 2012, 22:02   #4  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
Quote:
_SceneChange property (0 or 1)
Possibly a better model might be to tag the frames either side of a scene change, i.e. 0 frame is normal and in the middle of a scene, 1 frame is the end of a scene, 2 frame is the beginning of a scene, of course the sequence would always be ...,0,0,1,2,0,0,...

This would allow a random access to a temporal filter to avoid fetching a next/prior frame from the information of the current frame alone.


P.S. And Merry Christmas to all
IanB is offline   Reply With Quote
Old 25th December 2012, 01:48   #5  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,870
Quote:
Originally Posted by IanB View Post
Possibly a better model might be to tag the frames either side of a scene change, i.e. 0 frame is normal and in the middle of a scene, 1 frame is the end of a scene, 2 frame is the beginning of a scene, of course the sequence would always be ...,0,0,1,2,0,0,... if you go for the 0 1 2 0 system you also have to define 3 to mean scene change on both sides. Debate on. Im not sure which one will make script writing easier.

This would allow a random access to a temporal filter to avoid fetching a next/prior frame from the information of the current frame alone.


P.S. And Merry Christmas to all
The way I scribbled it down was to mark both the first and the last frame of a scene with 1. So it'd be 0 0 0 1 1 0 0 ... caching should make the multiple frame access mostly free anyway.

Why not simply show this concept with a script? There aready is a built in function to calculate the average difference for a frame.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet

Last edited by Myrsloik; 25th December 2012 at 01:51.
Myrsloik is offline   Reply With Quote
Old 25th December 2012, 07:25   #6  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Quote:
Originally Posted by Myrsloik View Post
Why not simply show this concept with a script? There aready is a built in function to calculate the average difference for a frame.
Like this?
Code:
#!/bin/env python3

import vapoursynth as vs
import sys

class SceneChangeDetect(object):
    def __init__(self, core):
        self.std = core.std
        self.resize = core.resize.Point
        self.th = 0

    def set_scene_change(self, n, f):
        fout = f[0].copy()
        fout.props._SceneChange = ((f[1].props.DiffPrev[0] > self.th) +
                                   (f[1].props.DiffNext[0] > self.th))
        return fout

    def detect(self, clip, th=0.0588, interval_h=None, interval_v=None):
        self.th = th
        if interval_h is None:
            interval_h = int(clip.width / 320)
        if interval_v is None:
            interval_v = int(clip.width / 320)
        curr = self.resize(clip, int(clip.width / interval_h),
                           int(clip.width / interval_v), vs.GRAY8)
        prev = curr[0] + curr
        next = curr[1:]
        curr = self.std.PlaneDifference([curr, prev], 0, 'DiffPrev')
        curr = self.std.PlaneDifference([curr, next], 0, 'DiffNext')
        return self.std.ModifyFrame([clip, curr], self.set_scene_change)

if __name__ == '__main__':
    core = vs.Core()
    core.std.LoadPlugin('G:/vsplugins/d2vsource_beta4_v2.dll')

    clip = core.d2v.Source('D:\HD_Source\sample_1080i_00.d2v", nocrop=True)
    clip = SceneChangeDetect(core).detect(clip)

    log = open('scenecange.log', 'w')
    for n in range(clip.num_frames):
        scene_change = clip.get_frame(n).props._SceneChange[0]
        if scene_change:
            print(n, file=log)
    log.close()
Because I’m used to thinking on C rather than Python

EDIT:
The true reason is for removing scene change detection from the code of temporalsoften(VS version).
I assumed that the biggest bottleneck of that is the part , and considered how to remove them.
As the results, I proposed adding _SceneChange to you, and wrote this plugin.
Since origin was written in C, I've never noticed the way written in Python.
__________________
my repositories

Last edited by Chikuzen; 25th December 2012 at 16:16.
Chikuzen is offline   Reply With Quote
Old 25th December 2012, 15:13   #7  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 770
Quote:
Originally Posted by IanB View Post
Possibly a better model might be to tag the frames either side of a scene change, i.e. 0 frame is normal and in the middle of a scene, 1 frame is the end of a scene, 2 frame is the beginning of a scene, of course the sequence would always be ...,0,0,1,2,0,0,...

This would allow a random access to a temporal filter to avoid fetching a next/prior frame from the information of the current frame alone.


P.S. And Merry Christmas to all
What if there's a 1-frame "scene"?
ajp_anton is offline   Reply With Quote
Old 25th December 2012, 23:02   #8  |  Link
IanB
Avisynth Developer
 
Join Date: Jan 2003
Location: Melbourne, Australia
Posts: 3,168
For a 1-frame scene I might expect something like ...,0,0,1,3,2,0,0,... or based on Myrsloik's comments ...,0,0,1,1,1,0,0,...

I was more or less thinking about the processes and decisions inside a get a frame call. Assuming all the frames get accessed, the cache makes discarding a frame fetch from an adjacent scene pretty costless to the VS system. So I guess for the standard windowed kernel model :-
Code:
...
  for (int i = n-k; i < n+k; i++) {
    accumulate frame i processing
  }
just having the left to right scene change status can be sufficient. I was trying to think a little out of the box that maybe some right to left scene change status might be useful to someone at sometime. And yes just having a single bit flag is sufficient, you just have to get all the frames involved first then track the transitions.

Debate on further required.
IanB is offline   Reply With Quote
Old 26th December 2012, 16:47   #9  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
updated to 0.1.0-2

* add temporalsoften2

seems about +45% faster than temporalsoften(VS version).
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 28th December 2012, 09:46   #10  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
updated to 0.1.0-3

* add SSE2 intrinsic code to temporalsoften.c.

about x5.2 faster than 0.1.0-2
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 28th December 2012, 13:21   #11  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,086
Hi Chikuzen, I was wondering if TemporalSoften2 only supports 8 bit for now? I tried a YUV420P16 source and it resulted in vertical stripes that alternate color with each frame.
Also, how about adding a "planes=[]" parameter? Seems like a good idea to me but you guys might think otherwise.
Reel.Deel is offline   Reply With Quote
Old 28th December 2012, 15:18   #12  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Hmm, I'm investigating the cause now.
__________________
my repositories

Last edited by Chikuzen; 28th December 2012 at 18:45.
Chikuzen is offline   Reply With Quote
Old 28th December 2012, 15:31   #13  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 1,870
I decided how to flag next/previous scenechanges. See the documentation. Basically there's one flag for next frame transition is a scenechange and one marking the previous transition as a scene change. (unset/0 of course means no scenechange)
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 28th December 2012, 18:43   #14  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
updated to 0.2.0

* change property's name from _SceneChange to _SceneChangePrev/_SceneChangeNext.
* fix temporalsoften2

now, tenporalsoften2 use SSE2 on 16bit process.
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 29th December 2012, 05:20   #15  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 493
In the example of temporalsoften2's readme, it's still using props._SceneChange. Is that correct?
HolyWu is offline   Reply With Quote
Old 29th December 2012, 06:22   #16  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Quote:
Originally Posted by HolyWu View Post
In the example of temporalsoften2's readme, it's still using props._SceneChange. Is that correct?
oops.
seems there are other typos, too.
This is sane version.
http://pastebin.com/H7Jfq184
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 30th December 2012, 03:29   #17  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 493
The very first/last frame of the clip returned by TemporalSoften2 are messed up when radius=1.


When radius>=2, the first/last radius frames' result of TemporalSoften2 are incorrect compared to TemporalSoften's(both AVS and VS version).
http://screenshotcomparison.com/comparison/166267

Last edited by HolyWu; 30th December 2012 at 04:43.
HolyWu is offline   Reply With Quote
Old 30th December 2012, 07:18   #18  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
updated to 0.2.0-2
fixed in case rasius=1
__________________
my repositories
Chikuzen is offline   Reply With Quote
Old 8th August 2013, 10:07   #19  |  Link
HolyWu
Registered User
 
HolyWu's Avatar
 
Join Date: Aug 2006
Location: Taiwan
Posts: 493
Code:
import vapoursynth as vs
core = vs.get_core()
core.std.LoadPlugin(r'C:\Program Files\VapourSynth\filters\scenechange.dll')
clip = core.std.BlankClip(format=vs.YUV420P8, length=2, color=[16, 128, 128]) + core.std.BlankClip(format=vs.YUV420P8, length=2, color=[235, 128, 128])
clip = core.scd.Detect(clip, 25)

========================================

clip.get_frame(0).props._SceneChangePrev == 1
clip.get_frame(1).props._SceneChangePrev == 0
clip.get_frame(2).props._SceneChangePrev == 1
clip.get_frame(3).props._SceneChangePrev == 0

clip.get_frame(0).props._SceneChangeNext == 0
clip.get_frame(1).props._SceneChangeNext == 1
clip.get_frame(2).props._SceneChangeNext == 0
clip.get_frame(3).props._SceneChangeNext == 1
Um...the first frame's _SceneChangePrev and the last frame's _SceneChangeNext is indicated as a scenechange. Is it intended or a mistake, since it has no previus/next frame to compare with?
HolyWu is offline   Reply With Quote
Old 12th August 2013, 13:06   #20  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 597
Quote:
Originally Posted by HolyWu View Post
Um...the first frame's _SceneChangePrev and the last frame's _SceneChangeNext is indicated as a scenechange. Is it intended or a mistake, since it has no previus/next frame to compare with?
intended.
_SceneChangePrev is always True at the first frame.
and _SceneChangeNext is always True at the last frame.
__________________
my repositories

Last edited by Chikuzen; 12th August 2013 at 13:09.
Chikuzen is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:28.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.