Identifying a video frame number or timestamp based on a source image?

madwoota · 20th March 2015, 09:12

I have a few hundred video files (.TS) that I have recorded (Free-To-Air) that I would like to batch trim down to save some space. Similar to the concept of ComSkip for identifying commercial breaks for removal.

Ideally, I'd like to use a sample image as a source, then identify either the timestamp or frame number of the video file when that frame first appears in the video. This might be a high % based match given the variations in video codecs vs image formats. The actual source image would just be a grab from the opening title sequence of the show.

Once the start & end frame numbers are found, I can easily batch trim the files to the content between the markers.

I'm really just looking for some suggestions on methods, tools, applications, plugins, scripts or libraries I could utilise to do this as I'm stumped.

ComSkip couldn't match either the start or end of this particular show as there is no scene break or black sequence to segment it. It also incorrectly tags the start of the credits as the start of a commercial break.

OpenCV (via OpenFrameworks) seemed promising, but I'll need to teach myself C++ first and it doesnt handle .TS (although thats solvable with a quick transcode).

StainlessS · 20th March 2015, 11:23

Google this:

Code:

A novel scheme for fast and efficient video sequence matching using compact signatures

White paper, identifying compressed frames.
Dont know if it is implemented in commercial or other software.

madwoota · 20th March 2015, 13:22

Thanks StainlessS, I didnt actually read the paper, but some of the search results I got gave me a few hints to look again scene change detection. Specifically, this post which pointed me at the Avisynth conditional & debug filters.

Turns out Avisynth has a Compare() function that outputs to a text file! Who knew? Lol.

It's a bit brute force, but I can make it work well enough for my needs. Just had to DGIndex the TS file to D2V then run the following Avisynth script:

Code:

LoadPlugin("dgdecode.dll")
clip1=MPEG2Source("test.d2v")

#This is in RGB24
clip2=ImageSource("snapshot.jpg", end=0, fps=25).ConvertToYV12()

Compare(clip1, clip2, "", "compare.log")

Process the subsequent compare.log and find the closest value to 0 in the Mean Deviation (frame #49)!

Code:

           Mean               Max    Max             
         Absolute     Mean    Pos.   Neg.            
 Frame     Dev.       Dev.    Dev.   Dev.  PSNR (dB) 
-----------------------------------------------------
    45   51.9287   +35.2766  205    -59     11.8964
    46   51.9408   +35.2722  207    -60     11.8930
    47   51.9392   +35.2412  206    -60     11.8935
    48   51.9580   +35.3230  209    -60     11.8908
    49    1.2914    +0.0085   29    -16     41.8406
    50    8.3169    -0.2046  149    -40     23.0569
    51   12.8632    -0.4066  143    -45     20.3819
    52   16.0608    -0.7585  140    -46     19.1283
    53   18.4752    -0.9800  143    -47     18.2948

Appreciate the nudge in the right direction. Cheers.

StainlessS · 20th March 2015, 13:53

Dont know if this of any use, MatchFrames and LocateFrames scripts: http://forum.doom9.org/showthread.ph...ht=matchframes
Updates to both are included in RT_Stats zip.

For Searching for many frames in single clip, might want to take a peek at RT_QwikScan.
Slow to create a DBase for the target clip, but only milliseconds to find frames afterwards.

Edit: for scene change detection search 'dbsc' on forum.

20th March 2015, 09:12	#1 \| Link
madwoota . Join Date: Oct 2005 Posts: 2	Identifying a video frame number or timestamp based on a source image? I have a few hundred video files (.TS) that I have recorded (Free-To-Air) that I would like to batch trim down to save some space. Similar to the concept of ComSkip for identifying commercial breaks for removal. Ideally, I'd like to use a sample image as a source, then identify either the timestamp or frame number of the video file when that frame first appears in the video. This might be a high % based match given the variations in video codecs vs image formats. The actual source image would just be a grab from the opening title sequence of the show. Once the start & end frame numbers are found, I can easily batch trim the files to the content between the markers. I'm really just looking for some suggestions on methods, tools, applications, plugins, scripts or libraries I could utilise to do this as I'm stumped. ComSkip couldn't match either the start or end of this particular show as there is no scene break or black sequence to segment it. It also incorrectly tags the start of the credits as the start of a commercial break. OpenCV (via OpenFrameworks) seemed promising, but I'll need to teach myself C++ first and it doesnt handle .TS (although thats solvable with a quick transcode).

20th March 2015, 11:23	#2 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	Google this: Code: A novel scheme for fast and efficient video sequence matching using compact signatures White paper, identifying compressed frames. Dont know if it is implemented in commercial or other software. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

20th March 2015, 13:22	#3 \| Link
madwoota . Join Date: Oct 2005 Posts: 2	Thanks StainlessS, I didnt actually read the paper, but some of the search results I got gave me a few hints to look again scene change detection. Specifically, this post which pointed me at the Avisynth conditional & debug filters. Turns out Avisynth has a Compare() function that outputs to a text file! Who knew? Lol. It's a bit brute force, but I can make it work well enough for my needs. Just had to DGIndex the TS file to D2V then run the following Avisynth script: Code: LoadPlugin("dgdecode.dll") clip1=MPEG2Source("test.d2v") #This is in RGB24 clip2=ImageSource("snapshot.jpg", end=0, fps=25).ConvertToYV12() Compare(clip1, clip2, "", "compare.log") Process the subsequent compare.log and find the closest value to 0 in the Mean Deviation (frame #49)! Code: Mean Max Max Absolute Mean Pos. Neg. Frame Dev. Dev. Dev. Dev. PSNR (dB) ----------------------------------------------------- 45 51.9287 +35.2766 205 -59 11.8964 46 51.9408 +35.2722 207 -60 11.8930 47 51.9392 +35.2412 206 -60 11.8935 48 51.9580 +35.3230 209 -60 11.8908 49 1.2914 +0.0085 29 -16 41.8406 50 8.3169 -0.2046 149 -40 23.0569 51 12.8632 -0.4066 143 -45 20.3819 52 16.0608 -0.7585 140 -46 19.1283 53 18.4752 -0.9800 143 -47 18.2948 Appreciate the nudge in the right direction. Cheers.

20th March 2015, 13:53	#4 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	Dont know if this of any use, MatchFrames and LocateFrames scripts: http://forum.doom9.org/showthread.ph...ht=matchframes Updates to both are included in RT_Stats zip. For Searching for many frames in single clip, might want to take a peek at RT_QwikScan. Slow to create a DBase for the target clip, but only milliseconds to find frames afterwards. Edit: for scene change detection search 'dbsc' on forum. __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 20th March 2015 at 14:35.

Thread Tools	Search this Thread
Show Printable Version Email this Page	Search this Thread: Advanced Search
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode