Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th February 2014, 10:52   #1  |  Link
magikarp99
Registered User
 
Join Date: Feb 2014
Posts: 16
Syncing Two Videos

I have two almost identical videos. One has a complete uninterrupted luma signal, but no chroma signal. The other video has both chroma and luma signals but has dropped frames. The dropped frames are just missing from the stream entirely, there are no blank frames in place.

By overlaying the two signals in difference mode and computing the average luma I can detect where the first dropped frame is. But because this first dropped frame puts the two videos out of sync, this will then incorrectly identify all proceeding frames as dropped.

How can I get Avisynth to identify every dropped frame in the video with a chroma signal, and insert the missing luma frame at that point?

I'm very grateful for any help.
magikarp99 is offline   Reply With Quote
Old 19th February 2014, 16:27   #2  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,582
This findcuts script by gavino might help

http://forum.videohelp.com/threads/310044-video-censor-detection?p=1912566#post1912566
poisondeathray is offline   Reply With Quote
Old 19th February 2014, 19:56   #3  |  Link
raffriff42
Retried Guesser
 
raffriff42's Avatar
 
Join Date: Jun 2012
Posts: 1,377
Another possibility is "Glitch Analyzer (detect dropped frames)" by jmac698
http://forum.doom9.org/showthread.php?p=1462931
raffriff42 is offline   Reply With Quote
Old 20th February 2014, 00:42   #4  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,862
I'm afraid Glitch Analyzer is for finding dropped frames, but it works by preparing a special video with a timecode.
If you could post some short samples where there's frame drops, I could take a look at it. I need the samples to tune the frame-matching parameters. I have an idea for this, and it's a good excuse to work on some frame sync code I've always been meaning to work on.

p.s. I mean samples from both videos in the same area. You can use virtual dub to cut them.
jmac698 is offline   Reply With Quote
Old 20th February 2014, 02:01   #5  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,056
I think that the DBase funcs of RT_stats could (maybe) be of great assistance in these type of problems,.

EDIT: alignment might best be done from End to Beginning, after detectiion.

EDIT: GScript is a given. (We love Big G {EDIT: Despite his reluctance to introduce a 'break' statement, for which, he will never be forgiven })

EDIT: Never, Ever !
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 20th February 2014 at 02:57.
StainlessS is offline   Reply With Quote
Old 20th February 2014, 22:25   #6  |  Link
magikarp99
Registered User
 
Join Date: Feb 2014
Posts: 16
Quote:
Originally Posted by poisondeathray View Post
This findcuts script by gavino might help

http://forum.videohelp.com/threads/310044-video-censor-detection?p=1912566#post1912566
Thank you, this is a good example of GScript. I have used this as a basis.

Quote:
Originally Posted by raffriff42 View Post
Another possibility is "Glitch Analyzer (detect dropped frames)" by jmac698
http://forum.doom9.org/showthread.php?p=1462931
Thanks for the suggestion.

Quote:
Originally Posted by StainlessS View Post
EDIT: GScript is a given. (We love Big G {EDIT: Despite his reluctance to introduce a 'break' statement, for which, he will never be forgiven })
Another possibility is "Glitch Analyzer (detect dropped frames)" by jmac698
http://forum.doom9.org/showthread.php?p=1462931
EDIT: Never, Ever !
Indeed, I had not come across GScript before and it appears to bring sorely need functionality to AVISynth! A break statement would be great.

Quote:
Originally Posted by jmac698 View Post
I'm afraid Glitch Analyzer is for finding dropped frames, but it works by preparing a special video with a timecode.
If you could post some short samples where there's frame drops, I could take a look at it. I need the samples to tune the frame-matching parameters. I have an idea for this, and it's a good excuse to work on some frame sync code I've always been meaning to work on.

p.s. I mean samples from both videos in the same area. You can use virtual dub to cut them.
I would really appreciate that.

I have got a short script, which works fairly well, going already; I based it upon the examples supplied here. There a few kinks to sort out though. I will upload a sample clip and my script soon.

Any suggestions on where to upload the video clips?

Last edited by magikarp99; 20th February 2014 at 22:32.
magikarp99 is offline   Reply With Quote
Old 20th February 2014, 22:30   #7  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,582
Quote:
Originally Posted by magikarp99 View Post

Any suggestions on where to upload the video clips?
There are plenty of free file hosts

mediafire.com or sendspace.com seem to be popular for hosting samples around here
poisondeathray is offline   Reply With Quote
Old 20th February 2014, 22:42   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,056
Quote:
Any suggestions on where to upload the video clips?
Groucho2004 recommends DropBox.com rather than Mediafire, I've recently registered but have yet to copy my stuff from MediaFire.com.
Recommended as being lightweight and unobtrusive.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 20th February 2014 at 22:42. Reason: speeling
StainlessS is offline   Reply With Quote
Old 21st February 2014, 17:49   #9  |  Link
magikarp99
Registered User
 
Join Date: Feb 2014
Posts: 16
Just uploading some video clips. I have removed the audio to save some file size.

I simplified the issue a bit before. Essentially the chroma video has 3 types of drops:
  1. A frame is just missing, there is no visual indication unless compared to another video.
  2. Two consecutive frames are missing, but a black frame is present in their place.
  3. Three consecutive frames are missing, but two black frames are in their place.

The script I currently have must start with the first frame of each video correctly in sync. The luma levels must also be matched very closely beforehand. It then proceeds to compute two values iteratively:
  1. The luma difference between the current frames in the luma and chroma videos.
  2. The luma difference between the current frame in the chroma video and the next frame in the luma video.
If the difference is lower for the second value, then it is assumed that a dropped frame has been reached. Another indication of a dropped frame is if average luma value for the frame is 16.

This proves to be very effective, but there are two issues:
  1. The gain/offset of the luma signal alters abruptly at points in the video. This can lead to a false positive detection of a dropped frame. Which then causes most succeeding frames to be false positives. e.g. at frame 1517 in the examples.
  2. Currently I am using overlays to add a chroma signal to the returned luma frames. This quickly eats up memory, so this isn't an ideal solution. I may have to split overlays into multiple scripts and concatenate the resulting videos unless someone can suggest a better solution.

It works well in this example all the way up to frame 1517 where it makes a false positive identifitication.

Here is the script, put this in a file named resync.avs:
Code:
# ReSync
# Based on FindCuts (Gavino, 13th Sept 2009)
# http://forum.videohelp.com/threads/310044-video-censor-detection?p=1912566#post1912566 
# 
# Resyncs two videos when luma channels are almost identical.
# Intended to sync video with chroma channel, but dropped frames
# with video without chroma, but an intact, uninterrupted luma signal
#
# Chroma video has following varieties of drops
# 1. Frame is dropped silently
# 2. Two frames are dropped, and one blank frame takes their place
# 3. Three frames are dropped, and two blank frames take their place
#
# Requires GScript (http://forum.doom9.org/showthread.php?t=147846)

function ReSync(clip full, clip cut, string "logFile") {
  log = Defined(logFile)
  txt = "Finding cuts ..."
  dropped = "Dropped"
  sep = ", "
  sep2 = ": "
  if (log) { WriteFileStart(full, logFile, "txt") }

  # Videos from to return frames
  show = full
  show_cut = cut
  # Crops noise from top and bottom of frames
  full = full.Crop(0, 14, 0, -8)
  cut = cut.Crop(0, 14, 0, -8)
  # Video shifted forward one frame
  next_full = full.Trim(1, 0)
  result = BlankClip(show, length=0)
  current_frame = 0
  limit = full.FrameCount

  # loop over sequence of cuts:
  while (current_frame < limit) {
    start = current_frame
    # find next dropped frame:
	# compares luma difference between current frames, and next frame
	# also checks frame is not blank
	 while (current_frame < limit && !(LumaDifference(full, cut) > (LumaDifference(next_full, cut))) && AverageLuma(cut) > 16) {
         end = current_frame
	     # Values for logging
	     if (log) {
	         diff_current = LumaDifference(full, cut)
             diff_next = LumaDifference(next_full, cut)
             WriteFileStart(full, logFile, "end", "sep2", "diff_current", "sep", "diff_next", append = true)
	         current_frame = end
	     }
	     current_frame = current_frame + 1
     }
	 missing = current_frame
	 if (log) {
	     diff_current = LumaDifference(full, cut)
         diff_next = LumaDifference(next_full, cut)
         WriteFileStart(full, logFile, "missing", "sep2", "diff_current", "sep", "diff_next", append = true)
	     current_frame = missing
	 }
	 avg_luma1 = AverageLuma(cut)
	 current_frame = current_frame + 1
	 avg_luma2 = AverageLuma(cut)
	 #current_frame = current_frame - 1
	 # if reached a blank frame
	 if (avg_luma1 == 16) { 
	     # if two successive blank frames
	     if (avg_luma2 == 16) {
		     if (log) {
		         WriteFileStart(full, logFile, "dropped", "sep2", "missing", append = true)
			     WriteFileStart(full, logFile, "dropped", "sep2", "missing + 1", append = true)
		     }
			 # Append uninterrupted section of chroma video to result
			 # Overlay previous and succeeding chroma frames with luma frames and append
			 result = result + show_cut.Trim(start, end) + Overlay(show.Trim(missing, missing + 2), show_cut.Trim(missing - 1, missing - 1) + show_cut.Trim(missing - 1, missing - 1) + show_cut.Trim(missing + 2, missing + 2), mode="chroma", opacity=1)
			 # Pad videos
			 cut = cut.Loop(2, 0, 0)
		     show_cut = show_cut.Loop(2, 0, 0)
			 current_frame = missing + 3
		 } else {
	         if (log) {
			     WriteFileStart(full, logFile, "dropped", "sep2", "missing", append = true)
		         WriteFileStart(full, logFile, "missing", "sep", "avg_luma1", append = true)
			 }
			 # Append uninterrupted section of chroma video to result
			 # Overlay previous and succeeding chroma frames with luma frames and append
		     result = result + show_cut.Trim(start, end) + Overlay(show.Trim(missing, missing + 1), show_cut.Trim(missing - 1, missing - 1) + show_cut.Trim(missing + 1, missing + 1), mode="chroma", opacity=1)
			 # Pad videos
		     cut = cut.Loop(2, 0, 0)
		     show_cut = show_cut.Loop(2, 0, 0)
		     current_frame = missing + 2
	     }
     # else single frame dropped silently
	 } else {
	     # Append uninterrupted section of chroma video to result
	     # Overlay previous chroma frame with luma frame and append
		 result = result + show_cut.Trim(start, end) + Overlay(show.Trim(missing, missing), show_cut.Trim(missing - 1, missing - 1), mode="chroma", opacity=1)
		 # Pad videos
	     cut = cut.Loop(2, 0, 0)
		 show_cut = show_cut.Loop(2, 0, 0)
	     if (log) {
             WriteFileStart(full, logFile, "missing", append = true)
             current_frame = missing + 1 # WriteFileStart sets current_frame to -1 !!!
         } else {
	      current_frame = current_frame + 2
	     }
     }
   }
   #return AudioDub(result, full)
   return result
 }
Use it like this:
Code:
chroma = AviSource("chroma_example.avi")
luma = AviSource("luma_example.avi")

#Match the luma signals
luma = ColorYUV(luma, off_y=-16, gain_y=-1, cont_y=0)
chroma = ColorYUV(chroma, off_y=-18, gain_y=0)
luma = Limiter(luma, 16, 235, 16, 240)
chroma = Limiter(chroma, 16, 218, 16, 240)

GImport("resync.avs")

ReSync(luma.ConvertToYV12(), chroma.ConvertToYV12(), "cutlog.txt")

#Uncomment this to see difference between frames and luma signals
#StackHorizontal(ScriptClip(Overlay(luma.Crop(0, 14, 0, -8), chroma.Crop(0, 14, 0, -8).Greyscale(), mode="Difference", opacity=1).ConvertToYV12(), "Subtitle(String(AverageLuma))"), Overlay(VideoScope(Greyscale(luma.Crop(0, 14, 0, -8)), "side", true, "Y").ConvertToYV12(), VideoScope(Greyscale(chroma.Crop(0, 14, 0, -8)), "side", true, "Y").ConvertToYV12(), mode="Difference", opacity=1).Crop(720, 0, 0, 0))
Will post sample videos when they have uploaded.

Last edited by magikarp99; 21st February 2014 at 17:51.
magikarp99 is offline   Reply With Quote
Old 21st February 2014, 19:34   #10  |  Link
magikarp99
Registered User
 
Join Date: Feb 2014
Posts: 16
http://www.mediafire.com/watch/2a7ndbzak5n8vaa/chroma_example.avi
http://www.mediafire.com/watch/2txf022cvx3l41l/luma_example.avi
magikarp99 is offline   Reply With Quote
Old 22nd February 2014, 23:41   #11  |  Link
magikarp99
Registered User
 
Join Date: Feb 2014
Posts: 16
Quote:
Originally Posted by magikarp99 View Post
The gain/offset of the luma signal alters abruptly at points in the video. This can lead to a false positive detection of a dropped frame. Which then causes most succeeding frames to be false positives. e.g. at frame 1517 in the examples.
I think my problem may be solved if I can normalise the luma of one video to the other on a frame to frame basis. Any suggestions?

This popped up in a search: http://forum.doom9.org/showthread.php?t=138662
I haven't had a chance to try any of the suggestions there yet.
magikarp99 is offline   Reply With Quote
Old 23rd February 2014, 08:15   #12  |  Link
jmac698
Registered User
 
Join Date: Jan 2006
Posts: 1,862
I'm sorry I don't think I'll have time to look at it, so it's up for grabs.
jmac698 is offline   Reply With Quote
Old 24th February 2014, 13:23   #13  |  Link
magikarp99
Registered User
 
Join Date: Feb 2014
Posts: 16
Quote:
Originally Posted by jmac698 View Post
I'm sorry I don't think I'll have time to look at it, so it's up for grabs.
Thanks anyway.

I think I have got it almost perfect now. Sorted out both my memory and levels issues I believe. Just doing a final test. If it all works I will clean up my script and put it up here

MergeChroma is much more efficient than using Overlay. It has a small memory footprint and is much faster.

To sort out the levels issues I am using ColorLike on every frame.
magikarp99 is offline   Reply With Quote
Old 28th February 2014, 11:17   #14  |  Link
magikarp99
Registered User
 
Join Date: Feb 2014
Posts: 16
This script works almost perfectly. It is a mess, and I don't really have the time to clean it up too much. I'm sure it could also be made more efficient.

File attached.
Attached Files
File Type: zip resync.zip (1.9 KB, 10 views)
magikarp99 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:00.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.