magikarp99
19th February 2014, 10:52
I have two almost identical videos. One has a complete uninterrupted luma signal, but no chroma signal. The other video has both chroma and luma signals but has dropped frames. The dropped frames are just missing from the stream entirely, there are no blank frames in place.
By overlaying the two signals in difference mode and computing the average luma I can detect where the first dropped frame is. But because this first dropped frame puts the two videos out of sync, this will then incorrectly identify all proceeding frames as dropped.
How can I get Avisynth to identify every dropped frame in the video with a chroma signal, and insert the missing luma frame at that point?
I'm very grateful for any help. :)
jmac698
20th February 2014, 00:42
I'm afraid Glitch Analyzer is for finding dropped frames, but it works by preparing a special video with a timecode.
If you could post some short samples where there's frame drops, I could take a look at it. I need the samples to tune the frame-matching parameters. I have an idea for this, and it's a good excuse to work on some frame sync code I've always been meaning to work on.
p.s. I mean samples from both videos in the same area. You can use virtual dub to cut them.
StainlessS
20th February 2014, 02:01
I think that the DBase funcs of RT_stats could (maybe) be of great assistance in these type of problems,.
EDIT: alignment might best be done from End to Beginning, after detectiion.
EDIT: GScript is a given. (We love Big G {EDIT: Despite his reluctance to introduce a 'break' statement, for which, he will never be forgiven :)})
EDIT: Never, Ever ! :)
magikarp99
20th February 2014, 22:25
This findcuts script by gavino might help
http://forum.videohelp.com/threads/310044-video-censor-detection?p=1912566#post1912566
Thank you, this is a good example of GScript. I have used this as a basis.
Another possibility is "Glitch Analyzer (detect dropped frames)" by jmac698
http://forum.doom9.org/showthread.php?p=1462931
Thanks for the suggestion.
EDIT: GScript is a given. (We love Big G {EDIT: Despite his reluctance to introduce a 'break' statement, for which, he will never be forgiven :)})
Another possibility is "Glitch Analyzer (detect dropped frames)" by jmac698
http://forum.doom9.org/showthread.php?p=1462931
EDIT: Never, Ever ! :)
Indeed, I had not come across GScript before and it appears to bring sorely need functionality to AVISynth! A break statement would be great.
I'm afraid Glitch Analyzer is for finding dropped frames, but it works by preparing a special video with a timecode.
If you could post some short samples where there's frame drops, I could take a look at it. I need the samples to tune the frame-matching parameters. I have an idea for this, and it's a good excuse to work on some frame sync code I've always been meaning to work on.
p.s. I mean samples from both videos in the same area. You can use virtual dub to cut them.
I would really appreciate that.
I have got a short script, which works fairly well, going already; I based it upon the examples supplied here. There a few kinks to sort out though. I will upload a sample clip and my script soon.
Any suggestions on where to upload the video clips?
StainlessS
20th February 2014, 22:42
Any suggestions on where to upload the video clips?
Groucho2004 recommends DropBox.com rather than Mediafire, I've recently registered but have yet to copy my stuff from MediaFire.com.
Recommended as being lightweight and unobtrusive.
magikarp99
21st February 2014, 17:49
Just uploading some video clips. I have removed the audio to save some file size.
I simplified the issue a bit before. Essentially the chroma video has 3 types of drops:
A frame is just missing, there is no visual indication unless compared to another video.
Two consecutive frames are missing, but a black frame is present in their place.
Three consecutive frames are missing, but two black frames are in their place.
The script I currently have must start with the first frame of each video correctly in sync. The luma levels must also be matched very closely beforehand. It then proceeds to compute two values iteratively:
The luma difference between the current frames in the luma and chroma videos.
The luma difference between the current frame in the chroma video and the next frame in the luma video.
If the difference is lower for the second value, then it is assumed that a dropped frame has been reached. Another indication of a dropped frame is if average luma value for the frame is 16.
This proves to be very effective, but there are two issues:
The gain/offset of the luma signal alters abruptly at points in the video. This can lead to a false positive detection of a dropped frame. Which then causes most succeeding frames to be false positives. e.g. at frame 1517 in the examples.
Currently I am using overlays to add a chroma signal to the returned luma frames. This quickly eats up memory, so this isn't an ideal solution. I may have to split overlays into multiple scripts and concatenate the resulting videos unless someone can suggest a better solution.
It works well in this example all the way up to frame 1517 where it makes a false positive identifitication.
Here is the script, put this in a file named resync.avs:
# ReSync
# Based on FindCuts (Gavino, 13th Sept 2009)
# http://forum.videohelp.com/threads/310044-video-censor-detection?p=1912566#post1912566
#
# Resyncs two videos when luma channels are almost identical.
# Intended to sync video with chroma channel, but dropped frames
# with video without chroma, but an intact, uninterrupted luma signal
#
# Chroma video has following varieties of drops
# 1. Frame is dropped silently
# 2. Two frames are dropped, and one blank frame takes their place
# 3. Three frames are dropped, and two blank frames take their place
#
# Requires GScript (http://forum.doom9.org/showthread.php?t=147846)
function ReSync(clip full, clip cut, string "logFile") {
log = Defined(logFile)
txt = "Finding cuts ..."
dropped = "Dropped"
sep = ", "
sep2 = ": "
if (log) { WriteFileStart(full, logFile, "txt") }
# Videos from to return frames
show = full
show_cut = cut
# Crops noise from top and bottom of frames
full = full.Crop(0, 14, 0, -8)
cut = cut.Crop(0, 14, 0, -8)
# Video shifted forward one frame
next_full = full.Trim(1, 0)
result = BlankClip(show, length=0)
current_frame = 0
limit = full.FrameCount
# loop over sequence of cuts:
while (current_frame < limit) {
start = current_frame
# find next dropped frame:
# compares luma difference between current frames, and next frame
# also checks frame is not blank
while (current_frame < limit && !(LumaDifference(full, cut) > (LumaDifference(next_full, cut))) && AverageLuma(cut) > 16) {
end = current_frame
# Values for logging
if (log) {
diff_current = LumaDifference(full, cut)
diff_next = LumaDifference(next_full, cut)
WriteFileStart(full, logFile, "end", "sep2", "diff_current", "sep", "diff_next", append = true)
current_frame = end
}
current_frame = current_frame + 1
}
missing = current_frame
if (log) {
diff_current = LumaDifference(full, cut)
diff_next = LumaDifference(next_full, cut)
WriteFileStart(full, logFile, "missing", "sep2", "diff_current", "sep", "diff_next", append = true)
current_frame = missing
}
avg_luma1 = AverageLuma(cut)
current_frame = current_frame + 1
avg_luma2 = AverageLuma(cut)
#current_frame = current_frame - 1
# if reached a blank frame
if (avg_luma1 == 16) {
# if two successive blank frames
if (avg_luma2 == 16) {
if (log) {
WriteFileStart(full, logFile, "dropped", "sep2", "missing", append = true)
WriteFileStart(full, logFile, "dropped", "sep2", "missing + 1", append = true)
}
# Append uninterrupted section of chroma video to result
# Overlay previous and succeeding chroma frames with luma frames and append
result = result + show_cut.Trim(start, end) + Overlay(show.Trim(missing, missing + 2), show_cut.Trim(missing - 1, missing - 1) + show_cut.Trim(missing - 1, missing - 1) + show_cut.Trim(missing + 2, missing + 2), mode="chroma", opacity=1)
# Pad videos
cut = cut.Loop(2, 0, 0)
show_cut = show_cut.Loop(2, 0, 0)
current_frame = missing + 3
} else {
if (log) {
WriteFileStart(full, logFile, "dropped", "sep2", "missing", append = true)
WriteFileStart(full, logFile, "missing", "sep", "avg_luma1", append = true)
}
# Append uninterrupted section of chroma video to result
# Overlay previous and succeeding chroma frames with luma frames and append
result = result + show_cut.Trim(start, end) + Overlay(show.Trim(missing, missing + 1), show_cut.Trim(missing - 1, missing - 1) + show_cut.Trim(missing + 1, missing + 1), mode="chroma", opacity=1)
# Pad videos
cut = cut.Loop(2, 0, 0)
show_cut = show_cut.Loop(2, 0, 0)
current_frame = missing + 2
}
# else single frame dropped silently
} else {
# Append uninterrupted section of chroma video to result
# Overlay previous chroma frame with luma frame and append
result = result + show_cut.Trim(start, end) + Overlay(show.Trim(missing, missing), show_cut.Trim(missing - 1, missing - 1), mode="chroma", opacity=1)
# Pad videos
cut = cut.Loop(2, 0, 0)
show_cut = show_cut.Loop(2, 0, 0)
if (log) {
WriteFileStart(full, logFile, "missing", append = true)
current_frame = missing + 1 # WriteFileStart sets current_frame to -1 !!!
} else {
current_frame = current_frame + 2
}
}
}
#return AudioDub(result, full)
return result
}
Use it like this:
chroma = AviSource("chroma_example.avi")
luma = AviSource("luma_example.avi")
#Match the luma signals
luma = ColorYUV(luma, off_y=-16, gain_y=-1, cont_y=0)
chroma = ColorYUV(chroma, off_y=-18, gain_y=0)
luma = Limiter(luma, 16, 235, 16, 240)
chroma = Limiter(chroma, 16, 218, 16, 240)
GImport("resync.avs")
ReSync(luma.ConvertToYV12(), chroma.ConvertToYV12(), "cutlog.txt")
#Uncomment this to see difference between frames and luma signals
#StackHorizontal(ScriptClip(Overlay(luma.Crop(0, 14, 0, -8), chroma.Crop(0, 14, 0, -8).Greyscale(), mode="Difference", opacity=1).ConvertToYV12(), "Subtitle(String(AverageLuma))"), Overlay(VideoScope(Greyscale(luma.Crop(0, 14, 0, -8)), "side", true, "Y").ConvertToYV12(), VideoScope(Greyscale(chroma.Crop(0, 14, 0, -8)), "side", true, "Y").ConvertToYV12(), mode="Difference", opacity=1).Crop(720, 0, 0, 0))
Will post sample videos when they have uploaded.
magikarp99
24th February 2014, 13:23
I'm sorry I don't think I'll have time to look at it, so it's up for grabs.
Thanks anyway.
I think I have got it almost perfect now. Sorted out both my memory and levels issues I believe. Just doing a final test. If it all works I will clean up my script and put it up here :)
MergeChroma is much more efficient than using Overlay. It has a small memory footprint and is much faster.
To sort out the levels issues I am using ColorLike on every frame.
vBulletin® v3.8.11, Copyright ©2000-2025, vBulletin Solutions Inc.