A script that automatically detect and replace broken frames?

[HappyLee]

Greetings, everyone. My first post here.

I have a video source that is full of broken frames, so I need to make a script to restore them automatically. And because I'm not good at writing avs script, I have to come and ask for a little help. Sorry for my poor English grammar.

The broken frames are nearly white (due to flashlights), and each broken frame comes alone (that is to say, no 2 broken frames are neighbours). So I'm guessing I can use those properties to replace them with the good frames right before them. Here's my plan:

1. Find a way to calculate the average brightness of every frames.
2. Compare every neighbouring 2 frames using the brightness information.
3. If frame (2) is greater than both frame (1) and frame (3) in brightness, and the difference is greater than a specified value, then it's been detected that frame (2) is a broken frame.
4. .DeleteFrame(that broken frame) and .DuplicateFrame(that broken frame-1)
5. Loop until every broken frames are replaced.

If anyone could help me with that script, I'll be much obliged. Thank you.

[StainlessS]

Try this (Requires RT_Stats, on first Avisynth Usage Page, few threads before this thread, at time of posting).


# FixBadWhiteFrame.avs
# https://forum.doom9.org/showthread.php?p=1623487#post1623487

AVISource("D:\avs\test.avi")

TestClip() # Make a TestClip with White Frame every 10 frames


Offset=DeleteFrame(Framecount-1).DuplicateFrame(0)

super = Offset.MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=2)
forward_vectors = MAnalyse(super, isb = false, delta=2)
inter = Offset.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)


LIMIT=200.0 # Average Luma above this is White Frame

CondS="""
ave=RT_AverageLuma()
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2)
clpn=(ave>LIMIT && alt < prv && alt < nxt) ? 1 : 0
# RT_Debug(String(current_frame)+"]","Ave="+String(ave),"Prv="+String(prv),"Nxt="+String(nxt),"Alt="+String(alt),"ClpN="+String(clpn))
clpn
"""

ConditionalSelect(Last,CondS,Last,inter) # Fix bad frames

return Last


Function TestClip(clip c) {
c
KillAudio()
WHT=Last.BlankClip(color=$FFFFFF)
A=SelectEvery(9,0)
B=SelectEvery(9,1)
C=SelectEvery(9,2)
D=SelectEvery(9,3)
E=SelectEvery(9,4)
F=SelectEvery(9,5)
G=SelectEvery(9,6)
H=SelectEvery(9,7)
I=SelectEvery(9,8)
Interleave(A,B,C,D,E,F,G,H,I,WHT)
Trim(0,999)
return last
}


Comment out the TestClip line, that just creates a testing clip.

Hope that makes you a happy chappy. :)

[johnmeyer]

I have two scripts that I use which you should be able to combine together to do what you ask. You say you are not an advanced programmer, but combining these two should be very straightforward. I think only one line will need to be changed in the second set of code below.

The first set of code is designed to find a "flash" frame. I use this all the time with video that I create from amateur 8mm and 16mm movies. The first frame of every scene that is filmed with an amateur camera is overexposed because the rotating shutter in the camera is still coming up to speed. To find these single, overexposed frames, all I do is compare the current frame to the frames immediately before and after. If the ratio of the averageluma is significantly greater than both of the adjacent frames, then this frame is a "flash frame."

Here is the code for finding a flash frame:

#Script to find "flash" frames

#Specify the name and location of the output file
filename = "m:\flash_frames.txt"
global flashthresh=1.15

AVISource("E:\fs.avi").killaudio()

global i=AssumeBFF
global p=selectevery(i, 1, -1) #Previous frame
global n=selectevery(i, 1, 1) #Next frame

# Temporarily un-comment next line to display the average luma value on screen to help determine threshold value
#ScriptClip(i,"""Subtitle(String(AverageLuma(i)/AverageLuma(p)) + " " + String(AverageLuma(i)/AverageLuma(n)))""" )

#This is the line that writes the frame number of any frame that falls above the threshold
WriteFileIf(last, filename, "(AverageLuma(i)/AverageLuma(p)>flashthresh)&&AverageLuma(i)/AverageLuma(n)>flashthresh", "current_frame", append = false)

The second function is one that replaces a frame with a motion-estimated frame, but only when a condition is met. In this case it is looking for a near-duplicate frame (or, if you set the comparison to 0.0 instead of 0.1, it will only find exact duplicates).

Here is that code:

function filldrops (clip c)
{
super=MSuper(c,pel=2)
vfe=manalyse(super,truemotion=true,isb=false,delta=1)
vbe=manalyse(super,truemotion=true,isb=true,delta=1)
filldrops = mflowinter(c,super,vbe,vfe,time=50)
fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.1")
return fixed
}

This function is for progressive video and is based on MugFunky's original idea from many years ago. If your video is interlaced, search these boards for my user name and the word "filldropsi()". That is a variation I created for interlaced video.

To combine these two together, I think all you need to do is take the code from the WriteFileIf function in the first code snippet, and put it into the ConditionalFilter line in the filldrops function. They both have similar syntax. So, what you will be doing is creating a "filldrops" function that, instead of substituting a sysnthesized ("estimated") video frame when a duplicate frame is detected, will instead substitute that synthesized frame when a frame is detected that doesn't match either the frame that precedes or the frame that follows.

One note: at first you might think this function would be activated at scene changes, but at a scene change the last frame of the earlier scene always matches the preceding frame, and the first frame of the new scene always matches the following frame, so the function doesn't get activated.

BTW, this same logic is useful for identifying photographer's flashes; for finding noise frames (from excessive dropouts, for instance) and any other situation that creates a "bad frame."

[edit] I took a long time creating this post, and didn't realize Stainless had posted essentially the same idea.

[StainlessS]

My previous post is for progressive, if your problem clip is Interlaced, then use John's script (EDIT: filldropsi) or say so here and I'll do an interlaced version.

[HappyLee]

Thank you both so much! You're really awesome! I hope one day I would be just like you, and I'm still learning.

[Boffee]

John's script did not work for me, received "only planar images as (YV12) supported!". Added the .ConvertToYV12() after killaudio(). This fixed the issue.

[StainlessS]

Here, slight mod of scripts posted elsewhere, replaces bad frames (single isolated frame eg black OR white)

Progressive, TweenFrames.avs


AVISource("D:\avs\test.avi")

# Pick a test below

#Return TestClip($FFFFFF) # White NOFIX
#Return TestClip($FFFFFF).TweenFrames(LimitLo=200.0,LimitHi=255.0,Show=true) # Fix White frames PROGRESSIVE

#Return TestClip($000000) # BLACK NOFIX
#Return TestClip($000000).TweenFrames(LimitLo=0.0,LimitHi=32.0,Show=true) # Fix Black frames PROGRESSIVE


Function TweenFrames(clip c,float "LimitLo",float "LimitHi",bool "show") {
# TweenFrames() by StainlessS
# PROGRESSIVE.
# Replace isolated bad frames eg Black or White with a frame tweened from those either side using MvTools, and RT_Stats.
# LimitLo: is minimum luma value of frame to replace. Allows select, eg black/White.
# LimitHi: is maximum luma value of frame to replace.
# Show: Puts indicator on fixed frames.
c
LimitLo=Float(Default(LimitLo,200.0)) # Average Luma greater or equal to this eligible for replacement
LimitHi=Float(Default(LimitHi,255.0)) # Average Luma lesser or equal to this eligible for replacement
show=Default(show,False)
Prev=DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("FRAME FIXED") : inter
CondS="""
ave=RT_AverageLuma()
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2)
clpn=(ave>=LimitLo && ave<=LimitHi && alt < prv && alt < nxt) ? 1 : 0
# RT_Debug(String(current_frame)+"]","Ave="+String(ave),"Prv="+String(prv),"Nxt="+String(nxt),"Alt="+String(alt),"ClpN="+String(clpn))
clpn
"""
CondS=RT_StrReplaceMulti(CondS,"LimitLo"+Chr(10)+"LimitHi",String(LimitLo)+Chr(10)+String(LimitHi))
ConditionalSelect(Last,CondS,Last,inter) # Fix bad frames
return Last
}


Function TestClip(clip c,int "Color") {
# Replace every 8th frame with blank.
c
Color=Default(Color,$FFFFFF)
BAD=Last.BlankClip(color=Color)
A=SelectEvery(8,0) B=SelectEvery(8,1) C=SelectEvery(8,2) D=SelectEvery(8,3)
E=SelectEvery(8,4) F=SelectEvery(8,5) G=SelectEvery(8,6)
Interleave(A,B,C,D,E,F,G,BAD) # every 8th frame replaced with BAD
Return Trim(0,1000)
}





Interlaced, TweenFields.avs

AVISource("D:\avs\test.avi")

# Pick a test below

#Return TestClipI($FFFFFF) # White NOFIX
#Return TestClipI($FFFFFF).TweenFields(LimitLo=200.0,LimitHi=255.0,Show=true) # Fix White frames INTERLACED

#Return TestClipI($000000) # BLACK NOFIX
#Return TestClipI($000000).TweenFields(LimitLo=0.0,LimitHi=32.0,Show=true) # Fix Black frames INTERLACED


Function TweenFields(clip c,float "LimitLo",float "LimitHi",bool "show") {
# TweenFields() by StainlessS
# INTERLACED.
# Replace isolated bad fields eg Black or White with a field tweened from those either side using MvTools, and RT_Stats.
# LimitLo: is minimum luma value of field to replace. Allows select, eg black/White.
# LimitHi: is maximum luma value of field to replace.
# Show: Puts indicator on fixed fields.
c
LimitLo=Float(Default(LimitLo,200.0)) # Average Luma greater or equal to this eligible for replacement
LimitHi=Float(Default(LimitHi,255.0)) # Average Luma lesser or equal to this eligible for replacement
show=Default(show,False)
SEP=SeparateFields()
Even=SEP.SelectEven()
Prev=Even.DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("EVEN FIELD FIXED") : inter
CondS="""
ave=RT_AverageLuma()
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2)
clpn=(ave>=LimitLo && ave<=LimitHi && alt < prv && alt < nxt) ? 1 : 0
# RT_Debug(String(current_frame)+"] EVEN","Ave="+String(ave),"Prv="+String(prv),"Nxt="+String(nxt),"Alt="+String(alt),"ClpN="+String(clpn))
clpn
"""
CondS=RT_StrReplaceMulti(CondS,"LimitLo"+Chr(10)+"LimitHi",String(LimitLo)+Chr(10)+String(LimitHi)) # Import explicit into script
EvenFixed=ConditionalSelect(Even,CondS,Even,inter) # Fix bad even fields
Odd=SEP.SelectOdd()
Prev=Odd.DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = Prev.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
inter = Prev.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
inter = (show) ? inter.Subtitle("ODD FIELD FIXED",align=9) : inter
CondS=RT_StrReplace(CondS,"EVEN","ODD")
OddFixed=ConditionalSelect(Odd,CondS,Odd,inter) # Fix bad Odd fields
Interleave(EvenFixed,OddFixed)
Weave()
return Last
}


Function TestClipI(clip c,int "Color") {
# Replace every 8th EVEN field and every 12th ODD field with a blank.
Color=Default(Color,$FFFFFF)
SEP=c.SeparateFields BAD=SEP.BlankClip(color=Color) EVEN=SEP.SelectEven()
A=EVEN.SelectEvery(8,0) B=EVEN.SelectEvery(8,1) C=EVEN.SelectEvery(8,2) D=EVEN.SelectEvery(8,3)
E=EVEN.SelectEvery(8,4) F=EVEN.SelectEvery(8,5) G=EVEN.SelectEvery(8,6) EVEN=Interleave(A,B,C,D,E,F,G,BAD)
ODD=SEP.SelectODD() A=ODD.SelectEvery(12,0) B=ODD.SelectEvery(12,1) C=ODD.SelectEvery(12,2)
D=ODD.SelectEvery(12,3) E=ODD.SelectEvery(12,4) F=ODD.SelectEvery(12,5) G=ODD.SelectEvery(12,6)
H=ODD.SelectEvery(12,7) I=ODD.SelectEvery(12,8) J=ODD.SelectEvery(12,9) K=ODD.SelectEvery(12,10)
ODD=Interleave(A,B,C,D,E,F,G,H,I,J,K,BAD) Interleave(EVEN,ODD) Weave()
Return Trim(0,1000)
}


Scripts included with RT_Stats.

[johnmeyer]

Stainless,

Thanks for that code. I have used the "filldrops()" code from MugFunky, and my updates to it, both to make it work with interlaced footage, and also to use MVTools2. However, having used your code for other projects, I'll bet this works better. I'll use it the next time I need to replace bad frames or duplicates (simple mod to make it work for that) and I'll report back what I find.

[StainlessS]

I'm guessin it works pretty good. (works great, with the only flash sample that I have).
It perhaps could be improved a little, but quite gud as is.

Will miss double frame flash'es (flash photography), but really quite resilient.
Those that take epileptic fits over flash photography may feel more comfortable, once processed.

Sample scripts as given able to provide I think much better control than original MugFunky (all hail) script,
and more easily modded.

EDIT: The flash photography sequence I used had already been temporally processed, and because of that,
may have resulted in a double flash frame (un-detected by above scripts). Not sure that, the double flash
would generally appear in real unprocessed clips, but no longer have the original clip.
EDIT: if need was there, scipts could be modified to also detect double and/or single flash frames at the expense of a little speed (methinks).

[Boffee]

I have a video made from an old 8mm cine film film where the first frame of each scene is over exposed. I have put this through John's script above using the "global flashthresh=1.15" and hey presto I have a text file of all the broken frames. So far so good. I then tried John's suggestion to amend the conditional filter in filldrops to eliminate these automatically, however Avisynth throws a syntax error. Can anyone tell me what the line fixed = ConditionalFilter....etc should be amended to. Thanks

[StainlessS]

Boffee, try this (Untested, no test clip)


# FixFlashAfterSceneCut.avs, https://forum.doom9.org/showthread.php?p=1669349#post1669349

Avisource("D:\avs\FLASHTEST.avi")

FlashThresh = 1.15
SETUP=True # Show f1 and f2 to setup FlashThresh
SHOWFIXED=True # No effect if SETUP == true, else show fixed frames if true

CondS="""
ave_p=Max(RT_AverageLuma(delta=-1), 0.01) # Avoid Division By Zero
ave_i=RT_AverageLuma()
ave_n=Max(RT_AverageLuma(delta=1), 0.01)
f1 = ave_i / ave_p
f2 = ave_i / ave_n
clpn = ((f1 > FlashThresh) && (f2 > FlashThresh)) ? 1 : 0
return clpn
"""

Next = DuplicateFrame(FrameCount-1).DeleteFrame(0)

Replace = (SHOWFIXED) ? Next.Subtitle("FRAME FIXED",Align=9,size=30) : Next

(SETUP)
\ ? Scriptclip("""
ave_p=Max(RT_AverageLuma(delta=-1), 0.01) # Avoid Division By Zero
ave_i=RT_AverageLuma()
ave_n=Max(RT_AverageLuma(delta=1), 0.01)
f1 = ave_i / ave_p
f2 = ave_i / ave_n
Subtitle(String(f1)+" "+String(f2)+((f1>FlashThresh&&f2>FlashThresh)?" FIXING":""),size=30)
""")
\ : ConditionalSelect(Last,CondS,Last,Replace)

Return Last

EDITED: Fixed division by zero.
EDIT: Fixed again alternative method

It replaces bad frames with following frame, no motion compensation.
(cannot tween with frame before scene change and MC not very good using two following frames although might be good in some cases).

Give it a try. (Based on mixture of previously given scripts in thread with Johns first clip for detection)

[Boffee]

Thanks very much StainlessS, this works brilliantly and does just what I want. Much appreciated.

[StainlessS]

You're welcome Boffee, however, be aware that there should be a guard again Division by Zero eg here


f1 = ave_i / RT_AverageLuma(delta=-1)


Ill fix in script a bit later, I have to leave in a moment or two.

EDIT: FIXED

[johnmeyer]

BTW, those "flash frames," each time the film camera starts, show up in most film transfers. They happen because the camera motor can't instantly get the rotating shutter up to full speed. As a result, the shutter speed on the first few frames is slower, resulting in over-exposure. The over-exposure on the first frame can sometimes be as much as three f-stops (or more), so the frame is useless.

However, my reason for posting is to point out that often the next 1-2 frames are also over-exposed, sometimes significantly. Therefore, the "ultimate" script for this application would detect the flash frame (that problem has been "solved" in this thread), but then look at the next 5-10 frames and make a determination as to how many additional frames after the initial flash frame should be eliminated.

In my existing film transfer workflow, I use my own script (which I will replace with the Stainless code) to output a list of flash frames. I then have a second script that operates inside of Vegas (my NLE) which cuts the video at each flash. I then use a keyboard shortcut to go to each flash frame and decide how many frames to cut. I could save myself a lot of time if the script gave me the option of deleting 1-3 of these subsequent over-exposed frames.

The only downside to what I am proposing (and please, Stainless, let me take a shot at this -- I am not asking for you to write additional code) is that sometimes those overexposed frames are important. For instance, I do film transfers from 1950s and 1960s American football games. These game films had the camera start just before each play. However, sometimes the camera operator forgot to start until the ball was hiked, so the play is already under way. In this case I only eliminate the flash frame (which usually has nothing but white), and then keep the additional frames.

So I definitely will use any modification which automatically cuts the flash frame because those are never useful, and I will work on a modification which provides a way to cut "n" number of additional frames if they are more than "m%" above the average exposure of frames 5-9 after the flash (I have never seen a camera take more than four frames to get up to speed).

[StainlessS]

let me take a shot at this

Arh, you like to spoil all my fun

Go for it John.

EDIT: I cannot guarentee that I will not leech off it, and perhaps mod. (Hopefully you will not mind too much if so)

[StainlessS]

Hi John, just something for you to consider,

The FrameSel: http://forum.doom9.org/showthread.php?t=167971&highlight=FrameSel

plugin allows you reject frames rather than select
so eg

FrameSel(cmd="Frames.txt",Reject=True)

Frames.txt

10 # reject frame 10
20,-1 # reject frame 20
30,-2 # reject 2 frames starting at frame 30
40,-3 # reject 3 frames starting at frame 40
50,53 # reject 4 frames, 50, 51, 52 and 53


perhaps of some use.

EDIT: Presumably no audio (it dont support audio, returns null audio)
EDIT: Can also use SPACE rather than COMMA as separator in frames.txt

[johnmeyer]

I'm in the midst, as I write this, of transferring eight small reels of film. I'll try out everything later today and let you know what I find.

Thanks!

[johnmeyer]

Well, this guy's dad had pro cameras: no flash frames on either the 16mm or 8mm film. If I get a chance, I'll try to find some older transfers and try out the scripts on that.

[johnmeyer]

I transferred about 1,000 feet of NFL football film last night and had a chance to try out the Stainless code. Unfortunately, the script doesn't work for a real-world situation. The reason is that flash frame are not overexposed compared to an absolute limit (i.e., above 200 luma) but instead are overexposed compared to their immediate neighbors. I think the mathematicians call it a "local maxima." Therefore, the detection must be done by comparing the test parameter to the averages on either side of each frame. Also, there are situations (fogging) where the luma value is moving up and down, but these frames should not be touched.

Here is a test clip where I have used VirtualDub (direct stream copy) to eliminate most of each scene, but left about half a dozen frames on each side of each flash. I've also included a small segment of fogged film, so you can so an example of luma variations that should not be rejected. Also note how the contrast and overall luma changes from one part of the film to the next. These short snippets were taken from several different reels of film, each having different exposure. Amateur film is always like this. You cannot guarantee any sort of exposure consistency.

Here is the sample clip (44 MB NTSC DV AVI file):

Flash Frame Example.avi (https://www.mediafire.com/?ub17hy1ajd8guue) (edit: changed from Dropbox to Mediafire)

Note: even though DV is always stored as interlaced, the actual content of this file is progressive, and it should be treated as progressive when doing any detection or modification.

Finally, once in awhile you do get a "big" flash frame that fits the description of what I think this script was designed to find and fix. I included this as the last "flash" in the test file.

I am going to spend a little more time on this and see if I can use the Stainless detection tools to do a better job than the script I have been using (posted a few weeks ago, earlier in this thread).

[StainlessS]

Thank you John, I'll have a play with your sample.

I saw a flv clip earler today (well late last night) that showed flash photography flashes extending up to 4 frames (30FPS),
where alternate frames started the flash at a particular vertical postion, and usually stopped about the same position
on the next frame (usually in paired Y postions). And also frames where flash area occupied only eg vertically central part
of a frame. Looks like these flashes can vary in characteristic somewhat.

PS, The Limits in previously given TweenFrames script are just to select between Black OR White frame detection, not to detect
the bad frames themselves (you could eg use LimitLo=128, LimitHi=255 for white and LimitLo=0, LimitHi=127 for black.

[johnmeyer]

P.S. One of the real challenges in the test clip I uploaded are frames 245-249 where the camera pans rapidly over a high contrast scene. In my old script, this produces a "flash frame detection" on each and every frame because the average luma value differences, for some reason, are very large when the camera is panning. Since these are supposed to be averages, I'm not sure I understand why that happens, but it does.

[johnmeyer]

Thank you John, I'll have a play with your sample.

I saw a flv clip earler today (well late last night) that showed flash photography flashes extending up to 4 frames (30FPS),
where alternate frames started the flash at a particular vertical postion, and usually stopped about the same position
on the next frame (usually in paired Y postions). And also frames where flash area occupied only eg vertically central part
of a frame. Looks like these flashes can vary in characteristic somewhat.Yes, there are a variety of different "flash frame" cases. The "pre-flash" used by some still cameras to reduce red eye is related, but is ultimately a different problem than the one presented here.

With movie film, most problems are shown in my test clip, but there is one more, namely the situation where the overexposure continues for more than one frame after the scene transition. This situation creates two problems. First, it makes the flash harder to detect, because if it takes 2-3 frames after the camera was re-started for the exposure to return to normal, then flash is no longer as isolated. To cope with this situation, more weight must be given to the exposure change from the last frames in previous scene than to the exposure change to the initial frames in the new scene.

Another problem is that the overall exposure from one scene to the next is not uniform at all. On every reel, you end up with all four possible permutations of:

last (goode exposure) frame from prev OVERexposed scene --> first (overexposed) frame in next OVERexposed scene
last (goode exposure) frame from prev OVERexposed scene --> first (overexposed) frame in next UNDERexposed scene
last (goode exposure) frame from prev UNDERexposed scene --> first (overexposed) frame in next OVERexposed scene
last (goode exposure) frame from prev UNDERexposed scene --> first (overexposed) frame in next UNDERexposed sceneThe flash detection must cope with each of these possible scenarios.

My script (which I've repeated below) is far too sensitive to motion (YDifference gets large when the camera pans quickly), and this creates false positives. I need a function that can "see" the obvious overall exposure differences that they eye can see, i.e., the "flash." The various lumadifference functions don't seem to do that. I've tried various blurs, but so far to no avail.

This is the script I am using, based on nothing more than native AVISynth calls:

#This script detects single bad frames

global blankthreshold = 25
filename = "e:\JHM_Scenes.txt"
source=AVISource("E:\Richards\Amateur NFL\Flash Frame Example.avi").convertTOYV12().colorYUV(autogain=true).killaudio().Crop(32,32,-32,-32)

#Uncomment the following lines, and comment out the WriteFileIf line in order to show the stats
#script = """Subtitle("\nprevious = " + String( YDifferenceFromPrevious(source)) + \
# "\nnext = " + String( YDifferenceToNext(source) ), lsp=0)"""
#final=Scriptclip(source, script)
#return final

WriteFileIf(source, filename, "(YDifferenceFromPrevious(source) > blankthreshold && YDifferenceToNext(source)> blankthreshold)", "current_frame", append = false)

It works, but it does miss some flashes, and it does signal for deletion many frames that are actually good. I still must edit each potential cut point manually, using the marker generated by this script as starting points.

[johnmeyer]

I'm not sure why taking the difference between averageluma values of adjacent frames returns a completely different value than YDifference (I am an idiot, and know nothing), but it does. Therefore, this script performs much, much better:

#This script detects single bad frames

global blankthreshold = 8
filename = "e:\JHM_Scenes.txt"
source=AVISource("E:\Richards\Amateur NFL\Flash Frame Example.avi").convertTOYV12().colorYUV(autogain=true).killaudio().Crop(32,32,-32,-32)

#Uncomment the following lines, and comment out the WriteFileIf line in order to show the stats
#script = """Subtitle("\nnextluma = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, 1)) ) + \
# "\nprevluma = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) ), lsp=0)"""

final=Scriptclip(source, script)
return final

WriteFileIf(source, filename, "(AverageLuma(source) - AverageLuma(selectevery(source, 1, 1)) > blankthreshold && \
AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) > blankthreshold)", "current_frame", append = false)


It still has problems when adjacent scenes have very different exposure levels (especially when the following scene is much darker). However, this works far better than what I posted before.

[Gavino]

I'm not sure why taking the difference between averageluma values of adjacent frames returns a completely different value than YDifference
YDifference gives the average (absolute) difference between pixels at the same position in the two frames, not the difference of the frame averages. So in a pan it will be relatively large, as every pixel will potentially have changed.

[johnmeyer]

YDifference gives the average (absolute) difference between pixels at the same position in the two frames, not the difference of the frame averages. So in a pan it will be relatively large, as every pixel will potentially have changed.Thanks for that information: I did not know that. Now that I do, I have several scripts I need to change, for the same reason I made the changes to this one.

[StainlessS]

Wilbert, big G's post above would be a very good addition to the YDifference() docs in Wiki.

EDIT:
John, I started downloading your sample last night and gave up when download seized up @ 15%,
resumed the download again just now, and still stuck @ 15% (dropbox).

Has anybody else successfully downloaded this sample ? (sample= ~42MB)

EDIT: John, I guess this has already occurred to you but if not,
Using FrameSel() with Extract = 3 or 5 or 7, will pull out all your suspect (with false alarms) flashes together with
surround frames, for your perusal, and setting Show=true will show original frame numbers.
I shall in next version see if I can give some additional indicator as to eg Target, or Pre1, Post1 where
extract is non 1.

EDIT: Perhaps you would find Extract = 9 or 11 also useful if implemented ?

[StainlessS]

John, Dont know if you saw this previous edit/ps

PS, The Limits in previously given TweenFrames script are just to select between Black OR White frame detection, not to detect
the bad frames themselves (you could eg use LimitLo=128, LimitHi=255 for white and LimitLo=0, LimitHi=127 for black.


Perhaps try this using base scripts already given eg post #11, need modify rest of script to suit,
not tested and will be a bit busy for some time.

Detect single frame flash photography without false detection when massive panning (Not at start of scene change or where camera start up)

CondS="""
ave_p=Max(RT_AverageLuma(delta=-1), 0.01) # Avoid Division By Zero
ave_i=RT_AverageLuma()
ave_n=Max(RT_AverageLuma(delta=1), 0.01)
f1 = ave_i / ave_p
f2 = ave_i / ave_n
Flashing = (f1 > FlashThresh) && f2 > FlashThresh)
prv=RT_YDifference(delta=-1)
nxt=RT_YDifference(delta=1)
alt=RT_YDifference(current_frame-1,delta=2) # difference between frames either side of current frame
NoPan = (alt < prv && alt < nxt)
clpn = (Flashing && NoPan) ? 1 : 0
return clpn
"""


Suggest that you use RT_Stats type funcs, at least until you get it working correctly, its a lot
easier than using built-ins for development work.

Download still stuck @ 15%

EDIT: Deleted download, re-started, got stuck @ 18%, not sure how impressed I am with Dropbox.

[Gavino]

Wilbert, big G's post above would be a very good addition to the YDifference() docs in Wiki.
Thanks, Mr S.
I think I expressed it better in this older post:
YDiff is not the difference between the average luma of the two frames. Instead, it is the average absolute luma difference between pixels at the same location. For each pixel position, the absolute luma difference is calculated; the results are then summed and divided by the total number of pixels to find the average.
This applies in a similar way to all the 'Difference' functions, eg LumaDifference(), etc.

[magikarp99]

It still has problems when adjacent scenes have very different exposure levels (especially when the following scene is much darker). However, this works far better than what I posted before.

I've been doing some similar work in synchronising two videos where one has dropped frames. I also had the same issue wherein the exposure of the scene would vary unpredictably. The solution I found to work well, was to use ColourLike on the two frames being compared to try and match the exposure of one to the other before then computing the difference between the two frames. This slowed the script down a bit, but proved incredibly effective, perhaps the same will work for you.

[StainlessS]

@ Big G, It was the summation that I liked best ie
So in a pan it will be relatively large, as every pixel will potentially have changed.

@magikarp99, Perhaps RT_LumaCorrelation() could be a possible alternative to ColorLike.

Here an update to post #7 scripts. Given different name as args changed.

TweenFlashFrames.avs

Function TweenFlashFrames(clip c,float "FlashThresh",bool "Show") {
# TweenFlashFrames() by StainlessS
# PROGRESSIVE.
# Replace single isolated bad frames eg Black or White with a frame tweened from those either side using MvTools, and RT_Stats.
# Will Likely fix a single Bright/Dark Flash frame after scene change where will be replaced with a blend of before and after frames.
# This due to NoPan failure, ie frames before and after flash are likely to be more similar (even though dif scenes) than to flash frame.
# There has been no attempt to fix this fortunate failure.
# FlashThresh Default 1.15 (white flash detect), is a threshold of AveLuma_Ratio, Flash_Frame_AveLuma / Adjacent_Frame_AveLuma.
# Values above 1.0 detect WHITE/Flash frames, Below 1.0 detect BLACK/Flash frames (suggest for Black eg 1.0/1.15 = 0.87)
# When FlashThresh above 1.0 (detecting WHITE flash) :
# If AveLuma_Ratio > FlashThresh, for both adjacent frames then is possible white flash
# When FlashThresh below 1.0 (detecting BLACK flash) :
# If AveLuma_Ratio < FlashThresh, for both adjacent frames then is possible black flash
# When FlashThresh < 0.0, will simultaneously fix both White and Black Flash frames, eg -1.15 will use equivalent to 1.15 for white
# and 1.0/1.15 for Black.
# 1.0 or -1.0 exactly, Throws an Error.
# Show: Puts indicator on fixed frames.
c
FlashThresh = Float(Default(FlashThresh,1.15)) # Default detects White flash
Show=Default(Show,False)
Assert(FlashThresh != 1.0 && FlashThresh != -1.0,"TweenFlashFrames: FlashThresh Cannot be 1.0 Nor -1.0")
(FlashThresh < 0.0 && FlashThresh > -1.0) ? 1.0 / FlashThresh : FlashThresh
CondS="""
ave_p = Max(RT_AverageLuma(delta=-1), 0.01) # Avoid Division By Zero
ave = RT_AverageLuma()
ave_n = Max(RT_AverageLuma(delta=1), 0.01)
rat_p = ave / ave_p # AveLuma_Ratio for Prev
rat_n = ave / ave_n # AveLuma_Ratio for Next
Flash =(FlashThresh<0.0)
\ ? ((rat_p > -FlashThresh && rat_n > -FlashThresh) || (rat_p < -1.0/FlashThresh && rat_n < -1.0/FlashThresh))
\ : (FlashThresh>1.0)
\ ? (rat_p > FlashThresh && rat_n > FlashThresh)
\ : (rat_p < FlashThresh && rat_n < FlashThresh)
dif_p=RT_YDifference(delta=-1) # Diff Prev <-> Curr
dif_n=RT_YDifference(delta=1) # Diff Curr <-> Next
dif_pn=RT_YDifference(current_frame-1,delta=2) # Diff Prev <-> Next (either side of current)
NoPan = (dif_pn < dif_p && dif_pn < dif_n) # (Prev<->Next < Prev<->Curr) AND (Prev<->Next < Curr<->Next)
clpn = (Flash && NoPan) ? 1 : 0
# RT_DebugF("%d ] Rat_p=%.2f Rat_n=%.2f Flash=%s Dif_pn=%.2f Dif_p=%.2f Dif_n=%.2f NoPan=%s Tween=%s",
# \ current_frame,rat_p,rat_n,(Flash)?"T":"F",Dif_pn,Dif_p,Dif_n,(NoPan)?"T":"F",(clpn==1)?"Y":"N")
clpn
"""
PrevC=DeleteFrame(FrameCount-1).DuplicateFrame(0) # Make clip where prev frames are aligned with curr frames (same length)
super = PrevC.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
TweenC = PrevC.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70) # Tweened clip
TweenC = (Show) ? TweenC.Subtitle("FRAME FIXED",size=30,text_color=$0000FF,align=5) : TweenC
CondS2=RT_StrReplace(CondS,"FlashThresh",String(FlashThresh)) # Import explicit FlashThresh into condition string
ConditionalSelect(Last,CondS2,Last,TweenC) # Fix bad frames
return Last
}


TweenFlashFrames_Client.avs

Import("TweenFlashFrames.avs")

AVISource("D:\avs\test.avi")

# Pick a simulated test below

#Return TweenFlashFrames_TestClip($FFFFFF) # White NOFIX
#Return TweenFlashFrames_TestClip($FFFFFF).TweenFlashFrames(FlashThresh=1.15,Show=true) # Fix White frames PROGRESSIVE

#Return TweenFlashFrames_TestClip($000000) # BLACK NOFIX
#Return TweenFlashFrames_TestClip($000000).TweenFlashFrames(FlashThresh=0.87,Show=true) # Fix Black frames PROGRESSIVE

Function TweenFlashFrames_TestClip(clip c,int "Color") {
# Replace every 8th frame with blank.
c
Color=Default(Color,$FFFFFF)
BAD=Last.BlankClip(color=Color)
A=SelectEvery(8,0) B=SelectEvery(8,1) C=SelectEvery(8,2) D=SelectEvery(8,3)
E=SelectEvery(8,4) F=SelectEvery(8,5) G=SelectEvery(8,6)
Interleave(A,B,C,D,E,F,G,BAD) # every 8th frame replaced with BAD
Return Trim(0,1000)
}

TweenFlashFields.avs

Function TweenFlashFields(clip c,float "FlashThresh",bool "Show") {
# Replace single isolated bad fields eg Black or White with a field tweened from those either side using MvTools, and RT_Stats.
# Will Likely fix a single Bright/Dark Flash field after scene change where will be replaced with a blend of before and after fields.
# This due to NoPan failure, ie fields before and after flash are likely to be more similar (even though dif scenes) than to flash field.
# There has been no attempt to fix this fortunate failure.
# FlashThresh Default 1.15 (white flash detect), is a threshold of AveLuma_Ratio, Flash_Field_AveLuma / Adjacent_Field_AveLuma.
# Values above 1.0 detect WHITE/Flash fields, Below 1.0 detect BLACK/Flash fields (suggest for Black eg 1.0/1.15 = 0.87)
# When FlashThresh above 1.0 (detecting WHITE flash) :
# If AveLuma_Ratio > FlashThresh, for both adjacent fields then is possible white flash
# When FlashThresh below 1.0 (detecting BLACK flash) :
# If AveLuma_Ratio < FlashThresh, for both adjacent fields then is possible black flash
# When FlashThresh < 0.0, will simultaneously fix both White and Black Flash frames, eg -1.15 will use equivalent to 1.15 for white
# and 1.0/1.15 for Black.
# 1.0 or -1.0 exactly, Throws an Error.
# Show: Puts indicator on fixed fields.
c
FlashThresh = Float(Default(FlashThresh,1.15)) # Default detects White flash
Show=Default(Show,False)
Assert(FlashThresh != 1.0 && FlashThresh != -1.0,"TweenFlashFrames: FlashThresh Cannot be 1.0 Nor -1.0")
(FlashThresh < 0.0 && FlashThresh > -1.0) ? 1.0 / FlashThresh : FlashThresh
CondS="""
ave_p = Max(RT_AverageLuma(delta=-1), 0.01) # Avoid Division By Zero
ave = RT_AverageLuma()
ave_n = Max(RT_AverageLuma(delta=1), 0.01)
rat_p = ave / ave_p # AveLuma_Ratio for Prev
rat_n = ave / ave_n # AveLuma_Ratio for Next
Flash =(FlashThresh<0.0)
\ ? ((rat_p > -FlashThresh && rat_n > -FlashThresh) || (rat_p < -1.0/FlashThresh && rat_n < -1.0/FlashThresh))
\ : (FlashThresh>1.0)
\ ? (rat_p > FlashThresh && rat_n > FlashThresh)
\ : (rat_p < FlashThresh && rat_n < FlashThresh)
dif_p=RT_YDifference(delta=-1) # Diff Prev <-> Curr
dif_n=RT_YDifference(delta=1) # Diff Curr <-> Next
dif_pn=RT_YDifference(current_frame-1,delta=2) # Diff Prev <-> Next (either side of current)
NoPan = (dif_pn < dif_p && dif_pn < dif_n) # (Prev<->Next < Prev<->Curr) AND (Prev<->Next < Curr<->Next)
clpn = (Flash && NoPan) ? 1 : 0
# RT_DebugF("%d ] EVEN Rat_p=%.2f Rat_n=%.2f Flash=%s Dif_pn=%.2f Dif_p=%.2f Dif_n=%.2f NoPan=%s Tween=%s",
# \ current_frame,rat_p,rat_n,(Flash)?"T":"F",Dif_pn,Dif_p,Dif_n,(NoPan)?"T":"F",(clpn==1)?"Y":"N")
clpn
"""
SepC=SeparateFields()
EvenC=SEPC.SelectEven()
PrevC=EvenC.DeleteFrame(FrameCount-1).DuplicateFrame(0) # Make clip where prev fields are aligned with curr fields (same length)
super = PrevC.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
TweenC = PrevC.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70) # Tweened clip
TweenC = (show) ? TweenC.Subtitle("EVEN FIELD FIXED",size=24,text_color=$0000FF,align=5,y=EvenC.Height/2-24) : TweenC
CondSE=RT_StrReplace(CondS,"FlashThresh",String(FlashThresh)) # Import explicit FlashThresh into condition string
EvenFixedC=ConditionalSelect(EvenC,CondSE,EvenC,TweenC) # Fix bad EVEN fields
OddC=SepC.SelectOdd()
PrevC=OddC.DeleteFrame(FrameCount-1).DuplicateFrame(0)
super = PrevC.MSuper()
backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2)
TweenC = PrevC.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70) # Tweened clip
TweenC = (show) ? TweenC.Subtitle("ODD FIELD FIXED",size=24,text_color=$0000FF,align=5,y=OddC.Height/2) : TweenC
CondSO=RT_StrReplace(CondSE,"EVEN","ODD")
OddFixedC=ConditionalSelect(OddC,CondSO,OddC,TweenC) # Fix bad Odd fields
Interleave(EvenFixedC,OddFixedC)
Weave()
return Last
}


TweenFlashFields_Client.avs

Import("TweenFlashFields.avs")

AVISource("D:\avs\test.avi")

# Pick a Simulated test below

#Return TweenFlashFields_TEST($FFFFFF) # White NOFIX
#Return TweenFlashFields_TEST($FFFFFF).TweenFlashFields(FlashThresh=1.15,Show=True) # Fix White fields INTERLACED

#Return TweenFlashFields_TEST($000000) # BLACK NOFIX
#Return TweenFlashFields_TEST($000000).TweenFlashFields(FlashThresh=0.87,Show=True) # Fix Black fields INTERLACED

Function TweenFlashFields_TEST(clip c,int "Color") {
# Replace every 8th EVEN field and every 12th ODD field with a blank.
Color=Default(Color,$FFFFFF)
SEP=c.SeparateFields BAD=SEP.BlankClip(color=Color) EVEN=SEP.SelectEven()
A=EVEN.SelectEvery(8,0) B=EVEN.SelectEvery(8,1) C=EVEN.SelectEvery(8,2) D=EVEN.SelectEvery(8,3)
E=EVEN.SelectEvery(8,4) F=EVEN.SelectEvery(8,5) G=EVEN.SelectEvery(8,6) EVEN=Interleave(A,B,C,D,E,F,G,BAD)
ODD=SEP.SelectODD() A=ODD.SelectEvery(12,0) B=ODD.SelectEvery(12,1) C=ODD.SelectEvery(12,2)
D=ODD.SelectEvery(12,3) E=ODD.SelectEvery(12,4) F=ODD.SelectEvery(12,5) G=ODD.SelectEvery(12,6)
H=ODD.SelectEvery(12,7) I=ODD.SelectEvery(12,8) J=ODD.SelectEvery(12,9) K=ODD.SelectEvery(12,10)
ODD=Interleave(A,B,C,D,E,F,G,H,I,J,K,BAD) Interleave(EVEN,ODD) Weave()
Return Trim(0,1000)
}

EDITED: Added simultaneouly fix White and Black flashes when -ve FlashThresh.

Work pretty good.

EDIT: Download still stuck @ 18%.

[johnmeyer]

I've been doing some similar work in synchronising two videos where one has dropped frames. I also had the same issue wherein the exposure of the scene would vary unpredictably. The solution I found to work well, was to use ColourLike on the two frames being compared to try and match the exposure of one to the other before then computing the difference between the two frames. This slowed the script down a bit, but proved incredibly effective, perhaps the same will work for you.The revised script I posted a few days ago that uses the difference in Averageluma between the current frame and adjacent frames works perfectly, so I am no longer looking for any improvements. However, I am curious, what is "Colorlike?" I hadn't heard about that before.

Stainless: Please don't spend any more time trying to figure out how to do any automatic cutting. The reason I say that is that in the real world, there are simply too many strange things the camera operator does just before and just after pushing the on/off button. This is true with current video cameras, but was even more of an issue with movie cameras which usually had a very strong spring-loaded button that you had to push to start the camera; hold while filming; and then release to stop the camera. The pushing and, later, the releasing of this button often caused the camera to jump up or down, and therefore the first and last frames (just before and after the flash frame) often have wild camera movement that also must be removed.

If I were doing thousands of reels of film instead of dozens, I would try to use motion estimation or some other method to determine wild camera movement just before and just after a flash frame. I seem to remember some script that was able to detect the blur from motion.

However, in the end, to get good results, you just have to do some manual work. On my current project, I had 600+ flash frames (four complete NFL football games), and this took about ninety minutes of editing to fix, once the script identified all the flash frames and put markers on the video editing timeline (Vegas) so I could instantly go to each one.

BTW, Gavino's insight into how YDifference works helps explain something that never made sense to me, namely how it can be used as a quick and dirty scene detection method. I thought YDifference just took the difference between the average luma value for the entire frame, and I couldn't see how that would change much between most scenes.

It all now makes perfect sense. Thanks again, Gavino.

[magikarp99]

The revised script I posted a few days ago that uses the difference in Averageluma between the current frame and adjacent frames works perfectly, so I am no longer looking for any improvements. However, I am curious, what is "Colorlike?" I hadn't heard about that before.

http://forum.doom9.org/showthread.php?t=96308

It is used to match the colours of one video to another. One use case being colour matching an analogue capture to an existing digital release.

[StainlessS]

IIRC, mg262, author of ColorLike said something like, "I am not very impressed with how ColorLike performs".

EDIT: 3rd attempt to download sample from Dropbox seized up @ 16%.

[johnmeyer]

3rd attempt to download sample from Dropbox seized up @ 16%.OK, I've uploaded the same file to MediaFire. Try this:

Flash Frame Example.avi (https://www.mediafire.com/?ub17hy1ajd8guue)

[StainlessS]

Thank you John, as always, no problems with MediaFire.

Modified post #30 scripts to simultaneously fix white and black flash frames/fields.

[johnmeyer]

I just discovered an obvious error in my script: when the film has several flash frames in a row, the script fails because the logic is only looking for the "hump on the camel," i.e., a single-frame local maxima. I am too lazy to correct this the right way (comparing the current frame to averages of multiple adjacent frames), so I simply added a second threshold and an OR logic operator so that the script will output a marker whenever the frame is almost completely white. This will output multiple frame numbers when there are multiple flash frames in a row, although I can easily decimate those, if needed. It may also output false positives during the bright frames you get at the beginning and end of some film due to fogging (when the film was loaded or unloaded in too-bright lighting conditions).

Here's a really tiny (4 MB) AVI file that shows not only this problem, but demonstrates the camera movement just prior to the end of the scene.

Multiple flash example.avi (https://www.mediafire.com/?tn0881mf19u689q)

It also shows the exposure issue (which I haven't mentioned until now) that happens when the camera capturing the film reacts to the flash and closes the aperture. This results in the first few frames after the flash being underexposed. There are various solutions during the capture that could reduce this, but they all involve manual exposure control which means I have to sit by the projector for hours at a time, something I'm not going to do.

At the moment, I am not correcting this under-exposure.

Here's the "improved" version of the script:

#This script detects single bad frames
#John Meyer
#March 1, 2014

global blankthreshold = 7
global flash_brightness = 220
filename = "e:\JHM_Scenes.txt"
source=AVISource("e:\fs.avi").convertTOYV12().colorYUV(autogain=true).killaudio().Crop(64,64,-64,-64)

#Uncomment the following lines, and comment out the WriteFileIf line in order to show the stats
#script = """Subtitle("\nnext_luma = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, 1)) ) + \
# "\nprev_luma = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) ) + \
# "\naverage_luma = " + String( AverageLuma(source) ), lsp=0)"""
#final=Scriptclip(source, script)
#return final

WriteFileIf(source, filename, "( (AverageLuma(source) - AverageLuma(selectevery(source, 1, 1)) > blankthreshold \
&& AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) > blankthreshold) ) \
|| AverageLuma(source) > flash_brightness", "current_frame", append = false)

[johnmeyer]

At the risk of beating a dead horse, here is a later "flash fame detection" script which will detect up to three flash frames in a row. This gets rid of the quick and dirty patch shown in my last post directly above, where I used an absolute AverageLuma as a catchall for multiple bright frames that might be missed. While this will certainly catch frames that are almost pure white, it will not catch multiple frames which are simply overexposed. The code below instead looks at the next three frames and, if there has been a big jump in luma from the last frame, then if there is a significant drop in luma in any of the next three frames, the current frame number is written to a file.

I might get slightly more consistent results by using ratios of AverageLuma values instead of differences, but this seems to work better than anything I've done so far.

The one remaining issue -- and perhaps one of StainlessS' functions might help -- is that rapid camera pans can still change the averageluma sufficiently to cause my code to sense an overall exposure change.

Why does this happen?

I think it is caused by something like this: Imagine five white circles against a black background: if the camera pans and there are now only four white circles in the next frame, this will cause the average luma value to drop. However, looking at the two images, there is no change in "exposure." I'm not sure if the solution to this is to use something like a standard deviation statistic, or if some other measurement needs to be used. I'm thinking of something like RT_YPlaneMedian or RT_YPlaneStdev.

I will experiment as time allows.
#This script detects flash (too bright) frames.
#It will detect up to three flash frames in a row.
#
#John Meyer © March 2, 2014

global blankthreshold = 13
filename = "e:\Flashes.txt"
source=AVISource("e:\fs.avi").convertTOYV12().killaudio().Crop(64,64,-64,-64)

#Uncomment the following lines, and comment out the WriteFileIf lines in order to show the stats (for setting "blankthreshold")
#script = """Subtitle("\nprev_luma = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) ) + \
# "\n" + \
# "\nnext_luma = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, 1)) ) + \
# "\nnext_luma2 = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, 2)) ) + \
# "\nnext_luma3 = " + String( AverageLuma(source) - AverageLuma(selectevery(source, 1, 3)) ) + \
# "\n" + \
# "\navg_luma = " + String( AverageLuma(source) ), lsp=0)"""
#final=Scriptclip(source, script)
#return final

#This will detect up to three flash frames in a row, and output frame number for first flash
WriteFileIf(source, filename, " \
(AverageLuma(source) - AverageLuma(selectevery(source, 1, 1)) > blankthreshold \
&& AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) > blankthreshold) \
|| \
(AverageLuma(source) - AverageLuma(selectevery(source, 1, 2)) > blankthreshold \
&& AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) > blankthreshold) \
|| \
(AverageLuma(source) - AverageLuma(selectevery(source, 1, 3)) > blankthreshold \
&& AverageLuma(source) - AverageLuma(selectevery(source, 1, -1)) > blankthreshold) \
", "current_frame", append = false)

[StainlessS]

John, you have comparison with previous frame in each of the three AND conditions,
compare with previous frame first and ONLY if that succeeds (use AND) compare with each of the following frames
separated by OR conditions.

something like this

ave = AverageLuma(source)
((ave - AverageLuma(selectevery(source, 1, -1)) > blankthreshold) && \ # Primary condition, if fails then no further tests necessary
( ave - AverageLuma(selectevery(source, 1, 1)) > blankthreshold || \
ave - AverageLuma(selectevery(source, 1, 2)) > blankthreshold || \
ave - AverageLuma(selectevery(source, 1, 3)) > blankthreshold))


just a bit faster

[johnmeyer]

John, you have comparison with previous frame in each of the three AND conditions,
compare with previous frame first and ONLY if that succeeds (use AND) compare with each of the following frames
separated by OR conditions.That is very useful because I just replaced Averageluma with RT_YPlaneMedian, and the script is now somewhat slower, so I was looking for ways to improve performance.

Each iteration does seem to reduce the number of missed flash frames, and decrease the number of false positives.

The following newer version uses your RT_YPlaneMedian function and I've tested it on ninety minutes of film transfers. It works better than any of the previous versions I posted.

I'll definitely take your suggestion and change the code below to stop further comparisons if the initial comparison (with the previous frame) fails.
#This script detects flash (too bright) frames.
#It will detect up to three flash frames in a row.
#
#John Meyer © March 2, 2014

global flashthreshold = 5
filename = "e:\Flashes.txt"
source=AVISource("e:\fs.avi").convertTOYV12().killaudio().Crop(64,64,-64,-64)

#Uncomment the following lines, and comment out the WriteFileIf lines in order to show the stats (for setting "flashthreshold")
#script = """Subtitle("\nprev_luma = " + String( RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) ) + \
# "\n" + \
# "\nnext_luma = " + String( RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 1)) ) + \
# "\nnext_luma2 = " + String( RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 2)) ) + \
# "\nnext_luma3 = " + String( RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 3)) ) + \
# "\n" + \
# "\nRT_Median = " + String( RT_YPlaneMedian(source) ), lsp=0)"""
#final=Scriptclip(source, script)
#return final

#This will detect up to three flash frames in a row, and output frame number for first flash
WriteFileIf(source, filename, " \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 1)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 2)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 3)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
", "current_frame", append = false)

[johnmeyer]

ave = AverageLuma(source)
((ave - AverageLuma(selectevery(source, 1, -1)) > blankthreshold) && \ # Primary condition, if fails then no further tests necessary
( ave - AverageLuma(selectevery(source, 1, 1)) > blankthreshold || \
ave - AverageLuma(selectevery(source, 1, 2)) > blankthreshold || \
ave - AverageLuma(selectevery(source, 1, 3)) > blankthreshold))Actually, I'm not sure I see how this code is going to stop executing if the primary condition is not true, although I'll admit that I am often surprise and puzzled by how AVISynth's runtime works. I assume that after the assignment in the first line, that the remaining code is designed to be put inside a conditional. Are you saying that AVISynth is smart enough to recognize that if the first element in an AND statement is false that no further elements need be evaluated?

[Gavino]

Are you saying that AVISynth is smart enough to recognize that if the first element in an AND statement is false that no further elements need be evaluated?
That's correct.
Similarly, if the first element in an OR is true, no further elements need to be evaluated.
This known as short-circuit evaluation and is a feature of many programming languages.
http://en.wikipedia.org/wiki/Short-circuit_evaluation

[johnmeyer]

That's correct.
Similarly, if the first element in an OR is true, no further elements need to be evaluated.
This known as short-circuit evaluation and is a feature of many programming languages.
http://en.wikipedia.org/wiki/Short-circuit_evaluationMy first programming used paper tape or a stack of cards as input. Most of it after that was in assembly language, controlling machinery, and I had to count clock cycles and know exactly how many milliseconds each operation would take. Therefore, I really love knowing these things. However, I guess I never got used to all of these modern programming conveniences where the interpreter or compiler is smart enough to optimize the code. Pretty nifty. Thanks for the useful education.

[johnmeyer]

Well, I tried the suggested improvements, but had a few problems, and the flash frame detection actually works differently.

The first problem is that I couldn't see how to do an assignment inside the conditional, and the runtime functions can't be used outside a conditional. So the line:

ave = AverageLuma(source)

doesn't seem to be possible to do. Perhaps I need to use some of Gavino's runtime stuff.

So, I didn't use that improvement and instead re-computed the luma calculation each time (I'm actually using RT_YPlaneMedian instead).

However, I ran into another more subtle problem: the proposed logic actually outputs a frame number for every flash frame, whereas my original code, by design, only outputs the first flash frame number if there are several in a row. This actually makes editing much easier because I don't have to spend time skipping past multiple markers on the timeline (these markers are generated from the frame numbers created by the script).

Finally, the speed improvement was undetectable: my code achieved 121 fps, and the alternate code achieved 122 fps. I'm actually not even sure that there is a difference.

So, unless there is a real speed improvement to be had, it probably isn't worth spending more time on this. However, just to be complete, the first block below shows my original code that works and gives me about 121 fps; the second block shows the original suggested code; and the third block shows my attempt at using that suggested code (with the change to RT_YPlaneMedian) inside my original conditional. This code does work, but as already noted it creates a frame number for every flash frame in a series.

#My original code
WriteFileIf(source, filename, " \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 1)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 2)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 3)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
", "current_frame", append = false)

#Suggested code
#ave = AverageLuma(source)
#((ave - AverageLuma(selectevery(source, 1, -1)) > blankthreshold) && \ # Primary condition, if fails then no further tests necessary
#( ave - AverageLuma(selectevery(source, 1, 1)) > blankthreshold || \
# ave - AverageLuma(selectevery(source, 1, 2)) > blankthreshold || \
# ave - AverageLuma(selectevery(source, 1, 3)) > blankthreshold))

#My adaptation of suggested code
#WriteFileIf(source, filename, " \
# (RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) && \
# RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 1)) > flashthreshold || \
# RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 2)) > flashthreshold || \
# RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 3)) > flashthreshold \
# ", "current_frame", append = false)

[StainlessS]

this is how I would do it

source = Avisource("FlashFrameExample.avi")
flashthreshold=25
filename="FileName.txt"

CondS = """
Med=RT_YPlaneMedian()
wr= Med-RT_YPlaneMedian(delta=-1)>flashthreshold && \
( \
Med-RT_YPlaneMedian(delta=1)>flashthreshold || \
Med-RT_YPlaneMedian(delta=2)>flashthreshold || \
Med-RT_YPlaneMedian(delta=3)>flashthreshold \
)
# RT_DebugF("%d ] WR=%s",current_frame,String(wr))
return wr
"""

WriteFileIf(source, filename, CondS, "current_frame", append = false)


Notes,
Use Last inside condition string, source was imported via WriteFileIf arg.
Might as well use RT_ delta arg rather than creating extra clips for Prev and Next etc.
Any error in condition string code will return TRUE and output all frames. (where RT_DebugF produces no output if enabled)
The 3 logical OR conditions need to be wrapped in their own parenthesis (all 3 group wrapped, EDIT: Marked in BLUE)
Logical AND && operator has higher precedence than logical OR ||.

EDIT: RT_YPlaneMedian is slower than RT_AverageLuma (RT_AverageLuma when processed alone, is optimized,
RT_YPlaneMedian requires counting of all pixels into count array, before deciding the median).

EDIT: If you wanted to use both RT_YPlaneMedian and RT_AverageLuma (and perhaps other RT_Y funcs) then
use RT_YStats to calc them (for same frame) simultaneously.

EDIT: 3rd code block in post #43 need parenthesis surrounding the 3 OR conditions, (same as marked in blue above).
The comment in 'suggested code' starting 'Primary condition' would cause error, as it occurs after line continuation char,
it was an afterthought to explain logic and should not have been included at that point in script.

[Gavino]

The first problem is that I couldn't see how to do an assignment inside the conditional, and the runtime functions can't be used outside a conditional. So the line:

ave = AverageLuma(source)

doesn't seem to be possible to do.
The 'condition' expression supplied to WriteFileIf is just an arbitrary script fragment that returns a boolean value. So (just like a 'ScriptClip' run-time script) it can contain multiple statements, including assignments, as StainlessS demonstrates in his latest code.

[johnmeyer]

I tried the StainlessS code on an entire reel of film and there were a few minor instances where it produced different frame numbers. I'm not sure why. It was about half a percent slower than my brute force code. It certainly is cleaner code.

I did try using RT_AverageLuma, and it is about half a percent faster (about 124 fps instead of 122 fps). However, the RT_YPlaneMedian function works much more reliably (i.e., fewer false positives).

I'm not going to try optimizing this any further. If I were to do any further work, it would be to try to enhance the code to find one type of "flash frame" that isn't detected by this approach, namely a flash that happens at the beginning of a scene that is much darker than the preceding scene. Remember, most film flash frames are not completely white, but instead are simply overexposed. In other words, you can still sometimes see an image. Some of the flash frames are only 1-2 f-stops overexposed. However, when the next scene is 1-2 f-stops below the average exposure of the current scene, then the flash may actually be darker than the frame which preceded it. To detect this, I would probably need to use an actual scene detection function, followed by flash detection logic which only looks forward from that point. Since this situation only happens rarely, and by definition isn't as visually distracting, I think I will just live with the script as is.

Thanks to everyone for all the help, and I hope this discussion also helped the OP.

[johnmeyer]

The 'condition' expression supplied to WriteFileIf is just an arbitrary script fragment that returns a boolean value. Got it. Thanks!

[StainlessS]

there were a few minor instances where it produced different frame numbers.

See last edit of my previous post.

EDIT: Arh, you mean this code (by Brute force code)


#My original code
WriteFileIf(source, filename, " \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 1)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 2)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 3)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
", "current_frame", append = false)


Cannot explain why you should get different frame numbers output, should be same, I think.

[johnmeyer]

Cannot explain why you should get different frame numbers output, should be same, I think.I can't explain it, but I did the test twice, using different input, just to make sure. I was going to try to reduce it down to a single case that I could illustrate with a really short test clip, but ran out of energy because everything works fine, and I've only got so many hours in the day ...

[StainlessS]

This might assist in locating problem clip frame numbers


source = Avisource("FlashFrameExample.avi")
flashthreshold=25
filename="FileName.txt"

CondS = """
Med=RT_YPlaneMedian(Last)
wr= Med-RT_YPlaneMedian(Last,delta=-1)>flashthreshold && \
( \
Med-RT_YPlaneMedian(Last,delta=1)>flashthreshold || \
Med-RT_YPlaneMedian(Last,delta=2)>flashthreshold || \
Med-RT_YPlaneMedian(Last,delta=3)>flashthreshold \
)

wr2= \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 1)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 2)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold) \
|| \
(RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, 3)) > flashthreshold \
&& RT_YPlaneMedian(source) - RT_YPlaneMedian(selectevery(source, 1, -1)) > flashthreshold)

(wr != wr2) ? RT_DebugF("%d ] WR=%s WR2=%d",current_frame,String(wr),String(wr2)) : NOP

return wr
"""

WriteFileIf(source, filename, CondS, "current_frame", append = false)


I only tried on your FlashFrameExample.avi, no difference detected.