Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
13th December 2016, 04:51 | #1 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
Finding individual "bad" frames in video; save frame number; or repair
[edit]Here is a link to a better version of the script, later in this thread: Better Version of Script [/end edit]
Long ago I created a script that I used to find individual blank frames. I needed this because my NLE (Vegas) has a nasty habit of creating random blank frames during certain types of renders, and you don't know if you have the problem without a tool which can look at every single frame. I then found I needed a script to find "flash" frames because I do movie film transfers, and because of the way home movie cameras work, the first frame of every single scene is horribly overexposed. These therefore need to be found and removed. As time went on, I found all sorts of other situations where a video can contain a single frame which does not match either adjacent frame. I recently had a request for this script, so I went back and cleaned it up a bit and added some comments. The main tool used for detection is the YDifference function built into AVISynth. It compares each pixel in the current frame to each corresponding pixel in the adjacent frame. The metric "blows up" whenever there is a big difference in lots of the pixels, something that happens at scene changes, but also when there is a single-frame corruption. What's more, the metrics blow up looking both backwards and forwards when there is an individual bad frame, whereas they only blow up in one direction at a scene change. Therefore, this function will not do anything at scene changes (which is what you want). Other types of detection could easily be substituted, using either the other stat functions built into AVISynth, or by using the myriad of compare functions built into StainlessS' excellent RT_Stats package. I look forward to any ideas for improving this. I couldn't figure out how to do the conditional without declaring one variable as global (Gavino always dings me on this). Also, I wasn't able to get it to both automatically fix bad frames AND output the bad frame numbers to a text file. I could get it to do one or the other, but not both. For me, this isn't a big deal, but some people might want to be able to easily go to the fixed video and check each and every new frame to make sure it looks OK. One other thing: I wrote this so it works on interlaced video. Since progressive is a special case of interlaced (with no temporal differences between fields), it will work just fine with progressive as well. Code:
#Find And (Optionally) Fix Bad Frames #John Meyer - December 12, 2016 #This script detects single bad frames. You have two options of what to do. #You can configure the script to write, to a file, the frame numbers of all frames which are detected as "bad". #As an alternative, you can configure it to automatically replace bad frames with a new #frame that is interpolated from its neighbors. This replacement is often near-perfect (no guarantees, however...) #This script will fail if the bad frame happens immediately before or after a scene change. #This script will also fail to find a bad frame if there is more than one bad frame in a row. #It works very well for finding both blank frames and also "flash" frames (like those caused #by a photographer's flash). It will also find single frames which have lots of #static or pixels. It can also find a frame with large x or y displacement from adjacent frames, like #a film frame that wasn't properly registered in the film gate, or an analog #video frame that lost vertical sync. #When using VirtualDub, to create the text file containing the bad frame numbers, first uncomment that code block. #Then, select "Run Video Analysis Pass" in the VirtualDub File menu. #The script uses ratios of the metrics for the current frame to the same metrics #on the two adjacent frames. Under normal circumstances, the metrics should be quite #similar, and therefore the ratio should be very near to unity (i.e., 1.00). #Run through the video with the "script" variable enabled, and look at the metrics #in order to determine an optimum threshold value. A larger threshhold will #catch fewer bad frames, and a lower threshold will eventually create false positives. #The replacement code works well enough that if you end up replacing a few frames that are #actually good, you probably won't notice it. #You need to un-comment the WriteFileIf lines to actually write the frame numbers to a file. #You need to un-comment the script lines to cause the metrics to appear on screen in order to #determine the optimum badthreshold value (2 is a good starting point, however). #You need to un-comment the ReplaceBadI lines to automatically replace bad frames with interpolated frames. #I recommend only having one of these three code blocks enabled at any one time. #----------------------------- loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll") global badthreshold = 2 showdot = false # set to true to add "***" to each replacment frame (for troubleshooting) filename = "e:\Bad.txt" source=AVISource("e:\fs.avi").convertTOYV12().killaudio() #TEMPORARILY remove comments from the following block in order to show the metrics. /* script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) \ / YDifferenceFromPrevious( selectevery(source, 1, -1) )) + \ "\nNext Ratio = " + String( YDifferenceToNext(source) / YDifferenceToNext(selectevery(source, 1, 1) )), lsp=0)""" final=Scriptclip(source, script) return final */ #Uncomment the code in the next two lines to create a file which contains the frame numbers of all bad frames /* WriteFileIf(source, filename, "YDifferenceFromPrevious(source) / YDifferenceFromPrevious( selectevery(source, 1, -1) ) > badthreshold && \ YDifferenceToNext(source) / YDifferenceToNext(selectevery(source, 1, 1) )>badthreshold", "current_frame", append = false) */ #Add comments to the two lines of code below to stop replacement of #each bad frame with a one that is interpolated from adjacent frames. #The "I" in the function name stands or Interlaced, because this will #work with interlaced video (as well as progressive) output=ReplaceBadI(source,showdot) return output #------------------------------ function ReplaceBadI (clip c, bool showdot) { even = c.SeparateFields().SelectEven() super_even = showdot ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2) vfe=manalyse(super_even,truemotion=true,isb=false,delta=2) vbe=manalyse(super_even,truemotion=true,isb=true,delta=2) filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50) odd = c.SeparateFields().SelectOdd() super_odd = showdot ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2) vfo=manalyse(super_odd,truemotion=true,isb=false,delta=2) vbo=manalyse(super_odd,truemotion=true,isb=true,delta=2) filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50) Interleave(filldrops_e,filldrops_o) Replacement = Weave() global original = c fixed = ConditionalSelect(c, "\ Prev = YDifferenceFromPrevious(original)" + \ chr(13) + " Prev1 = YDifferenceFromPrevious(SelectEvery(original,1,-1))" + \ chr(13) + " Next = YDifferenceToNext(original)" + \ chr(13) + " Next1 = YDifferenceToNext(SelectEvery(original,1,1))" + \ chr(13) + " Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \ original, selectevery(Replacement,1,-1)) return fixed } Last edited by johnmeyer; 13th December 2016 at 21:13. Reason: Added link to better version of scrip, in later post in this thread |
13th December 2016, 11:13 | #2 | Link | ||
Avisynth language lover
Join Date: Dec 2007
Location: Spain
Posts: 3,431
|
Quote:
However, the global variable 'original' inside the function is a potential source of errors. In particular, if the function is called more than once in a script, all the instances of the function will use the same clip for 'original', even if they are called with different inputs. It turns out that the variable 'original' is actually unnecessary. Since 'c' is the source clip for the ConditionalSelect, it can be referenced as 'last' inside the run-time script. And since 'last' is the implicit default in clip functions, it can just be left out. Code:
fixed = ConditionalSelect(c, " Prev = YDifferenceFromPrevious() Prev1 = YDifferenceFromPrevious(SelectEvery(1,-1)) Next = YDifferenceToNext() Next1 = YDifferenceToNext(SelectEvery(1,1)) Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \ c, selectevery(Replacement,1,-1)) Quote:
source = WriteFileIf(source, ...) The reason is that for WriteFile(If) to produce anything, it must be in the filter chain contributing to the final script output. EDIT: Actually, for this part to work, the WriteFileIf() call must also be simplified along the same lines, removing 'source' from the run-time script (otherwise an endless recursion would occur at run-time - see here). Code:
source = WriteFileIf(source, filename, " YDifferenceFromPrevious() / YDifferenceFromPrevious( selectevery(1, -1) ) > badthreshold && \ YDifferenceToNext() / YDifferenceToNext(selectevery(1, 1) )>badthreshold", "current_frame", append = false) Last edited by Gavino; 13th December 2016 at 11:58. Reason: change to WriteFileIf code |
||
13th December 2016, 17:19 | #3 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
Many thanks for the advice. I've made the changes and everything works, including the ability to create both the text file and the video at the same time. I wish you had a cubicle down the hall from me. Two minutes with you twice a day would save me hours of work, and everything would work better.
I'll wait a few days to see if I get other advice and then post an updated script (it would be too confusing to modify the original in post #1). |
13th December 2016, 20:06 | #4 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
How bout summick like this (fixed your divide by zero's).
Code:
#source=AVISource("F:\V\StarWars.avi").convertTOYV12().killaudio() source=AVISource("e:\fs.avi").convertTOYV12().killaudio() global badthreshold = 2 filename = "e:\Bad.txt" METRICS = True # True, Show Metrics ONLY (overrides other selctions) SHOWDOT = False # True, set to true to add "***" to each replacment frame (for troubleshooting) FILEWRITE = False # True, create a file which contains the frame numbers of all bad frames REPLACE = False # True, Replace each bad frame with a one that is interpolated from adjacent frames. # The "I" in the function name stands or Interlaced, because this will work with interlaced video (as well as progressive) ###### script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) / Max(YDifferenceFromPrevious( selectevery(source, 1, -1)),0.00001) ) + \ "\nNext Ratio = " + String( YDifferenceToNext(source) / Max(YDifferenceToNext(selectevery(source, 1, 1)),0.00001)), lsp=0)""" MetClip = Scriptclip(source, script) Source2 = (FILEWRITE) \ ? WriteFileIf(source, filename, " \ YDifferenceFromPrevious() / Max(YDifferenceFromPrevious( selectevery(1, -1)),0.00001) > badthreshold && \ YDifferenceToNext() / Max(YDifferenceToNext(selectevery(1, 1)),0.00001)>badthreshold", "current_frame", append = false) \ : Source output = (METRICS) ? MetClip : (REPLACE) ? ReplaceBadI(source2,showdot) : Source2 return output #------------------------------ Function ReplaceBadI (clip c, bool showdot) { even = c.SeparateFields().SelectEven() super_even = showdot ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2) vfe=manalyse(super_even,truemotion=true,isb=false,delta=2) vbe=manalyse(super_even,truemotion=true,isb=true,delta=2) filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50) odd = c.SeparateFields().SelectOdd() super_odd = showdot ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2) vfo=manalyse(super_odd,truemotion=true,isb=false,delta=2) vbo=manalyse(super_odd,truemotion=true,isb=true,delta=2) filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50) Interleave(filldrops_e,filldrops_o) Replacement = Weave() fixed = ConditionalSelect(c, " Prev = YDifferenceFromPrevious Prev1 = Max(YDifferenceFromPrevious(SelectEvery(1,-1)),0.00001) Next = YDifferenceToNext Next1 = Max(YDifferenceToNext(SelectEvery(1,1)),0.00001) Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \ c, selectevery(Replacement,1,-1)) return fixed } EDIT: Removed Global Original assignment.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 13th December 2016 at 20:12. |
13th December 2016, 20:21 | #5 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
StainlessS,
I never liked the word "robust" to describe the quality of a computer algorithm, but since that is the word everyone uses, what you have done is to make my script more robust. I need to remember to use the Max function, in future scripts, to make sure the denominator doesn't go to zero. I also much prefer having metrics that you can set at the beginning of the script rather than forcing the user to edit comment blocks. I'll add all those things to the script. Thanks!!! |
13th December 2016, 20:59 | #6 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
Thanks to Gavino and StainlessS!!
Here is the script, revised to include the suggestions in posts #2 and #4. I cleaned up the formatting to make it look prettier, and also added a variable for the file name. You can now, thanks to these changes, control the operation of the script simply by setting the values in the variable listed at the beginning of the script. Code:
#Find And (Optionally) Fix Bad Frames #John Meyer - December 13, 2016 #Rev. 2.0 #Thanks to Gavino and StainlessS for making the script more professional. #This script detects single bad frames. #You can configure the script to write, to a file, the frame numbers of all frames which are detected as "bad". #You can also configure it to automatically replace each bad frame with a new #frame interpolated from its neighbors. #This script will fail if the bad frame happens immediately before or after a scene change. #This script will also fail to find a bad frame if there is more than one bad frame in a row. #It works very well for finding both blank frames and also "flash" frames (like those caused #by a photographer's flash). It will also find single frames which have lots of #static or pixels. It can also find a frame with large x or y displacement from adjacent frames, like #a film frame that wasn't properly registered in the film gate, or an analog #video frame that lost vertical sync. #When using VirtualDub, to create the text file containing the bad frame numbers, #select "Run Video Analysis Pass" in the VirtualDub File menu. If you are simultaneously #creating a fixed video file, you don't need to do this because the file will #be created simultaneously as the fixed video file is created. #The script uses ratios of the metrics for the current frame to the same metrics #on the two adjacent frames. Under normal circumstances, the metrics should be quite #similar, and therefore the ratio should be very near to unity (i.e., 1.00). #Run through the video with the "METRICS" variable set to "True" and look at the metrics #in order to determine an optimum threshold value. A larger threshhold will #catch fewer bad frames, and a lower threshold will eventually create false positives. #The replacement code works well enough that if you end up replacing a few frames that are #actually good, you probably won't notice it. #----------------------------- loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll") #Control script operation by changing the following values : #===================================================================== VideoFile = "E:\fs.avi" global badthreshold = 2 # Set METRICS=TRUE to determine best value METRICS = FALSE # TRUE will show Metrics ONLY (i.e., TRUE overrides all other selctions) SHOWDOT = FALSE # TRUE will add "***" to each replacment frame (for troubleshooting) REPLACE = TRUE # TRUE will replace each bad frame with a one that is interpolated from adjacent frames FILEWRITE = TRUE # TRUE will create a file which contains the frame numbers of all bad frames filename = "E:\Bad.txt" # Set to name and location where you want the frame numbers stored #===================================================================== source = AVISource(VideoFile).convertTOYV12().killaudio() script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) / \ Max(YDifferenceFromPrevious( selectevery(source, 1, -1)),0.00001) ) + \ "\nNext Ratio = " + String( YDifferenceToNext(source) / \ Max(YDifferenceToNext(selectevery(source, 1, 1)),0.00001)), lsp=0)""" MetClip = Scriptclip(source, script) FileFixed = (FILEWRITE) \ ? WriteFileIf(source, filename, " \ YDifferenceFromPrevious() / Max(YDifferenceFromPrevious( selectevery(1, -1)),0.00001) \ > badthreshold && YDifferenceToNext() / Max(YDifferenceToNext(selectevery(1, 1)),0.00001) \ > badthreshold", "current_frame", append = false) : Source output = (METRICS) ? MetClip : (REPLACE) ? ReplaceBadI(FileFixed,showdot) : FileFixed return output #------------------------------ function ReplaceBadI (clip c, bool SHOWDOT) { even = c.SeparateFields().SelectEven() super_even = SHOWDOT ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2) vfe = manalyse(super_even,truemotion=true,isb=false,delta=2) vbe = manalyse(super_even,truemotion=true,isb=true,delta=2) filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50) odd = c.SeparateFields().SelectOdd() super_odd = SHOWDOT ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2) vfo = manalyse(super_odd,truemotion=true,isb=false,delta=2) vbo = manalyse(super_odd,truemotion=true,isb=true,delta=2) filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50) Interleave(filldrops_e,filldrops_o) Replacement = Weave() fixed = ConditionalSelect(c, " Prev = YDifferenceFromPrevious() Prev1 = Max(YDifferenceFromPrevious(SelectEvery(1,-1)),0.00001) Next = YDifferenceToNext() Next1 = Max(YDifferenceToNext(SelectEvery(1,1)),0.00001) Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \ c, selectevery(Replacement,1,-1)) return fixed } Last edited by johnmeyer; 13th December 2016 at 22:23. |
14th December 2016, 02:47 | #7 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
Here's a link to a very short, small test video showing how the script automatically removes photographer flashes at a wedding:
Photo Flash Removal There are two flashes, a fraction of a second apart, about midway through this five-second clip. Here is one frame from that clip, with the left side showing the before, and the right side showing the video which results from the script's automatic replacement: The image on the right is a completely synthesized frame. The one thing I did find out -- and I completely expected this -- is that the script does not work for certain types of photo flashes. The problem is that modern strobes often fire multiple times, especially during their "pre-flash" routine. As I said in my notes in the script, the script is not designed to handle two bad frames in a row. However, when the bad frame is all by itself, the script does really good things. Last edited by johnmeyer; 10th January 2020 at 17:00. Reason: fix photobucket missing image |
14th December 2016, 06:08 | #8 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
I do have a question about my MVTools2 code: did I use the correct Delta? I tried using Delta=1 (default), but that ended up using part of the bad frame I was trying to replace. OK, so I make the reference the frame after the bad frame, and from the perspective of that frame, have it look backwards two frames. So I think I need to use Delta=2 for isb=true (isb="is backwards"). However, should I be using Delta=1 for the forward vector? And, if I do that, do I use a time of 66 or 33?
The synthesized frames look great, and appear to be from the correct moment in time, but I'm wondering if I would get cleaner results if I did this differently. This part of MVTools2 still befuddles me (not hard to do these days). Thanks to anyone who can shed some light on this. |
14th December 2016, 07:47 | #9 | Link | |||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Delta=2, looks OK to me John. (comparing with what I've got here, Included in RT_Stats demos)
Code:
Function TweenFlashFields(clip c,float "FlashThresh",bool "Show") { # Replace single isolated bad fields eg Black or White with a field tweened from those either side using MvTools, and RT_Stats. # Will Likely fix a single Bright/Dark Flash field after scene change where will be replaced with a blend of before and after fields. # This due to NoPan failure, ie fields before and after flash are likely to be more similar (even though dif scenes) than to flash field. # There has been no attempt to fix this fortunate failure. # FlashThresh Default 1.15 (white flash detect), is a threshold of AveLuma_Ratio, Flash_Field_AveLuma / Adjacent_Field_AveLuma. # Values above 1.0 detect WHITE/Flash fields, Below 1.0 detect BLACK/Flash fields (suggest for Black eg 1.0/1.15 = 0.87) # When FlashThresh above 1.0 (detecting WHITE flash) : # If AveLuma_Ratio > FlashThresh, for both adjacent fields then is possible white flash # When FlashThresh below 1.0 (detecting BLACK flash) : # If AveLuma_Ratio < FlashThresh, for both adjacent fields then is possible black flash # When FlashThresh < 0.0, will simultaneously fix both White and Black Flash frames, eg -1.15 will use equivalent to 1.15 for white # and 1.0/1.15 for Black. # 1.0 or -1.0 exactly, Throws an Error. # Show: Puts indicator on fixed fields. c FlashThresh = Float(Default(FlashThresh,1.15)) # Default detects White flash Show=Default(Show,False) Assert(FlashThresh != 1.0 && FlashThresh != -1.0,"TweenFlashFrames: FlashThresh Cannot be 1.0 Nor -1.0") (FlashThresh < 0.0 && FlashThresh > -1.0) ? 1.0 / FlashThresh : FlashThresh CondS=""" ave_p = Max(RT_AverageLuma(delta=-1), 0.01) # Avoid Division By Zero ave = RT_AverageLuma() ave_n = Max(RT_AverageLuma(delta=1), 0.01) rat_p = ave / ave_p # AveLuma_Ratio for Prev rat_n = ave / ave_n # AveLuma_Ratio for Next Flash =(FlashThresh<0.0) \ ? ((rat_p > -FlashThresh && rat_n > -FlashThresh) || (rat_p < -1.0/FlashThresh && rat_n < -1.0/FlashThresh)) \ : (FlashThresh>1.0) \ ? (rat_p > FlashThresh && rat_n > FlashThresh) \ : (rat_p < FlashThresh && rat_n < FlashThresh) dif_p=RT_YDifference(delta=-1) # Diff Prev <-> Curr dif_n=RT_YDifference(delta=1) # Diff Curr <-> Next dif_pn=RT_YDifference(current_frame-1,delta=2) # Diff Prev <-> Next (either side of current) NoPan = (dif_pn < dif_p && dif_pn < dif_n) # (Prev<->Next < Prev<->Curr) AND (Prev<->Next < Curr<->Next) clpn = (Flash && NoPan) ? 1 : 0 # RT_DebugF("%d ] EVEN Rat_p=%.2f Rat_n=%.2f Flash=%s Dif_pn=%.2f Dif_p=%.2f Dif_n=%.2f NoPan=%s Tween=%s", # \ current_frame,rat_p,rat_n,(Flash)?"T":"F",Dif_pn,Dif_p,Dif_n,(NoPan)?"T":"F",(clpn==1)?"Y":"N") clpn """ SepC=SeparateFields() EvenC=SEPC.SelectEven() PrevC=EvenC.DeleteFrame(FrameCount-1).DuplicateFrame(0) # Make clip where prev fields are aligned with curr fields (same length) super = PrevC.MSuper() backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2) forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2) TweenC = PrevC.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70) # Tweened clip TweenC = (show) ? TweenC.Subtitle("EVEN FIELD FIXED",size=24,text_color=$0000FF,align=5,y=EvenC.Height/2-24) : TweenC CondSE=RT_StrReplace(CondS,"FlashThresh",String(FlashThresh)) # Import explicit FlashThresh into condition string EvenFixedC=ConditionalSelect(EvenC,CondSE,EvenC,TweenC) # Fix bad EVEN fields OddC=SepC.SelectOdd() PrevC=OddC.DeleteFrame(FrameCount-1).DuplicateFrame(0) super = PrevC.MSuper() backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2) forward_vectors = MAnalyse(super, isb = false,truemotion=true, delta=2) TweenC = PrevC.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70) # Tweened clip TweenC = (show) ? TweenC.Subtitle("ODD FIELD FIXED",size=24,text_color=$0000FF,align=5,y=OddC.Height/2) : TweenC CondSO=RT_StrReplace(CondSE,"EVEN","ODD") OddFixedC=ConditionalSelect(OddC,CondSO,OddC,TweenC) # Fix bad Odd fields Interleave(EvenFixedC,OddFixedC) Weave() return Last } docs Quote:
Quote:
Quote:
clip to create synth frames whereas you do shift after detection, ie Code:
fixed = ConditionalSelect(c, " Prev = YDifferenceFromPrevious() Prev1 = Max(YDifferenceFromPrevious(SelectEvery(1,-1)),0.00001) Next = YDifferenceToNext() Next1 = Max(YDifferenceToNext(SelectEvery(1,1)),0.00001) Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \ c, selectevery(Replacement,1,-1)) size delta, creates synth frame at always frame n. Trouble is, we want to replace bad frame at synth frame n+1, so have to shift over 1. this probably dont make any more sense than the docs
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 14th December 2016 at 11:34. |
|||
14th December 2016, 17:59 | #10 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
I'm still trying to wrap my head around how delta is really used. I did some searching, and found this very useful post:
Generate broken frames from neighbour frames using MVFlow "pbristow" provides an explanation of how the motion vectors are created. He also provides some code that may permit fixing two bad frames in a row. If this works, it would be interesting (although I don't know if I am up for the challenge) to include some conditional logic which, when two bad frames in a row are detected, uses the alternative code. For the moment, I'll be happy if I can just better understand how this works. It sure seems to me that setting the backward vector to one and the forward vector to two (or vice versa) should produce better results. However, having tried it, I know that it does not work at all, and that doesn't make any sense. |
14th December 2016, 18:13 | #11 | Link | |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,493
|
Quote:
So in the two "vector" clips, frame 0 will represent the vectors between frame 0 and -3 (in the backwards clip) and frame 0 and 3 in the forwards clip. Frame 1 will represent the vectors between frame 1 (bad frame) and frame -2 in the backwards clip, and frame 1 and frame 4 in the forwards clip (but since frame 1 is bad, this info won't get used). Does that help? You're not calculating vectors between x-3 and x+3, but between x and x-3, and also between x and x+3. |
|
14th December 2016, 19:01 | #12 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
Yes, that helps. However, here is why I am puzzled. The following visually shows how motion vectors for bad frame B4 are calculated using the delta=2 setting that I used in my script:
Code:
___________________ | | | v v v G1 G2 G3 B4 G5 G6 I think I understand this part. The next diagram shows where I get confused. Code:
______________ | | | v v v G1 G2 G3 B4 G5 G6 This seems like it would produce better results because G2 is closer in time to the reference frame G3 and therefore the vectors should be more reliable. However, using the different delta=1 for the ISB=false vector and delta=2 for the ISB=true vector doesn't work. My guess is that, even thought the MVTools2 documentation is mute on the subject, the delta values for both the forward and backward vectors have to be the same. This is the only explanation for why it fails even though, in theory, better results should be obtained using the closest possible good frames. So, I think what is going on is that the way the function works requires skipping a nearer, better neighbor frame in order to make the internal logic of MVTools2 work. |
14th December 2016, 19:40 | #13 | Link | |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
This is how I believe that it works, everything is from the point of view of the synth frame (because that is what you are doing).
Code:
50% | <---------> | | v v G1 G2 G3 B4 G5 G6 n n+1 n+2 delta=2 Frame n synth frame created using bidirectional vectors between n and n+delta(2), however, frame requiring fixing is n+1 (B4, 50% between n and n+2) and so requires shift over. Frames are synthesised from point of view of the synth frames not the bad frame. Synth frame to fix B4 @ 50% is generated still at frame n (reqires shift forward 1 frame replacing n+1) 33% 66% | | <---------------> | | v v G1 G2 G3 B4 B5 G6 G7 n n+1 n+2 n+3 delta=3 Synth frame to fix B4 (@ 33% of the way between n and n+3) is generated still at frame n (reqires shift forward 1 frames replacing n+1) Synth frame to fix B5 (@ 66% of the way between n and n+3) is generated still at frame n (reqires shift forward 2 frames replacing n+2) The 'between' percentage does NOT need to be directionally worked out by the user, it is looked after by MVTools and governed again by isb, percentage is relative the synth frame n->n+delta. (Worrying too much about what is in which particular vector clip, and at which frame number, will lead to madness, dont do it , but for the already insane, frame n vectors will contain vectors for n<->n+delta depending upon isb direction) EDIT: I think some of what was posted in the PBristow thread was less than accurate. (not blaming you PB ) EDIT: Oops, delta=3, we have a B5,G5,G6, should have been B5,G6,G7, but I hope you get the drift. EDIT: Quote:
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 5th August 2018 at 11:57. |
|
14th December 2016, 20:56 | #14 | Link |
Registered User
Join Date: Feb 2002
Location: California
Posts: 2,691
|
StainlessS,
Thanks for that. It does help further understand the sometimes murky world of MVTools2. However, I'm still left wondering why I can't use different vectors for isb=true than the ones I use for isb=false in order to be able to use frames which are closer in time to the frame I want to replace. My conclusion at this point is that this cannot be done because MVTools2 is "hard-wired" to use vectors that are symmetrical, backwards and forwards, in time. I can live with that, and will at this point get on with other things, giving up trying to wring one last ounce of quality out of my script. |
14th December 2016, 22:08 | #15 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Can't say that I've tried that John, perhaps it would work, think the bi-di vectors used to compare results, and if too different then blur or whatever. Both forwords and backwards vectors would have to coincide to the same synthsized frame exactly otherwise blurring due to mismatch.
Don't see any reason why would not work, both delta and isb for both separately relevant. Edit, although I can't think of a scenario where useful. Mobile Edit, above, total guesswork.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 14th December 2016 at 22:42. |
15th December 2016, 00:29 | #16 | Link |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,493
|
I'm just guessing here, but when trying to certain things to generate frame x, mvtools will use the backwards and forwards vector clips. For example, if frame x is meant to be an interpolation between source frames y and z, it will need to look at the forward vectors which go from y to z, and at the backward vectors which go from z to y. If the vector clips were generated with different deltas, there'll be no such matching pair of forward and backward vectors.
|
15th December 2016, 00:40 | #17 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
John. Methinks is BAD idea to do frames produced by diff vectors. They
Are a check on each other, as did forward from n match with backwards from n+z, ie self checking, sort of and reason for it. If you eg wanted to recreate eg 120fps from25, and 30, then fix and recreate 25 & 30 separately, and then do whatever you can with both results. Edit, pub shuts in 20 mins, life's a bitch.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 17th December 2016 at 12:36. Reason: Added, 's' :), Merry XMAS every body. |
5th February 2021, 14:48 | #19 | Link |
Registered User
Join Date: Dec 2012
Posts: 65
|
Is it possible using this script to compare frames from different sources and insert a frame from source 1 into source 2 when a difference is detected? Or maybe there's some other way?
I have two sources, one of better quality than the other, but it is censored, the problem is to replace places with censored in the source file 1 with places without censorship from source file 2. I do it manually now, but it would be great to automate the process.
__________________
Ryzen 2700x | ASUS ROG Strix GTX 1080 Ti | 16 Gb DDR4
Windows 10 x64 20H2 KD-55XE9005 | Edifier R2800 |
5th February 2021, 16:40 | #20 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Are they exact same size/cropping and framerate ? [spatial and temporal aligned]
If not then is far far more difficult problem. [where probably a lot less work to do it by hand, after matching spatial and temporal].
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 5th February 2021 at 16:47. |
Thread Tools | Search this Thread |
Display Modes | |
|
|