Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th December 2016, 04:51   #1  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Finding individual "bad" frames in video; save frame number; or repair

[edit]Here is a link to a better version of the script, later in this thread: Better Version of Script [/end edit]

Long ago I created a script that I used to find individual blank frames. I needed this because my NLE (Vegas) has a nasty habit of creating random blank frames during certain types of renders, and you don't know if you have the problem without a tool which can look at every single frame.

I then found I needed a script to find "flash" frames because I do movie film transfers, and because of the way home movie cameras work, the first frame of every single scene is horribly overexposed. These therefore need to be found and removed.

As time went on, I found all sorts of other situations where a video can contain a single frame which does not match either adjacent frame.

I recently had a request for this script, so I went back and cleaned it up a bit and added some comments. The main tool used for detection is the YDifference function built into AVISynth. It compares each pixel in the current frame to each corresponding pixel in the adjacent frame. The metric "blows up" whenever there is a big difference in lots of the pixels, something that happens at scene changes, but also when there is a single-frame corruption. What's more, the metrics blow up looking both backwards and forwards when there is an individual bad frame, whereas they only blow up in one direction at a scene change. Therefore, this function will not do anything at scene changes (which is what you want).

Other types of detection could easily be substituted, using either the other stat functions built into AVISynth, or by using the myriad of compare functions built into StainlessS' excellent RT_Stats package.

I look forward to any ideas for improving this. I couldn't figure out how to do the conditional without declaring one variable as global (Gavino always dings me on this). Also, I wasn't able to get it to both automatically fix bad frames AND output the bad frame numbers to a text file. I could get it to do one or the other, but not both. For me, this isn't a big deal, but some people might want to be able to easily go to the fixed video and check each and every new frame to make sure it looks OK.

One other thing: I wrote this so it works on interlaced video. Since progressive is a special case of interlaced (with no temporal differences between fields), it will work just fine with progressive as well.

Code:
#Find And (Optionally) Fix Bad Frames
#John Meyer - December 12, 2016

#This script detects single bad frames. You have two options of what to do.
#You can configure the script to write, to a file, the frame numbers of all frames which are detected as "bad".
#As an alternative, you can configure it to automatically replace bad frames with a new
#frame that is interpolated from its neighbors. This replacement is often near-perfect (no guarantees, however...)

#This script will fail if the bad frame happens immediately before or after a scene change.
#This script will also fail to find a bad frame if there is more than one bad frame in a row.

#It works very well for finding both blank frames and also "flash" frames (like those caused
#by a photographer's flash). It will also find single frames which have lots of 
#static or pixels. It can also find a frame with large x or y displacement from adjacent frames, like
#a film frame that wasn't properly registered in the film gate, or an analog
#video frame that lost vertical sync.

#When using VirtualDub, to create the text file containing the bad frame numbers, first uncomment that code block.
#Then, select "Run Video Analysis Pass" in the VirtualDub File menu.

#The script uses ratios of the metrics for the current frame to the same metrics 
#on the two adjacent frames. Under normal circumstances, the metrics should be quite
#similar, and therefore the ratio should be very near to unity (i.e., 1.00). 

#Run through the video with the "script" variable enabled, and look at the metrics
#in order to determine an optimum threshold value. A larger threshhold will
#catch fewer bad frames, and a lower threshold will eventually create false positives. 
#The replacement code works well enough that if you end up replacing a few frames that are
#actually good, you probably won't notice it.

#You need to un-comment the WriteFileIf lines to actually write the frame numbers to a file.
#You need to un-comment the script lines to cause the metrics to appear on screen in order to 
#determine the optimum badthreshold value (2 is a good starting point, however).
#You need to un-comment the ReplaceBadI lines to automatically replace bad frames with interpolated frames.

#I recommend only having one of these three code blocks enabled at any one time.

#-----------------------------
loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
global badthreshold = 2
showdot = false           # set to true to add "***" to each replacment frame (for troubleshooting)
filename = "e:\Bad.txt"

source=AVISource("e:\fs.avi").convertTOYV12().killaudio()


#TEMPORARILY remove comments from the following block in order to show the metrics. 
/*
script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) \
         / YDifferenceFromPrevious( selectevery(source, 1, -1) )) + \
         "\nNext Ratio        = " + String( YDifferenceToNext(source) / YDifferenceToNext(selectevery(source, 1, 1) )), lsp=0)"""
final=Scriptclip(source, script)
return final
*/


#Uncomment the code in the next two lines to create a file which contains the frame numbers of all bad frames

/*
WriteFileIf(source, filename, "YDifferenceFromPrevious(source) / YDifferenceFromPrevious( selectevery(source, 1, -1) ) > badthreshold && \
           YDifferenceToNext(source) / YDifferenceToNext(selectevery(source, 1, 1) )>badthreshold", "current_frame", append = false)
*/


#Add comments to the two lines of code below to stop replacement of
#each bad frame with a one that is interpolated from adjacent frames.
#The "I" in the function name stands or Interlaced, because this will
#work with interlaced video (as well as progressive)

output=ReplaceBadI(source,showdot)
return output


#------------------------------

function ReplaceBadI (clip c, bool showdot)
{
  even = c.SeparateFields().SelectEven()
  super_even = showdot ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2) 
  vfe=manalyse(super_even,truemotion=true,isb=false,delta=2)
  vbe=manalyse(super_even,truemotion=true,isb=true,delta=2)
  filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)

  odd  = c.SeparateFields().SelectOdd()
  super_odd = showdot ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2) 
  vfo=manalyse(super_odd,truemotion=true,isb=false,delta=2)
  vbo=manalyse(super_odd,truemotion=true,isb=true,delta=2)
  filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)

  Interleave(filldrops_e,filldrops_o)
  Replacement = Weave()

  global original = c

  fixed = ConditionalSelect(c, "\
                     Prev = YDifferenceFromPrevious(original)" + \
                     chr(13) + "
                     Prev1 = YDifferenceFromPrevious(SelectEvery(original,1,-1))" + \
                     chr(13) + "
                     Next = YDifferenceToNext(original)" + \
                     chr(13) + "
                     Next1 = YDifferenceToNext(SelectEvery(original,1,1))" + \
                     chr(13) + "
                     Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \
                     original, selectevery(Replacement,1,-1))

  return fixed
}

Last edited by johnmeyer; 13th December 2016 at 21:13. Reason: Added link to better version of scrip, in later post in this thread
johnmeyer is offline   Reply With Quote
Old 13th December 2016, 11:13   #2  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by johnmeyer View Post
I couldn't figure out how to do the conditional without declaring one variable as global (Gavino always dings me on this).
The global 'badthreshold' isn't a problem, since it's a global constant.
However, the global variable 'original' inside the function is a potential source of errors.
In particular, if the function is called more than once in a script, all the instances of the function will use the same clip for 'original', even if they are called with different inputs.

It turns out that the variable 'original' is actually unnecessary.
Since 'c' is the source clip for the ConditionalSelect, it can be referenced as 'last' inside the run-time script.
And since 'last' is the implicit default in clip functions, it can just be left out.
Code:
fixed = ConditionalSelect(c, "
                     Prev = YDifferenceFromPrevious()
                     Prev1 = YDifferenceFromPrevious(SelectEvery(1,-1))
                     Next = YDifferenceToNext()
                     Next1 = YDifferenceToNext(SelectEvery(1,1))
                     Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \
                     c, selectevery(Replacement,1,-1))
Note that Avisynth directly supports newlines inside string literals, so I have also removed the string concatenation, backslashes and chr(13) from this code.

Quote:
Also, I wasn't able to get it to both automatically fix bad frames AND output the bad frame numbers to a text file. I could get it to do one or the other, but not both.
To get both at the same time, it should work if you replace WriteFileIf(source, ...) by
source = WriteFileIf(source, ...)
The reason is that for WriteFile(If) to produce anything, it must be in the filter chain contributing to the final script output.

EDIT: Actually, for this part to work, the WriteFileIf() call must also be simplified along the same lines, removing 'source' from the run-time script (otherwise an endless recursion would occur at run-time - see here).
Code:
source = WriteFileIf(source, filename, "
    YDifferenceFromPrevious() / YDifferenceFromPrevious( selectevery(1, -1) ) > badthreshold && \
    YDifferenceToNext() / YDifferenceToNext(selectevery(1, 1) )>badthreshold", "current_frame", append = false)
__________________
GScript and GRunT - complex Avisynth scripting made easier

Last edited by Gavino; 13th December 2016 at 11:58. Reason: change to WriteFileIf code
Gavino is offline   Reply With Quote
Old 13th December 2016, 17:19   #3  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Many thanks for the advice. I've made the changes and everything works, including the ability to create both the text file and the video at the same time. I wish you had a cubicle down the hall from me. Two minutes with you twice a day would save me hours of work, and everything would work better.

I'll wait a few days to see if I get other advice and then post an updated script (it would be too confusing to modify the original in post #1).
johnmeyer is offline   Reply With Quote
Old 13th December 2016, 20:06   #4  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
How bout summick like this (fixed your divide by zero's).

Code:
#source=AVISource("F:\V\StarWars.avi").convertTOYV12().killaudio()
source=AVISource("e:\fs.avi").convertTOYV12().killaudio()

global badthreshold = 2
filename  = "e:\Bad.txt"
METRICS   = True            # True, Show Metrics ONLY (overrides other selctions)
SHOWDOT   = False           # True, set to true to add "***" to each replacment frame (for troubleshooting)
FILEWRITE = False           # True, create a file which contains the frame numbers of all bad frames
REPLACE   = False           # True, Replace each bad frame with a one that is interpolated from adjacent frames.
                            #   The "I" in the function name stands or Interlaced, because this will work with interlaced video (as well as progressive)
######

script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) / Max(YDifferenceFromPrevious( selectevery(source, 1, -1)),0.00001) ) + \
         "\nNext Ratio        = " + String( YDifferenceToNext(source) / Max(YDifferenceToNext(selectevery(source, 1, 1)),0.00001)), lsp=0)"""
MetClip = Scriptclip(source, script)

Source2 = (FILEWRITE)
    \ ? WriteFileIf(source, filename, "
    \      YDifferenceFromPrevious() / Max(YDifferenceFromPrevious( selectevery(1, -1)),0.00001) > badthreshold &&
    \      YDifferenceToNext() / Max(YDifferenceToNext(selectevery(1, 1)),0.00001)>badthreshold", "current_frame", append = false)
    \ : Source

output = (METRICS) ? MetClip : (REPLACE) ? ReplaceBadI(source2,showdot) : Source2
return output

#------------------------------

Function ReplaceBadI (clip c, bool showdot) {
    even = c.SeparateFields().SelectEven()
    super_even = showdot ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2)
    vfe=manalyse(super_even,truemotion=true,isb=false,delta=2)
    vbe=manalyse(super_even,truemotion=true,isb=true,delta=2)
    filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)

    odd  = c.SeparateFields().SelectOdd()
    super_odd = showdot ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2)
    vfo=manalyse(super_odd,truemotion=true,isb=false,delta=2)
    vbo=manalyse(super_odd,truemotion=true,isb=true,delta=2)
    filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)

    Interleave(filldrops_e,filldrops_o)
    Replacement = Weave()

    fixed = ConditionalSelect(c, "
             Prev  = YDifferenceFromPrevious
             Prev1 = Max(YDifferenceFromPrevious(SelectEvery(1,-1)),0.00001)
             Next  = YDifferenceToNext
             Next1 = Max(YDifferenceToNext(SelectEvery(1,1)),0.00001)
             Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \
             c, selectevery(Replacement,1,-1))

    return fixed
}
EDIT: Little testing, dont have test clip handy (should be OK I think).
EDIT: Removed Global Original assignment.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 13th December 2016 at 20:12.
StainlessS is offline   Reply With Quote
Old 13th December 2016, 20:21   #5  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
StainlessS,

I never liked the word "robust" to describe the quality of a computer algorithm, but since that is the word everyone uses, what you have done is to make my script more robust. I need to remember to use the Max function, in future scripts, to make sure the denominator doesn't go to zero. I also much prefer having metrics that you can set at the beginning of the script rather than forcing the user to edit comment blocks.

I'll add all those things to the script.

Thanks!!!
johnmeyer is offline   Reply With Quote
Old 13th December 2016, 20:59   #6  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Thanks to Gavino and StainlessS!!

Here is the script, revised to include the suggestions in posts #2 and #4. I cleaned up the formatting to make it look prettier, and also added a variable for the file name.

You can now, thanks to these changes, control the operation of the script simply by setting the values in the variable listed at the beginning of the script.
Code:
#Find And (Optionally) Fix Bad Frames
#John Meyer - December 13, 2016
#Rev. 2.0
#Thanks to Gavino and StainlessS for making the script more professional.

#This script detects single bad frames. 

#You can configure the script to write, to a file, the frame numbers of all frames which are detected as "bad".
#You can also configure it to automatically replace each bad frame with a new
#frame interpolated from its neighbors. 

#This script will fail if the bad frame happens immediately before or after a scene change.
#This script will also fail to find a bad frame if there is more than one bad frame in a row.

#It works very well for finding both blank frames and also "flash" frames (like those caused
#by a photographer's flash). It will also find single frames which have lots of 
#static or pixels. It can also find a frame with large x or y displacement from adjacent frames, like
#a film frame that wasn't properly registered in the film gate, or an analog
#video frame that lost vertical sync.

#When using VirtualDub, to create the text file containing the bad frame numbers,
#select "Run Video Analysis Pass" in the VirtualDub File menu. If you are simultaneously
#creating a fixed video file, you don't need to do this because the file will
#be created simultaneously as the fixed video file is created.

#The script uses ratios of the metrics for the current frame to the same metrics 
#on the two adjacent frames. Under normal circumstances, the metrics should be quite
#similar, and therefore the ratio should be very near to unity (i.e., 1.00). 

#Run through the video with the "METRICS" variable set to "True" and look at the metrics
#in order to determine an optimum threshold value. A larger threshhold will
#catch fewer bad frames, and a lower threshold will eventually create false positives. 
#The replacement code works well enough that if you end up replacing a few frames that are
#actually good, you probably won't notice it.


#-----------------------------
loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#Control script operation by changing the following values :
#=====================================================================
VideoFile           = "E:\fs.avi"
global badthreshold = 2              # Set METRICS=TRUE to determine best value
METRICS             = FALSE          # TRUE will show Metrics ONLY (i.e., TRUE overrides all other selctions)
SHOWDOT             = FALSE          # TRUE will add "***" to each replacment frame (for troubleshooting)
REPLACE             = TRUE           # TRUE will replace each bad frame with a one that is interpolated from adjacent frames
FILEWRITE           = TRUE           # TRUE will create a file which contains the frame numbers of all bad frames
filename            = "E:\Bad.txt"   # Set to name and location where you want the frame numbers stored
#=====================================================================

source = AVISource(VideoFile).convertTOYV12().killaudio()

script = """Subtitle("\nPrevious Ratio = " + String( YDifferenceFromPrevious(source) /
         \ Max(YDifferenceFromPrevious( selectevery(source, 1, -1)),0.00001) ) + 
         \ "\nNext Ratio        = " + String( YDifferenceToNext(source) /
         \ Max(YDifferenceToNext(selectevery(source, 1, 1)),0.00001)), lsp=0)"""
MetClip = Scriptclip(source, script)

FileFixed = (FILEWRITE)
    \ ? WriteFileIf(source, filename, "
    \ YDifferenceFromPrevious() / Max(YDifferenceFromPrevious( selectevery(1, -1)),0.00001) 
    \ > badthreshold && YDifferenceToNext() / Max(YDifferenceToNext(selectevery(1, 1)),0.00001)
    \ > badthreshold", "current_frame", append = false) : Source

output = (METRICS) ? MetClip : (REPLACE) ? ReplaceBadI(FileFixed,showdot) : FileFixed
return output

#------------------------------

function ReplaceBadI (clip c, bool SHOWDOT)
{
  even        = c.SeparateFields().SelectEven()
  super_even  = SHOWDOT ? even.subtitle("***").MSuper(pel=2) : even.MSuper(pel=2) 
  vfe         = manalyse(super_even,truemotion=true,isb=false,delta=2)
  vbe         = manalyse(super_even,truemotion=true,isb=true,delta=2)
  filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)

  odd         = c.SeparateFields().SelectOdd()
  super_odd   = SHOWDOT ? odd.subtitle("***").MSuper(pel=2) : odd.MSuper(pel=2) 
  vfo         = manalyse(super_odd,truemotion=true,isb=false,delta=2)
  vbo         = manalyse(super_odd,truemotion=true,isb=true,delta=2)
  filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)

  Interleave(filldrops_e,filldrops_o)
  Replacement = Weave()

  fixed       = ConditionalSelect(c, "
                Prev  = YDifferenceFromPrevious()
                Prev1 = Max(YDifferenceFromPrevious(SelectEvery(1,-1)),0.00001)
                Next  = YDifferenceToNext()
                Next1 = Max(YDifferenceToNext(SelectEvery(1,1)),0.00001)
                Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \
                c, selectevery(Replacement,1,-1))

  return fixed
}

Last edited by johnmeyer; 13th December 2016 at 22:23.
johnmeyer is offline   Reply With Quote
Old 14th December 2016, 02:47   #7  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Here's a link to a very short, small test video showing how the script automatically removes photographer flashes at a wedding:

Photo Flash Removal

There are two flashes, a fraction of a second apart, about midway through this five-second clip.

Here is one frame from that clip, with the left side showing the before, and the right side showing the video which results from the script's automatic replacement:



The image on the right is a completely synthesized frame.

The one thing I did find out -- and I completely expected this -- is that the script does not work for certain types of photo flashes. The problem is that modern strobes often fire multiple times, especially during their "pre-flash" routine. As I said in my notes in the script, the script is not designed to handle two bad frames in a row. However, when the bad frame is all by itself, the script does really good things.

Last edited by johnmeyer; 10th January 2020 at 17:00. Reason: fix photobucket missing image
johnmeyer is offline   Reply With Quote
Old 14th December 2016, 06:08   #8  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
I do have a question about my MVTools2 code: did I use the correct Delta? I tried using Delta=1 (default), but that ended up using part of the bad frame I was trying to replace. OK, so I make the reference the frame after the bad frame, and from the perspective of that frame, have it look backwards two frames. So I think I need to use Delta=2 for isb=true (isb="is backwards"). However, should I be using Delta=1 for the forward vector? And, if I do that, do I use a time of 66 or 33?

The synthesized frames look great, and appear to be from the correct moment in time, but I'm wondering if I would get cleaner results if I did this differently.

This part of MVTools2 still befuddles me (not hard to do these days).

Thanks to anyone who can shed some light on this.
johnmeyer is offline   Reply With Quote
Old 14th December 2016, 07:47   #9  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Delta=2, looks OK to me John. (comparing with what I've got here, Included in RT_Stats demos)

Code:
Function TweenFlashFields(clip c,float  "FlashThresh",bool "Show") {
# Replace single isolated bad fields eg Black or White with a field tweened from those either side using MvTools, and RT_Stats.
# Will Likely fix a single Bright/Dark Flash field after scene change where will be replaced with a blend of before and after fields.
#    This due to NoPan failure, ie fields before and after flash are likely to be more similar (even though dif scenes) than to flash field.
#    There has been no attempt to fix this fortunate failure.
# FlashThresh Default 1.15 (white flash detect), is a threshold of AveLuma_Ratio, Flash_Field_AveLuma / Adjacent_Field_AveLuma.
#   Values above 1.0 detect WHITE/Flash fields, Below 1.0 detect BLACK/Flash fields (suggest for Black eg 1.0/1.15 = 0.87)
#   When FlashThresh above 1.0 (detecting WHITE flash) : 
#       If AveLuma_Ratio > FlashThresh, for both adjacent fields then is possible white flash
#   When FlashThresh below 1.0 (detecting BLACK flash) : 
#       If AveLuma_Ratio < FlashThresh, for both adjacent fields then is possible black flash
#   When FlashThresh < 0.0, will simultaneously fix both White and Black Flash frames, eg -1.15 will use equivalent to 1.15 for white
#       and 1.0/1.15 for Black.
#   1.0 or -1.0 exactly, Throws an Error.
# Show: Puts indicator on fixed fields.
    c
    FlashThresh = Float(Default(FlashThresh,1.15))                  # Default detects White flash
    Show=Default(Show,False)
    Assert(FlashThresh != 1.0 && FlashThresh != -1.0,"TweenFlashFrames: FlashThresh Cannot be 1.0 Nor -1.0")
    (FlashThresh < 0.0 && FlashThresh > -1.0) ? 1.0 / FlashThresh : FlashThresh
    CondS="""
        ave_p   = Max(RT_AverageLuma(delta=-1), 0.01)               # Avoid Division By Zero
        ave     = RT_AverageLuma()
        ave_n   = Max(RT_AverageLuma(delta=1), 0.01)
        rat_p   = ave / ave_p                                       # AveLuma_Ratio for Prev
        rat_n   = ave / ave_n                                       # AveLuma_Ratio for Next
        Flash   =(FlashThresh<0.0)
            \ ? ((rat_p > -FlashThresh && rat_n > -FlashThresh) || (rat_p < -1.0/FlashThresh && rat_n < -1.0/FlashThresh))
            \ : (FlashThresh>1.0)
            \ ? (rat_p > FlashThresh && rat_n > FlashThresh)
            \ : (rat_p < FlashThresh && rat_n < FlashThresh)
        dif_p=RT_YDifference(delta=-1)                              # Diff Prev <-> Curr
        dif_n=RT_YDifference(delta=1)                               # Diff Curr <-> Next
        dif_pn=RT_YDifference(current_frame-1,delta=2)              # Diff Prev <-> Next (either side of current)
        NoPan = (dif_pn < dif_p && dif_pn < dif_n)                  # (Prev<->Next < Prev<->Curr) AND (Prev<->Next < Curr<->Next) 
        clpn = (Flash && NoPan) ? 1 : 0
#       RT_DebugF("%d ] EVEN Rat_p=%.2f Rat_n=%.2f Flash=%s Dif_pn=%.2f Dif_p=%.2f Dif_n=%.2f NoPan=%s Tween=%s",
#               \ current_frame,rat_p,rat_n,(Flash)?"T":"F",Dif_pn,Dif_p,Dif_n,(NoPan)?"T":"F",(clpn==1)?"Y":"N")
        clpn
    """
    SepC=SeparateFields()
    EvenC=SEPC.SelectEven()
    PrevC=EvenC.DeleteFrame(FrameCount-1).DuplicateFrame(0) # Make clip where prev fields are aligned with curr fields (same length)
    super = PrevC.MSuper()
    backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
    forward_vectors  = MAnalyse(super, isb = false,truemotion=true, delta=2)
    TweenC = PrevC.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)     # Tweened clip
    TweenC = (show) ? TweenC.Subtitle("EVEN FIELD FIXED",size=24,text_color=$0000FF,align=5,y=EvenC.Height/2-24) : TweenC
    CondSE=RT_StrReplace(CondS,"FlashThresh",String(FlashThresh))   # Import explicit FlashThresh into condition string
    EvenFixedC=ConditionalSelect(EvenC,CondSE,EvenC,TweenC)         # Fix bad EVEN fields
    OddC=SepC.SelectOdd()
    PrevC=OddC.DeleteFrame(FrameCount-1).DuplicateFrame(0)
    super = PrevC.MSuper()
    backward_vectors = MAnalyse(super, isb = true,truemotion=true, delta=2)
    forward_vectors  = MAnalyse(super, isb = false,truemotion=true, delta=2)
    TweenC = PrevC.MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)     # Tweened clip
    TweenC = (show) ? TweenC.Subtitle("ODD FIELD FIXED",size=24,text_color=$0000FF,align=5,y=OddC.Height/2) : TweenC
    CondSO=RT_StrReplace(CondSE,"EVEN","ODD")
    OddFixedC=ConditionalSelect(OddC,CondSO,OddC,TweenC)                # Fix bad Odd fields
    Interleave(EvenFixedC,OddFixedC)
    Weave()
    return Last
}
With Delta=1, would be partially using bad frames.
docs
Quote:
Motion interpolation function. It is not the same (but similar) as MVInterpolate function of older MVTools version. It uses backward mvbw and forward mvfw motion vectors to create picture at some intermediate time moment between current and next (by delta) frame. It uses pixel-based (by MFlow method) motion compensation from both frames. Internal forward and backward occlusion masks (MMask kind = 2 method) and time weighted factors are used to produce the output image with minimal artefacts. True motion estimation is strongly recommended for this function.
Quote:
time

Interpolation time position between frames, in percent. Default value of 50.0 is half-way.
Quote:
To recreate bad frames by interpolation with MFlowInter:

AVISource("c:\test.avi") # or MPEG2Source, DirectShowSource, some previous filter, etc
super = MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=2)
forward_vectors = MAnalyse(super, isb = false, delta=2)
inter = MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
# Assume bad frames are 50 and 60
trim(0,49) ++ inter.trim(49,-1) \
++ trim(51,59) ++ inter.trim(59,-1) ++ trim(61,0)
EDIT: Note Above, @ bad frame 50, replace with synth frame from 49 (I know, it's weird), I have used pre-shift-over
clip to create synth frames whereas you do shift after detection, ie
Code:
fixed       = ConditionalSelect(c, "
                Prev  = YDifferenceFromPrevious()
                Prev1 = Max(YDifferenceFromPrevious(SelectEvery(1,-1)),0.00001)
                Next  = YDifferenceToNext()
                Next1 = Max(YDifferenceToNext(SelectEvery(1,1)),0.00001)
                Prev/Prev1 < badthreshold && Next/Next1 < badthreshold ? 0 : 1", \
                c, selectevery(Replacement,1,-1))
From a synth frame point of view, creating synth frame at frame n, uses two vectors n<--->(n+delta), so no matter what
size delta, creates synth frame at always frame n. Trouble is, we want to replace bad frame at synth frame n+1, so
have to shift over 1. this probably dont make any more sense than the docs
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 14th December 2016 at 11:34.
StainlessS is offline   Reply With Quote
Old 14th December 2016, 17:59   #10  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
I'm still trying to wrap my head around how delta is really used. I did some searching, and found this very useful post:

Generate broken frames from neighbour frames using MVFlow

"pbristow" provides an explanation of how the motion vectors are created. He also provides some code that may permit fixing two bad frames in a row. If this works, it would be interesting (although I don't know if I am up for the challenge) to include some conditional logic which, when two bad frames in a row are detected, uses the alternative code.

For the moment, I'll be happy if I can just better understand how this works. It sure seems to me that setting the backward vector to one and the forward vector to two (or vice versa) should produce better results. However, having tried it, I know that it does not work at all, and that doesn't make any sense.
johnmeyer is offline   Reply With Quote
Old 14th December 2016, 18:13   #11  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,493
Quote:
It sure seems to me that setting the backward vector to one and the forward vector to two (or vice versa) should produce better results.
You want to calculate the vectors between the good frames - from frame 0 (good) to frame 3 (good) (delta=3) and also from frame 3 to frame 0 (also delta=3, isb=true).

So in the two "vector" clips, frame 0 will represent the vectors between frame 0 and -3 (in the backwards clip) and frame 0 and 3 in the forwards clip.
Frame 1 will represent the vectors between frame 1 (bad frame) and frame -2 in the backwards clip, and frame 1 and frame 4 in the forwards clip (but since frame 1 is bad, this info won't get used).

Does that help?

You're not calculating vectors between x-3 and x+3, but between x and x-3, and also between x and x+3.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 14th December 2016, 19:01   #12  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
Yes, that helps. However, here is why I am puzzled. The following visually shows how motion vectors for bad frame B4 are calculated using the delta=2 setting that I used in my script:

Code:
 ___________________
|         |         |
v         v         v
G1   G2   G3   B4   G5   G6
As can be seen, the reference good frame is G3 and the bad frame that we want to replace is B4. ("G"=Good; "B"=Bad). In the MVTools2 sample code, on which I based my code, it uses delta=2. This skips the adjacent frame and instead uses frames that are two frames away in time. Thus, the ISB=true (IS Backward) vector is calculated from G5 back to G3 and the delta=2, ISB=false is calculated from G1 to G3. Then, using MFlowInter, these vectors are used to create a replacement for frame B4 which is 50% of the way between G3 and G5.

I think I understand this part.

The next diagram shows where I get confused.

Code:
      ______________
     |    |         |
     v    v         v
G1   G2   G3   B4   G5   G6
Once again, the reference frame is "G3". The delta=2, ISB=true (IS Backward) vector is calculated from G5 back to G3 but in this example, I use delta=1, for the ISB=false vector (calculated from G2 to G3). Then, using MFlowInter, these vectors are used to create a replacement for frame B4 which is 50% of the way between G3 and G5.

This seems like it would produce better results because G2 is closer in time to the reference frame G3 and therefore the vectors should be more reliable.

However, using the different delta=1 for the ISB=false vector and delta=2 for the ISB=true vector doesn't work. My guess is that, even thought the MVTools2 documentation is mute on the subject, the delta values for both the forward and backward vectors have to be the same. This is the only explanation for why it fails even though, in theory, better results should be obtained using the closest possible good frames.

So, I think what is going on is that the way the function works requires skipping a nearer, better neighbor frame in order to make the internal logic of MVTools2 work.
johnmeyer is offline   Reply With Quote
Old 14th December 2016, 19:40   #13  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
This is how I believe that it works, everything is from the point of view of the synth frame (because that is what you are doing).

Code:
              50%
               |
          <--------->
          |         |
          v         v
G1   G2   G3   B4   G5   G6
          n    n+1  n+2

delta=2
Frame n synth frame created using bidirectional vectors between n and n+delta(2),
however, frame requiring fixing is n+1 (B4, 50% between n and n+2) and so requires shift over. Frames are synthesised
from point of view of the synth frames not the bad frame.
Synth frame to fix B4 @ 50% is generated still at frame n (reqires shift forward 1 frame replacing n+1)


              33%   66%
               |     |
          <--------------->
          |               |
          v               v
G1   G2   G3   B4   B5    G6   G7
          n    n+1  n+2   n+3

delta=3
Synth frame to fix B4 (@ 33% of the way between n and n+3) is generated still at frame n (reqires shift forward 1 frames replacing n+1)
Synth frame to fix B5 (@ 66% of the way between n and n+3) is generated still at frame n (reqires shift forward 2 frames replacing n+2)
The 'between' percentage does NOT need to be directionally worked out by the user, it is looked after by MVTools
and governed again by isb, percentage is relative the synth frame n->n+delta.
EDIT: Delta is always +ve, isb sets whether it is a forward or backward vector between n and n + delta.
(Worrying too much about what is in which particular vector clip, and at which frame number, will lead to madness, dont do it ,
but for the already insane, frame n vectors will contain vectors for n<->n+delta depending upon isb direction)
EDIT: I think some of what was posted in the PBristow thread was less than accurate. (not blaming you PB )

EDIT: Oops, delta=3, we have a B5,G5,G6, should have been B5,G6,G7, but I hope you get the drift.
EDIT:
Quote:
but for the already insane, frame n vectors will contain vectors for n<->n+delta depending upon isb direction)
EDIT: Actually, backwards vectors will be at frame n, and forwards vector at n+delta.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 5th August 2018 at 11:57.
StainlessS is offline   Reply With Quote
Old 14th December 2016, 20:56   #14  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,691
StainlessS,

Thanks for that. It does help further understand the sometimes murky world of MVTools2. However, I'm still left wondering why I can't use different vectors for isb=true than the ones I use for isb=false in order to be able to use frames which are closer in time to the frame I want to replace. My conclusion at this point is that this cannot be done because MVTools2 is "hard-wired" to use vectors that are symmetrical, backwards and forwards, in time. I can live with that, and will at this point get on with other things, giving up trying to wring one last ounce of quality out of my script.
johnmeyer is offline   Reply With Quote
Old 14th December 2016, 22:08   #15  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Can't say that I've tried that John, perhaps it would work, think the bi-di vectors used to compare results, and if too different then blur or whatever. Both forwords and backwards vectors would have to coincide to the same synthsized frame exactly otherwise blurring due to mismatch.
Don't see any reason why would not work, both delta and isb for both separately relevant.

Edit, although I can't think of a scenario where useful.

Mobile

Edit, above, total guesswork.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 14th December 2016 at 22:42.
StainlessS is offline   Reply With Quote
Old 15th December 2016, 00:29   #16  |  Link
wonkey_monkey
Formerly davidh*****
 
wonkey_monkey's Avatar
 
Join Date: Jan 2004
Posts: 2,493
I'm just guessing here, but when trying to certain things to generate frame x, mvtools will use the backwards and forwards vector clips. For example, if frame x is meant to be an interpolation between source frames y and z, it will need to look at the forward vectors which go from y to z, and at the backward vectors which go from z to y. If the vector clips were generated with different deltas, there'll be no such matching pair of forward and backward vectors.
__________________
My AviSynth filters / I'm the Doctor
wonkey_monkey is offline   Reply With Quote
Old 15th December 2016, 00:40   #17  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
John. Methinks is BAD idea to do frames produced by diff vectors. They
Are a check on each other, as did forward from n match with backwards from n+z, ie self checking, sort of and reason for it.

If you eg wanted to recreate eg 120fps from25, and 30, then fix and recreate 25 & 30 separately, and then do whatever you can with both results.

Edit, pub shuts in 20 mins, life's a bitch.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 17th December 2016 at 12:36. Reason: Added, 's' :), Merry XMAS every body.
StainlessS is offline   Reply With Quote
Old 15th December 2016, 00:41   #18  |  Link
Gavino
Avisynth language lover
 
Join Date: Dec 2007
Location: Spain
Posts: 3,431
Quote:
Originally Posted by davidhorman View Post
For example, if frame x is meant to be an interpolation between source frames y and z, it will need to look at the forward vectors which go from y to z, and at the backward vectors which go from z to y.
That's exactly right, and that's what StainlessS's first diagram in post #13 shows (using vectors from G3 to G5 and from G5 to G3).
__________________
GScript and GRunT - complex Avisynth scripting made easier
Gavino is offline   Reply With Quote
Old 5th February 2021, 14:48   #19  |  Link
Shinkiro
Registered User
 
Join Date: Dec 2012
Posts: 65
Is it possible using this script to compare frames from different sources and insert a frame from source 1 into source 2 when a difference is detected? Or maybe there's some other way?
I have two sources, one of better quality than the other, but it is censored, the problem is to replace places with censored in the source file 1 with places without censorship from source file 2.
I do it manually now, but it would be great to automate the process.
__________________
Ryzen 2700x | ASUS ROG Strix GTX 1080 Ti | 16 Gb DDR4
Windows 10 x64 20H2
KD-55XE9005 | Edifier R2800
Shinkiro is offline   Reply With Quote
Old 5th February 2021, 16:40   #20  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
Are they exact same size/cropping and framerate ? [spatial and temporal aligned]
If not then is far far more difficult problem. [where probably a lot less work to do it by hand, after matching spatial and temporal].
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???

Last edited by StainlessS; 5th February 2021 at 16:47.
StainlessS is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.