Inverse of Decimate() ...?

[Gavino]

Originally Posted by johnmeyer

Unfortunately, this created a new source of confusion, and now makes me think that both the filldrops function and even the example given in the MVTools2 documentation are incorrect.

See if you agree.

No, I believe the example in the docs is correct.

You're right that any vector derived from a 'bad' frame is of little value. But if you look at my explanation of MFlowInter again, you will see that with delta=2, neither the forward nor the backward vector used actually involves the 'bad' frame. Using your notation, the vectors used would be F2->F4 and F4->F2, and to replace F3 you would use the frame of MFlowInter corresponding to F2.
Your filldrops function written this way becomes:

function filldrops(clip c)
{
  previous = Loop(c,2,0,0) # needed to select correct frame
  super=MSuper(previous,pel=2)
  vfe=manalyse(super,truemotion=true,isb=false,delta=2)
  vbe=manalyse(super,truemotion=true,isb=true,delta=2)
  filldrops = mflowinter(previous,super,vbe,vfe,time=50)
  fixed = ConditionalFilter(c, filldrops, c, "YDifferenceFromPrevious()", "lessthan", "0.1")
  return fixed
}

In fact, I remember now I posted similar code in response to a similar question in this thread.

Your original code does work correctly (possibly by accident) as long as the detected frame is an exact duplicate.
So does your filldropsnext function, and for the same reason. Because of the Loop function, it replaces frame n by interpolating this time between n-1 and n, and since n is a duplicate of n+1, this produces the required interpolation between n-1 and n+1.

[johnmeyer]

Well, I have now spent several hours on this and can say with a fairly high degree of confidence that the code example in the MVTools2 documentation does NOT work. I am not trying to be argumentative. Instead, I am just reporting what I have found from actually using the code on a real piece of video that has a totally bad frame.

I created a sample clip where I took one frame from a completely unrelated piece of video and replaced a good frame in a video where someone was moving their hand rapidly. The moving hand makes it easy to see what is going on with the interpolated video.

Here is that clip:

http://dl.dropbox.com/u/1561578/test...ame%20clip.avi

I then copied the code from the MVTools2 documentation, and modified it to point to my video and to compare the original video with the synthesized video:

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

source=AVISource("E:\Disney Films\test bad frame clip.avi").ConvertToYV12
super = MSuper(source)
backward_vectors = MAnalyse(super, isb = true, delta=2)
forward_vectors = MAnalyse(super, isb = false, delta=2)
inter = MFlowInter(source,super, backward_vectors, forward_vectors, time=50, ml=70)
#return inter
stackvertical(source,inter)

This should be very easy for anyone to duplicate. Just download the 5 MB clip (uses HufYUV codec), modify the script above to point wherever you put the video, and modify (or delete) the line which loads the MVTools2 plugin.

What you will see is that when the "bad" frame appears in the top of the stacked video, the interpolated frame in the bottom actually contains part of the bad frame. This is obviously NOT what we want. We don't want to see any part of that bad frame.

I have tried many, many permutations to try to get this to work, including using delta=1, delta=2, and "mixed deltas" where I use a different delta for the forward vector than the backward vector. I also have tried using the previous frame as the reference and also the next frame as the reference. This actually gets me quite close and I think will be the way I ultimate solve this puzzle. This is what I was proposing in my previous post.

I am getting close to the correct solution to this problem and will post it when I get it working correctly.

BTW, the conditional code to only use the interpolated frame when needed works just fine (I haven't included that above), and as I posted earlier in this thread, if it occasionally uses an interpolated frame rather than the original, it doesn't matter much for most frames (because the interpolated version is usually pretty close to the original).

Quick P.S.: The code you posted does work, and it is exactly what I am working on. However, it is NOT the code in the documentation because it uses the previous frame for the vectors. I am trying to get it to work delta=1 for the backwards vector and delta=2 for the forward vector in order to get a more accurate estimation. It seems wrong to look at something two frames away in the backwards direction when the frame immediate preceding that frame is perfectly OK.

[Gavino]

Originally Posted by johnmeyer

I then copied the code from the MVTools2 documentation, and modified it to point to my video and to compare the original video with the synthesized video:

source=AVISource("E:\Disney Films\test bad frame clip.avi").ConvertToYV12
super = MSuper(source)
backward_vectors = MAnalyse(super, isb = true, delta=2)
forward_vectors = MAnalyse(super, isb = false, delta=2)
inter = MFlowInter(source,super, backward_vectors, forward_vectors, time=50, ml=70)
stackvertical(source,inter)

You are missing an important point from the docs example - to replace frame 50, it uses inter.Trim(49, -1). So the last line of your code should be
stackvertical(source, inter.Loop(2,0,0))

Quick P.S.: The code you posted does work, and it is exactly what I am working on. However, it is NOT the code in the documentation because it uses the previous frame for the vectors.

So (in effect) does the documentation, as I just explained above.

I am trying to get it to work delta=1 for the backwards vector and delta=2 for the forward vector in order to get a more accurate estimation. It seems wrong to look at something two frames away in the backwards direction when the frame immediate preceding that frame is perfectly OK.

It doesn't look two frames backwards from the bad frame, it looks two frames back from the next good frame to the current one. Remember I said it uses vectors F2->F4 and F4->F2.

To quote the documentation:
"It uses backward "mvbw" and forward "mvfw" motion vectors to create picture at some intermediate time moment between current and next (by delta) frame. It uses pixel-based (by MFlow method) motion compensation from both frames."

There are only two frames involved (current and current+delta) and it uses vectors in both directions between those frames.

[johnmeyer]

It doesn't look two frames backwards from the bad frame, it looks two frames back from the next good frame to the current one. Remember I said it uses vectors F2->F4 and F4->F2.

Well, I just read every one of your posts in other threads about how "delta" works, and I think my old, stupid brain finally "got it." My problem is that I thought the delta, when isb=true, was looking backwards from the current frame. Instead, now that I understand your explanation, everything is referenced to frames after the current frame. Put another way, the backwards vector refers to vectors that point backwards to the current frame, and not vectors backwards from the current frame.

Now everything makes sense.

I'll spend a little more time on this, just to make sure it all makes sense and make sure the resulting code works, and then I'll post the result, just in case someone stumbles on this thread later on. The only contribution to the world as we know it that will come from posting that code is that it integrates the conditional code with the interpolation code.

Thank you for helping me.

[Gavino]

Originally Posted by johnmeyer

Well, I just read every one of your posts in other threads about how "delta" works, and I think my old, stupid brain finally "got it."
...
Thank you for helping me.

You're welcome - it has also helped me to order my thoughts about what MFlowInter actually does in detail, which is not entirely clear from the documentation.
For anyone else still not sure, reread my posts above (also the relevant explanations in this thread).

I'll spend a little more time on this, just to make sure it all makes sense and make sure the resulting code works, and then I'll post the result, just in case someone stumbles on this thread later on. The only contribution to the world as we know it that will come from posting that code is that it integrates the conditional code with the interpolation code.

Isn't that just what I did in post #21? I guess you could also provide similarly revised versions of filldropsI and filldropsnext.

[johnmeyer]

johnmeyer said: I'll spend a little more time on this, just to make sure it all makes sense and make sure the resulting code works, and then I'll post the result ...

Gavino replied: Isn't that just what I did in post #21? I guess you could also provide similarly revised versions of filldropsI and filldropsnext.

Yes, you did post that in #21, but that conditional code isn't sufficient because it will trigger a frame replacement at most scene changes, which is definitely not what we want. Fortunately, a single bad frame is quite different from a scene change because it is different from BOTH of its neighbors. This also makes it different from other types of frame-to-frame variations that normally occur. Thus, it is actually pretty easy to identify with fairly high reliability, but it requires two tests, not one: you need to compare to both the previous and next frames and only if both differences are large for BOTH should the frame be replaced.

So, bottom line, without this double test, you don't get very good identification of a bad frame.

Doing two comparisons instead of one is easy and I'm sure you could do that in ten seconds (it took me ten minutes because I am stupid, and also because I didn't realize you have to declare a variable before it can be used inside of a conditional). But I like to always strive for perfection. So, what I'd like to do, especially given the nature of the original post and also given the number of other posts in this forum about similar problems, is to derive a "local threshold" using a moving average of the YDifference value. Ideally, that average should be reset after a scene change. Given how AVISynth works, I don't know if I can do that. It turns out that my script works pretty well without it, but I thought I'd take a little time an try to add this.

One way or the other, I'll post the result.

[Gavino]

Yes, sorry John, I was forgetting that replacing duplicate frames wasn't the original purpose of this thread.

Originally Posted by johnmeyer

I didn't realize you have to declare a variable before it can be used inside of a conditional

I don't want to hijack this thread even further, but I'm not sure what you mean by this - could you elaborate?

[johnmeyer]

Originally Posted by Gavino

I don't want to hijack this thread even further, but I'm not sure what you mean by this - could you elaborate?

OK. The problem is that this line won't work:

ConditionalFilter(clip, filldrops, clip, "YDifferenceFromPrevious(clip)>10 && YDifferenceToNext(clip)>10", "greaterthan", "0")

unless you make "clip" global:

global clip = c

I initially just used "c," which is declared in the function header, as the argument to YDifferenceToNext. This doesn't work. Once I stumbled on this, I remembered seeing it somewhere, either in the AVISynth documentation, the AVISynth Wiki, or maybe here in the forum.

Also, while we have posted a lot, I think what we've done should help the original poster, so we haven't completely taken this thread away from its original purpose (although we have concentrated on only one aspect of his problem).

Finally, I may have to take back everything I said about how MVTools "delta" works. Both the documentation as well as the tests I am doing today make it seem that my original understand of how MAnalyze uses Delta is correct, namely that it does look at the preceding frame. I think our confusion comes from the fact that the Mflowinter function DOES only look forward and DOES use the delta for determining how many frames ahead to interpolate. Thus delta is being used in one way by MAnalyse, but in a different way by MFlowInter. This is why you have to use delta=2 so it (MFlowInter) looks beyond the bad frame. And it is why you can't use a different delta for the forward and backward vectors in MAnalyze because this causes MFlowinter to interpolate to the wrong frame.

However, it seems increasingly clear to me that my initial understanding of MAnalyse is correct: the isb vectors look to the previous frame and the next frame rather than to the next frame and from the next frame. The more I think about that, the more I realize that it MUST be correct. After all, what would be the difference between a vector to a frame, and a vector back from that same frame, other than a negative sign?

I have now actually been able to prove this with the solution I am creating. This solution uses the motion vectors from the previous frame and from two frames ahead, and then feeds these to MFlowInter. This does produces output that is different, and better, than the code in the documentation which I now once again believe to be wrong, or at least less than optimal.

The solution I am developing uses the conditional filter to create a frame stream that replaces the bad frame with the next frame. This in essence creates the scenario that I've dealt with in the past, namely a perfect, exact duplicate in place of the bad frame. Once you have this, you can use delta=1 for both the forward and backward vectors, but the delta=1 for the forward vector is now actually pointing to a frame that is two frames ahead. MFlowInter is "happy" because it is being told to interpolate to the next frame, but since the "next" frame is now actually two frames ahead, I can use 50 for the time parameter and get the correct outcome.

Now, anticipating what you are thinking, no this is NOT the equivalent of what you've posted, with delta=2 because the vector which points to the previous frame (and I really do think it points to the previous frame) is now pointing to the previous frame, and not the frame prior to the previous frame. Thus, I get a much more accurate interpolation because the isb=true vector is constructed from the immediately adjacent previous frame rather than the frame before that.

The more I thought about it, and the more times I read the MVTools2 documentation, the more I have become convinced that your explanation of the isb delta vectors cannot be correct for MAnalyse. The MVTools2 documentation for MAnalyse states: delta : set the frame interval between the reference frame and the current frame. By default, it's 1, which means that the motion vectors are searched between the current frame and the previous ( or next ) frame.

More to come ...

[Gavino]

Originally Posted by johnmeyer

OK. The problem is that this line won't work:

ConditionalFilter(clip, filldrops, clip, "YDifferenceFromPrevious(clip)>10 && YDifferenceToNext(clip)>10", "greaterthan", "0")

unless you make "clip" global

No need to do that. Since 'clip' is the first parameter to ConditionalFilter, it becomes 'last' inside the run-time script.

ConditionalFilter(clip, filldrops, clip, "YDifferenceFromPrevious()>10 && YDifferenceToNext()>10", "greaterthan", "0")

I'll answer your other points when I've digested them fully.

[Didée]

Originally Posted by johnmeyer

the isb vectors look to the previous frame and the next frame rather than to the next frame and from the next frame.

Frankly, I'm not sure if I'd understand it from that explanation. But OK, it's a bit dry to explain with just words. Let's use a visual representation.

With "delta=1", the vectors produced by MAnalyse for frame "c" look like

       (isb = false)        (isb = true) 
(c-1) --------------> (c) <-------------- (c+1)
       (forward vec)       (backward vec)

And yes: "forward" and "backward" refer to the orientation of the vector.

I think our confusion comes from the fact that the Mflowinter function DOES only look forward and DOES use the delta for determining how many frames ahead to interpolate. Thus delta is being used in one way by MAnalyse, but in a different way by MFlowInter.

Partly incorrect. MFlowInter does look both forward and backward. However, it is using a little trick while doing so.

The "vector situation" for MFlowInter (with delta=2 vectors) :

           <----------
    A     B     C     D     E
           ---------->

The trick is: in order to interpolate between frame X and frame X+1 (or, more precisely, between frame X and frame X+[delta]), MFlowInter uses the "backward" vector of frame X, and the "forward" vector of frame X+(delta).
Well, call it a "trick", or rather a "technical necessity" ... it is done this way because it can't be done in any other way.

while we have posted a lot, I think what we've done should help the original poster, so we haven't completely taken this thread away from its original purpose

I reckon that the original problem already has been solved.

Look at post#18. pbristow hasn't visited since then so I don't know if it works for him. For me, it certainly does work for exactly the posed problem.

[johnmeyer]

ConditionalFilter(clip, filldrops, clip, "YDifferenceFromPrevious(clip)>10 && YDifferenceToNext(clip)>10", "greaterthan", "0")

No need to do that. Since 'clip' is the first parameter to ConditionalFilter, it becomes 'last' inside the run-time script.

You are correct about not needing the first "clip" in this ConditionalFilter call, and also about not needing the "(clip)" arguments in the YdifferenceToNext calls ...

except ...

when clip is something other than "last." As my script has evolved, some versions used "last" and others used frames other than last, so I definitely did need it in earlier versions of the script.

I see that while I was preparing this post that Didée provided an excellent explanation of how vectors are created and used within MVTools2. His explanation reinforces what I always believed to be true prior to this thread and, after a temporary misunderstanding by me, was what I tried to explain in my most recent post. Even when Didée said I was partially incorrect, it was only because I used the word "look" to describe what Mflowinter was doing. Obviously it has to use the vectors from previous as well as future frames, so in that sense it is "looking" in both directions. However, the interpolation is only done in the forward direction (i.e., it does not interpolate a frame at a time prior to the current frame). So, I totally agree with Didée's explanation, and very much appreciate his post.

Now, to hopefully finish off my contributions here, the following is my final (for now) script for fixing bad frames. It uses a much better (in most ways) method of finding a bad frame (although it of course fails on the first and last two frames of the input video).

To detect bad frames the script looks at ratios of YDifference values between this frame and the previous frame, and the previous frame and the frame before that. It also does the same thing with the following frames. This achieves something similar to the moving average idea, but without having to compute a moving average. In other words, it normalizes the YDifference values to whatever is "normal" for this portion of this clip, and only takes action when there is a discontinuity from the norm in BOTH the positive and negative direction from the current frame. Thus, scene changes are ignored (because the frame AFTER the first frame of a scene change has little difference), and it doesn't really matter whether YDifference values are large are small, as long as they are relatively similar from frame to frame.

This code uses this logic to determine when to replace a "bad" frame with a duplicate of the frame following. This corrected stream is then fed to MAnalyse which then feeds its vectors to Mflowinter. By feeding the stream which now contains duplicates in place of the bad frames, the "filldrops" logic developed years ago by MugFunky now works just as it always has.

The advantage this approach has over what is shown in the MVTools documentation is that it avoids using a delta of 2 in the backwards direction. Using this backwards delta of 2 skips the previous frame which, being closer in time is going to provide a better starting point for the estimation vectors. Indeed, I was able to prove this by comparing the frame I got using the code shown in the MVTools2 documentation with what I got with this code.

This produces a better result.

So, I am finished here, and I hope this has helped someone.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

global bad_thresh = 20
source=AVISource("E:\frameserver.avi").ConvertToYV12
corrected=replacebad(source).AssumeFPS(29.97, false) 

return corrected


function replacebad (clip c)
{
  global clip = c

# Replace bad frame with a duplicate of the frame immediately after the bad frame
  goodframes = ConditionalFilter(clip, trim(clip,1,0), clip, \
               "(YDifferenceFromPrevious(clip)/YDifferenceFromPrevious(loop(clip,2,0,0))) * \
               (YDifferenceToNext(clip)/YDifferenceToNext(trim(clip,1,0)))", "greaterthan", "bad_thresh")

# This next line gets the previous frame in the stream that now contains duplicates instead of bad frames
  previousframe = Loop(goodframes,2,0,0)

  super=MSuper(previousframe,pel=2)

  vfe=manalyse(super,truemotion=true,isb=false,delta=1)
  vbe=manalyse(super,truemotion=true,isb=true,delta=1)

  replacement = mflowinter(previousframe,super,vbe,vfe,time=50)


  fixed = ConditionalFilter(clip, replacement, clip, "(YDifferenceFromPrevious(clip)/YDifferenceFromPrevious(loop(clip,2,0,0))) * \
          (YDifferenceToNext(clip)/YDifferenceToNext(trim(clip,1,0)))", "greaterthan", "bad_thresh")

  return fixed
}

[Didée]

Nitpick:

Originally Posted by johnmeyer

However, the interpolation [by MFlowInter] is only done in the forward direction (i.e., it does not interpolate a frame at a time prior to the current frame).

That's the point which I was referring to with "partially incorrect". Or at least, "misunderstandable".
It's correct the the final interpolation of MFLowInter is only "forward in time", in relation to the "current" frame. But in order to produce that interpolation, MFlowInter indees does both a forward interpolation and a backward interpolation, and mixes them acc. to the internal metrics. (After all, you are handing *two* vectors over to MFlowInter ... the 2nd vector isn't bogus. It is used.)
The twist is just that when creating the interpolation between this-and-next-frame, MFlowInter does not use the forward and backward vector of the current frame. It uses the backward vector of the current frame, and the forward vector of the (current+delta) frame.

[Gavino]

Originally Posted by johnmeyer

confusion comes from the fact that the Mflowinter function DOES only look forward

No, as I said (and Didée has since explained much more clearly), it looks both backwards and forwards, but from different starting points.
Remember "F2->F4 and F4->F2"? Note that these vectors are not simply the reverse of each other (unless the motion analysis is perfect and no object moves into or out of the picture).

However, it seems increasingly clear to me that my initial understanding of MAnalyse is correct: the isb vectors look to the previous frame and the next frame rather than to the next frame and from the next frame.

Actually (again as Didée explained), the vectors (for a given frame) look forward from the previous frame and back from the next frame.
But of course each vector is a clip, so the vector for current->next can be found in the next frame of the forward vector, and so on.

Now, anticipating what you are thinking, no this is NOT the equivalent of what you've posted, with delta=2 because the vector which points to the previous frame (and I really do think it points to the previous frame) is now pointing to the previous frame, and not the frame prior to the previous frame. Thus, I get a much more accurate interpolation because the isb=true vector is constructed from the immediately adjacent previous frame rather than the frame before that.

No, as I pointed out, MFlowInter only uses two frames and using delta=2 makes it use the frames immediately around the bad frame, which is the best you can get. The backward vector goes from the frame after the bad one to the one before the bad one (cf Didée's diagram).

The more I thought about it, and the more times I read the MVTools2 documentation, the more I have become convinced that your explanation of the isb delta vectors cannot be correct for MAnalyse.

My explanation may not have been very clear (Didée's is much better), but which part do you think was wrong?

EDIT: Just seen you have posted again in the interim. Perhaps more to come...

[Gavino]

Originally Posted by johnmeyer

You are correct about not needing the first "clip" in this ConditionalFilter call, and also about not needing the "(clip)" arguments in the YdifferenceToNext calls ... except ... when clip is something other than "last."

I didn't say anything about not needing 'clip' as the first argument.
I said that, whatever the first argument is, it is available as 'last' within the 'run-time' script (the bit inside the quotes). So there is no need to use 'clip' there and the global is not needed. This is independent of whether 'clip' itself is equal to 'last' in the 'outside' script.

This corrected stream is then fed to MAnalyse which then feeds its vectors to Mflowinter. By feeding the stream which now contains duplicates in place of the bad frames, the "filldrops" logic developed years ago by MugFunky now works just as it always has.

I thought we'd established that the "filldrops" logic is completely equivalent to the delta=2 approach (when you have an exact duplicate), but works in a less clear way. It seems to me you are doing twice as much work to arrive at the same result.

The advantage this approach has over what is shown in the MVTools documentation is that it avoids using a delta of 2 in the backwards direction. Using this backwards delta of 2 skips the previous frame which, being closer in time is going to provide a better starting point for the estimation vectors. Indeed, I was able to prove this by comparing the frame I got using the code shown in the MVTools2 documentation with what I got with this code.

Now that you have, I hope, a clearer understanding of how MFlowInter works, do you still believe this?
You can't get any 'closer in time' to the bad frame than the immediately surrounding frames, which is what MFlowInter (with delta=2) uses.

[johnmeyer]

johnmeyer said: The advantage this approach has over what is shown in the MVTools documentation is that it avoids using a delta of 2 in the backwards direction. Using this backwards delta of 2 skips the previous frame which, being closer in time is going to provide a better starting point for the estimation vectors. Indeed, I was able to prove this by comparing the frame I got using the code shown in the MVTools2 documentation with what I got with this code.

Gavino replied: Now that you have, I hope, a clearer understanding of how MFlowInter works, do you still believe this?

It is not a matter of belief; it is something that I was able to prove by doing many, many tests. There IS a difference in my approach, and it produces better frames, as it should.

As long as we both agree that Didee's explanation is correct, which I do, then I don't think we need to go further. Thank you VERY much for all your help.

[Gavino]

Didée's explanation is the same as mine (but clearer!), and would suggest that the delta=2 approach cannot be improved upon. Your results are therefore puzzling and invite further explanation, which I believe I have found.

By looking at the source code, I have discovered that for occlusion areas, MVFlowInter also uses information from the two further surrounding frames (current-delta and current+2*delta).
So the diagram becomes, when interpolating between C and C+d,

       ----> <----
    C-d     C     C+d     C+2d
             ---->   <----

It is this additional information that will be more accurate when delta=1, explaining your findings and making your two-stage approach a better one. Well done.

Just to clarify further my remark about ConditionalFilter, what I am saying is this.
In ConditionalFilter(c1, c2, c3, "...", "...", "..."), 'last' is always set to the value of 'c1' before the expression strings are evaluated (on each frame). Therefore your code

  goodframes = ConditionalFilter(clip, trim(clip,1,0), clip, \
               "(YDifferenceFromPrevious(clip)/YDifferenceFromPrevious(loop(clip,2,0,0))) * \
               (YDifferenceToNext(clip)/YDifferenceToNext(trim(clip,1,0)))", "greaterthan", "bad_thresh")

can be replaced by

  goodframes = ConditionalFilter(c, trim(c,1,0), c, \
               "(YDifferenceFromPrevious()/YDifferenceFromPrevious(loop(2,0,0))) * \
               (YDifferenceToNext()/YDifferenceToNext(trim(1,0)))", "greaterthan", "bad_thresh")

and similarly for 'fixed', with the global variable being removed.
Using a global variable for this purpose in a function is in general a bad idea as the function will not work if called more than once.

[johnmeyer]

Therefore your code ... can be replaced by ... and similarly for 'fixed', with the global variable being removed.
Using a global variable for this purpose in a function is in general a bad idea as the function will not work if called more than once.

Thanks for that tip! I don't like the idea of not being able to call the function more than once.

Interesting what you found in the source code. That confirms the behavior I have been observing.

I went ahead and modified the script to remove the global variable. This works for all ConditionalFilter calls that use "c", but I had a few debug calls (which I didn't post, and am not posting below) where I used ScriptClip to show the YDifference vectors so I could see what threshold value to use. This was done later in the script at a point where "last" was now referencing the stream where the bad frame had been replaced with a duplicate. Obviously the YDifference values for this stream no longer can identify the bad frame. So, at your suggestion, I removed the global variable for the normal working script, but I still have to use it for debugging because it IS needed for ScriptClip even though it isn't for ConditionalFilter.

Finally, I decided that my YDifference ratio approach wasn't such a good idea, because it can't differentiate between slightly bad frames and scene changes. So, I went back to the approach I've used for several years. It works great for finding single frame anomalies, but it isn't "automatic" so you have to spend a few minutes to determine the threshold value for each particular video stream.

Here's the script with these two revisions:

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

global bad_thresh = 20

source=AVISource("E:\frameserver.avi").ConvertToYV12.killaudio()
corrected=replacebad(source).AssumeFPS(29.97, false) 

return corrected


function replacebad (clip c)
{
# This next statement replaces bad frame with a duplicate of the frame immediately after the bad frame
  goodframes = ConditionalFilter(c, trim(c,1,0), c, \
               "YDifferenceFromPrevious()>bad_thresh && YDifferenceToNext()>bad_thresh", "greaterthan", "0")

# This next statement gets the previous frame in the stream that now contains duplicates instead of bad frames
  previousframe = Loop(goodframes,2,0,0)

  super=MSuper(previousframe,pel=2)

  vfe=manalyse(super,truemotion=true,isb=false,delta=1)
  vbe=manalyse(super,truemotion=true,isb=true,delta=1)

  replacement = mflowinter(previousframe,super,vbe,vfe,time=50)

  fixed = ConditionalFilter(c, replacement, c, "YDifferenceFromPrevious()>bad_thresh && \
          YDifferenceToNext()>bad_thresh", "greaterthan", "0")

  return fixed
} # End function replacebad

[Gavino]

Originally Posted by johnmeyer

I removed the global variable for the normal working script, but I still have to use it for debugging because it IS needed for ScriptClip even though it isn't for ConditionalFilter.

Note that ScriptClip (indeed all 'run-time' filters) works in a similar way.

ScriptClip(clip, "
# here 'last' refers to 'clip', regardless of the value of 'last' outside
...
")

Or did you mean you needed to refer to a different clip 'inside', one not the same as (or derivable from) ScriptClip's input clip?
That's an example of what I call the 'variable binding problem' (alluded to here) and I developed the GRunT plugin specifically to solve this (and other usability issues with the standard run-time filters).

Incidentally, being a 'language lawyer' guy, I was suprised to see that this works in ConditionalFilter:

"YDifferenceFromPrevious()>bad_thresh && YDifferenceToNext()>bad_thresh", "greaterthan", "0"

It appears ConditionalFilter allows comparison of a boolean value with an integer, whereas the script language itself does not.
It obviously works, but it might be clearer to rewrite this as

"YDifferenceFromPrevious()>bad_thresh && YDifferenceToNext()>bad_thresh", "==", "true"

[johnmeyer]

Or did you mean you needed to refer to a different clip 'inside', one not the same as (or derivable from) ScriptClip's input clip?

I didn't provide, in my earlier posts, the code which causes the problem. So, here it is. In this code fragment:

global clip = c
fixed = ConditionalFilter(c, replacement.ScriptClip("""subtitle(string(YDifferenceFromPrevious(clip) ) \
        + ", " + string(YDifferenceToNext(clip) )   )"""), c, \
        "YDifferenceFromPrevious()>bad_thresh && YDifferenceToNext()>bad_thresh", "greaterthan", "0")

If I do not include clip in the above, then the ScriptClip call will overlay the WRONG YDifference values onto the clip. The YDifference values used by the Conditional filter itself (i.e., the ones on the bottom line above) are the ones associated with "c," but the YDifference values in the ScriptClip call will be associated with "last" (which in the script I just posted is the video stream where the bad frames are replaced with duplicates) unless I pass the clip parameter explicitly, and to do that I must declare it as global. I didn't believe this until I experienced it myself, but it is true. In other words, if I eliminate "clip" from the first two lines so that all the YDifference calls look identical, those inside of the ScriptClip section return different results from those used by the conditional logic.

Incidentally, being a 'language lawyer' guy ... it might be clearer to rewrite this as ...

"YDifferenceFromPrevious()>bad_thresh && YDifferenceToNext()>bad_thresh", "==", "true"

Well, I accept that although, in the same vein, based on my early computer programming in the 1960s, I was taught to never compare using an equal operator unless absolutely necessary. I guess with Booleans this isn't necessary, but when comparing numbers, you can often end up with one variable equal to 4.000000 and the comparison equal to 4.0000001 due to roundoff error, and the equality will fail. So, it is that old (almost forty-year-old) habit that leads me to use the inequality operator.

Using "==" and ">" etc. is probably a good idea. I can't remember whether that was allowed in early versions of AVISynth. There must be some reason I used the non-symbolic version, but I can't remember. A lot of my code is "cut/pasted" from various places and, if it works, I don't mess with it.

[Gavino]

Originally Posted by johnmeyer

In other words, if I eliminate "clip" from the first two lines so that all the YDifference calls look identical, those inside of the ScriptClip section return different results from those used by the conditional logic.

That's right, since 'last' inside the ConditionalFilter refers to 'c', while inside the ScriptClip it refers to 'replacement'.

If you need to do something like this in the future, it can be written like this using GRunT:

fixed = ConditionalFilter(c, replacement.ScriptClip("""subtitle(string(YDifferenceFromPrevious(c) ) \
        + ", " + string(YDifferenceToNext(c) )   )""", args="c"), c, \
        "YDifferenceFromPrevious()>bad_thresh && YDifferenceToNext()>bad_thresh", "greaterthan", "0")

Using "==" and ">" etc. is probably a good idea. I can't remember whether that was allowed in early versions of AVISynth. There must be some reason I used the non-symbolic version, but I can't remember.

It wasn't the use of "greaterthan" instead of ">" that struck me, it was the idea of comparing a boolean (true/false) with an integer (0), something the script language itself does not allow.
(Nor, if I recall, even did Fortran IV - yeah, I started back in the punched card era too )
In fact, GRunT's ConditionalFilter make this simpler too, by allowing the three separate strings to be replaced by a single boolean expression.

ConditionalFilter(..., "YDifferenceFromPrevious()>bad_thresh && YDifferenceToNext()>bad_thresh")

Thanks for your replacebad function.