Display difference between two frames

[signal]

For demonstration purposes I'm trying to figure out how to show the difference between frames in a single clip.

The source avi is a capture of EIA colorbars (Huffyuv encoded) and I want to show the interference or noise that is captured on a clip after trying different capture cards, computers, sources, etc.

I tried subtract with:

clip = avisource("d:\test.avi")
subtract(clip.trim(1,0),clip)

I must be doing this wrong as I'm getting an error (AVISynth 2.5.7):
CAVIStreamSynth: System exception - Access Violation at 0x0, reading from 0x0

I'd really like to get something like the LUV Metric example that MSU demonstrates with their MSU Denoiser examples but not sure how they generated that. Their Denoiser doesn't have a mode to show this.

[edit: they used their MSU Video Quality Measurement Tool but requires a before/after clip for comparison. This causes problems on captured clips as they don't align with the source clip.]

http://www.compression.ru/video/denoising/index_en.html

Any ideas?

[Leak]

Any ideas?
I noticed on a test video that your script got slower and slower the further it moved from a keyframe, both using AviSynth 2.57 and the 2.58 alpha release...
clip = avisource("d:\test.avi")
subtract(clip.trim(1,0),clip)
Simply swapping subtract's arguments fixed that:
clip = avisource("d:\test.avi")
subtract(clip,clip.trim(1,0))
Obviously, the first script caused a lot of seeking - maybe that caused your HuffYUV decoder to crash?

Then again, the order of the clips shouldn't really matter as far as the scripts speed is concerned... that looks like a bug to me - exchanging "Subtract" with "StackVertical" for example shows no slowdown at all...

np: Thomas Fehlmann - Arbeitstitel (Honigpumpe)

[squid_80]

Obviously, the first script caused a lot of decoding from the last keyframe - maybe that caused your HuffYUV decoder to crash?But huffyuv is all keyframes...

[Leak]

But huffyuv is all keyframes...
Errrm... I was testing it with an XviD file - I guess I wouldn't even have noticed the seeking otherwise... :D

Also, I meant that maybe the excessive seeking that seems to be going on irregardless of the format might be causing the crash. Or that there's something else going on that's a bug.

EDIT: Okay, thanks to IanB I now know that it's Subtract overwriting the frame of the first clip it gets. But the excessive re-decoding of frames might still have made the codec crash...

np: Thomas Fehlmann - Atlas (Honigpumpe)

[signal]

Yea, it does seem to be causing Huffyuv (or itself) to crash for some reason.
And maybe my imagination but subtract seems to be struggling with this clip more so than others (i.e. even in another format).

Here's a sample clip: cb4sub.avi (47MB Huffyuv) (http://64.202.175.38/examples/cb4sub.avi)

Any other ideas to get an analysis and view of the pixel variations with AVISynth??

[signal]

I found histogram(mode="luma") which seems to be more what I'm after but after some use it ended up crashing with the same exception error as subtract.

This was after trying to do some stepping through the video.

Turns out Subsequent tries just playing the clip crash it as well.
Sometimes immediatly, sometimes after a few seconds of play.

And yes, I rebooted just to make sure things were "clean" to start with.

I'll post something in the dev forum as something is buggy here.

Any other ideas for visually showing the difference between frames?

[signal]

With VDub I took the original example clip, turned off the audio, and Direct Stream copied it to an "old avi" format.

Seems to work fine this way.

"Meanwhile... back at the ranch..."

Histogram(mode="luma") is more of what I'm after but need to be able to select the threshold or highlight the larger variances.

Trying to find a way to only display a range of luminance and highlight what's left.

[ChiDragon]

If the audio was somehow the problem you might also try the audio=false argument of AviSource.

[signal]

Thanks for the tip, that does solve it.

[signal]

So is there anyway to weigh the differences in pixels or set the threshold of histogram?

The translated docs for histogram (luma mode) only mention that "1 pixel luminance difference will show as a 16 pixel luminance pixel".
I take it this means that for a single luminance difference in 1 pixel between frames, the lumiance will be brought up by 16 for that pixel. Is that right?

It seems to be the case as you can see single pixel variations in my sample clip when put through histogram.

If so, if there is a two or three luminance difference than does it bring it up by 32 and 64?

Ultimately what I need is to weigh ranges of variations into groups (or another tool to do so).
An example might be say a variance of 1 to 10 is switched green, 11 to 20 is switch to blue, and 21 to 30 is switched to red. Obviously those ranges would need to be tweaked.

Is that possible?

[Lefungus]

Not sure it's exactly the answer you're looking for, but I'm using this script to compare pictures ( masktools inside(tm) )

mt_lutxy(clip_a, clip_b, "x y - abs 192 *", y = 3, u = 0, v = 0)


Check documentation if you want more details

[signal]

Thanks Lefungus.

So for a single clip to compare against frames I do:clip_a=AviSource("cb4sub.avi",audio=false).trim(0,300).ConvertToYV12()
clip_b=clip_a.trim(1,301)

mt_lutxy(clip_a, clip_b, "x y - abs 192 *", y = 3, u = 0, v = 0)

After looking at the v2.0 docs I'm afraid my skills on what to use for the arguments are slim.

Is it a matter of just changing y to see the difference between luma values between each frame (doesn't seem to be as 3 is the only one I see the "noise" on).

[Didée]

Ultimately what I need is to weigh ranges of variations into groups (or another tool to do so).
An example might be say a variance of 1 to 10 is switched green, 11 to 20 is switch to blue, and 21 to 30 is switched to red. Obviously those ranges would need to be tweaked.

Colorizing different ranges of luma differences with different colors can be done, but its a bit cumbersome to script: there are no straightforward filters with luma-chroma-linking, like "if we have [this] on luma, then do [that] on chroma".

If ranges in different greyscales are sufficient, then it's a quickie:

r0 = "1" # range0: up to this consider as "no difference"
r1 = "11" # range1: upper threshold
r2 = "21" # range2: upper threshold
r3 = "31" # range3: upper threshold. Bigger than this maps to "255"
absD = "x y - abs "

mt_lutxy(clip_a,clip_b,
\ absD+r0+" <= 0 "+absD+r1+" <= 64 "+absD+r2+" <= 128 "+absD+r3+" <= 192 255 ? ? ? ?",Y=3,U=-128,V=-128)

[signal]

Cool! Greyscale will be fine.

I'll give this a shot.

[signal]

Many Thanks Didée! That works great! =)

It turns out that that using the cb4sub.avi clip (link in post #5) the highest variance is 7 which only appears as a single pixel in a single frame going from frame 0-300.

The majority seems to be in the 1 to 4 range.

What I'm working on now are some tests to see what variances is perceptable to the eyes (well my eyes anyway).

One thing I just noticed is that black is showing no variances at all.
If the source was the cause of the noise then wouldn't it show up there as well?

It appears that the "noise" is dropping luminance rather than a random gain/loss scenario.

[Guest]

Colorizing different ranges of luma differences with different colors can be done, but its a bit cumbersome to script: there are no straightforward filters with luma-chroma-linking, like "if we have [this] on luma, then do [that] on chroma". Actually there is such a filter that I made for VirtualDub:

http://neuron2.net/pseudocolor/pseudocolor.html

It could be imported into Avisynth, or a native port would be pretty easy.

[signal]

Thanks neuron2, I'll try that out.
That should complete what I was originally looking for.

I've been using the script suggested by Didée quite a bit on the various systems and capture devices I have.
It's been very useful to me.

I also tried to break out the variances into short clips over a range.

I'm sure this isn't the best way to do it and it probably needs a sanity check:

LoadModule("avslib", "array", "core")
LoadModule("avslib", "array", "operators")

Function lumadiff(val dof, clip c) {

d=c.trim(1,60)
sdof=string(dof)

return mt_lutxy(c, d, "x y - abs "+sdof+" <= 0 255 ?",Y=3,U=-128,V=-128).subtitle("Luma Difference of "+string(dof))

}

global testclip=AviSource("cb4sub.avi",audio=false).trim(0,60).ConvertToYV12

pmvalues = ArrayRange(1, 6, 1)
pmclips = pmvalues.ArrayOpFunc("lumadiff", "testclip")
return pmclips.ArraySum()

Look ok?

edit: Just saw the problem with starting the range at 0 (changed that to 1) but don't seem to get the same output for a r1=1 as with Didée's script.. back to the drawing board.

[gzarkadas]

@signal,
Try this function, it will make things easier (I hope :)):

LoadPackage("avslib", "string")
LoadPlugin("mt_masktools.dll")

function colorise_diff(clip test, clip ref, int low_th, int high_th, int diff_color, bool "hard_color") {
hard_color = Default(hard_color, true)
Assert(low_th >= 0 && low_th <= 255 \
&& high_th >= 0 && high_th <= 255, "thresholds must be in [0..255] range")
Assert(low_th < high_th, "low threshold must be smaller than high threshold")

absdiff = mt_lutxy(test, ref, "x y - abs", y=3, u=-128, v=-128)
lut_exp = "x %i < 0 x %i > 0 255 ? ?" # makes all pixels in range white & outside black
cmask = mt_lut(absdiff, StrPrint(lut_exp, low_th, high_th), y=3, u=-128, v=-128)
return hard_color \
? Overlay(cmask, test.BlankClip(color=diff_color), mask=cmask, mode="blend", opacity=1.0) \
: Overlay(cmask, Overlay(test, test.BlankClip(color=diff_color), mask=cmask, mode="chroma", \
opacity=1.0), mask=cmask, mode="blend", opacity=1.0)
}


Now you will only have to call (for as many times needed to cover your target range):

mask1 = colorise_diff(test, ref, lo1, hi1, color1)
mask2 = colorise_diff(test, ref, hi1+1, hi2, color2)
mask3 = colorise_diff(test, ref, hi2+1, hi3, color3)
...

and then combine all mask... clips in one (possibly by using:

combined = Overlay(ref, mask1, mask=mask1.ColorYUV(levels="tv->pc"))
combined = Overlay(combined, mask2, mask=mask2.ColorYUV(levels="tv->pc"))
...
but this depends on the application and thus I leave it up to you; you may only need to see them one by one, which is much simpler than overlaying them all together).

If you want to use arrays to code the diff creation and combining by Overlay, don't use ArraySum(...sum_func="Overlay"...) to sum the masks because you can't pass the array element both as overlay and mask. You will have to use a global clip as an accumulator and ArrayOpFunc with a custom function that assigns to the global (and returns a dummy value).

[signal]

Thanks gzarkadas.

I was now looking at doing both a one by one via a loop (what I was trying in my last script) and then throw in the combined. I'm finding both methods useful.

I'll see what I can do with your suggestions.

BTW - Am I doing this right?
If I simply want to select a single variance instead of a range (say of 1 for example) I would use the mt_lutxy expression:

x y - abs 1 <= 0 x y - abs 1 => 0 255 ? ?

[gzarkadas]

If I simply want to select a single variance instead of a range (say of 1 for example) I would use the mt_lutxy expression:

x y - abs 1 <= 0 x y - abs 1 => 0 255 ? ?

For start, the => must become >= . After this correction, the expression above translates to:

abs(x - y) <= 1 ? 0 : (abs(x - y) >= 1 ? 0 : 255)


which will always output 0 because the conditions overlap. What you need is probably:

abs(x - y) < 1 ? 0 : (abs(x - y) > 1 ? 0 : 255)


thus the lut expression

x y - abs 1 < 0 x y - abs 1 > 0 255 ? ?