"Peak Hold" filter across range of frames?

[jbeale]

Is there an avisynth filter that has a "peak hold" function? I want to accumulate the maximum value that any given pixel ever reaches as the video clip plays, so in a normal video the screen would eventually become brighter and brighter everywhere, as bright areas move across the screen.

Actually I want a "minimum hold", but I figured one filter would probably have both options. I want to use it in conjunction with VideoScope() in "UV" vector plot mode, to map out the (Cb,Cr) gamut captured by a camcorder as I expose it to a wide range of scene colors and intensities.

I've seen someone ask for this before (to remove twinkle from astro starfield) but in 2005, at least there wasn't any clean way to do it:
"'Luma Peak' Hold, Store, and Aggregate?" http://forum.doom9.org/showthread.php?t=96235

The suggested workaround included a bunch of lines like
output=output.overlay(o8,mode="lighten") but it seems to require one script line for every frame of video, which is painful to say the least.

[Gavino]

The suggested workaround included a bunch of lines like
output=output.overlay(o8,mode="lighten") but it seems to require one script line for every frame of video, which is painful to say the least.
The core of the processing can be done using ScriptClip to accumulate the results:
...
b = BlankClip(last)
ScriptClip("""
b = b.Overlay(last, mode="lighten")
return b
""")

[jbeale]

Thanks very much for the help! I admit I don't really understand how ScriptClip() works, especially in this case. I tried simply appending your code to my script as below, but I'm not sure it's working properly. When I open this script in VirtualDub, it gets increasingly slow, and the VirtualDub.exe process is taking up more memory with each frame (up to 1.3 GB after 450 frames). I think my PC is about to crash... is it trying to store every separate frame in RAM? In principle, this can be done with a single (x,y) framebuffer holding the peak value of each pixel seen so far (not sure if that's possible through current Avisynth scripting though.)

--
# Avisynth script to load DGindex'ed AVC file and display chroma histogram

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGAVCDecode.dll")
AVCSource("C:\misc\video\20100519\20100518_220835.dga")

BilinearResize(480,270) # make the image a reasonable size relative to the scope display
ConvertToYUY2() # VideoScope works only in YUY2 mode
VideoScope("both", false, "U", "V", "UV") # show (u,v) = (Cb, Cr) vectorscope display at lower right

# code to do multiframe "peak hold" from Gavino http://forum.doom9.org/showthread.php?p=1411115

b = BlankClip(last)
ScriptClip("""
b = b.Overlay(last, mode="lighten")
return b
""")
# --- end of script

[Gavino]

When I open this script in VirtualDub, it gets increasingly slow, and the VirtualDub.exe process is taking up more memory with each frame (up to 1.3 GB after 450 frames). I think my PC is about to crash... is it trying to store every separate frame in RAM?
When I saw this, I had a case of déja vu and remembered this thread, which did something similar.

The solution is similar - replace
b = b.Overlay(last, mode="lighten")
by
b = b.Loop(2,0,0).Overlay(last, mode="lighten")

Explanation (including use of ScriptClip) is given in the other thread.

[jbeale]

Thanks again! The code below works as expected, up to frame 741, at which point it stopped with an error.
LoadPlugin("DGAVCDecode.dll")
AVCSource("file.dga")
BilinearResize(480,270) # make the image a reasonable size

b = BlankClip(last)
ScriptClip("""
b = b.Loop(2,0,0).Overlay(last, mode="lighten")
return b
""")
stopped at frame 741 with this:

Avisynth read error:
GetFrameBuffer: Returned a VFB with a 0 data pointer!
size=3110464, max=258639872, used=181624032
I think we have run out of memory folks!

by the way, that code and also the "darken" version below, can generate some interesting "smeared" effects!
b = BlankClip(last, color=$FFFFFF)
ScriptClip(""" b = b.Loop(2,0,0).Overlay(last, mode="darken") return b """)

[Gavino]

The code below works as expected, up to frame 741, at which point it stopped with an error.
...I think we have run out of memory folks!
As Overlay is quite a memory hog, better results can probably obtained by using mt_logic from MaskTools:
ScriptClip(""" b = b.Loop(2,0,0).mt_logic(last, mode="max", chroma="copy")
return b
""")
However, this gives a greyscale result, which might not be what you want.

To get chroma as well, you could use instead
ScriptClip("""
b = b.Loop(2,0,0)
mask = mt_lutxy(b, last, "y x > 255 0 ?")
b = mt_merge(b, last, mask, luma=true, chroma="process")
return b
""")
but this is likely to be slower and use more memory (though probably still less than Overlay).

[jbeale]

This is slightly different from your version, but the below code works, using MaskTools from http://manao4.free.fr/MaskTools-v1.5.8.zip
Memory use still grows continually as the clip plays, but much more slowly than with the previous scripts. The output does remain in color. (UPDATE: I tried removing all this code, and memory grew the same amount. Memory use might be something from AVCSource when reading a .dga indexed AVC file. )

ConvertToYV12() # MaskTools works only in YV12 mode (4:2:0)
b = BlankClip(last, length=2, color=$FFFFFF) # solid white
ScriptClip(""" b = b.Loop(2,0,0).Logic(last, mode="min") return b """) # hold minimum value each pixel has ever been

[Gavino]

The output does remain in color.
But is it the right color?
For each position in the frame, should it be:
a) the chroma of the pixel with the largest luma value attained at that position over the length of the clip; or,
b) the largest chroma (U and V) values attained at that position, independently of luma?

I was assuming you want (a), but I think mt_logic (or Logic) will give you (b) if chroma is processed (I could be wrong).

[jbeale]

Well, it's a fairly strange color response; the image starts normal with the first frame and becomes progressively darker but with ghost-images of saturated color that remain visible wherever there is saturated color in the current frame. Anyway, it worked for what I needed it for.

[Gavino]

That suggests you are getting (b), or possibly just the chroma from the current frame. So in general each pixel will have luma from one frame and chroma from another, which doesn't seem useful. But if it suits your purpose, no problem.