Extended blend.

[Boris9]

Hi Doomers,
I would like to be able to blend (say) ten seconds of video to obtain an average single frame repeated for those 10 seconds. The camera is still as is most of the picture. I want to obtain the locus of the moving objects. I think that avisynth is probably the right tool. Can anyone point me in the right direction?
Thanks
Boris

[raffriff42]

I for one don't think Avisynth is the right tool, although I would be glad to hear I'm wrong. It seems to me you need a feedback loop for this, and Avisynth, being a frameserver, can't (???) do feedback.

That said, just to start the discussion, here is a crude script to blend 24 frames together. (EDIT and another one to blend 46 frames) Add more frames to taste. :)
## Overlay mode = "blend" by default. Experiment.
## opacity (by trial and error) = (frame no) / (tot frames + 1) ^ 0.7

function BLURRR24(C)
{
return
\Overlay(C.Trim(00,0),
\Overlay(C.Trim(01,0),
\Overlay(C.Trim(02,0),
\Overlay(C.Trim(03,0),
\Overlay(C.Trim(04,0),
\Overlay(C.Trim(05,0),
\Overlay(C.Trim(06,0),
\Overlay(C.Trim(07,0),
\Overlay(C.Trim(08,0),
\Overlay(C.Trim(09,0),
\Overlay(C.Trim(10,0),
\Overlay(C.Trim(11,0),
\Overlay(C.Trim(12,0),
\Overlay(C.Trim(13,0),
\Overlay(C.Trim(14,0),
\Overlay(C.Trim(15,0),
\Overlay(C.Trim(16,0),
\Overlay(C.Trim(17,0),
\Overlay(C.Trim(18,0),
\Overlay(C.Trim(19,0),
\Overlay(C.Trim(20,0),
\Overlay(C.Trim(21,0),
\Overlay(C.Trim(22,0),
\Overlay(C.Trim(23,0),
\Overlay(C.Trim(24,0), C,
\opacity=0.12),
\opacity=0.14),
\opacity=0.16),
\opacity=0.22),
\opacity=0.24),
\opacity=0.26),
\opacity=0.32),
\opacity=0.34),
\opacity=0.36),
\opacity=0.42),
\opacity=0.44),
\opacity=0.46),
\opacity=0.52),
\opacity=0.54),
\opacity=0.56),
\opacity=0.62),
\opacity=0.64),
\opacity=0.66),
\opacity=0.72),
\opacity=0.74),
\opacity=0.76),
\opacity=0.82),
\opacity=0.84),
\opacity=0.86),
\opacity=0.92)
}

function BLURRR46(C)
{
return
\Overlay(C.Trim(1,0),
\Overlay(C.Trim(2,0),
\Overlay(C.Trim(3,0),
\Overlay(C.Trim(4,0),
\Overlay(C.Trim(5,0),
\Overlay(C.Trim(6,0),
\Overlay(C.Trim(7,0),
\Overlay(C.Trim(8,0),
\Overlay(C.Trim(9,0),
\Overlay(C.Trim(10,0),
\Overlay(C.Trim(11,0),
\Overlay(C.Trim(12,0),
\Overlay(C.Trim(13,0),
\Overlay(C.Trim(14,0),
\Overlay(C.Trim(15,0),
\Overlay(C.Trim(16,0),
\Overlay(C.Trim(17,0),
\Overlay(C.Trim(18,0),
\Overlay(C.Trim(19,0),
\Overlay(C.Trim(20,0),
\Overlay(C.Trim(21,0),
\Overlay(C.Trim(22,0),
\Overlay(C.Trim(23,0),
\Overlay(C.Trim(24,0),
\Overlay(C.Trim(25,0),
\Overlay(C.Trim(26,0),
\Overlay(C.Trim(27,0),
\Overlay(C.Trim(28,0),
\Overlay(C.Trim(29,0),
\Overlay(C.Trim(30,0),
\Overlay(C.Trim(31,0),
\Overlay(C.Trim(32,0),
\Overlay(C.Trim(33,0),
\Overlay(C.Trim(34,0),
\Overlay(C.Trim(35,0),
\Overlay(C.Trim(36,0),
\Overlay(C.Trim(37,0),
\Overlay(C.Trim(38,0),
\Overlay(C.Trim(39,0),
\Overlay(C.Trim(40,0),
\Overlay(C.Trim(41,0),
\Overlay(C.Trim(42,0),
\Overlay(C.Trim(43,0),
\Overlay(C.Trim(44,0),
\Overlay(C.Trim(45,0), C,
\opacity=0.07),
\opacity=0.11),
\opacity=0.15),
\opacity=0.18),
\opacity=0.21),
\opacity=0.24),
\opacity=0.27),
\opacity=0.29),
\opacity=0.32),
\opacity=0.34),
\opacity=0.37),
\opacity=0.39),
\opacity=0.41),
\opacity=0.43),
\opacity=0.46),
\opacity=0.48),
\opacity=0.50),
\opacity=0.52),
\opacity=0.54),
\opacity=0.56),
\opacity=0.58),
\opacity=0.60),
\opacity=0.62),
\opacity=0.63),
\opacity=0.65),
\opacity=0.67),
\opacity=0.69),
\opacity=0.71),
\opacity=0.72),
\opacity=0.74),
\opacity=0.76),
\opacity=0.78),
\opacity=0.79),
\opacity=0.81),
\opacity=0.83),
\opacity=0.84),
\opacity=0.86),
\opacity=0.87),
\opacity=0.89),
\opacity=0.91),
\opacity=0.92),
\opacity=0.94),
\opacity=0.95),
\opacity=0.97),
\opacity=0.98)
}

blurrr off:
https://www.dropbox.com/s/8361f8axclp1pi0/GTR2-Paris-sm-BLURRR-off.jpg?raw=1

blurrr 24 frames:
https://www.dropbox.com/s/ncbetgozmlh2sh7/GTR2-Paris-sm-BLURRR24.jpg?raw=1

blurrr 46 frames:
https://www.dropbox.com/s/alkku1ak6f6lzp0/GTR2-Paris-sm-BLURRR46.jpg?raw=1

Going in the right direction...still it would be easier with feedback or reentry of some kind. Plus the script gets very sluggish at 46 frames.

[StainlessS]

I think Jmac968 did something along those lines, but,

Find here temp plugin ClipBlend(), Planar, YUY2, RGB24/32.
Blends all frames played so far.
Resets itself at frame 0.

All frames blended given equal weight.


ClipBlend(clip,"delay"=0) # 8 bit Planar, YUY2, RGB
ClipBlend16(clip,"delay"=0) # 16 bit, Stack16 Planar, YUY2, RGB (see Dither Tools)

Added Delay arg to plugin.

Delay, default = 0, == ALL frames played so far blended.

1 = blend with previous frame.
10 = blend with 10 previous frames, etc.

Accumulator resets at frame 0 and if you jump about.



to get frame 300 only
# ------------------------ Requires GScript, RT_Stats
Avisource("D:\avs\test.avi")

ConvertToYV24() # v2.60
#ConvertToYV12() # v2.58

ClipBlend() # ClipBlend(clip)

FRM =300 # Required frame number

GSCript("""
for(i=0,FRM) {
RT_yankChain(i) # force process frames up to FRM
}
""")

Trim(FRM,-1) # Get Required frame

return ConvertToRGB32() # For Viewing YV24
# ------------------------
StainlessS

EDIT: Added Delay arg
EDIT: Added YUY2, RGB24/32
EDIT: Added 16 bit ClipBlend16()

here ClipBlend TAKE 4 (v2.5 & v2.6, with source):
LINK REMOVED, SEE HERE:-
http://forum.doom9.org/showthread.php?t=168048

you could use eg Loop to repeat frame for as long as you like.


EDIT: With a delay of maybe 7, produces effect that I think they used to use on Top Of The Pops.

Should play in real time.

EDIT: Might be useful for isolating static opaque logos.

[StainlessS]

Previous post plugin Updated, TAKE 3.

Added YUY2, RGB24/32.

[Reel.Deel]

I think Jmac968 did something along those lines

Good memory :). There's good information in his "Average all frames in a clip" (http://forum.doom9.org/showthread.php?t=165229) thread.
A little further down -Vit- (http://forum.doom9.org/showpost.php?p=1579509&postcount=7) also came up with a 16bit solution.
I don't know if it's too much work but maybe you too can add 16bit output to ClipBlend? Regardless thanks for the plugin StainlessS.

[StainlessS]

Reel.Deel,

I've never touched 16bit before and dont have any samples, please post a short 10 sec sample, and I'll give it a bash.

Also, how do you tell it's 16 bit ?, any kind of flag or is user flagged (arg) as 16 bit.

[Reel.Deel]

16-bit processing uses DitherTools (http://forum.doom9.org/showpost.php?p=1386559&postcount=3)' stacked MSB/LSB format known as "Stack16". MSB on top and LSB on the bottom. Stack16 clips are always double in height.
Most 16-bit filters/scripts have a lsb_in and and lsb_out parameter (true or false). I've also seen input_depth=xx and output_depth=xx, but I guess it's just a matter of choice.
If lsb_in/out=true then the input/output clip is stack16, if false the input/output is a regular 8-bit clip.
Other filters have a separate functions for 8-bit and 16-bit processing. For example, SmoothAdjust (http://forum.doom9.org/showthread.php?t=154971) has Smoothlevels and SmoothLevels16.
In SmoothLevels the input *has* to be 8-bit, and although internal processing is done in 32-bit float the output will always be dithered down to 8-bit.
In SmoothLevels16 input/output are always stack16. The 16-bit clip can later processed by other 16-bit filters or be dithered down to the desired bit depth using DitherTools or Flash3kyuu Deband (http://forum.doom9.org/showthread.php?t=161411).
IIRC All 16-bit filters can only work with planar YUV.

Honestly, I don't have any native high bit depth video. All of the video I work with is 8-bit.
For testing purposes you can use DitherTools to obtain a stack16 clip. But to be sure, this is what a stack16 clip looks like (https://www.dropbox.com/s/xqbjo7yi51k759p/Stacked%2016-bit.png).

I Hope this helps. :)

[StainlessS]

Thank you very much, I'll look into it.
If I do do it, I shall try to implement 16 bit for RGB and YUY2 too, see no reason no to just because nobody
else supports at present, dont mean they never will.

EDIT: 16 bit would mean a limitation of ~ 32,000 frames unless I bumped the accumulator to 64 bit, might slow it down a tad.

[StainlessS]

@ReelDeal,

Just to clarify, the reason for VIT doing 16 bit was because of lack of precision when merging lots of clips, where
a little precision is lost at each merge. The ClipBlend() func, uses an Accumulater (32 bit) for each pixel, summing
for every frame encountered, and divides by frames done, eg


dstp[x] = ((*p++ *2) + FramesDone) / (FramesDone*2);


where p points to 32bit accumulator, and +FramesDone is rounding to nearest whole bit.

Cannot get more accurate than that for 8 bit return (equivalent to a single merge of all frames simultaneously).

Using 32 bit accumulator for 8 bits provides 23 bits (- 1 for sign) about 8,000,000 frames maximum usage.

I'll still do the 16bit version (about to do it now) but maybe the above is the reason you asked for it.

EDIT: Oops, the multiply by 2 on *p would halve maximum frames usage and so would actually be more like 4,000,000 frames.

[StainlessS]

Added 16 bit ClipBlend to post #3.

Used this as test (not familiar with Dither Tools)

Avisource("D:\avs\test.avi")
Dither_convert_8_to_16 ()
#Dither_resize16 (720, 480)
ConvertToRGB32() # testing Upside down RGB
ClipBlend16(delay=7) # ClipBlend(clip)
#return ConvertToYV12() .DitherPost ()
return Last


Might feel happier if Cretindesalpes could give it the once over.

[Boris9]

Many thanks to all of you who contributed to answering my question, especially StainlessS. Some of the discussion was out of my league, but I have the plugins and script working and the outcome is exactly what I need. :) I luv it when a plan comes together!@#!
While it is now acedemic, raffriff's comment "I for one don't think Avisynth is the right tool.." does beg the question; "What is the right tool?
Again, many thanks. :D
Boris.

[Reel.Deel]

... maybe the above is the reason you asked for it.

Yes, that's it. Thank you for the additional information. I really don't understand any of the internals but -Vit-'s comments on rounding errors was mainly it. I tested ClipBlend16() and everything works fine.
I did not test RGB or YUY2, because I don't know how to properly handle that with DitherTools.

Might feel happier if Cretindesalpes could give it the once over.

Yes, I would like to hear his thoughts on the 16-bit RGB situation. I've not seen him around lately, I hope he's doing good.

Once again thank you for being so helpful StainlessS.