How to replicate in Avisynth: multiple movies at once using "Lighter Color blending"

[Sparktank]

Hi again,

I found another project on youtube that just blew me away and would love to know how to replicate it in avisynth.

It's real messy to watch, but it just tickles my inner nerd.
Watching with ChromaDepth 3D (https://en.wikipedia.org/wiki/ChromaDepth) glasses is a real brain twister.

The link:
I selected the comment where the uploader responded to someone asking him how he did it.
The question and his response should be top of the list of comments.

Star Wars Wars: All 6 Films At Once (720p HD)
https://www.youtube.com/watch?v=HTHTEKwaYdI&lc=z123wnjawtmvzlcpg04civoyeoymu5pjsxc0k

The (Imgur) Gallery:
https://imgur.com/a/8gS2O?gallery
https://i.imgur.com/2YLsNVfl.jpg

The Response:
" Thanks for the question! Pretty good guess +Curtis Dominguez.

It's actually pretty simple.

It is accomplished using the Lighter Color blending mode (also known as Max or Lighter depending on your composting software).

It mathematically selects the brightest pixel of all layers of video.

This is why a brighter landscape, like Hoth, will totally dominate the frame, and lightsabers and blasters will always show up.

I added my own level of contrast to each layer equally to make increase the falloff into darkness of each video (increasing overlay potential), and keeping it from just looking all grey/muddy.

I'm working on a how to video but not quite finished. "


So it seems he used a NLE like Adobe Premiere or other.
But I want to do this via Avisynth. :p

I would love to do this to a nice boxset one of these decades.
Maybe the theatrical Alien Quadrilogy.
Maybe Pirates of the Caribbean trilogy.

The Plea:
Help me, Obi-Wan Doom9'ers. You're my only hope. :thanks:

EDIT: The solutions:

Function maxY (clip1, clip2)
{
Dif = mt_makediff (mt_logic (clip1.converttoy8 (), clip2.converttoy8 (), "max"), clip1.converttoy8 ())
Mask = mt_lut (Dif, "x 128 == 0 255 ?").Blur (1)
Y = mt_merge (clip1.converttoy8 (), clip2.converttoy8 (), Mask)
U = mt_merge (clip1.utoy8 (), clip2.utoy8 (), Mask)
V = mt_merge (clip1.vtoy8 (), clip2.vtoy8 (), Mask)
YtoUV (U, V, Y)
}


this one blends stuff softly (blurred mask instead of binary mask), should work better and make the weird thing a little bit less weird...

The last four lines can be replaced by:
mt_merge (clip1, clip2, Mask, luma=true, U=3, V=3)

Function maxY (clip1, clip2)
{
Dif = mt_makediff (mt_logic (clip1.converttoy8 (), clip2.converttoy8 (), "max"), clip1.converttoy8 ())
Mask = mt_lut (Dif, "x 128 == 0 255 ?").Blur (1)
mt_merge (clip1, clip2, Mask, luma=true, U=3, V=3)
}

[feisty2]

"It mathematically selects the brightest pixel of all layers of video."
clip1
clip2
clip3
...
clipn

max = mt_logic (clip1, clip2, "max").mt_logic (clip3, "max")...mt_logic (clipn, "max")
return max

[Sparktank]

Interesting!

Buffy season 2 (didn't do detelecine/deinterlace).

SetMemoryMax(512)
LoadPlugin("C:\AVS\dgdecnv2049\DGDecodeNV.dll")
clip1 = DGSource("E:\MakeMKV\BUFFY_SEASON2_DISC3\09- What's My Line - Part 1.dgi").AssumeFPS(29.97)
clip2 = DGSource("E:\MakeMKV\BUFFY_SEASON2_DISC3\10- What's My Line - Part 2.dgi").AssumeFPS(29.97)
clip3 = DGSource("E:\MakeMKV\BUFFY_SEASON2_DISC3\11- Ted.dgi")

max = mt_logic (clip1,clip2, "max").mt_logic (clip3, "max")
return max

Frame(ish) 13446
http://i.imgur.com/zwHa3cN.png (http://imgur.com/zwHa3cN)

Except clip1 is the only one with colors.

Clip 1: http://i.imgur.com/AMKm41Y.png
Clip 2: http://i.imgur.com/06bJkdZ.png
Clip 3: http://i.imgur.com/nLyX4JZ.png

[feisty2]

chroma stuff should get guided by luma

Function maxY (clip1, clip2)
{
Y = mt_logic (clip1.converttoy8 (), clip2.converttoy8 (), "max")
Dif = mt_makediff (Y, clip1.converttoy8 ())
Mask = mt_lut (Dif, "x 128 == 0 255 ?")
U = mt_merge (clip1.utoy8 (), clip2.utoy8 (), Mask)
V = mt_merge (clip1.vtoy8 (), clip2.vtoy8 (), Mask)
YtoUV (U, V, Y)
}

clip1=clip1.ConvertToYV24 ()
clip2=clip2.ConvertToYV24 ()
clip3=clip3.ConvertToYV24 ()
...
clipn=clipn.ConvertToYV24 ()

max = maxY (clip1, clip2).maxY (clip3)...maxY (clipn)
return max

[feisty2]

and here we go!


clip1=clip1.converttoyv24 ()
clip2=clip2.converttoyv24 ()

Function maxY (clip1, clip2)
{
Y = mt_logic (clip1.converttoy8 (), clip2.converttoy8 (), "max")
Dif = mt_makediff (Y, clip1.converttoy8 ())
Mask = mt_lut (Dif, "x 128 == 0 255 ?")
U = mt_merge (clip1.utoy8 (), clip2.utoy8 (), Mask)
V = mt_merge (clip1.vtoy8 (), clip2.vtoy8 (), Mask)
YtoUV (U, V, Y)
}

maxY (clip1,clip2)



http://i.imgur.com/kEk0Mt0.png

dunno if it's the kinda stuff u want tho

[Sparktank]

EDIT: using post #4 (i do things slowly)

i LOVE this!

13446
http://i.imgur.com/cl2K90O.png (http://imgur.com/cl2K90O)

03175
http://i.imgur.com/QkUbMqe.png (http://imgur.com/QkUbMqe)

57603
http://i.imgur.com/uGc6sQL.png (http://imgur.com/uGc6sQL)

Thanks a lot!

One caveat is to expect clips 1&2 to take precedence over proceeding clips in the event that clips 1&2 end sooner than any other clips?

Some creepy stuff to do to:
http://i.imgur.com/uBA4MOd.png (http://imgur.com/uBA4MOd)


fill_clip = blankclip(3,720,480,"YV12",fps=29.97).killaudio()
clip1 = DGSource("E:\MakeMKV\BUFFY_SEASON2_DISC3\09- What's My Line - Part 1.dgi").AssumeFPS(29.97)
clip2 = fill_clip+clip1

[Sparktank]

Using post #5:


maxY (clip1,clip2)
http://i.imgur.com/YJzz5em.png (http://imgur.com/YJzz5em)


maxY (clip1,clip2).maxY (clip3)
http://i.imgur.com/2Nw8a1u.png (http://imgur.com/2Nw8a1u)

[feisty2]

One caveat is to expect clips 1&2 to take precedence over proceeding clips in the event that clips 1&2 end sooner than any other clips?


dunno about that, I just converted "It mathematically selects the brightest pixel of all layers of video." to a piece of avisynth script :)

[Sparktank]

Thanks a bunch! This is perfect! :thanks::thanks:

In the event that clips are shorter than others, I'll just add a blank clip to the end so they all match same runtime.

The results will be much better with HD sources (or properly handled SD sources).

This will provide endless hours of fun. :)

[feisty2]

Function maxY (clip1, clip2)
{
Dif = mt_makediff (mt_logic (clip1.converttoy8 (), clip2.converttoy8 (), "max"), clip1.converttoy8 ())
Mask = mt_lut (Dif, "x 128 == 0 255 ?").Blur (1)
Y = mt_merge (clip1.converttoy8 (), clip2.converttoy8 (), Mask)
U = mt_merge (clip1.utoy8 (), clip2.utoy8 (), Mask)
V = mt_merge (clip1.vtoy8 (), clip2.vtoy8 (), Mask)
YtoUV (U, V, Y)
}


this one blends stuff softly (blurred mask instead of binary mask), should work better and make the weird thing a little bit less weird...

[Sparktank]

soft blend (post #10):

Used an HD source for clarity.

Frame #164560
http://i.imgur.com/wTxDIml.png (http://imgur.com/wTxDIml)

SetMemoryMax(512)
LoadPlugin("C:\AVS\dgdecnv2049\DGDecodeNV.dll")

fill_clip = blankclip(3,1280, 534,"YV24",fps=23.976).killaudio()
clip1 = DGSource("F:\MakeMKV\V For Vendetta.dgi", crop_t=140, crop_b=148, crop_l=0, crop_r=0).AssumeFPS(23.976)
clip1 = clip1.ConvertToYV24(matrix="Rec709").Spline36Resize(1280, 534)
clip2 = fill_clip+clip1

Function maxY (clip1, clip2)
{
Dif = mt_makediff (mt_logic (clip1.converttoy8 (), clip2.converttoy8 (), "max"), clip1.converttoy8 ())
Mask = mt_lut (Dif, "x 128 == 0 255 ?").Blur (1)
Y = mt_merge (clip1.converttoy8 (), clip2.converttoy8 (), Mask)
U = mt_merge (clip1.utoy8 (), clip2.utoy8 (), Mask)
V = mt_merge (clip1.vtoy8 (), clip2.vtoy8 (), Mask)
YtoUV (U, V, Y)
}

max = maxY (clip1, clip2)
return max


Forwarding a bit in Avspmod, I can see this is something to do for isolated shots (like in some movies when someone is sick or drugged).
Would make a really fun addition to 'fan made trailers'.

That shot where the camera tilts up to V, it's perfect for this weird effect!
Trim it down, weird it out, add it main project. ;)

Thanks again for this!

[Sparktank]

Here's very short sample of that shot/scene.
Resized to 500xHeight (for browser/gif), lossless then uploaded to gycat.
(interesting they support UtVideo codec for conversion)

Original clip:
HTML5 video: http://gfycat.com/PoshUnacceptableDogwoodtwigborer
GIF: http://zippy.gfycat.com/PoshUnacceptableDogwoodtwigborer.gif
http://zippy.gfycat.com/PoshUnacceptableDogwoodtwigborer.gif (https://gfycat.com/PoshUnacceptableDogwoodtwigborer)

Edited clip (with "ripple effect" / delay_3frames):
HTML5 video: http://gfycat.com/EmptyUncommonBustard
GIF: http://zippy.gfycat.com/EmptyUncommonBustard.gif
http://zippy.gfycat.com/EmptyUncommonBustard.gif (https://gfycat.com/EmptyUncommonBustard)

[kuchikirukia]

Star Wars Wars: All 6 Films At Once

But there are only 3 Star Wars films.

[feisty2]

But there are only 3 Star Wars films.

6 if with prequel trilogy

[wonkey_monkey]

There are only 3 Star Wars films. :D

Anyway, back on topic, SparkTank, can I suggest that you try, after overlaying all the movies, overlaying the results over itself, but flipped horizontally? And then also flipped vertically, if you like. The result can be pretty trippy.

[duedel]

6 if with prequel trilogy

Whoosh ;)

[Sparktank]

flipped horizontally? And then also flipped vertically

:devil: I should do this to the animated How the Grinch Stole Christmas!
(There is only one of those!)

Since we're still in GD, I find with current releases, fan projects aside (of which there a millions of builds), there are Zero Star Wars.

Until Disney decides to create a DeSpecialized OT release, I only have the boxset because it was 60% off during the Christmas sales.
Though, PT has its visual qualities.

This whole thing can really help with new animated avatars (where supported).
And I think it's time to give my tumblr a nice revamp.
Tumblr people love this kind of stuff. Just add pretty hues for a border, throw in some glittery text and Twilight will live again.

[Gavino]

Function maxY (clip1, clip2)
{
...
Y = mt_merge (clip1.converttoy8 (), clip2.converttoy8 (), Mask)
U = mt_merge (clip1.utoy8 (), clip2.utoy8 (), Mask)
V = mt_merge (clip1.vtoy8 (), clip2.vtoy8 (), Mask)
YtoUV (U, V, Y)
}

The last four lines can be replaced by:
mt_merge (clip1, clip2, Mask, luma=true, U=3, V=3)

[Sparktank]

:cool::thanks:

Neat to see how things can work differently.
One day I just may learn something lol.