"The Matrix" Look [while keeping skin tones]

[joshbm]

I was wondering in AviSynth how I could make the green tones come out like the Matrix screenshot below:

[before] (http://www.videocdpro.com/josh/matrix01.jpg)

[desired look] (http://www.videocdpro.com/josh/matrix02.jpg)

Regards,
joshbm

[Wilbert]

Making the green tones is not the problem. Keeping the tones of the face is the problem, at least for my attempts :)

[Soulhunter]

Not really what you looking for, but...

Here is a "matrix style" filter Manao wrote for me!

function rec_add_horizontal_bar(clip c, int bar_height)
{
h = height(c)

bar = c.blankclip(height = bar_height)
full_bar = bar.yv12lut(y=-255,u=-255,v=-255)
null_bar = bar.yv12lut(y=-0,u=-0,v=-0)
double_bar = stackvertical(full_bar, null_bar)

result = h >= 2 * bar_height ? double_bar : double_bar.crop(0,0,0,h-2*bar_height)

return h <= 2 * bar_height ? result : stackvertical(result, rec_add_horizontal_bar(c.crop(0,0,0,-2*bar_height), bar_height))
}


function matrix_style1(clip c, int bar_height, float strength, float strength_u, float strength_v)
{
bar_height = default(bar_height, 2)
strength = default(strength, 1.0)
u_strength = default(strength_u, 1.0)
v_strength = default(strength_v, 1.0)

str = string(strength)
str_u = string(u_strength)
str_v = string(v_strength)

deviation_str = "256 x 128 - 128 / abs " + str + " ^ 256 * -"

deviation_u = "x 128 - y * " + str_u + " * 256 / 128 +"
deviation_v = "x 128 - y * " + str_v + " * 256 / 128 +"

deviation = c.yv12lut(yexpr = deviation_str, y = 3, u = 1, v = 1).lanczosresize(width(c) / 2, height(c) / 2)
deviationuv = ytouv(deviation, deviation)
light_bar = yv12lutxy(c, deviationuv, uexpr = deviation_u, vexpr = deviation_v, y = 2, u = 3, v = 3)

overshoot = c.yv12lut(yexpr = "x 255 / 1.1 ^ 255 *", u = -128, v = -128)
dark_bar = yv12lutxy(overshoot, deviationuv, uexpr = deviation_u, vexpr = deviation_v, y = 2, u = 3, v = 3)

mask = c.rec_add_horizontal_bar(bar_height)

return maskedmerge(light_bar, dark_bar, mask, y=3, u=3, v=3)
}


function matrix_style2(clip c, int bar_height, float strength, float overshoot)
{
bar_height = default(bar_height, 2)
strength = default(strength, 1.0)
overshoot = default(overshoot, 1.2)

str = string(strength)
ovr = string(overshoot)

deviation_str = "256 x 128 - 128 / abs " + str + " ^ 256 * -"

deviationuv = c.yv12lut(uexpr = "x 128 - abs 2 *", vexpr = "x 128 - abs 2 * ", y=1, u=3, v=3)
deviationuv = logic(deviationuv, deviationuv.swapuv(), mode = "max", y = 1, u = 3, v = 3)
deviation = c.yv12lut(yexpr = deviation_str, y = 3, u = 1, v = 1).lanczosresize(width(c) / 2, height(c) / 2)
deviationuv2 = ytouv(deviation, deviation)
resultuv = yv12lutxy(deviationuv, deviationuv2, uexpr = "128 x y * 512 / -", vexpr = "128 x y * 512 / -", y = 1, u = 3, v = 3)

light_bar = mergechroma(c, resultuv, 1)
overshootc = c.yv12lut(yexpr = "x 255 / " + ovr + " ^ 255 *", u = -128, v = -128)
dark_bar = mergechroma(overshootc, resultuv, 1)

mask = c.rec_add_horizontal_bar(bar_height)

return maskedmerge(light_bar, dark_bar, mask, y=3, u=3, v=3)
}
Bye

[joshbm]

Thanks Soulhunter, I may use that in a couple scenes,

Any suggestions on what filters/plugins to use? Because I REALLY need this, a client is asking for a video with a Matrix look to it and I'm in dire need of a method to do so.

Maybe, Wilbert, you could share your attempts and possibly come up with a solution to somewhat satisfy. And maybe someone has an idea of how we can keep the skins tones.

Regards,
joshbm

[joshbm]

What do you think of this?

Before (http://www.videocdpro.com/josh/matrix01.jpg) and After (http://www.videocdpro.com/josh/matrix03.jpg)

I think that my skin tones are so red that it looks fine when the filter is applied. Any other input?

Regards,
joshbm

[joshbm]

Anyone know how to get the Matrix look while preserving skin tones and other colors?

Regards,
joshbm

[brabbudu]

I think the problem of "colorizing" to green is due to the colorisation of high saturation colors. We can try to shift only the colors near to gray to mantain the normal appearence of skin etc.
I go try this ..

[Wilbert]

Ok, here's my attempt:

the one of yesterday:
http://www.geocities.com/wilbertdijkhof/mat_green.jpg

the one of today:
http://www.geocities.com/wilbertdijkhof/mat_final.jpg

I will post the scripts later today, since i've to eat first :)

But i like the attempt of soulhunter more (but he should increase the green a bit) :)

[Gurm]

I personally think you should ramp down the skin tones BEFORE applying the green matrix-y filter, but that's my taste.

[joshbm]

That's what I did in my go at it.

http://www.videocdpro.com/josh/matrix03.jpg

Regards,
joshbm

[mg262]

@joshbm

Your attempt looks pretty good to me. I was going to suggest trying to mask out the stuff that you couldn't risk changing the colour of too much (especially skin) with a *gradated* mask and then changing the colour of the rest. I think that's equivalent to merging the original with a 'greenified version' using a weight that varies from pixel to pixel. (That may be what @Soulhunter has done... I'm not terribly good at reading scripts.) I'm not very good with complex scripts, or I'd try writing something like that myself...

The other thing I thought of is that it might be worth taking a scene from the matrix and throwing it into @trevlac's vectorscope to get an idea of exactly what colours it is made up of.

[Soulhunter]

...but he should increase the green a bit :)
Well, you have to tweak/edit it to match the source...

Source:
http://img290.imageshack.us/img290/5323/8445orig4ij.png

Tweaked Filtering:
http://img290.imageshack.us/img290/152/8445filter2tv.png

Filtering + Subtitles + Tweaking:
http://img267.imageshack.us/img267/8469/8445filter8oq.png

Bye

[joshbm]

Any luck with your new filter brabbudu?

Regards,
joshbm

[Wilbert]

@Soulhunter,

There's must be a chroma bug in one those filters somewhere (with those ugly horizontal lines as a result).

[Backwoods]

@Soulhunter,

There's must be a chroma bug in one those filters somewhere (with those ugly horizontal lines as a result).

Aren't they supposed to be there? Either way I'd like to know how to get rid of it myself. The result look very good.

EDIT: damn these eyes!

[mg262]

Either way I'd like to know how to get rid of myself.Not until you've explained EDLs to me, you don't. :p

mergechroma( Blur(0,1) )

might do the fix the chroma weirdness. Or you could convert to YV12!

[Soulhunter]

@ Wilbert

Nah, have a look @ the "rec_add_horizontal_bar" function... XD

It simulates this "scanline effect" from the intro of Matrix Reloaded !!!

http://img35.imageshack.us/img35/4576/02ld.png

http://img35.imageshack.us/img35/3084/15rf.png

Bye

[brabbudu]

Here i am!

This is my result :

matrix0.JPG

i've applied this formula to Green:

green=green(1+(1-sqrt(sat/(lum+1))))

hence the most colorized areas will be those with low saturation and low luminance

[joshbm]

A mod anyone? I wanna see the pic :rolleyes:.

[joshbm]

Oh Mod? Come out, come out where ever you are.

[Guest]

You are always welcome to PM me if there is a pending attachment. I don't always visit the mod panel as often as I visit the forum, and sometimes I miss the thread.

[joshbm]

@brabbudu:

What is the AviSynth code I would use for that? I think I'll enhance my code with yours.

Regards,
joshbm

[Bester]

Perhaps you could play with this:


a=avisource("blabla")
b=overlay(a,a,mode="multiply",opacity=0.25)

overlay(c,a.RGBAdjust(0, 1, 0, 1),mode="chroma",opacity=0.15)

It works for me with your pic.

[Wilbert]

@Soulhunter and others,

I'd like to have some help with rec_add_horizontal_bar. I'm using the following script


function rec_add_horizontal_bar(clip c)
{
h = height(c)
bar = c.blankclip(height = 2)
full_bar = bar.yv12lut(y=-255,u=-255,v=-255)
null_bar = bar.yv12lut(y=-0,u=-0,v=-0)
double_bar = stackvertical(full_bar, null_bar)

result = h >= 4 ? double_bar : double_bar.crop(0,0,0,h-4)

return h <= 4 ? result : stackvertical(result, rec_add_horizontal_bar(c.crop(0,0,0,-4)))
}


LoadPlugin("F:\CompilingPlugins\ColorLooks\Release\ColorLooks.dll")

ImageSource("F:\CompilingPlugins\ColorLooks\girl.jpg") # 384x288
ConvertToYV12()
rec_add_horizontal_bar()

VDub and WMP6.4 just closes. yv12lut works, i've checked that.

Any idea why it doesn't work?

[mg262]

LoadPlugin("F:\CompilingPlugins\ColorLooks\Release\ColorLooks.dll")
Ummm... I know this is going to turn out to be a stupid question, but what is this thing + what function is it serving in the script?

(I just tried to look at the docs for ImageSource and they don't seem to be there (v2.55)... I think my computer is trying to me to go back to sleep ;-) )

[Soulhunter]

@ Wilbert

So, you just wanna add the scanlines without changing the color?

Try -> Matrix_Style1(2,1,1,1).Mergechroma(YourSource)


Bye

[Wilbert]

Ummm... I know this is going to turn out to be a stupid question, but what is this thing + what function is it serving in the script?
It's reduntant. I fogot to delete that line. If its documentation is ready, you will see what it is :)

So, you just wanna add the scanlines without changing the color?
Yup.

Try -> Matrix_Style1(2,1,1,1).Mergechroma(YourSource)
Same problem of course. VDub just quits when loading the script.

Soulhunter, can you just grab the script above and try whether it should work?

[mg262]

If it is any use, with avs 2.5.5, Masktools 1.5.6, VirtualDubMod 1.5.10.1, that script doesn't cause a crash for me. (I get a screen full of horizontal bars.) Also works in the avsedit preview.

[Wilbert]

Yes, it doesn't crash with v2.55. v2.56b3 is the first version it crashes.

[Soulhunter]

Yeah, I still use v 2.55...

In MPC and VDubMod it works, just MPlayer2 crashes!


Bye

[IanB]

Wilbert,

Yes, Somebody asked for an increase in the number of arguments to a call, so I bumped it from 60 to 1024. Now every call to Invoke pushes 1024 AVSValues (16Kb) onto the stack instead of just 60 (0.94kb). Recursive scripts like this one will fall out of the sky pretty quickly with a stack overflow. I guess I gotta fix this.

IanB

[Backwoods]

There is a scanlines filter if you want to try that out.

[IanB]

Okay it's fixed. But man is this a slow way of doing this. I couldn't believe how long the GetFrame() chain generated by such a simple script was, I almost started chasing it as a bug.

A very much faster way is to use nested SeparateField()'s and Weave()'s, something like thish=height(c)
h4=(h+3)/4*4
AddBorders(0, 0, 0, h4-h) # mod 4 height
AssumeFrameBased().SeparateFields()
AssumeFrameBased().SeparateFields()
L04=SelectEvery(4, 0) # Lines 0,4,8,...
L26=SelectEvery(4, 1) # Lines 2,6,10,...
L15=SelectEvery(4, 2) # Lines 1,5,9,...
L37=SelectEvery(4, 3) # Lines 3,7,11,...

# Do whatever to each line group
# maybe Weave(L04, L15) and Weave(L26, L37)

Interleave(L04, L26, L15, L37)
AssumeFieldBased().Weave()
AssumeFieldBased().Weave()
Crop(0,0,0,h) # Put back to original heightSeeing your playing with YV12 you probably need to process the chroma separately, but the ideas is there. Remember SeparateFields() and SelectEvery() are zero cost filters and Weave() costs a blit.

IanB

[mg262]

Okay it's fixed. But man is this a slow way of doing this. I couldn't believe how long the GetFrame() chain generated by such a simple script was, I almost started chasing it as a bug.Height/4 GetFrame calls, I think... if the 16kB bit is still true, it would mean 4kB*Height being pushed onto the stack -- typically 1-2MB per frame!

Would you mind if I ask whether 1024 AVSValues are still being pushed onto the stack for every GetFrame call? It makes other things noticeably more expensive* which is worth knowing about...

*I ought to justify that ... for example in part of an unreleased plugin I use small-dimension clips to simulate streams of Boolean values (i.e. one Boolean value per frame) or of floats or of some compound structures (or of pretty much anything I feel like), which lets you do most things that ScriptClip/ConditionalFilter is used for but with nicer syntax. But that requires throwing around quite a lot of GetFrames.

[IanB]

@mg262,

Fortunatly it doesn't work like this. The 16kb+ was the stack frame size of Invoke. Invoke is called for every expression element while the script is being "compiled", i.e. for calling the constructor of each filter, it is not called as part of the GetFrame chain. The GetFrame chain renders your video frames. Normally the length of the chain is approximately twice the number of filter statements, of course recursive scripts like Wilberts dramatically increase this. 100's of statements is okay, 1000's start to be a problem.

Now invoke has about a 70 byte stack frame and keeps only the amount currently required, not the maximum, on the heap.

IanB

[mg262]

@IanB,

Brilliant. Thank you for all your hard work.

[brabbudu]

@joshbm

Sorry but these weeks i was busy :rolleyes:
I will put very soon this filter and related source code in my SF page.