Automated Colorization Script (Now ChromaJig)

[MWilson]

Edit: Updated - See Post # 157 (http://forum.doom9.org/showthread.php?p=1775733#post1775733)

I've written a script that I think is pretty interesting. I should clean this up, but I'm away from my normal computer and I'm very used to using AVSPMod.

Requirements are:
defreq
Masktools2


Function Cmx7Mod(clip c, Float "Sat",Bool "DownSz",Int "Pad") { # (c) MWilson, http://forum.doom9.org/showthread.php?t=173364
# Progressive Only : MUST be MOD 2 Width and Height, Otherwise will crop Mod 2
Sat = Default(Sat,0.33) # Default 0.33, 0.0 -> 1.0
DownSz = Default(DownSz,True) # Default true, True=Qwiker, False Precise
Pad = (Default(Pad,2)+1)/2*2 # Round up next multiple of 2, Avoid green border.
c = c.Crop(0,0,c.Width/2*2,c.Height/2*2).ConvertToYV12.GrayScale # Mod 2
smallc = (DownSz) ? c.sincresize((c.Width+3)/4*2,(c.Height+3)/4*2) : c # At least half size, Mod 2
Padc = (Pad>0) ? smallc.pointresize(smallc.Width+Pad*2,smallc.Height+Pad*2,-Pad,-Pad,smallc.Width+Pad*2,smallc.Height+Pad*2) : smallc
r = Padc.defreq(fy=0,fx=39,sharp=100,dx=50,dy=50,cutx=0,cuty=0).Crop(Pad,Pad,-Pad,-Pad)
r1 = Padc.defreq(fy=0,fx=47,sharp=100,dx=50,dy=50,cutx=0,cuty=0).Crop(Pad,Pad,-Pad,-Pad)
g = Padc.defreq(fy=0,fx=37,sharp=100,dx=50,dy=50,cutx=0,cuty=0).Crop(Pad,Pad,-Pad,-Pad)
g1 = Padc.defreq(fy=0,fx=85,sharp=100,dx=50,dy=50,cutx=0,cuty=0).Crop(Pad,Pad,-Pad,-Pad)
b = Padc.defreq(fy=0,fx=65,sharp=100,dx=50,dy=50,cutx=0,cuty=0).Crop(Pad,Pad,-Pad,-Pad)
b1 = Padc.defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0,fx2=100,fy2=0).Crop(Pad,Pad,-Pad,-Pad)
x1 = merge(r,r1)
y1 = merge(smallc.invert(),merge(g,g1))
z1 = merge(smallc.invert(),merge(b,b1))
y2 = overlay(smallc,y1,mode="multiply")
z2 = overlay(smallc,z1,mode="multiply")
x=mt_makediff(smallc,x1,Chroma="-128").converttorgb() # Full range RGB
y=mt_makediff(smallc,y2,Chroma="-128").converttorgb()
z=mt_makediff(smallc,z2,Chroma="-128").converttorgb()
mergergb(x,y,z).ConvertToYV12 # RGB to YV12 TV Levels
lanczos4resize(c.Width,c.Height)
mergechroma(c,ColorYUV(autowhite=true))
return (Sat>0.0) ? mergechroma(c,tweak(sat=7.5,interp=32),Sat) : Last
}

Thanks to StainlessS for the mod :)


The latest version:
function fc1(clip v,float weight)

{# (c) MWilson, http://forum.doom9.org/showthread.php?t=173364

v=v.greyscale().converttoyv12()
af=v.defreq(fy=-.29,fx=.29,sharp=0,dx=.50,dy=.50,fy2=-.88,fx2=.88,dx2=.50,dy2=.50)
av=v.defreq(fy=-29,fx=29,sharp=0,dx=50,dy=50,fy2=-88,fx2=88,dx2=50,dy2=50)
ax=v.defreq(fy2=-88,fx2=88,sharp=0,dx2=50,dy2=50,fy=-58,fx=58,dx=0,dy=50)
a=mt_makediff(merge(af,av),v).converttorgb().converttoyv12()
m1=ytouv(av,a).converttoyv12().sincresize(width(v),height(v))
y2=ytouv(m1.invert(),av).converttoyv12().sincresize(width(v),height(v))
merge(m1,y2,.35)
converttoyv12(last)
coloryuv(autowhite=true)
tweak(sat=5,maxsat=59.5,interp=20)
merge(v.invert().converttoyv12(),last.converttoyv12())
converttorgb()
sb=showblue(last).invert()
sr=showred(last).invert()
sg=showgreen(last).invert()
mergergb(sb,sr,sg)
ConvertToYV12()
sx=last.converttorgb().invert("G").converttoyv12()
sy=last.converttoyv12()
mergechroma(sy,sx,.50).converttoyv12()
merge(last,v.invert().converttoyv12()).invert("UV").swapuv()
tweak(sat=7.5,maxsat=59.5,interp=20)
mergechroma(v.converttoyv12(),last,weight)
coloryuv(autowhite=true)
converttorgb24()

return last

}



function fc2(clip v)

{# (c) MWilson, http://forum.doom9.org/showthread.php?t=173364

v=v.greyscale().converttoyv12()
merge(v.invert().fc1(1).converttoyv12().coloryuv(cont_u=-128,cont_v=-128),v.fc1(1).converttoyv12())
converttoyv12()
tweak(sat=5,interp=32)
mergechroma(v,last.awarpsharp2(depth=48,chroma=4),.333)

return last

}



function chromajig(clip y)

{# (c) MWilson, http://forum.doom9.org/showthread.php?t=173364

y=y.greyscale().converttoyv12()
y1=y.gaussresize(72,48)
y1
fc2()
#hqdn3d(0,28,0,4.5)
gaussresize(width(y),height(y))
subtract(y.fc2(),last)
mergeluma(y)
return last
}


function cjbx(clip v)

{# (c) MWilson, http://forum.doom9.org/showthread.php?t=173364

v=v.converttoyv12().greyscale()
v1=v.spline36resize(240,240)
y=v1.converttorgb()
y.pointresize(8,8)
invert()
pointresize(16,16)
chromajig()
blur(1,1)
invert()
swapuv()
#temporalsoften(10,0,20)
spline36resize(width(y),height(y))

a=last

y.pointresize(32,32)
chromajig()
blur(1,1)
invert()
spline36resize(width(y),height(y))

b=last

y.pointresize(180,180)
chromajig()
blur(1,1)
invert()
swapuv()
spline36resize(width(y),height(y))

c=last

subtract(a,b)
subtract(last,c)
invert()
spline36resize(width(y),height(y))
mergechroma(y.converttoyv12().greyscale(),last,.75)
coloryuv(autowhite=true)
spline36resize(width(v),height(v))
blur(1,1)
mergechroma(v,last)
return last
}



function colorfox(clip y)
{# (c) MWilson, http://forum.doom9.org/showthread.php?t=173364
y=y.greyscale().converttoyv12()
y.cjbx()
a=last
c1=a.converttorgb()

r=showred(c1)
g=showgreen(c1)
b=showblue(c1)

c=merge(r,merge(g,b).invert())
m=merge(g,merge(r,b).invert())
ye=merge(b,merge(r,g).invert())

mergergb(ye,m,c)

subtract(last,c1)
converttoyv12()
invert("UV")
subtract(a,last)
swapuv()
}


function colorjig(clip v, float "Weight")
{# (c) MWilson, http://forum.doom9.org/showthread.php?t=173364
Weight= Default(Weight,1)
v=v.greyscale().converttoyv12()
b=v.greyscale().colorfox().tweak(hue=90).invert("UV").coloryuv(autowhite=true).invert("V").converttoyv12()

merge(b,b.chromajig())
coloryuv(autowhite=true)
mergechroma(v,last,Weight)
}

function cfcf2(clip y)
{
y=y.greyscale().converttoyv12().invert().turnleft()
a=y.cjbx().utoy()
b=y.invert().TurnLeft().cjbx().TurnRight().invert().utoy()
ytouv(a,b,y)
mergechroma(y,last,.45)
converttoyv12()
c=last
subtract(c,y.cjbx())
invert("UV")
medianblur(0,10,10)
mergechroma(c,last,.33)
turnright()
invert("Y")
return last
}

:thanks:

Using the latest script, will update this post later.
http://i64.tinypic.com/8xlmvr.jpg

Edit: Updated to the latest version. A note: this is for colorization, don't expect miracles! Sorry if I was misleading in the beginning of this thread, it wasn't on purpose. I was confused as well!

Edit: Sorry folks, as davidhorman pointed out this needs to have a call to greyscale! Code updated.

Edit: Code Updated - See Posts #34 & #35

Edit: Updated - See Post #37

Edit: Updated - See Posts #66 & #69

Edit: Updated 4/18/16

Edit: Minor Changes

[StainlessS]

I'm not gonna pretend that I know how it does it, but it is rather intriguing.
I provided it with a grayscale StarWars (not necessary as it GrayScales it anyway), and I got out a golden C-3PO.
Just like magic :)

EDIT: This is quite amazing, I tried with Jurassic Park and did a StackVertical of original and result, and
the colors and color patterns on peoples shirts etc were 'sort of' the same, green trees etc.
I say again, "Just like Magic".
Would you care to elaborate upon how it does this (some kind of residual color stored in luma ?).

[StainlessS]

EDIT: The first two examples are misleading, I did not GrayScale beforehand and bug in script merged with original colored version.
The 3rd example was already Gray and so does not suffer from this problem.

Here we go, get a load of this (original on top, recolored GrayScale on bottom)


Not entirely successfull but "Worth waiting for". (from VCD B&W movie)
https://s20.postimg.cc/enzxknvct/Worth_Waiting_For_zpscvdf3qbs.png (https://postimg.cc/image/enzxknvcp/)

EDIT: Misleading images removed.

[johnmeyer]

Trying to understand this script reminds me of reading through some of Didée's genius code: I know it works, but I'll be darned if I can figure out how.

This one is beyond my pay grade.

[MWilson]

Sure! It's all dependent on the greyscale. The photographs are records of light. The light contains all the information for color, but splitting it into chunks is/was mainly trial and error. My HDD can attest to that!

Edit: johnmeyer, I wish....

[Sparktank]

Interesting indeed!

linked images:

Original (1080p bluray)
https://i.imgur.com/WuLA3eol.png (https://imgur.com/WuLA3eo)

cmfull
https://i.imgur.com/CSnQVe8l.png (https://imgur.com/CSnQVe8)

cmfullshift
https://i.imgur.com/DPun8Wel.png (https://imgur.com/DPun8We)

Greyscale().cmfullshift
https://i.imgur.com/kWU5Dtql.png (https://imgur.com/kWU5Dtq)

spline36resize(854,480).cmfullshift
for the good ol' SD look.
https://i.imgur.com/uDQgdXBl.png (https://imgur.com/uDQgdXB)

There were some other TV-appearance functions I played with before... combining them all, I could make a nostalgic conversion.
Make it look like a sdtv rip or something. Just need to make up a logo. :D:devil:

EDIT: I used RGtools instead of RemoveGrain v1.0b.

[StainlessS]

How on earth does the script know that the beer crate in back RHS of first image is blue ?
(pretty much everything on the main table is very near original color).

Searching Google on Automatic Colorization bring up quite a bit on using Neural Networks and teaching
software what color something 'could be', but this is entirely different.

A very big welcome to the forum MWilson, England Expects great things of you :)

[MWilson]

Thanks StainlessS!:eek: I'll try not to let England down!:D

[StainlessS]

Are you 'Doc W Mastering' ?

From YouTube: "Doctor Who Automatically Colorized Using Avisynth"
I hope you enjoy the video! No audio, as you can tell. I've been working on this for four years, and what you are seeing is the raw untouched output from the original greyscale video. Feedback is not only welcome but encouraged!
P.S. I am not affiliat

Here:- https://www.youtube.com/watch?v=idpNPicSzxU

I am not able to view above, just keep getting errors from YouTube (not just for this clip).

Google search which discovered above:- https://www.google.co.uk/?gfe_rd=cr&ei=KRd1VKG6N5HCVND3gGA&gws_rd=ssl#q=automatic+colorization+dr+who

EDIT: I thought I remembered something about recovering color from B&W Dr Who, some time ago. (think mentioned in a thread somewhere)

EDIT: Another one here:- https://www.youtube.com/watch?v=qvx69sJInP8

[MWilson]

Yep. I also tweet as analogmoe.

Edit: Check the comments on the first link...

[StainlessS]

Yep, saw the comment, also managed to Download the clip even if I cant view on-line.
Wondering if bbc has managed to recover that one.

Great job, well done.

[StainlessS]

It seems to favour Gold, (although the Daleks come out a little bit pinkish).
Even catches the gold rim on top of the Carlsberg glass in B&W movie clip.
I'm wondering how Shirley Eaton would turn out if turned to B&W (her big scene).

Could not resist, here she is :)
https://s20.postimg.cc/nx23us48t/Shirley0_zpshcip3byy.png (https://postimg.cc/image/8bksatsah/)

EDIT: I can see the BBC taking this up.

EDIT: Actually I dont think there is an identical color version of that frame, looks to have been done several
times where a little bit different setting.

[johnmeyer]

Wow, I just saw that movie on BBC America. How the heck can it know that her skin is gold and not flesh-colored??

This is FM (the M stands for magic; you can insert your own word for the F).

[johnmeyer]

StainlessS,

I won't be able to get to my editing computer until tomorrow. When you tested this code, were the colors in the resulting video fairly stable from frame-to-frame, or did they pulse, flicker, or change as the video played?

[edit] Never mind. I forgot to look at his Dr. Who video. That answers my question.

[feisty2]

I'm just curious how this thing works,
Say Luma=c1*R+c2*G+c3*B while Luma, c1, c2 and c3 are constants, and R, G, B are the unknowns
You simply cannot get a constant solution of RGB from the indeterminate equation above..

[MWilson]

It's not perfect, but as far as completely automated colorization, I think it works well.

Here's a quick test:


c=Blankclip().converttorgb().invert()## Quick white....
r=c.rgbadjust(b=0,g=0)
g=c.rgbadjust(r=0,b=0)
b=c.rgbadjust(g=0,r=0)

original=stackhorizontal(r,g,b).sincresize(256,256)
filter=original.greyscale().cmfullshift()
stackvertical(original.converttoyv12(),filter)


Bring the saturation up. Red and Blue look fairly close, green will show up orange-yellow.

[wonkey_monkey]

How on earth does the script know that the beer crate in back RHS of first image is blue ?
(pretty much everything on the main table is very near original color).

Hang on a minute...

StainlessS, did you pass the full colour version to cmfullshift?

function cmfullshift (clip v)
{

v=v.converttoyv12() # input clip - full colour?

cmfull(v)
u=last.utoy()
v2=last.vtoy()
ytouv(u,v2,v)
tweak(sat=3.5,interp=32)
coloryuv(autowhite=true)
u3=repair(v.lanczosresize(width(v)/2,height(v)/2),u)
v3=repair(v.lanczosresize(width(v)/2,height(v)/2),v2)
ytouv(u3,v3,v)
tweak(sat=7.5,interp=32)

mergechroma(v,last,.33) # 2/3 of original chroma is restored

coloryuv(autowhite=true)
return last
}

v (the input clip) never gets reassigned, so that last bolded line is just putting back 2/3 of the original chroma.

[MWilson]

^^^^^^^^
He's right :(

[wonkey_monkey]

^^^^^^^^
He's right :(

This would've made an awesome April Fool, though.

[MWilson]

Example: Left is untouched, Middle is cmfullshift with a greyscale call first, Right is without the greyscale call.

https://drive.google.com/open?id=0B72PmD8lKDzScVN1OFpURHhJOU0

https://drive.google.com/file/d/0B72PmD8lKDzSOXRXOG1PaTYtOUE/view?usp=sharing (https://drive.google.com/file/d/0B72PmD8lKDzSOXRXOG1PaTYtOUE/view?usp=sharing)

[MWilson]

This would've made an awesome April Fool, though.

The Doctor Who video should give a better idea of how this works.
Because it does work , albeit not as well as I was hoping. It wasn't a joke.

[Sparktank]

This is what the JP clip would look like greyscaled().

ImageSource("D:\Downloads\JP0_zpstys30smw(1).PNG")
Crop(0, 0, -0, -552) # crop out the previous result
Greyscale()
cmfullshift
https://i.imgur.com/ZHYT13j.png (https://imgur.com/ZHYT13j)

[wonkey_monkey]

The Doctor Who video should give a better idea of how this works.
Because it does work , albeit not as well as I was hoping. It wasn't a joke.

Oh, no, I didn't mean to suggest it was - only that a deliberately obfuscated function similar to that could've fooled some people.

It is pretty amazing how well it manages to separate objects with such a seemingly simple script.

[MWilson]

Here's an example from a greyscale input ran through cmfullshift and given a minute in colormill.
http://i1053.photobucket.com/albums/s461/Morgan_Dale_Wilson/dead%20alive%20recolor%20-%20colormill_zpsb2gckjqb.png (http://s1053.photobucket.com/user/Morgan_Dale_Wilson/media/dead%20alive%20recolor%20-%20colormill_zpsb2gckjqb.png.html)

And raw output from an old Sherlock Holmes episode.
http://i1053.photobucket.com/albums/s461/Morgan_Dale_Wilson/New%20File%209004326_zpsvczazmlq.png (http://s1053.photobucket.com/user/Morgan_Dale_Wilson/media/New%20File%209004326_zpsvczazmlq.png.html)
http://i1053.photobucket.com/albums/s461/Morgan_Dale_Wilson/New%20File%209011292_zpsdpole7my.png (http://s1053.photobucket.com/user/Morgan_Dale_Wilson/media/New%20File%209011292_zpsdpole7my.png.html)
http://i1053.photobucket.com/albums/s461/Morgan_Dale_Wilson/New%20File%209030429_zpsr7uhwauj.png (http://s1053.photobucket.com/user/Morgan_Dale_Wilson/media/New%20File%209030429_zpsr7uhwauj.png.html)

[johnmeyer]

This technology sure is intriguing. While every tool has limitations, and that is surely the case here, I nonetheless think this could work even better if I could better understand a little about how it works. In particular, if I could both tune the script parameters AND also modify my B&W source prior to modifying it with this script so that it would be more acceptable to the script, I think I could get better results.

For instance ...

Here is a before/after snap of some B&W football film I happened to have lying around from some work I did for a major sports film collector:

Before Colorization
http://i177.photobucket.com/albums/w208/johnmeyer/Test%20Original%20BampW_zpsjes3qrqh.jpg

After Colorization
http://i177.photobucket.com/albums/w208/johnmeyer/test%20Uncorrected%20Colorized_zpssnfhgbv7.jpg

And here is a short video clip of the colorized version:

Colorized B&W Football Film (https://youtu.be/l-6RWz8ZKpQ)

As you can see, the script does a pretty good job on the grass on the edge of the frame, and a credible job on the jerseys, but screws up on the grass in the middle of the frame. In looking at the original B&W (above), whatever exposure variation is causing this change in behavior is not obvious to me, but I have to think that if I could understand a little of the theory of operation, I could tune the script or prep the source to get the script to produce green across the entire width of the frame.

[StainlessS]

I would think that 8 bit Avisynth limitation would not help matters, wonder what a Vapoursynth mod would produce. (I can hear some scurrying around right now).

StainlessS, did you pass the full colour version to cmfullshift?
Mea Culpa, after the first test I did not bother with initial GreyScale.

The Shirley Eaton Golden Girl frame was already B&W, source here (about 2048x1nnn):- https://www.google.co.uk/search?q=shirley+eaton&biw=1280&bih=839&tbm=isch&imgil=8cXZYPqx62E4dM%253A%253B2LFWJRk_HmaIzM%253Bhttp%25253A%25252F%25252Fwww.topbondgirls.com%25252Fjill-masterson-shirley-eaton%25252F&source=iu&pf=m&fir=8cXZYPqx62E4dM%253A%252C2LFWJRk_HmaIzM%252C_&usg=__rxd6C8lu_EVwpk5s-8oevL44atc%3D&dpr=1&ved=0ahUKEwi4ifOz3PXLAhVD0hoKHWWfC7wQyjcIggE&ei=EcECV_i8HMOka-W-ruAL#imgrc=8cXZYPqx62E4dM%3A

The movie though was in color.

Shirley refused to be re-painted for the intro and exit credits, so is a different girl.

The script also needs amending for the resizing (width/2 --->>> width/4*2) or similar.
Might also check suitable size before conversion to YV12, eg Crop(0,0,Width/4*2,Height/4/*2).ConvertToYV12.GrayScale

EDIT: There is at least one frame in given link of the color version of B&W Golden Girl image.
EDIT: Actually dont think there is a color version of exact same golden girl frame, looks like they made the most
of that paint job, and took a number of shots in slightly different locations.

[Wilbert]

The idea is that greyscale is actually not luma sans any color, but luma and chroma together unfiltered.
Please expand, what exactly do you mean by this?

[StainlessS]

MW, I assume this is an error (I'm trying to mod your script a little just to make it clearer [for me] what its doing)


z1=merge(b,b1) # !!! CHANGED b1 to b !!! was z1=merge(b1,b1) which dont really make sense

[MWilson]

The idea is that we are viewing a record of the light captured at the time. Technicolor used prisms as well as color filters to separate the light into base components of color inside of the camera. My idea was that if this was possible, it might be possible to make a sort of "inverse" process from the recorded light in the greyscale.

Edit: StainlessS, I saw that just a minute ago. Yeah, it was a brain fart :P

[Groucho2004]

Mia Culpa
Is that your girlfriend? :D

Sorry, couldn't resist.

[StainlessS]

Nah, I like my girls good, ol' fashioned, and silver.

EDIT: This aint me, he's a cheap fake. https://www.google.co.uk/search?q=tin+man&biw=1280&bih=839&tbm=isch&imgil=POoTTehiqTlv_M%253A%253BD8RUg_IOnXU1PM%253Bhttp%25253A%25252F%25252Fhardcoversandheroines.com%25252F2014%25252F08%25252F25%25252Fmarry-date-or-dump-the-scarecrow-the-tin-man-the-lion%25252F&source=iu&pf=m&fir=POoTTehiqTlv_M%253A%252CD8RUg_IOnXU1PM%252C_&usg=__23CKP07jxidLzoT_hCyiHKnITfk%3D&ved=0ahUKEwixgObk7fXLAhXInBoKHUQIANIQyjcIKQ&ei=S9MCV_GuGci5asSQgJAN#imgrc=POoTTehiqTlv_M%3A

And oops, should be Mea Culpa (I know a girl named Mia [and yes, she is silver, we would make beautiful little tin cans]).

[johnmeyer]

Not that it is relevant to this discussion, but since StainlessS seems obsessed with Shirley Eaton and his old girlfriend Mia Culpa (not to be confused with her mother, Mamma Mia), I thought he would appreciate this link to the scene I still remember when I saw this film, first run, in a theater back in 1964:

"Jill Masterson" painted gold (https://www.youtube.com/watch?v=USD2Y7wRNgk)

As for this "color from nothing" script, it reminds me of something else I saw a few times, at about the same time as I saw that film (i.e., very early 1960s). It was a device to turn your B&W TV into a color television. It was a large piece of clear plastic, tinted blue at the top, green at the bottom, and flesh colored in the middle. It was intended to "colorize" outdoor scenes.

Instant Color TV (https://trulyskrumptious.wordpress.com/2012/11/29/instant-color-tv-screen/)

This script obviously does a lot better job, and may actually be useful for certain situations, but only if we can collectively figure out how to tweak it. I think intelligent tweaking is the key for moving forward: how do we put controls or "dials" on the front end so we can tune it?

Finally, just to provide that last bit of my knowledge about strange ways to get color from B&W, there is the Fechner effect where rapidly flashing B&W images can cause your brain to perceive color. There were even a few commercials that attempted to exploit this ("our product is so great that you may be seeing color right now on your black and white TV!").

[MWilson]

johnmeyer, I can't say I appreciate the likening:devil:

But, what you said about people getting together and finding an intelligent solution is exactly the kind of thing I was hoping for.:)

[StainlessS]

This is what I've got so far (can you check it over).
Also, need some kind of direction on how to treat crop/resize/colorspace, there have to be some rules.


Function cmfull2(clip v2) { # (c) MWilson, http://forum.doom9.org/showthread.php?t=173364
v2 = v2.grayscale()
BordW = v2.width /4*2 # Must be even, Avoid using Width and Height as variables (misleading)
BordH = v2.height/4*2
v=v2.sincresize(256,256).addborders(BordW,BordH,BordW,BordH)

r =v.defreq(fy=0,fx=39,sharp=100,dx=50,dy=50,cutx=0,cuty=0) # DeFreq YV12 and YUY2, Plane default 0 = Y
r1=v.defreq(fy=0,fx=47,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g =v.defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g1=v.defreq(fy=0,fx=51,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b =v.defreq(fy=0,fx=65,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b1=v.defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0,fx2=100,fy2=0)

x1=merge(r,r1)
y1=merge(g,g1)
z1=merge(b,b1) # !!! CHANGED b1 to b !!! was z1=merge(b1,b1)

z2=overlay(v,z1,mode="multiply") # x & y default 0
y2=overlay(v,y1,mode="multiply")

# x=mt_makediff(v,x1,Chroma="128").converttorgb().crop(BordW,BordH,-BordW,-BordH) # Chroma planes = 128 (same as Grayscale), Else Garbage
# y=mt_makediff(v,y2,Chroma="128").converttorgb().crop(BordW,BordH,-BordW,-BordH)
# z=mt_makediff(v,z2,Chroma="128").converttorgb().crop(BordW,BordH,-BordW,-BordH)
# mergergb(x,y,z)

x=mt_makediff(v,x1) # Garbage in U and V
y=mt_makediff(v,y2)
z=mt_makediff(v,z2)
mergergb(x,y,z) # Merge Luma planes to RGB
crop(BordW,BordH,-BordW,-BordH) # Remove borders

lanczos4resize(v2.Width,v2.Height)
converttoyv12()
ColorYUV(autowhite=true)
# tweak(sat=1.0) # What is this doing ?, Sat defaults 1.0, only affect Coring=True.
# ax=last.sincresize(v2.Width,v2.Height) # I think this may have NO EFFECT other than ax=Last (ie is already v2.width,v2.height)
ax=Last
bx=mergechroma(v2,ax,1.0)
coloryuv(bx,autowhite=true)
#Blank = merge(last,last.invert("YUV")) # YUV=$808080
#Blank = last.BlankClip(Pixel_type="YV12",Color_YUV=$808080) # Does nothing
#mergechroma(Blank,last,1.0) # Does nothing (result chroma is from last, below luma discarded)
mergechroma(v2,last,1.0)
return last
}



EDIT: We seem to be missing BLUE.

EDIT: "Jill Masterson Painted Gold", This Video Is Not Available, Sorry about that.

[Gavino]

Thanks, MWilson.
I've been trying this out on some old B&W family photos and the results, though clearly not perfect, are quite amazing.

I've no idea how the method works, but there seems to be something wrong in the script:

function cmfullshift (clip v)
{
v=v.greyscale().converttoyv12()
cmfull(v)
u=last.utoy()
v2=last.vtoy()
ytouv(u,v2,v)
tweak(sat=3.5,interp=32)
coloryuv(autowhite=true)
u3=repair(v.lanczosresize(width(v)/2,height(v)/2),u)
v3=repair(v.lanczosresize(width(v)/2,height(v)/2),v2)
ytouv(u3,v3,v)
tweak(sat=7.5,interp=32)
mergechroma(v,last,.33)
coloryuv(autowhite=true)
return last
}

The lines in red have no effect, since their result is left in 'last', which is not used before being overwritten at the subsequent ytouv(u3,v3,v).
Perhaps it doesn't matter and these lines aren't really needed anyway, or is there something missing?

[MWilson]

StainlessS: Thanks, that's amazing detective work. I ran a greyscaled clip through, it's amazing what you can find with a second pair of eyes!:p

Gavino: Thank you, and yes those lines do absolutely nothing. Sorry for any confusion. Mine is enough to go around:p

[StainlessS]

Is cmfull() intended to be called on its own, or only through cmfullshift2 ?

EDIT: I've upped two frames here, intended so that we all have same test frames [EDIT: Added color frames].
http://www.mediafire.com/download/3gprxpkzmom1bqj/AutomaticColorization_Test.7z

Both already border cropped and GrayScaled. RGB24 BMP.

and test script for comparison

AutomaticColorization.avs

Function cmfull2(clip v2) { # (c) MWilson, http://forum.doom9.org/showthread.php?t=173364
# MUST be MOD 2 Width and Height
v2 = v2.ConvertToYV12.GrayScale
BordW = v2.width /4*2 # Must be even, Avoid using Width and Height as variables (misleading)
BordH = v2.height/4*2
v=v2.sincresize(256,256).addborders(BordW,BordH,BordW,BordH)
r =v.defreq(fy=0,fx=39,sharp=100,dx=50,dy=50,cutx=0,cuty=0) # DeFreq YV12 and YUY2, Plane default 0 = Y
r1=v.defreq(fy=0,fx=47,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g =v.defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g1=v.defreq(fy=0,fx=51,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b =v.defreq(fy=0,fx=65,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b1=v.defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0,fx2=100,fy2=0)
x1=merge(r,r1)
y1=merge(g,g1)
z1=merge(b,b1)
y2=overlay(v,y1,mode="multiply") # x & y default 0
z2=overlay(v,z1,mode="multiply")
# Went back to this, Need Expand range YUV TV -> RGB Full range, and then convert back to YV12 TV levels.
# (previous mod missed out TV -> full range convert)
C="-128"
x=mt_makediff(v,x1,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH) # Chroma planes = -(-128) (same as Grayscale), Else Garbage
y=mt_makediff(v,y2,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH)
z=mt_makediff(v,z2,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH)
mergergb(x,y,z).ConvertToYV12 # RGB to YV12 TV Levels
lanczos4resize(v2.Width,v2.Height)
ColorYUV(autowhite=true)
mergechroma(v2,last,1.0)
return last
}

Function cmfullshift2 (clip v) {
# MUST be MOD 2 Width and Height
v=v.ConvertToYV12.GrayScale
cmfull2(v)
u =last.utoy()
v2=last.vtoy()
u3=repair(v.lanczosresize(width(v)/2,height(v)/2),u)
v3=repair(v.lanczosresize(width(v)/2,height(v)/2),v2)
ytouv(u3,v3,v)
tweak(sat=7.5,interp=32)
mergechroma(v,last,0.33)
coloryuv(autowhite=true)
return last
}



Mod of original script, use for comparison of future changes (pretty much identical to above script)

org_cm.avs

Function cmfull(clip v2) {
# MUST be MOD 2 Width and Height
v2 = v2.ConvertToYV12.GrayScale
BordW = v2.width /4*2 # Must be even, Avoid using Width and Height as variables (misleading)
BordH = v2.height/4*2
v=v2.sincresize(256,256).addborders(BordW,BordH,BordW,BordH)
r =v.defreq(fy=0,fx=39,sharp=100,dx=50,dy=50,cutx=0,cuty=0) # DeFreq YV12 and YUY2, Plane default 0 = Y
r1=v.defreq(fy=0,fx=47,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g =v.defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g1=v.defreq(fy=0,fx=51,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b =v.defreq(fy=0,fx=65,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b1=v.defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0,fx2=100,fy2=0)
x1=merge(r,r1)
y1=merge(g,g1)
z1=merge(b,b1)
y2=overlay(v,y1,mode="multiply") # x & y default 0
z2=overlay(v,z1,mode="multiply")
# Went back to this, Need Expand range YUV TV -> RGB Full range, and then convert back to YV12 TV levels.
# (previous mod missed out TV -> full range convert)
C="-128"
x=mt_makediff(v,x1,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH) # Chroma planes = -(-128) (same as Grayscale), Else Garbage
y=mt_makediff(v,y2,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH)
z=mt_makediff(v,z2,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH)
mergergb(x,y,z).ConvertToYV12 # RGB to YV12 TV Levels
lanczos4resize(v2.Width,v2.Height)
ColorYUV(autowhite=true)
# bx=mergechroma(v2,Last,1.0) # AutoWhite twice, second time does nothing
# coloryuv(bx,autowhite=true)
mergechroma(v2,last,1.0)
return last
}

Function cmfullshift (clip v) {
# MUST be MOD 2 Width and Height
v=v.ConvertToYV12.GrayScale
cmfull(v)
u =last.utoy()
v2=last.vtoy()
u3=repair(v.lanczosresize(width(v)/2,height(v)/2),u)
v3=repair(v.lanczosresize(width(v)/2,height(v)/2),v2)
ytouv(u3,v3,v)
tweak(sat=7.5,interp=32)
mergechroma(v,last,0.33)
coloryuv(autowhite=true)
return last
}


AutomaticColorization_Client.avs

Import("org_cm.avs") # Original script (with a few small mods)
Import("AutomaticColorization.avs") # Mod


Imagesource("DQ-Tools_Color.BMP",end=0).convertToYV12
#Imagesource("JP_Test_Color.BMP",end=0).convertToYV12
Org=Last # Display Color

AMP=true

GrayScale
#Org=Last # Display GrayScale

A=cmfullshift
B=cmfullshift2

D=ClipDelta(B,A,amp=AMP)
L=StackVertical(ORG,D)
R=StackVertical(A,B)
Return StackHorizontal(L,R)

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}


Delta frame shows difference between original script and supplied, if eg more red then mod is more red.

EDIT: We dont seem to be getting any blues at all. Cyan in Delta frame would indicate Lack Of Red rather than Too Much Blue.

EDIT: Added color BMP's to the 7z

Updated

[SSH4]

https://richzhang.github.io/colorization/

[MWilson]

SSH4: That paper honestly freaked me out. But from what I could tell it used neural networks, which are way above my head.

[StainlessS]

Post #37 updated and new Original script added (Identical to current, can then compare any changes with present state).

Had a few hiccoughs, but hopefully back with the program.

[StainlessS]

I've just put the first Doctor clip through the mod, and to all intents and purposes came out exactly the same as YouTube version
(taking into account that it was encoded to MP4 between re-colorings)


Import("AutomaticColorization.avs")

AviSource("Doctor.avi")
Org=Last
GrayScale
cmfullshift2
D1=ClipDelta(Last,Org,False)
D2=ClipDelta(Last,Org,True)
T=StackHorizontal(ORG ,D1)
B=StackHorizontal(Last,D2)
Return StackVertical(T,B)

# Return Clip Difference of input clips (amp==true = Amplified, show==true = show background)
Function ClipDelta(clip clip1,clip clip2,bool "amp",bool "show") {
amp=Default(amp,false)
show=Default(show,false)
c2=clip1.levels(128-32,1.0,128+32,128-32,128+32).greyscale()
c1=clip1.subtract(clip2)
c1=(amp)?c1.levels(127,1.0,129,0,255):c1
return (show)?c1.Merge(c2):c1
}


EDIT: Anybody got any Laurel & Hardy, Buster Keaton or similar to try out ?
(they never did anything but extract a grimace out of me, but might be interesting to see some real old footage colorized).
Or maybe the guy I used to see at Saturday morning matinee, said Shazam a lot (Captain Marvel or something like that),
he was in B&W I think.

And just to clarify for John Meyer, the girls in Ice Cold In Alex and Gold Finger are two different lassies, Sylvia Syms and Shirley Eaton (JFYI).

EDIT: I'm just trying with the 2nd YouTube example (Test 1) and its coming out more pink that the original Yellow cast clip.
Was anything else done to that clip ?

EDIT: After the initial scene in photo studio (which I think looked better on YouTube) the remaining scenes all looked better with current
script (or original supplied script, not quite so yellow).
Obviously quite ancient footage, thank you :)
Now all we gotta do is squeeze some blue out of them there clips.

[MWilson]

https://jordibatllecaminal.files.wordpress.com/2014/12/harold-lloyd.jpg

The first clip put on youtube was completely inaccurate.
I've got several scripts based on this and other ideas, most are bad, but there are a few that I like.

Example:

function wcf(clip v,float weight)
{
v=v.greyscale().converttoyv12()
af=v.defreq(fy=-.29,fx=.29,sharp=0,dx=.50,dy=.50,fy2=-.88,fx2=.88,dx2=.50,dy2=.50)
av=v.defreq(fy=-29,fx=29,sharp=0,dx=50,dy=50,fy2=-88,fx2=88,dx2=50,dy2=50)
ax=v.defreq(fy2=-88,fx2=88,sharp=0,dx2=50,dy2=50,fy=-58,fx=58,dx=0,dy=50)
a=mt_makediff(merge(af,av),v).converttorgb().converttoyv12()
m1=ytouv(av,a).converttoyv12().sincresize(width(v),height(v))
y2=ytouv(m1.invert(),av).converttoyv12().sincresize(width(v),height(v))
merge(m1,y2,.35)
converttoyv12(last)
coloryuv(autowhite=true)
tweak(sat=5,maxsat=59.5,interp=20)
merge(v.invert().converttoyv12(),last.converttoyv12())
converttorgb()
sb=showblue(last).invert()
sr=showred(last).invert()
sg=showgreen(last).invert()
mergergb(sb,sr,sg)
ConvertToYV12()
sx=last.converttorgb().invert("G").converttoyv12()
sy=last.converttoyv12()
mergechroma(sy,sx,.50).converttoyv12()
merge(last,v.invert().converttoyv12()).invert("UV").swapuv()
tweak(sat=7.5,maxsat=59.5,interp=20)
mergechroma(v.converttoyv12(),last,weight)
coloryuv(autowhite=true)
return last
}

function fc(clip v)
{
v=v.converttoyv12()
merge(v.invert().wcf(1).converttoyv12().coloryuv(cont_u=-128,cont_v=-128),v.wcf(1).converttoyv12().medianblur(0,12,12))
converttoyv12()
tweak(sat=5,interp=32)
mergechroma(v,last,.333)
return last
}

Note: This is something I wrote quite a while ago. It may look alright depending on what you're going for. And yes, redundancies abound.

[StainlessS]

You really are quite a magician.


ImageSource("Harold-Lloyd.jpg",end=0)
Crop(0,0,Width/4*4,Height/4*4)
wcf(1.0)
Return ConvertToRGB32


This squeezes out quite a bit of blue, if only we could get a better compromise :)

Wish I could understand that DeFreq thing, I've no idea at all how to use it (ignorance is seldom bliss).

Well think its time for bed 08:40AM.
bye.

EDIT: Can I make a suggestion, when you next up a clip to YouTube, why not do a side by side demo, B&W left and mod on right
much more impressive to see the result. You might even want to replace current clips if you are able to delete already uploaded clips.
I love the "Fathers Present" scene, looks great (the current script version not youtube upload).
EDIT: Maybe they resize to some standard size which would mess up your demo, perhaps a stack four with explanation
as to why it was necessary. (I've never uploaded anything to YouTube)

EDIT: Top Youtube source for GrayScale, bottom current script recolored from GrayScale
https://s20.postimg.org/6xt5fit19/Fathers_Present_2_zpsznpvsbgm.png (https://postimg.org/image/clzg6exdl/)

EDIT: Removed black border

And to get David Horman a little fired up, here a couple more
https://s20.postimg.cc/8qw23ue7x/Doctor_zpsrvligw9k.png (https://postimg.cc/image/x7e7ybeyh/)

and his nemesis (a herd of pinkish Daleks)

https://s20.postimg.cc/uel0eaem5/Daleks_zpsnzw7elpb.png (https://postimg.cc/image/urcekgwvt/)

EDIT: There is a chroma shift in the Doctor frame, we need to establish some rules for cropping I think, also same for Green edges.

[MWilson]

https://archive.org/details/Cmfull2Test

Quick test with the mod.

As for green edges (as well as other small noise),try medianblur with a small radius.;)

Edit: Tested a basic merge of cmfull2 and fc.

http://i1053.photobucket.com/albums/s461/Morgan_Dale_Wilson/BK%2022000027_zpsbjuqg9lb.png (http://s1053.photobucket.com/user/Morgan_Dale_Wilson/media/BK%2022000027_zpsbjuqg9lb.png.html)

http://i1053.photobucket.com/albums/s461/Morgan_Dale_Wilson/BK%2022001940_zpsrndjnfnk.png (http://s1053.photobucket.com/user/Morgan_Dale_Wilson/media/BK%2022001940_zpsrndjnfnk.png.html)

[johnmeyer]

Anybody got any Laurel & Hardy, Buster Keaton or similar to try out ? (they never did anything but extract a grimace out of me, but might be interesting to see some real old footage colorized). Or maybe the guy I used to see at Saturday morning matinee, said Shazam a lot (Captain Marvel or something like that), he was in B&W I think.I have a huge collection of both amateur and stock movie B&W footage from 1928 onward:

John Meyer Film Transfers (https://www.youtube.com/playlist?list=PL230DED9513ECA2CB)

I would be happy to contribute any of these to the cause, especially if we can get blue to appear (as it does in the examples in one of the links posted that goes to a site where they describe a similar process), and if we can figure out why most of the results posted so far (including my own) seem to have green at the edges and purple in the center.

[MWilson]

function cmfull3(clip v2)
{
v2 = v2.ConvertToYV12.GrayScale
BordW = v2.width /4*2 # Must be even, Avoid using Width and Height as variables (misleading)
BordH = v2.height/4*2
v=v2.sincresize(256,256).addborders(BordW,BordH,BordW,BordH)
r =v.defreq(fy=0,fx=44,sharp=100,dx=50,dy=50,cutx=0,cuty=0) # DeFreq YV12 and YUY2, Plane default 0 = Y
r1=v.defreq(fy=0,fx=47,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g =v.invert().defreq(fy=0,fx=59,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
g1=v.defreq(fy=0,fx=51,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b =v.invert().defreq(fy=0,fx=74,sharp=100,dx=50,dy=50,cutx=0,cuty=0)
b1=v.defreq(fy=0,fx=68,sharp=100,dx=50,dy=50,cutx=0,cuty=0,fx2=100,fy2=0) ##Almost entirely trial and error
x1=merge(r,r1)
y1=merge(g,g1)
z1=merge(b,b1)
y2=overlay(v,y1,mode="multiply") # x & y default 0 ##Overlay was originally used to cover artifacts
z2=overlay(v,z1,mode="multiply")
# Went back to this, Need Expand range YUV TV -> RGB Full range, and then convert back to YV12 TV levels.
# (previous mod missed out TV -> full range convert)
C="-128"
x=mt_makediff(v,x1,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH) # Chroma planes = -(-128) (same as Grayscale), Else Garbage
y=mt_makediff(v,y2,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH)
z=mt_makediff(v,z2,Chroma=C).converttorgb().crop(BordW,BordH,-BordW,-BordH) ##Try to make a difference signal
mergergb(x,y,z).ConvertToYV12 # RGB to YV12 TV Levels
lanczos4resize(v2.Width,v2.Height)
ColorYUV(autowhite=true)
tweak(sat=7.5,interp=32)
mergechroma(v2,last,.33)
last
}

Some modifications.

Edit:This may be silly. But if it helps it's worth it right?

http://i1053.photobucket.com/albums/s461/Morgan_Dale_Wilson/New%20File%205004600_zpsdhdicl9z.png (http://s1053.photobucket.com/user/Morgan_Dale_Wilson/media/New%20File%205004600_zpsdhdicl9z.png.html)

Script Used:

function cmfullx (clip v) ###Originally written for a different script a while back
{
v=v.converttoyv12()
cmfull2(v)
u=last.utoy()
v2=last.vtoy()
u4=mt_merge(u,u.invert(),v.lanczosresize(width(v)/2,height(v)/2)) ##Merge the positive and negative values according to the mask
v4=mt_merge(v2,v2.invert(),v.lanczosresize(width(v)/2,height(v)/2)) ### Was looking at U and V channels of full color images to try to "match"
ytouv(u4,v4,v)
tweak(sat=7.5)
mergechroma(v,last,.75)
return last
}
merge(cmfull3(v),cmfullx(v)) ###Quick and dirty combination
medianblur(0,3,3) ### Cleaning up small noise

Edit:Removed image.

[StainlessS]

Would be good if you can in some way reveal the secrets that you have uncovered,
Me dont know if by trial an error or some kind of physical basis,
but to have the clever guys assist there has to be some kind of direction.
Request that you in-depth add as many comments on "EVERYTHING" in current script.
Give the lads some clue as to how your magic works. I am unlikely to be of further assistance,
me just too stupid. I can clean up and stuff like that but for the big-brain guys to get involved,
then they gotta have some idea what they are doin.

Really.

EDIT: If you dont know how your stuff works, then just give as much as you know or understand it.

EDIT: Perhaps Fizick could give his take on this, would love to hear how he interpret results.

[MWilson]

https://en.wikipedia.org/wiki/Visible_spectrum

The idea was that light could be separated by frequency.

By the way, I would also like to hear Fizick's interpretation.

:thanks:

[pwnsweet]

I have a short clip (1mb) (https://mega.nz/#!ZZ5HCQYZ!vQZE31rhcPIfgEEaUDMp6sE68DE3pMgJEpQXM282x5k) (below, left) where I would like to change the color of the eyes into darker shade like the image below. Could this script be used for something like this? (ignore the contrast and brightness change).

http://i68.tinypic.com/2yzms8y.jpghttp://i64.tinypic.com/2h2fjgo.png

[MWilson]

pwnsweet, I'm sorry, this was written trying to get semi-natural color from black and white. But FWIW, the shirt and hair color are close... :o