Automatic Colorization - Tutorial (Windows 10 edition)

color · 10th March 2018, 15:00

LAST UPDATED: 2019-12-11 - Updated the install script and to the newer version on Ubuntu Shell in Windows and added new images. Step 2 has a little bit more to do than before. If you get error try:

Code:

$ sudo apt --fix-broken install python-pycurl python-apt

EDIT: There might be different between different countries(?) wierdly. Stick with the "Ubuntu" version (not the 18.04 or 16.04)

Only works with 64-bit version on Windows 10.

This works great on scenes that are filmed outdoors, some indoors works great as well mostly skin colors but some backgrounds indoors may be colorized a bit wrong. Cartoons does sadly not work at all, it has problem to regonize drawings. If you want to colorize cartoons i recomend Chromajig, CFCF2 or HaldCUT

1.) First off all, see that your computer has the latest updates, then go in to Control Panel > Programs > Turn Windows Features On Or Off. Enable the"Windows Subsystem for Linux" and click "OK". Then restart your computer (important step)

Then go in to Microsoft Store and install Ubuntu. Then open Ubuntu in the start menu. You will get the step to write a nickname and password

2.)Before installing, you need to update ubuntu to the latest package.

Code:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade
$ grep -r lucid /etc/apt/sources.list*

3.) INSTALLING TORCH

Code:

$ git clone https://github.com/torch/distro.git ~/torch --recursive
$ cd ~/torch; bash install-deps;
$ ./install.sh
$ source ~/.bashrc
$ ./clean.sh
$ luarocks install image
$ luarocks list
$ sudo apt install torch-repl

(It will take a while to install, and you will need to write your password a few times and answer "yes" for some installed)

then test it:

Code:

$ th

It should look like this:

4.) INSTALL LET Let There Be Color!
Download here. Extract the files in a folder called "color" in the download folder.

Code:

$  cd /mnt/c/Users/(YOUR USERNAME)/Downloads/color
$  ./download_model.sh

change to your username "(YOUR USERNAME)". It is sensitive with capital letters.

Now this is done its time to get the files to colorize.

5.) Open a video in virtualdub, change the resolution so the width is 220px or 226px and export it as image sequence (put in the "Minimum number of digits in" so many numbers that is in the movie, if its 3400 frames you need to write 4, this is so you easy can transfer the colorized images if you need to stop and start another time). in the color folder you extracted the files in. Also see so its PNG, that will colorize the images faster.

6.) SCRIPT TO CONVERT ALL IMAGES

Code:

$ cd /mnt/c/Users/(YOUR USERNAME)/Downloads/color
$ for file in * ; do th colorize.lua "$file" "${file/./-color.}" ; done

It takes hours and hours. but it will be done.

Here is one before and after image:

As you see here there is an error, its no problem, that is couse its trying to colorize all files in the folder, and it can only colorize images files not anything else.

7.) When it has colorized the images you need to do a avisynthscript to merge the color (overlay) on the high quality greyscale image.

I don't get a good script for how to use the image directly (like with ImageSource()), so I import the first image "*-colorized.png", change the framerate so it matches the original (important) and export it.

Now its time to merge the colorized video over the original grayscale video. Use the script that is best for you, I use:

Code:

B=DirectShowSource("original_video.mp4")
C=DirectShowSource("colorized_video.avi")
Overlay(B, C.BicubicResize(B.Width, B.Height), mode="chroma")

If you want to do a bigger quality 1080p or higher, I suggest you to upscale the colorized photo to 720p (use Lanczose3) and use warpsharp (Depth in about 114 is good, do not change the Blur passes), otherwise there will be some color bleeding.

A sample, klick on the image to watch it on youtube. (I don't know how to post so you can see the video directly here)

I'm don't know realy how to write the script with the imagesource. import the images in avisynth and export it with the right framerate. But if you know how please help me with the script:

Code:

B=DirectShowSource("high quality movie.mkv")
C=ImageSource("00000.jpg")
Overlay(B, C.BicubicResize(B.Width, B.Height), mode="chroma")

color · 10th March 2018, 15:42

I will update with more images soon.

StainlessS · 10th March 2018, 18:02

Quote:

I'm don't know realy how to write the script with the imagesource. import the images in avisynth and export it with the right framerate. But if you know how please help me with the script:

Not sure if something like this is all that you want. (untested)

Code:

B=DirectShowSource("high quality movie.mkv")
FC=B.FrameCount # Assuming same number of frames
C=ImageSource("00000.jpg",End=FC-1)
Overlay(B, C.BicubicResize(B.Width, B.Height), mode="chroma") 
AssumeFPS(B.FrameRate,True)
# ReSampleAudio(B.audioRate)

# Export in VirtualDubfilterMod

PS, DirectShowSource is rather frowned upon as being Frame Inaccurate.

EDIT: Added in blue.

color · 10th March 2018, 19:51

I will try it out tomorrow, thank you for the script. What is better to use instead of directshowsource? I want to make the best tutorial.

Btw, i know there is some spelling error. I will fix it as soon as possible.

Mounir · 10th March 2018, 21:15

Lsmash is best imo

johnmeyer · 10th March 2018, 22:01

If I want to do this, can I just boot from an Ubuntu CD/DVD? That's how I've been running Ubuntu up until now.

color · 10th March 2018, 23:11

Mounir, I will check that out.

johnmeyer. Yes it work with booting from Ubuntu CD/DVD or USB, but you need to do a "sudo apt-get update" so you get the latest packages in terminal first. It does take a bit longer in "pure install" of Ubuntu wierdly. I will do a tutorial in ubuntu also. But its almost the same, just start from the 3.) and that you are in the root folder when you install torch.

I did have some problem in Ubuntu, so after I installed torch i did:

./clean.sh #before the code "th" to see that its correctly installed. (on Ubuntu, no need in Windows Ubuntu install)

johnmeyer · 11th March 2018, 02:06

I will be very interested to see what people can do with this. I don't have time to play with it right now. StainlessS and others spent a lot of time on the goofy colorization code that was posted here a couple of years ago. It was tantalizing, but if one was honest, it really didn't work very well.

I just looked at the samples on the download page you linked to, and they look incredibly good. This would be a big deal for me if I could actually colorize some of the work I do for clients.

poisondeathray · 11th March 2018, 02:17

Has anyone used larger than 224px width images successfully ? If so can you post some more examples? The "thumbnails" look decent, but I'm curious if it's actually usable for decent image sizes

Quote:

While the model works on any size image, we trained it on 224x224 pixel images and thus it works best on small images. Note that you can process a small imageto obtain the chrominance map and then rescale it and combine it with the original grayscale image for higher quality.

color · 11th March 2018, 09:59

poisondeathray, yes you can do bigger than 224px, but it takes longer time to let it colorize the photo. For be the smaler one takes 6 seconds per image in that small size. I used 1280px it tooked more than a minute. The best way is to use 220px or 224 px that it is trained for. I do merge the color only over the original high quality picture and it look great, I'm colorizing a 1080p movie and it tooks great.

I will post some examples soon. I have not been home so mutch lately. I think it will be up under this day or latest tomorrow.

color · 11th March 2018, 10:03

Quote:

Originally Posted by johnmeyer

I will be very interested to see what people can do with this. I don't have time to play with it right now. StainlessS and others spent a lot of time on the goofy colorization code that was posted here a couple of years ago. It was tantalizing, but if one was honest, it really didn't work very well.

I just looked at the samples on the download page you linked to, and they look incredibly good. This would be a big deal for me if I could actually colorize some of the work I do for clients.

Yes it does an amazing work. There is one problem tho', It works better with outdoor images. Inside does work but it has sometimes need to adjust the brightness and contrast.

Also there might be some flickering, I do use CNR2 in virtualdub and a deflicker filter, I haven't tried the CNR-filter for avisynth yet. But I guess that might be faster and work a bit better. It does give you more controll.

EDIT: Just added a link to a sample, the original video is found here and is legal to use, change and publish without any permissions.

color · 11th March 2018, 13:12

I do want to put out the limitations, as you can see when the cows get blurry, the script recognize them as gras so they get a little bit green. But CNR2 will fix that

Gser · 11th March 2018, 13:17

What's this whole install linux on windows business?

Yanak · 11th March 2018, 13:22

Nice tutorial, i might give it a try with a live CD as not having win10, never tried to play a bit with linux but might be the occasion to try when i'll have a lot of free time to burn.

This said i had used this in a form of online service some years ago, had forgot about this until seeing this thread : https://colorize.dev.kaisou.misosi.ru/ ( pics resized to fit 800px max on the largest side )

examples where it does a nice job :
https://colorize.dev.kaisou.misosi.r...1-691bb96ddfdb
https://colorize.dev.kaisou.misosi.r...c-12ffbe23b23d
https://colorize.dev.kaisou.misosi.r...c-80764b2a52e0

For some footage it's really impressive, outdoors where it was trained especially, for some others it used to give some things not really better than the colorize script i saw a few times in the avisynth section of the forum.

Thanks to "color" for reminding me this exists, one could only wish it was easier and directly available on win without the linux steps

color · 11th March 2018, 15:05

Gser, its needed for the colorization. The "program" is made for linux.

Yanak, thank you. What windows version do you have? Its a good page, but they have made a bit approvment after that page you liked to, they have an officiell online colorization here, it is mutch better than colorize.dev.kaisou that uses the older trained file. Yes I wish it was easier, but to install the ubuntu terminal in windows isn't that hard and works great. It does work with cygwin also (works with older windows versions), but there is some other scripts like "sudo apt-get" does not work its something else for that command, its kind of hard to install. There is win-bash also, but I have not tried it. I don't know about Windows 7, it might have the option also with latest update to in "Windows Subsystem for Linux" I don't use 7, 8 or 8.1 so I can not check.

StainlessS · 11th March 2018, 15:14

You can also get a live image linux on USB flash stick with added persistence, which can be used to store settings/updates (Read/Write partition).

EDIT: on Google:- https://www.google.co.uk/search?sour....0.FwIvKMcaA6Y

Yanak · 11th March 2018, 15:26

I'm on win7 and there isn't anything like this sadly, so a usb stick or live CD are probably the only options, something i will need to try and explore one of those days.

Thanks for the link on them page, had not used this one before, but looks like they resize more compared to the other allowing 800px, will need ot compare both results and see if it is really improved, especially when there isn't much landscape where it seems to be quite good..

Well i'm not into linux stuff at all but will have to try this one of those days, what StainlessS said seems like a good option, especially curious about higher resolutions and compare original grayscale input with the tool does on output.

Thank again.

color · 11th March 2018, 15:29

Outdoor it looks the same. But there is better with humans in photos indoors. It gives more different colors than all similar as the other one. But it does also sometimes help to adjust the brightness and contrast. But yes it has some limitations.

StainlessS · 11th March 2018, 15:43

Further to links for live linux USB persistence, (from one of the links on posted google results), and those wanting persistence greater than usual
limitation of 4GB, see here (I found it of particular interest wanting to install on 64GB USB3 with full drive usage):- https://askubuntu.com/questions/3974...-more-than-4gb

smok3 · 12th March 2018, 06:44

Quick test (using web service) on some cgi (1. orig, 2. gimp grayscaled, 3. ai colorized)

https://cdn.scrot.moe/images/2018/03...Icolorized.jpg

p.s. Getting this error in Debian 9

Code:

 th colorize.lua ~/Downloads/vintageRocket11d_gimp2.png 
/home/b/source/torch/install/bin/luajit: cannot open <colornet.t7> in mode r  at /home/b/source/torch/pkg/torch/lib/TH/THDiskFile.c:673
stack traceback:
	[C]: at 0x7f7ea2f4a3a0
	[C]: in function 'DiskFile'
	/home/b/source/torch/install/share/lua/5.1/torch/File.lua:405: in function 'load'
	colorize.lua:24: in main chunk
	[C]: in function 'dofile'
	...urce/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
	[C]: at 0x5621a1c52610

10th March 2018, 15:00	#1 \| Link
color Registered User Join Date: May 2016 Posts: 235	Automatic Colorization - Tutorial (Windows 10 edition) LAST UPDATED: 2019-12-11 - Updated the install script and to the newer version on Ubuntu Shell in Windows and added new images. Step 2 has a little bit more to do than before. If you get error try: Code: $ sudo apt --fix-broken install python-pycurl python-apt EDIT: There might be different between different countries(?) wierdly. Stick with the "Ubuntu" version (not the 18.04 or 16.04) Only works with 64-bit version on Windows 10. This works great on scenes that are filmed outdoors, some indoors works great as well mostly skin colors but some backgrounds indoors may be colorized a bit wrong. Cartoons does sadly not work at all, it has problem to regonize drawings. If you want to colorize cartoons i recomend Chromajig, CFCF2 or HaldCUT 1.) First off all, see that your computer has the latest updates, then go in to Control Panel > Programs > Turn Windows Features On Or Off. Enable the"Windows Subsystem for Linux" and click "OK". Then restart your computer (important step) Then go in to Microsoft Store and install Ubuntu. Then open Ubuntu in the start menu. You will get the step to write a nickname and password 2.)Before installing, you need to update ubuntu to the latest package. Code: $ sudo apt-get update $ sudo apt-get upgrade $ sudo apt-get dist-upgrade $ grep -r lucid /etc/apt/sources.list* 3.) INSTALLING TORCH Code: $ git clone https://github.com/torch/distro.git ~/torch --recursive $ cd ~/torch; bash install-deps; $ ./install.sh $ source ~/.bashrc $ ./clean.sh $ luarocks install image $ luarocks list $ sudo apt install torch-repl (It will take a while to install, and you will need to write your password a few times and answer "yes" for some installed) then test it: Code: $ th It should look like this: 4.) INSTALL LET Let There Be Color! Download here. Extract the files in a folder called "color" in the download folder. Code: $ cd /mnt/c/Users/(YOUR USERNAME)/Downloads/color $ ./download_model.sh change to your username "(YOUR USERNAME)". It is sensitive with capital letters. Now this is done its time to get the files to colorize. 5.) Open a video in virtualdub, change the resolution so the width is 220px or 226px and export it as image sequence (put in the "Minimum number of digits in" so many numbers that is in the movie, if its 3400 frames you need to write 4, this is so you easy can transfer the colorized images if you need to stop and start another time). in the color folder you extracted the files in. Also see so its PNG, that will colorize the images faster. 6.) SCRIPT TO CONVERT ALL IMAGES Code: $ cd /mnt/c/Users/(YOUR USERNAME)/Downloads/color $ for file in * ; do th colorize.lua "$file" "${file/./-color.}" ; done It takes hours and hours. but it will be done. Here is one before and after image: As you see here there is an error, its no problem, that is couse its trying to colorize all files in the folder, and it can only colorize images files not anything else. 7.) When it has colorized the images you need to do a avisynthscript to merge the color (overlay) on the high quality greyscale image. I don't get a good script for how to use the image directly (like with ImageSource()), so I import the first image "-colorized.png", change the framerate so it matches the original (important) and export it. Now its time to merge the colorized video over the original grayscale video. Use the script that is best for you, I use: Code: B=DirectShowSource("original_video.mp4") C=DirectShowSource("colorized_video.avi") Overlay(B, C.BicubicResize(B.Width, B.Height), mode="chroma") If you want to do a bigger quality 1080p or higher, I suggest you to upscale the colorized photo to 720p (use Lanczose3) and use warpsharp (Depth* in about 114 is good, do not change the Blur passes), otherwise there will be some color bleeding. A sample, klick on the image to watch it on youtube. (I don't know how to post so you can see the video directly here) I'm don't know realy how to write the script with the imagesource. import the images in avisynth and export it with the right framerate. But if you know how please help me with the script: Code: B=DirectShowSource("high quality movie.mkv") C=ImageSource("00000.jpg") Overlay(B, C.BicubicResize(B.Width, B.Height), mode="chroma") __________________ Automatic Colorization Last edited by color; 11th December 2018 at 09:12. Reason: updated and added new video

10th March 2018, 15:42	#2 \| Link
color Registered User Join Date: May 2016 Posts: 235	I will update with more images soon. __________________ Automatic Colorization

10th March 2018, 19:51	#4 \| Link
color Registered User Join Date: May 2016 Posts: 235	I will try it out tomorrow, thank you for the script. What is better to use instead of directshowsource? I want to make the best tutorial. Btw, i know there is some spelling error. I will fix it as soon as possible. __________________ Automatic Colorization

10th March 2018, 23:11	#7 \| Link
color Registered User Join Date: May 2016 Posts: 235	Mounir, I will check that out. johnmeyer. Yes it work with booting from Ubuntu CD/DVD or USB, but you need to do a "sudo apt-get update" so you get the latest packages in terminal first. It does take a bit longer in "pure install" of Ubuntu wierdly. I will do a tutorial in ubuntu also. But its almost the same, just start from the 3.) and that you are in the root folder when you install torch. I did have some problem in Ubuntu, so after I installed torch i did: ./clean.sh #before the code "th" to see that its correctly installed. (on Ubuntu, no need in Windows Ubuntu install) __________________ Automatic Colorization

11th March 2018, 09:59	#10 \| Link
color Registered User Join Date: May 2016 Posts: 235	poisondeathray, yes you can do bigger than 224px, but it takes longer time to let it colorize the photo. For be the smaler one takes 6 seconds per image in that small size. I used 1280px it tooked more than a minute. The best way is to use 220px or 224 px that it is trained for. I do merge the color only over the original high quality picture and it look great, I'm colorizing a 1080p movie and it tooks great. I will post some examples soon. I have not been home so mutch lately. I think it will be up under this day or latest tomorrow. __________________ Automatic Colorization Last edited by color; 11th March 2018 at 10:19.

10th March 2018, 21:15	#5 \| Link
Mounir Registered User Join Date: Nov 2006 Posts: 773	Lsmash is best imo

10th March 2018, 22:01	#6 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	If I want to do this, can I just boot from an Ubuntu CD/DVD? That's how I've been running Ubuntu up until now.

11th March 2018, 02:06	#8 \| Link
johnmeyer Registered User Join Date: Feb 2002 Location: California Posts: 2,695	I will be very interested to see what people can do with this. I don't have time to play with it right now. StainlessS and others spent a lot of time on the goofy colorization code that was posted here a couple of years ago. It was tantalizing, but if one was honest, it really didn't work very well. I just looked at the samples on the download page you linked to, and they look incredibly good. This would be a big deal for me if I could actually colorize some of the work I do for clients.

11th March 2018, 13:12	#12 \| Link
color Registered User Join Date: May 2016 Posts: 235	I do want to put out the limitations, as you can see when the cows get blurry, the script recognize them as gras so they get a little bit green. But CNR2 will fix that __________________ Automatic Colorization

11th March 2018, 13:17	#13 \| Link
Gser Registered User Join Date: Apr 2008 Posts: 418	What's this whole install linux on windows business?

11th March 2018, 13:22	#14 \| Link
Yanak Registered User Join Date: Oct 2011 Posts: 275	Nice tutorial, i might give it a try with a live CD as not having win10, never tried to play a bit with linux but might be the occasion to try when i'll have a lot of free time to burn. This said i had used this in a form of online service some years ago, had forgot about this until seeing this thread : https://colorize.dev.kaisou.misosi.ru/ ( pics resized to fit 800px max on the largest side ) examples where it does a nice job : https://colorize.dev.kaisou.misosi.r...1-691bb96ddfdb https://colorize.dev.kaisou.misosi.r...c-12ffbe23b23d https://colorize.dev.kaisou.misosi.r...c-80764b2a52e0 For some footage it's really impressive, outdoors where it was trained especially, for some others it used to give some things not really better than the colorize script i saw a few times in the avisynth section of the forum. Thanks to "color" for reminding me this exists, one could only wish it was easier and directly available on win without the linux steps Last edited by Yanak; 11th March 2018 at 13:56.

11th March 2018, 15:05	#15 \| Link
color Registered User Join Date: May 2016 Posts: 235	Gser, its needed for the colorization. The "program" is made for linux. Yanak, thank you. What windows version do you have? Its a good page, but they have made a bit approvment after that page you liked to, they have an officiell online colorization here, it is mutch better than colorize.dev.kaisou that uses the older trained file. Yes I wish it was easier, but to install the ubuntu terminal in windows isn't that hard and works great. It does work with cygwin also (works with older windows versions), but there is some other scripts like "sudo apt-get" does not work its something else for that command, its kind of hard to install. There is win-bash also, but I have not tried it. I don't know about Windows 7, it might have the option also with latest update to in "Windows Subsystem for Linux" I don't use 7, 8 or 8.1 so I can not check. __________________ Automatic Colorization

11th March 2018, 15:14	#16 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	You can also get a live image linux on USB flash stick with added persistence, which can be used to store settings/updates (Read/Write partition). EDIT: on Google:- https://www.google.co.uk/search?sour....0.FwIvKMcaA6Y __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 11th March 2018 at 15:20.

11th March 2018, 15:26	#17 \| Link
Yanak Registered User Join Date: Oct 2011 Posts: 275	I'm on win7 and there isn't anything like this sadly, so a usb stick or live CD are probably the only options, something i will need to try and explore one of those days. Thanks for the link on them page, had not used this one before, but looks like they resize more compared to the other allowing 800px, will need ot compare both results and see if it is really improved, especially when there isn't much landscape where it seems to be quite good.. Well i'm not into linux stuff at all but will have to try this one of those days, what StainlessS said seems like a good option, especially curious about higher resolutions and compare original grayscale input with the tool does on output. Thank again.

11th March 2018, 15:29	#18 \| Link
color Registered User Join Date: May 2016 Posts: 235	Outdoor it looks the same. But there is better with humans in photos indoors. It gives more different colors than all similar as the other one. But it does also sometimes help to adjust the brightness and contrast. But yes it has some limitations. __________________ Automatic Colorization

11th March 2018, 15:43	#19 \| Link
StainlessS HeartlessS Usurer Join Date: Dec 2009 Location: Over the rainbow Posts: 10,980	Further to links for live linux USB persistence, (from one of the links on posted google results), and those wanting persistence greater than usual limitation of 4GB, see here (I found it of particular interest wanting to install on 64GB USB3 with full drive usage):- https://askubuntu.com/questions/3974...-more-than-4gb __________________ I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???