Automatic Colorization

[Macanudo]

Here is a paper on automatic colorization (http://www.cs.huji.ac.il/~yweiss/Colorization/). It includes links to the matlab code of the algorithms used. There are several examples of grayscale colorization, but I found the recoloring examples more interesting.

As I am not a programmer or developer, I don't know if this has any potential to be developed in Avisynth. I just thought some of you might find it interesting.

Additionally, I thought some of the techniques used may be of some value in the development of your own filters and plugins.:)

[Backwoods]

This is some pretty amazing stuff. I don't know how this can relate to AVISynth. Maybe have the DLL call upon 5 marked frames (BMPs on the HDD) and go from there?

[DeepDVD]

I'm an unprofessional, too. ;)
But in my experience being on this forum I think the algorithms and the matlab thing are very useful. It's like having all necessary parts to build a house. You just have to use the bricks and concrete to do the rest. The metaphor is a li'l bit blurry but I hope everyone can imagine what i wanna say ;)

I find it amazing how far general recoloring's developed in the last few decades. Remember the cheap Technicolor©-colorization of ancient hollywood films. :D

The method described and showed on the webside is able to do almost realistic recoloring, a dream every fan of B/W-films dream of. ;)

Me as an Photoshop-fan it would be very interesting to see a plugin with that method. I pray for it. Old family-photos could be more realistic again.

Although, it seems you should be able to find the right colors you wanna fill an object with.
The film-part is a bit heavier as you can see among the picture examples. You have to choose the colors for (almost) every frame... I don't want to imagine how long it would take to recolor Casablanca. :D :D :D

[kururu]

Oh, I didn't know it was Matlab. I may have to take a look at it myself...

[trolltuning]

Is this something that would let us turn DV footage shot with too little light so there is no color into colored footage?

[Mug Funky]

it probably wouldn't be satisfactory on nightshot material - infra red borks things up a tad - black cloth comes out white, but black polished wood comes out black. people's clothes in general will come out far too light compared to everything else, and people will have "eyebeams" from the IR lamps.

very cool technology though. i don't know about using it to colourise old films - as far as i see it there's nothing to be gained artistically but plenty to be lost aesthetically if it comes out wrong. that is unless the process was supervised by the film's original director...

[trolltuning]

Generally if I have low lighting there's no infared or anything like that. I rarely play with the shutter speed as it seems to give a nasty strobe effect to the motion so I just settle for missing color. Some other clips I'd like to colorize almost all the participants are now deceased so there's no one to comlain if the colors are correct as long as they look reasonable...
Unfortunately I don't have access to a univerity system with matlab. I tried to look into purchasing it from Prentiss Hall, but couldn't find it on their site.

[kassandro]

Originally posted by Macanudo
Here is a paper on automatic colorization (http://www.cs.huji.ac.il/~yweiss/Colorization/).

This is not about automatic colorization nor do the authors claim that. Interactive assistance is always needed, whence this method is not suitable for Avisynth. If somebody claims that he can do automatic colorization, then he is truely a charlatan. Having said that, I do think that the approach of the three Israeli authors seems to be very promising.

[Mug Funky]

yeah. if this were combined with some of the efforts to plug avisynth into manual touch-up apps, this could become an extremely fast and relatively labour-light way to colourize footage.

what was that little program that allowed near-seamless frame painting with avisynth? LBkiller or something. there was also talk about integrating avs with CinePaint (film gimp).

[kassandro]

Originally posted by Mug Funky
yeah. if this were combined with some of the efforts to plug avisynth into manual touch-up apps, this could become an extremely fast and relatively labour-light way to colourize footage.

If you are serious, then this is the best you can hope for. Certainly many frame need not be checked by a human and the program should be smart enough to alert a human whenever neccessary. Nevertheless several days will still be necessary to colorize a movie, whence colorization will never make sense for indivduals.

[morsa]

I'm thinking that code will be really usefull to correct damaged color channels. (e.g. Sometimes VHS has lots of problem with UV when getting old)

[akupenguin]

Hmm... parametric colorization for very low bitrate video... like parametric stereo for audio?

[Soulhunter]

Originally posted by akupenguin
Hmm... parametric colorization for very low bitrate video... like parametric stereo for audio?

Would that be like "object based chroma subsampling" or so ???


Bye

[guada 2]

Apparently, this method only exists to succeed to this tranfert of coloration..
Decidedly, months can parade to see to appear an one outlines.

So, I applied it some filters avisynth, that will only improve the picture.

One cannot imagine a more astute scheme, like the partnership of the creator of codec and the pictorial expert.
Are not able to make a program of transition.
It is an idea, no.

[morsa]

Really difficult to understand your english, sorry...
(take note I'm Spanish native speaker)

[guada 2]

I am sorry Morsa, I would stain to make better.
I must leave to work
To later. ;)

[Kagura]

If the colourization algo is fast enough, one could have it colourize with every x pixels in chroma from the original and apply the same colours to regions of low luma difference and spread out until there's an abrupt change in luma.

Somehow, methinks this will work better in analog anime sources where areas of solid colour should be solid but aren't.

[Joel Cairo]

I thought I'd resurrect this thread because a) it's a feature I'd dearly love to see adapted to Avisynth, and b) it appears two guys at the Univ. of Minnesota have really made a quantum leap in the development, speed and simplicity of the process that was the original inspiration of the thread.

Their webpage is here:

http://mountains.ece.umn.edu/~liron/colorization/

Another closely related approach is detailed in the abstract available here:

http://www.cs.nott.ac.uk/~qiu/Colorization/ICIP2005Color-Final.pdf


Do we have any hope that someone can code this sort of thing for Avisynth?

Thanks for any thoughts, folks!

-Kevin

[Joel Cairo]

And, did anything come of **this** thread, which talks about (and evidently implements!) a different approach:

http://forum.doom9.org/showthread.php?t=91344&highlight=colorization

The source page they're referring to (with the released C++ code) is here:

http://www.ece.northwestern.edu/~mya671/Colorization.htm

-Kevin