colorization code used as denoiser?

[Hero--]

can the method used here be used to 'normalize' larger areas
of 'noisy' color section be used to reduce noise by enforcing
uniform color areas?

http://www.cs.huji.ac.il/~yweiss/Colorization/


(also isn't it time there was a gui interface (with selecting masks etc) for avisynth? ie when will there be a 'mother' application where you can mark/select areas, easily split screen to view effects of different filters etc)

thanks,


/H

[E-Male]

interesting idea

also this colorizing algorithm alone looks pretty nice

i daubt i can port it, because i never used matlab
but i'll have a look


EDIT:
i had a look at it and daubt it'll be much use as a chroma-denoise
but CNRF uses a similar idea and should be the way to go in avisynth

but it would be really cool if this paper/matlab-code could be ported to avisynth

[HighInBC]

I would love to have an AVISynth filter that can do this! The 'recoloring' especially...

I am already pictureing strange alien landscapes!

[E-Male]

would be cool
but seems like it'll take a real pro to port this (if it's possible at all)

[HighInBC]

If instead of using the matlab code, one could use the math in the paper they released... The actual formula is rather simple. The paper describes the proccess in plain english and math.

The ability to selectively color, or recolorize would really boost the functionality of avisynth. I am afraid it is a bit beyond my programming abilities.

[E-Male]

rather simple??
the formula maybe is, but the whole process...

implementing it in c++ and making it run efficiently seems quite complex to me


EDIT1:

Now given a set of locations ri where the colors are specied
by the user u(ri) = ui; v(ri) = vi we minimize J(U); J(V) subject
to these constraints. Since the cost functions are quadratic and
the constraints are linear, this optimization problem yields a large,
sparse system of linear equations, which may be solved using a
number of standard methods.

EDIT2:
gues i spoke to soon
http://www.ece.northwestern.edu/~mya...lorization.htm
nice and small c++ colorizing tool


i've e-mailed the authors of both projects to see if they will publish the c++ code we need


EDIT3:
maybe one of our pros (if any reads this) can use this code already:
http://www.ece.northwestern.edu/~mya...inalReport.pdf

[E-Male]

we won't get any code or help for the method in the first post
the university is going to use it comercially
the people who wrote it can't do anything about it

[HighInBC]

Hmm.. Here is hoping... I would not care about speed for something like this, if it take 5 days to colorize a scene I don't mind, as I am making my own films and only have to do this sort of thing rarely.

Hmmm remember that green chick from original star trek? What if I made Picard green in all the TNG ep's ahaha.

[Mug Funky]

the university is going to use it comercially
the people who wrote it can't do anything about it

dammit, that annoys me.

a lot of universities (ie the one i went to and probably all of them, truth be told) make you sign an IP waiver before you are allowed to start the course - anything you submit can be used by the uni in a manner of their choosing.

and with the slump in government funding for education, more and more universities are having to pay their way - that means stealing their students' hard work and making money off it.

[edit]
a colouriser would be very cool though. i'm thinking fixing chroma dropouts in composite/component analog captures. with the last good frame combined with avisynth's existing motion-compensation functionality (MVtools), plus this handy colourisation stuff, there'd be no percievable difference from the original.

[E-Male]

well, there is those 2nd project (partially based on the other) :
http://www.ece.northwestern.edu/~mya...lorization.htm

nice small c++ program is there for dl :
http://www.ece.northwestern.edu/~mya...lorization.rar

and some core-source-code in the paper, too (for dl) :
http://www.ece.northwestern.edu/~mya...inalReport.pdf

and i'm still waiting for an e-mail reply from the author

if he doesn't reply we might be able to do it with the code from the paper (experienced programmers, i'm looking at you guys)

that OpenCV library is news to me, so it's somewhat cryptic to me



here is the code from the paper:

#define MAX_BLEND_COLOR 3 //Max number of color to be blended
#define MAX_DISTANCE 65535 //Max intrinsic distance of pixels
///////////////////////////////////////////////////////////////////////////////////////
// Pixels representation for blending colorization algorithm
///////////////////////////////////////////////////////////////////////////////////////
class CBlendPixel
{
int m_x; // x-coordinate of the pixel
int m_y; // y-coordinate of the pixel
COLORREF m_c[MAX_BLEND_COLOR]; // maintained chrominance
float m_d[MAX_BLEND_COLOR]; // maintained intrinsic distance
};
typedef set<CBlendPixel*> BlendPixelSet;

///////////////////////////////////////////////////////////////////////////////////////
// Colorization Algorithm
// Blending the color with the intrinsic distance as the weights
///////////////////////////////////////////////////////////////////////////////////////
class CBlendColorize : public CColorize
{
public:
BOOL Colorize(IplImage* pColorImage,const IplImage* pGreyImage, const IplImage* pLayerImage);
CBlendColorize();
virtual ~CBlendColorize();
protected:
void BlendColor();
void PropagateColor();
void GenerateColoredSet();
BOOL ModifyPixel(CBlendPixel& testPixel,const CBlendPixel & curPixel);
BOOL Internal8(int x,int y);
inline double WeightDistance(double d);
void AllocateArray();
void ReleaseArray();
int m_iColored;
BlendPixelSet m_ActiveSet;
CBlendPixel*** m_PixelArray;
};

// Iteratively propagate the active colored pixels to their adjacent pixels until the equilibrium
void CBlendColorize::PropagateColor()
{
CString str;
BlendPixelSet::iterator it =m_ActiveSet.begin();
BlendPixelSet::iterator it_temp;
while (!m_ActiveSet.empty())
{
it =m_ActiveSet.begin();
while (it!=m_ActiveSet.end())
{
CBlendPixel * pPixel = *it;
for(int y=pPixel->m_y-1;y<=pPixel->m_y+1;y++)
for(int x=pPixel->m_x-1;x<=pPixel->m_x+1;x++)
{
if (x==pPixel->m_x||y==pPixel->m_y)
if (x>=0&&x<=m_width-1&&y>=0&&y<=m_height-1)
{
CBlendPixel * testPixel = m_PixelArray[y][x];
if (ModifyPixel(*testPixel,*pPixel))
m_ActiveSet.insert(testPixel);
}
}
it_temp = it; it++;
m_ActiveSet.erase(it_temp);
}
}
}

//Modify the color list of curPixel according to testPixel
BOOL CBlendColorize::ModifyPixel(CBlendPixel& testPixel,const CBlendPixel & curPixel)
{
if (cvGetReal2D(m_pY,testPixel.m_y,testPixel.m_x)==1)
return FALSE;
double y1 = cvGetReal2D(m_pGreyImage,curPixel.m_y,curPixel.m_x);
double y2 = cvGetReal2D(m_pGreyImage,testPixel.m_y,testPixel.m_x);
double d = abs(y1-y2);
BOOL result = FALSE;
for(int i=0;i<MAX_BLEND_COLOR;i++)
{
if (curPixel.m_d[i]<MAX_DISTANCE)
{
int j=0;
while (((curPixel.m_d[i]+d)>=testPixel.m_d[j])&&(j<=MAX_BLEND_COLOR))
{
if (curPixel.m_c[i]==testPixel.m_c[j])
{
j=MAX_BLEND_COLOR;
break;
}
j++;
}
if (j<MAX_BLEND_COLOR)
{
if (curPixel.m_c[i]!=testPixel.m_c[j])
for(int k=MAX_BLEND_COLOR-1;k>j;k--)
{
if (testPixel.m_c[k-1]!=curPixel.m_c[i])
{
testPixel.m_c[k]=testPixel.m_c[k-1];
testPixel.m_d[k]=testPixel.m_d[k-1];
}
}
testPixel.m_c[j]=curPixel.m_c[i];
testPixel.m_d[j]=curPixel.m_d[i]+d;
result = TRUE;
}
else
break;
}
}
return result;
}

///////////////////////////////////////////////////////////////////////////////////
// Command Pattern to support Undo and Redo functions
///////////////////////////////////////////////////////////////////////////////////
class CCommand
{
public:
virtual BOOL Execute(IplImage *pImage,WPARAM wParam, LPARAM lParam);
virtual BOOL UnExecute(IplImage *pImage);
virtual BOOL ReExecute(IplImage* pImage);
CCommand();
virtual ~CCommand();
protected:
BOOL m_bExecuted;
};



samples:

[guada 2]

Very interesting, if one wishes to restore old photographs.

But somebody has it to try to remove the color of an image (VDM+BMP+Flitre MSU OLD CINEMA) to treat it with this process?

It would be interesting to make the comparison of the 2 images:
the original with its own colors, and the image treated with absence of color.

[E-Male]

>>Very interesting, if one wishes to restore old photographs.

-try the sample program


>>But somebody has it to try to remove the color of an image (VDM+BMP+Flitre MSU OLD CINEMA) to treat it with this process?

-no, at least not in the example program
i guess in most implementations the colors of the picture would just be ignored


>>It would be interesting to make the comparison of the 2 images:
the original with its own colors, and the image treated with absence of color.

- http://www.ece.northwestern.edu/~mya...lorization.htm
(i just didn't link those to keep the post from getting to wide)

[guada 2]

Thank you, E-Male.
However, you will excuse me I had not opened your first link.

[HighInBC]

ok, here is a side effect that I just realized.

If you colorize only a bouncing ball in a movie with blue, then took that blue and turned it to white and everything else the black then you would have automatic mask detection for moving objects... If anyone has ever done any mask related special effects they will surely see the advantage of this.

The program seems to be able to track objects that are moving, while the camera is moving, ontop of a moving background.

This is very hard to mask normally, and ussually must be done by hand frame by frame.

[E-Male]

so you colorize the ball frame per frame and use for masking
doesn't seem to help much IMO
(in case my irony is bad as always: this method for now is for stills only, we'll have to add video automation ourselves, maybe include mvtools funcionality, but let's do still colorizing first)

I think Hero-- or a mod should change the thread titls, in order to have the coding pros look in here

else i'll have to try to make sense of that code, which won't be pretty

[HighInBC]

Originally posted by E-Male
so you colorize the ball frame per frame and use for masking
doesn't seem to help much IMO
(in case my irony is bad as always: this method for now is for stills only, we'll have to add video automation ourselves, maybe include mvtools funcionality, but let's do still colorizing first)

I think Hero-- or a mod should change the thread titls, in order to have the coding pros look in here

else i'll have to try to make sense of that code, which won't be pretty

If you look lower on the page(where it says 'Video Clip Colorization Examples'): http://www.cs.huji.ac.il/~yweiss/Colorization/

You will see that the formula does traverse space and time, needing only occasional updates as it loses the shape.

[E-Male]

if you look higher in this thread you will see that we can't use that algo, or at least won't get any help doing it

EDIT: sorry for my tone, i was in a bad mood when i replied (just got up)

i think we should first focus on getting code for still-colorisation working, when we got that we can expand in any direction we want

[insanedesio]

Sounds interesting to me. I wouldn't mind coding this, especially since I've been looking for something to code. As it stands, though, I'm new to AVS development, or any kind of coding that has actual applications, for that matter :-\. Given that I'm swamped in finals right now I don't have the patience to think about that paper, but after finals hopefully I should have more thought to give it... I read it through and thought about it a bit and can see how to implement most of it.. just missing the main part I think. If I get a chance I'll ask some people that I know who might be able to help.

[E-Male]

nice

i've been doing some more research on the topic and found some more papers and algos
i'll post a summary with all links here after i'm sure which paper is based on which others and after i've mailed all authors and waited some days

then i'll also make a new thread to attract some more experienced programmers from this board
together we should be able to make open-source colorization possible

[E-Male]

got e-mail reply no. 2 (of 3)
again university patenting
so only one left to put hopes on
if it's an other 'no' we have to do the coding from scratch with the avaible information