Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th March 2005, 23:16   #1  |  Link
Hero--
Registered User
 
Join Date: Nov 2002
Posts: 8
colorization code used as denoiser?

can the method used here be used to 'normalize' larger areas
of 'noisy' color section be used to reduce noise by enforcing
uniform color areas?

http://www.cs.huji.ac.il/~yweiss/Colorization/


(also isn't it time there was a gui interface (with selecting masks etc) for avisynth? ie when will there be a 'mother' application where you can mark/select areas, easily split screen to view effects of different filters etc)

thanks,


/H
Hero-- is offline   Reply With Quote
Old 20th March 2005, 21:05   #2  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
interesting idea

also this colorizing algorithm alone looks pretty nice

i daubt i can port it, because i never used matlab
but i'll have a look


EDIT:
i had a look at it and daubt it'll be much use as a chroma-denoise
but CNRF uses a similar idea and should be the way to go in avisynth

but it would be really cool if this paper/matlab-code could be ported to avisynth

Last edited by E-Male; 20th March 2005 at 23:29.
E-Male is offline   Reply With Quote
Old 12th April 2005, 20:30   #3  |  Link
HighInBC
Registered User
 
HighInBC's Avatar
 
Join Date: Jan 2003
Location: Victoria, BC, Canada
Posts: 144
I would love to have an AVISynth filter that can do this! The 'recoloring' especially...

I am already pictureing strange alien landscapes!
HighInBC is offline   Reply With Quote
Old 13th April 2005, 05:25   #4  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
would be cool
but seems like it'll take a real pro to port this (if it's possible at all)
E-Male is offline   Reply With Quote
Old 13th April 2005, 15:29   #5  |  Link
HighInBC
Registered User
 
HighInBC's Avatar
 
Join Date: Jan 2003
Location: Victoria, BC, Canada
Posts: 144
If instead of using the matlab code, one could use the math in the paper they released... The actual formula is rather simple. The paper describes the proccess in plain english and math.

The ability to selectively color, or recolorize would really boost the functionality of avisynth. I am afraid it is a bit beyond my programming abilities.
HighInBC is offline   Reply With Quote
Old 13th April 2005, 15:34   #6  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
rather simple??
the formula maybe is, but the whole process...

implementing it in c++ and making it run efficiently seems quite complex to me


EDIT1:
Quote:
Now given a set of locations ri where the colors are specied
by the user u(ri) = ui; v(ri) = vi we minimize J(U); J(V) subject
to these constraints. Since the cost functions are quadratic and
the constraints are linear, this optimization problem yields a large,
sparse system of linear equations, which may be solved using a
number of standard methods.

EDIT2:
gues i spoke to soon
http://www.ece.northwestern.edu/~mya...lorization.htm
nice and small c++ colorizing tool


i've e-mailed the authors of both projects to see if they will publish the c++ code we need


EDIT3:
maybe one of our pros (if any reads this) can use this code already:
http://www.ece.northwestern.edu/~mya...inalReport.pdf

Last edited by E-Male; 13th April 2005 at 16:33.
E-Male is offline   Reply With Quote
Old 13th April 2005, 22:08   #7  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
we won't get any code or help for the method in the first post
the university is going to use it comercially
the people who wrote it can't do anything about it
E-Male is offline   Reply With Quote
Old 14th April 2005, 13:58   #8  |  Link
HighInBC
Registered User
 
HighInBC's Avatar
 
Join Date: Jan 2003
Location: Victoria, BC, Canada
Posts: 144
Hmm.. Here is hoping... I would not care about speed for something like this, if it take 5 days to colorize a scene I don't mind, as I am making my own films and only have to do this sort of thing rarely.

Hmmm remember that green chick from original star trek? What if I made Picard green in all the TNG ep's ahaha.
HighInBC is offline   Reply With Quote
Old 17th April 2005, 15:22   #9  |  Link
Mug Funky
interlace this!
 
Mug Funky's Avatar
 
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,547
Quote:
the university is going to use it comercially
the people who wrote it can't do anything about it
dammit, that annoys me.

a lot of universities (ie the one i went to and probably all of them, truth be told) make you sign an IP waiver before you are allowed to start the course - anything you submit can be used by the uni in a manner of their choosing.

and with the slump in government funding for education, more and more universities are having to pay their way - that means stealing their students' hard work and making money off it.

[edit]
a colouriser would be very cool though. i'm thinking fixing chroma dropouts in composite/component analog captures. with the last good frame combined with avisynth's existing motion-compensation functionality (MVtools), plus this handy colourisation stuff, there'd be no percievable difference from the original.
__________________
sucking the life out of your videos since 2004

Last edited by Mug Funky; 17th April 2005 at 15:24.
Mug Funky is offline   Reply With Quote
Old 17th April 2005, 15:35   #10  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
well, there is those 2nd project (partially based on the other) :
http://www.ece.northwestern.edu/~mya...lorization.htm

nice small c++ program is there for dl :
http://www.ece.northwestern.edu/~mya...lorization.rar

and some core-source-code in the paper, too (for dl) :
http://www.ece.northwestern.edu/~mya...inalReport.pdf

and i'm still waiting for an e-mail reply from the author

if he doesn't reply we might be able to do it with the code from the paper (experienced programmers, i'm looking at you guys)

that OpenCV library is news to me, so it's somewhat cryptic to me



here is the code from the paper:

#define MAX_BLEND_COLOR 3 //Max number of color to be blended
#define MAX_DISTANCE 65535 //Max intrinsic distance of pixels
///////////////////////////////////////////////////////////////////////////////////////
// Pixels representation for blending colorization algorithm
///////////////////////////////////////////////////////////////////////////////////////
class CBlendPixel
{
int m_x; // x-coordinate of the pixel
int m_y; // y-coordinate of the pixel
COLORREF m_c[MAX_BLEND_COLOR]; // maintained chrominance
float m_d[MAX_BLEND_COLOR]; // maintained intrinsic distance
};
typedef set<CBlendPixel*> BlendPixelSet;

///////////////////////////////////////////////////////////////////////////////////////
// Colorization Algorithm
// Blending the color with the intrinsic distance as the weights
///////////////////////////////////////////////////////////////////////////////////////
class CBlendColorize : public CColorize
{
public:
BOOL Colorize(IplImage* pColorImage,const IplImage* pGreyImage, const IplImage* pLayerImage);
CBlendColorize();
virtual ~CBlendColorize();
protected:
void BlendColor();
void PropagateColor();
void GenerateColoredSet();
BOOL ModifyPixel(CBlendPixel& testPixel,const CBlendPixel & curPixel);
BOOL Internal8(int x,int y);
inline double WeightDistance(double d);
void AllocateArray();
void ReleaseArray();
int m_iColored;
BlendPixelSet m_ActiveSet;
CBlendPixel*** m_PixelArray;
};

// Iteratively propagate the active colored pixels to their adjacent pixels until the equilibrium
void CBlendColorize::PropagateColor()
{
CString str;
BlendPixelSet::iterator it =m_ActiveSet.begin();
BlendPixelSet::iterator it_temp;
while (!m_ActiveSet.empty())
{
it =m_ActiveSet.begin();
while (it!=m_ActiveSet.end())
{
CBlendPixel * pPixel = *it;
for(int y=pPixel->m_y-1;y<=pPixel->m_y+1;y++)
for(int x=pPixel->m_x-1;x<=pPixel->m_x+1;x++)
{
if (x==pPixel->m_x||y==pPixel->m_y)
if (x>=0&&x<=m_width-1&&y>=0&&y<=m_height-1)
{
CBlendPixel * testPixel = m_PixelArray[y][x];
if (ModifyPixel(*testPixel,*pPixel))
m_ActiveSet.insert(testPixel);
}
}
it_temp = it; it++;
m_ActiveSet.erase(it_temp);
}
}
}

//Modify the color list of curPixel according to testPixel
BOOL CBlendColorize::ModifyPixel(CBlendPixel& testPixel,const CBlendPixel & curPixel)
{
if (cvGetReal2D(m_pY,testPixel.m_y,testPixel.m_x)==1)
return FALSE;
double y1 = cvGetReal2D(m_pGreyImage,curPixel.m_y,curPixel.m_x);
double y2 = cvGetReal2D(m_pGreyImage,testPixel.m_y,testPixel.m_x);
double d = abs(y1-y2);
BOOL result = FALSE;
for(int i=0;i<MAX_BLEND_COLOR;i++)
{
if (curPixel.m_d[i]<MAX_DISTANCE)
{
int j=0;
while (((curPixel.m_d[i]+d)>=testPixel.m_d[j])&&(j<=MAX_BLEND_COLOR))
{
if (curPixel.m_c[i]==testPixel.m_c[j])
{
j=MAX_BLEND_COLOR;
break;
}
j++;
}
if (j<MAX_BLEND_COLOR)
{
if (curPixel.m_c[i]!=testPixel.m_c[j])
for(int k=MAX_BLEND_COLOR-1;k>j;k--)
{
if (testPixel.m_c[k-1]!=curPixel.m_c[i])
{
testPixel.m_c[k]=testPixel.m_c[k-1];
testPixel.m_d[k]=testPixel.m_d[k-1];
}
}
testPixel.m_c[j]=curPixel.m_c[i];
testPixel.m_d[j]=curPixel.m_d[i]+d;
result = TRUE;
}
else
break;
}
}
return result;
}

///////////////////////////////////////////////////////////////////////////////////
// Command Pattern to support Undo and Redo functions
///////////////////////////////////////////////////////////////////////////////////
class CCommand
{
public:
virtual BOOL Execute(IplImage *pImage,WPARAM wParam, LPARAM lParam);
virtual BOOL UnExecute(IplImage *pImage);
virtual BOOL ReExecute(IplImage* pImage);
CCommand();
virtual ~CCommand();
protected:
BOOL m_bExecuted;
};



samples:






Last edited by E-Male; 17th April 2005 at 16:10.
E-Male is offline   Reply With Quote
Old 17th April 2005, 18:00   #11  |  Link
guada 2
Registered User
 
Join Date: Feb 2005
Location: Lyon
Posts: 718
Very interesting, if one wishes to restore old photographs.

But somebody has it to try to remove the color of an image (VDM+BMP+Flitre MSU OLD CINEMA) to treat it with this process?

It would be interesting to make the comparison of the 2 images:
the original with its own colors, and the image treated with absence of color.
guada 2 is offline   Reply With Quote
Old 17th April 2005, 18:48   #12  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
>>Very interesting, if one wishes to restore old photographs.

-try the sample program


>>But somebody has it to try to remove the color of an image (VDM+BMP+Flitre MSU OLD CINEMA) to treat it with this process?

-no, at least not in the example program
i guess in most implementations the colors of the picture would just be ignored


>>It would be interesting to make the comparison of the 2 images:
the original with its own colors, and the image treated with absence of color.

- http://www.ece.northwestern.edu/~mya...lorization.htm
(i just didn't link those to keep the post from getting to wide)

Last edited by E-Male; 17th April 2005 at 18:58.
E-Male is offline   Reply With Quote
Old 17th April 2005, 21:33   #13  |  Link
guada 2
Registered User
 
Join Date: Feb 2005
Location: Lyon
Posts: 718
Thank you, E-Male.
However, you will excuse me I had not opened your first link.
guada 2 is offline   Reply With Quote
Old 17th April 2005, 21:55   #14  |  Link
HighInBC
Registered User
 
HighInBC's Avatar
 
Join Date: Jan 2003
Location: Victoria, BC, Canada
Posts: 144
ok, here is a side effect that I just realized.

If you colorize only a bouncing ball in a movie with blue, then took that blue and turned it to white and everything else the black then you would have automatic mask detection for moving objects... If anyone has ever done any mask related special effects they will surely see the advantage of this.

The program seems to be able to track objects that are moving, while the camera is moving, ontop of a moving background.

This is very hard to mask normally, and ussually must be done by hand frame by frame.
HighInBC is offline   Reply With Quote
Old 17th April 2005, 22:09   #15  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
so you colorize the ball frame per frame and use for masking
doesn't seem to help much IMO
(in case my irony is bad as always: this method for now is for stills only, we'll have to add video automation ourselves, maybe include mvtools funcionality, but let's do still colorizing first)

I think Hero-- or a mod should change the thread titls, in order to have the coding pros look in here

else i'll have to try to make sense of that code, which won't be pretty
E-Male is offline   Reply With Quote
Old 18th April 2005, 00:17   #16  |  Link
HighInBC
Registered User
 
HighInBC's Avatar
 
Join Date: Jan 2003
Location: Victoria, BC, Canada
Posts: 144
Quote:
Originally posted by E-Male
so you colorize the ball frame per frame and use for masking
doesn't seem to help much IMO
(in case my irony is bad as always: this method for now is for stills only, we'll have to add video automation ourselves, maybe include mvtools funcionality, but let's do still colorizing first)

I think Hero-- or a mod should change the thread titls, in order to have the coding pros look in here

else i'll have to try to make sense of that code, which won't be pretty
If you look lower on the page(where it says 'Video Clip Colorization Examples'): http://www.cs.huji.ac.il/~yweiss/Colorization/

You will see that the formula does traverse space and time, needing only occasional updates as it loses the shape.
HighInBC is offline   Reply With Quote
Old 18th April 2005, 11:33   #17  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
if you look higher in this thread you will see that we can't use that algo, or at least won't get any help doing it

EDIT: sorry for my tone, i was in a bad mood when i replied (just got up)

i think we should first focus on getting code for still-colorisation working, when we got that we can expand in any direction we want

Last edited by E-Male; 18th April 2005 at 13:37.
E-Male is offline   Reply With Quote
Old 18th April 2005, 23:48   #18  |  Link
insanedesio
Registered User
 
Join Date: May 2003
Posts: 27
Sounds interesting to me. I wouldn't mind coding this, especially since I've been looking for something to code. As it stands, though, I'm new to AVS development, or any kind of coding that has actual applications, for that matter :-\. Given that I'm swamped in finals right now I don't have the patience to think about that paper, but after finals hopefully I should have more thought to give it... I read it through and thought about it a bit and can see how to implement most of it.. just missing the main part I think. If I get a chance I'll ask some people that I know who might be able to help.

Last edited by insanedesio; 18th April 2005 at 23:50.
insanedesio is offline   Reply With Quote
Old 19th April 2005, 07:12   #19  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
nice

i've been doing some more research on the topic and found some more papers and algos
i'll post a summary with all links here after i'm sure which paper is based on which others and after i've mailed all authors and waited some days

then i'll also make a new thread to attract some more experienced programmers from this board
together we should be able to make open-source colorization possible
E-Male is offline   Reply With Quote
Old 19th April 2005, 19:03   #20  |  Link
E-Male
mad computer-scientist
 
Join Date: Mar 2002
Posts: 1,375
got e-mail reply no. 2 (of 3)
again university patenting
so only one left to put hopes on
if it's an other 'no' we have to do the coding from scratch with the avaible information
E-Male is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.