Image gradient map

[luquinhas0021]

Possibly, I will use this thread for us discussing about image gradient subjects. The ask I want make now seems silly, but I`m interested about it: why, in every gradient map, the image edges color is different than image non-edge color? An example is Prewitt gradient map, which has image edges appearing in white and image non-edges appearing in black. Course, the edges the gradient operator can discover. I`m talking about gradient map, not edge map.

[raffriff42]

>why, in every gradient map, the image edges color is different than image non-edge color?
Because a large gradient is one definition of 'edge' ?

https://en.wikipedia.org/wiki/Edge_detection#ApproachesThere are many methods for edge detection, but most of them can be grouped into two categories, search-based and zero-crossing based. The search-based methods detect edges by first computing a measure of edge strength, usually a first-order derivative expression such as the gradient magnitude, and then searching for local directional maxima of the gradient magnitude using a computed estimate of the local orientation of the edge, usually the gradient direction. The zero-crossing based methods search for zero crossings in a second-order derivative expression computed from the image in order to find edges, usually the zero-crossings of the Laplacian or the zero-crossings of a non-linear differential expression. As a pre-processing step to edge detection, a smoothing stage, typically Gaussian smoothing, is almost always applied (see also noise reduction).

https://en.wikipedia.org/wiki/Prewitt_operatorIn simple terms, the operator calculates the gradient of the image intensity at each point, giving the direction of the largest possible increase from light to dark and the rate of change in that direction. The result therefore shows how "abruptly" or "smoothly" the image changes at that point, and therefore how likely it is that part of the image represents an edge, as well as how that edge is likely to be oriented. In practice, the magnitude (likelihood of an edge) calculation is more reliable and easier to interpret than the direction calculation.

A 'gradient map' maps (https://en.wikipedia.org/wiki/Map_%28mathematics%29#Maps_as_functions) gradient values (magnitude or direction) to colors - usually, higher gradients (https://en.wikipedia.org/wiki/Image_gradient) are mapped to lighter colors.

https://en.wikipedia.org/wiki/Image_gradient
https://upload.wikimedia.org/wikipedia/commons/6/67/Intensity_image_with_gradient_images.png (https://en.wikipedia.org/wiki/Image_gradient)

EDIT to add to the confusion, Photoshop uses the term "gradient map" to mean something COMPLETELY different (https://www.google.com/search?q=Photoshop+gradient+map): an effect which maps input colors to a predefined color gradient, something I would call 'solarization' or 'custom LUT effect'

[luquinhas0021]

thank you, raffriff. I`m worry about how the gradient (gx, gy and gradient magnitude) is computed. Most of operators are based on sum of differences of adjacent pixels, which is pretty close to finite difference method, but it is, specially in distant values, proper to errors. I`m thinking about use splines to interpolate between pixel values, however I have some doubts: I) interpolate as splines in 3D space (f(x, y) = z) or transform a k x m matrix in a 1 x (k x m) matrix, and, this way, make a splines in 2D space (f(x) = y)? The other doubts I will post in other moment, after we discussing your and my ideas about this question.

[raffriff42]

I have no ideas about this question. Too stupid to have ideas, but smart enough to know it. However I will gladly make use of the ideas of Prewitt, Sobel et al.

>but it is, specially in distant values, proper to errors
That is why smoothing is applied. With smoothing, distant values begin to affect local values.

[colours]

Most of operators are based on sum of differences of adjacent pixels, which is pretty close to finite difference method

Yeah, the common gradient operators are pretty much just that.

I`m thinking about use splines to interpolate between pixel values, however I have some doubts: I) interpolate as splines in 3D space (f(x, y) = z) or transform a k x m matrix in a 1 x (k x m) matrix, and, this way, make a splines in 2D space (f(x) = y)?

What's x? What's y? What's z? What kind of splines do you want to use? How do you expect this to differ from using finite differences?

[luquinhas0021]

Before answer the Colours doubts, I wish talk that my operator will work only with YCbCr 4:4:4, YCbCr over RGB 'cause YCbCr is the better color space for do interpolation and find gradient, because it preserves better the edges and textures and has a special component for luminance, what is so good for gradient operator. RGB and YCbCr, as I know, are the only two color systems which is device invariant. If some other exists, please, advice me. The chose of YCbCr 4:4:4 over YCbCr 4:x:x (x different to 4) is given by YCbCr 4:4:4 has same topological information in all it components.
But if content is in different color space? It will be converted, following this rules:
RGB 101010: convert to RGB 888 and convert to YCbCr 4:4:4
RGB 888: convert to YCbCr 4:4:4
RGB xxx (x different from 8 and 10): convert to RGB 888 and convert to YCbCr 4:4:4
YUV 4:4:4: convert to YCbCr 4:4:4
YUV 4:x:x (x different from 4): convert to YUV 4:4:4 and convert to YCbCr 4:4:4
YCbCr 4:x:x (x different from 4): convert to YCbCr 4:4:4
The conversion of YCbCr 4:x:x (x different from 4) to YCbCr 4:4:4 or from YUV 4:x:x (x different from 4) to YUV 4:4:4 will be done in special way: how, in this sub sample types, the Y component is untouched, the conversion will look at gradient informations of Y component and try replicate them into other components.
I prefer YCbCr over YUV because YUV is for analogical and digital content, however, as my operator will be only for digital content, I chose a color system that is optimized to it.
RGB, YUV and YCbCr will be in it full range.
The initial test of my operator will be done in an image which was originally captured in RGB 888, and, after, passed to YCbCr 4:4:4. This means the all types of color system conversion I talked about before will be revised or created after I finish my operator, and all conversions will be done in single or double floating point precision.

Well, now let's answers you, Colours! In "f(x,y) = z", (x,y) is simply the pixel's matrix coordinates; z is pixel value of some YCbCr component (My operator will be done separately in each component and, finally, join the results in a full component image). In "f(x) = y", x is the cardinal number that is associated with ordinal position of an element Ai,j on a matrix Pmxn (Suppose a matrix 5 x 7 and one it element, A3,5. It is the nineteenth element of this matrix, then x = 19); y is the same of z is.
The reason I think my algorithm will differ from common gradient operators is, instead use only discrete pixel values, I will use proper calculated intermediate values between two pixel values. This will give me the correct, or best possible approximated, derivative of each one pixel value.
The type of spline I will choose will depend of the intermediate values, of what spline gives me the derivative of first element of (K + 1)ª spline be equal to derivative of last element of Kª spline, and of what type of spline turns possible the operator to be scale-space invariant, shift invariant, rotational invariant, contrast invariant, illumination invariant and chromatic invariant. Yeah, this will be so robust gradient operator, and will spend me a lot of work. Any suggestion or corrections are welcome.

[colours]

RGB and YCbCr, as I know, are the only two color systems which is device invariant.

Technically, neither RGB nor YCbCr are device-independent. There's XYZ for that.

But if content is in different color space? It will be converted, following this rules: [snip]

Colour space conversion is one of those very well-known topics and you didn't really have to explain it here, but okay. Your methods of conversion are sometimes suboptimal, by the way.

The conversion of YCbCr 4:x:x (x different from 4) to YCbCr 4:4:4 or from YUV 4:x:x (x different from 4) to YUV 4:4:4 will be done in special way: how, in this sub sample types, the Y component is untouched, the conversion will look at gradient informations of Y component and try replicate them into other components.

You actually cannot do this because there's no reason for chroma to share the same "gradient" as luma. Also, watch out for chroma placement problems.

I prefer YCbCr over YUV because YUV is for analogical and digital content, however, as my operator will be only for digital content, I chose a color system that is optimized to it.

"YUV" is nowadays commonly used as a synonym for YCbCr, by the way.

In "f(x,y) = z", (x,y) is simply the pixel's matrix coordinates; z is pixel value of some YCbCr component. In "f(x) = y", x is the cardinal number that is associated with ordinal position of an element Ai,j on a matrix Pmxn; y is the same of z is.

Okay, now that's an explanation that makes some sense. Flattening the image into one row sounds like an immensely dumb thing to try, because the left and right edges don't continuously blend into each other.

The reason I think my algorithm will differ from common gradient operators is, instead use only discrete pixel values, I will use proper calculated intermediate values between two pixel values. This will give me the correct, or best possible approximated, derivative of each one pixel value.

If your spline is linear in the input, then the resulting derivative operator will also be linear, in which case you should optimise the "fit a spline to the values" part away and find a way to calculate the coefficients directly.

what type of spline turns possible the operator to be scale-space invariant, shift invariant, rotational invariant, contrast invariant, illumination invariant and chromatic invariant. Yeah, this will be so robust gradient operator, and will spend me a lot of work.

Some of these invariants you're asking for don't make sense in the context of spline interpolation. Scale invariance is meaningless because your input values come in only one scale, and likewise rotational invariance is meaningless because your input values come in a fixed rectangular grid. (I might be misunderstanding what you mean by scale/rotational invariance here.) Contrast and brightness invariance automatically come with any linear operator, and hue invariance (sort of) comes with using YCbCr.

[luquinhas0021]

"Technically, neither RGB nor YCbCr are device-independent. There's XYZ for that."

Maybe CIEuvw (CIE 1964) is the best color space for gradient operator and upscaling. What you think about?

"(...) Your methods of conversion are sometimes sub-optimal, by the way."

What my color space conversion methods are sub-optimal and why?

"You actually cannot do this because there's no reason for chroma to share the same "gradient" as luma. Also, watch out for chroma placement problems."

Yeah, you are plenty of reason: I saw some images splitted into it YCbCr components and, really, it neither haves the same perceptible edges.
I'm thinking about mark the pixels that were changed by sub-sample process, analyse it neighbors and replace it component values the best possible way. I'm thinking about too other method, based on components edge perceptible difference; but, as I told, it is a future subject. If I won't work with YCbCr for my gradient operator, at least I do the conversion for restoring image, before gradient operator applying.

""YUV" is nowadays commonly used as a synonym for YCbCr, by the way."

Look on Wikipedia for you see that is, technically, difference.

"Okay, now that's an explanation that makes some sense. Flattening the image into one row sounds like an immensely dumb thing to try, because the left and right edges don't continuously blend into each other."

Well, so I will have to use 3D splines, i.e, make a set of functions of form f(x,y) = z. Perhaps it is so hard to do... I search on Google about interpolation of two variables functions, but i didn't find anything plausible. Can you give an advice or indicate some site?

"...,in which case you should optimize the "fit a spline to the values" part away and find a way to calculate the coefficients directly."

If the optimization doesn't create error, yes, I will do it.

"Some of these invariants you're asking for don't make sense in the context of spline interpolation. Scale invariance is meaningless because your input values come in only one scale, and likewise rotational invariance is meaningless because your input values come in a fixed rectangular grid. (I might be misunderstanding what you mean by scale/rotational invariance here.)..."

Let's put rotational invariant in affine invariant group. So can you give me and advice about where start from for finding a function which is scale-space invariant, shift invariant, affine invariant, contrast invariant, illumination invariant, chromatic invariant and, somehow, sharpness invariant?

P.S.: I did a small change in post you comment.

[colours]

What my color space conversion methods are sub-optimal and why?

Use float operations to convert directly; don't convert from 10 bpc to 8 bpc first and then change colour space. Your suggested method is both slower and less accurate.

Look on Wikipedia for you see that is, technically, difference.

Language is descriptive, not prescriptive. People use "YUV" to mean YCbCr; it doesn't matter that "YUV" once-upon-a-time meant something else.

I search on Google about interpolation of two variables functions, but i didn't find anything plausible. Can you give an advice or indicate some site?

Look up radial basis functions. Note that using rbf for interpolation is relatively slow as it involves inverting sort-of-sparse matrices, and, for square grids, it converges to sinc interpolation as the radius increases anyway.

Alternatively, given that we're still assuming that we're sampling on a rectangular grid, you can just interpolate in one direction then interpolate in the other.

Let's put rotational invariant in affine invariant group. So can you give me and advice about where start from for finding a function which is scale-space invariant, shift invariant, affine invariant, contrast invariant, illumination invariant, chromatic invariant and, somehow, sharpness invariant?

This still doesn't make any sense to me, sorry. (Also, scaling and shifting are affine transforms too.)

[luquinhas0021]

"Use float operations to convert directly; don't convert from 10 bpc to 8 bpc first and then change color space. Your suggested method is both slower and less accurate."

So what optimal (The most possible accurate, without speed worry) color space conversion methods you suggest me?

"Alternatively, given that we're still assuming that we're sampling on a rectangular grid, you can just interpolate in one direction then interpolate in the other."

How make that?

"This still doesn't make any sense to me, sorry. (Also, scaling and shifting are affine transforms too.)"

More invariational attributes the gradient operator has, more universal it is, IMHO. The precision is on pixel values and your relationship.