Smooth hue adjustment using Masktools?

[HappyLee]

Hi. I'm dealing with a video source that needs to turn the blue color a bit to green, leaving other color untouched. The result should look a bit similar with this:
Tweak(hue=30,starthue=325,endhue=15)

But unfortunately it's not smooth using Tweak. There's a filter MaskHS which can create a mask using starthue and endhue, but it's not smooth either, cause the mask would only be pure black and white.

I want to create a mask using Masktools2, but I don't know how to calculate the value of hue using U and V values and put the formula into mt_lut function.

Once the calculation is done, then it would be easy to create a mask, for example, making parts where the hue is equal to 350 pure white, and parts greater than 20 or smaller than 320 pure black, and making rest of the parts in gradient grey. Then I'll be able to use mt_merge(Tweak(30),mask,luma=true) to adjust the hue smoothly.

I've been looking everywhere but couldn't find a solution. Please help. Thanks in advance.

[Gavino]

Hue is calculated by Avisynth as atan2(v-128, u-128) * 180.0 / Pi

Masktools doesn't have atan2(), but it does have atan().
Note that to be able to access both u and v at the same time, you will have to use mt_lutxy(UToY(), VToY(), ...).

[HappyLee]

Originally Posted by Gavino

Hue is calculated by Avisynth as atan2(v-128, u-128) * 180.0 / Pi

Masktools doesn't have atan2(), but it does have atan().
Note that to be able to access both u and v at the same time, you will have to use mt_lutxy(UToY(), VToY(), ...).

Thank you. How do you use atan2() in mt_polish()? I found the atan2() formula quite complicated, which depends on whether x=0 and such. I wonder if there's an easier way to calculate all this. Thanks.

Edit: Should I use this (http://upload.wikimedia.org/math/5/7/1/571efb70f630041f9e2b00019025171e.png) or this (http://upload.wikimedia.org/math/5/6/0/560894456327bac6cd16e27339df5c32.png)? Sorry my math's not that good...

[HappyLee]

Well, I just wrote a function that solves this problem using the stupidest way I know, but the result looks good as expected:

Function HLM(clip o,clip c,int s,float d,int n) {
o2=o.Overlay(c.MaskHS(s-d*n,(s+d*n>360)? s+d*n-360:s+d*n,coring=false).Levels(0,1,255,0,8,coring=false),mode="Add")
return (n==31)? o2:HLM(o2,c,s,d,n+1)
}
Function HLSH(clip c,int s,float d,int t) {  #s for the middle Hue, d for minimum range, t for Hue in function Tweak()
c.ConverttoYV24()
m=HLM(MaskHS(s,s,coring=false).Levels(0,1,255,0,8,coring=false),c.ConverttoYV24(),s,d,1).blur(1.58).blur(1.58).blur(1.58).blur(1.58).blur(1.58).blur(1.58)
mt_merge(c,c.Tweak(t),m,luma=true)
#m  #You can check what final mask looks like here
}

How to use: HLSH(350,0.8,30), which would turn color blue a bit to green. Simple, right?

[Gavino]

That's a neat solution.
I'm not sure about the final series of blurs applied to the mask though. I can see that this eliminates banding in the mask, but since the blur is applied in the spatial domain rather than the 'hue domain', doesn't it lead to the mask 'leaking' into adjacent parts of the image that are in a completely different hue, and hence potentially changing those too?

[HappyLee]

Originally Posted by Gavino

That's a neat solution.
I'm not sure about the final series of blurs applied to the mask though. I can see that this eliminates banding in the mask, but since the blur is applied in the spatial domain rather than the 'hue domain', doesn't it lead to the mask 'leaking' into adjacent parts of the image that are in a completely different hue, and hence potentially changing those too?

Yeah you're right, but I haven't found a better way to avoid banding in the mask. Maybe I should leave banding there since it's almost impossible to catch it with our eyes. Thanks again.

[Gavino]

Looking more closely, there are a couple of bugs lurking in your code:

- MaskHS(s, s, ...) actually selects all pixels rather than just those with hue exactly equal to s.
(Not sure if this is a bug in MaskHS, but that's the way it works).
Also, MaskHS(0, 0, ...) gives an error (endHue must be greater than zero).

- For small values of s, startHue can go negative during the iteration, producing an error.

Fixed version:

Function HLM(clip o,clip c,int s,float d,int n) {
o2=o.Overlay(c.MaskHS((s-d*n<0)? s-d*n+360:s-d*n,(s+d*n>360)? s+d*n-360:s+d*n,coring=false).Levels(0,1,255,0,8,coring=false),mode="Add")
return (n==31)? o2:HLM(o2,c,s,d,n+1)
}
Function HLSH(clip c,int s,float d,int t) {  #s for the middle Hue, d for minimum range, t for Hue in function Tweak()
c.ConverttoYV24()
m=HLM(MaskHS(s,s+.001,coring=false).Levels(0,1,255,0,8,coring=false),c.ConverttoYV24(),s,d,1).blur(1.58).blur(1.58).blur(1.58).blur(1.58).blur(1.58).blur(1.58)
mt_merge(c,c.Tweak(t),m,luma=true)
#m  #You can check what final mask looks like here
}

[raffriff42]

Originally Posted by HappyLee

I want to create a mask using Masktools2, but I don't know how to calculate the value of hue using U and V values and put the formula into mt_lut function.

Once the calculation is done, then it would be easy to create a mask, for example, making parts where the hue is equal to 350 pure white, and parts greater than 20 or smaller than 320 pure black, and making rest of the parts in gradient grey. Then I'll be able to use mt_merge(Tweak(30),mask,luma=true) to adjust the hue smoothly.

I've been looking everywhere but couldn't find a solution. Please help. Thanks in advance.

I've been hacking away at this problem, off and on, since you posted about it. Now I think I have a solution. The mt_lutxy expression is not pretty, to say the least. This solution is not really better than your iterative one (which is actually quite clever), but that's OK! At least it's not worse!

#Last = YUV

## TEST
return SmoothMaskedTweak(
\   120, /* h - center hue */
\   030, /* d - range or spread */
\   180, /* hue shift - see Tweak */
\   1.5, /* saturation - see Tweak */ 
\   0>0, /* if true, invert the mask */
\   1>0, /* if true, show vector scope (slow) */
\   0>0) /* if true, show mask; else show effect */


##################################
### Wrap call to Tweak, masked with SmoothHueMask
##
## @ h   - the middle Hue to select
## @ d   - approximate range in degrees
## @ hue - Tweak: hue +/- (default 0)
## @ sat - Tweak: sat (default 1.0)
## @ negate     - if true, invert the mask
## @ showvector - if true, add Histogram(mode="color")
## @ showmask   - if true, return the mask
## @ bypass     - if true, bypass effect but allow
##                 showvector & showmask
##
## @ raffriff42 Feb-2014
##
function SmoothMaskedTweak(clip C, int h, int d,
\            int hue, float sat, bool "negate",
\            bool "showvector", bool "showmask", bool "bypass")
{
    negate     = Default(negate,     false)
    showvector = Default(showvector, false)
    showmask   = Default(showmask,   false)
    bypass     = Default(bypass,   false)

    c_00 = c
    c  = (showvector==false) ? c
    \  : c.Histogram(mode="color").AddBorders(0, 40, 0, 0)
    \     .Crop(0, 0, 0, C.Height+40)

    mc = c.SmoothHueMask(h, d, minsat=0, showargs=showvector)
    mc = (negate) ? mc.Invert : mc

    ##################
    ## modify this function with different effects here!
    fx = c.mt_merge(
    \        c.Tweak(hue=hue, sat=sat, coring=false),
    \        mc.ConvertToY8,
    \        luma=true)
    ##################

    rc = (bypass) ? c : fx
    return (showmask)
    \  ? mc.BicubicResize(c_00.Width, c_00.Height).ConvertToYV12()
    \  : (showvector==false)
    \    ? rc
    \    : rc.Overlay(
    \         mc.Crop(mc.Width-256, 0, 256, 40+256),
    \         x=mc.Width-256, y=0, mode="add", opacity=0.25)
    \        .Overlay(mc.Crop(0, 0, mc.Width-256, 40))
    \        .hist_color_labels(y=40)
    \        .BicubicResize(c_00.Width, c_00.Height)
}

##################################
### mask a smooth range of source hues
##
## @ h  - the middle Hue to select
## @ d  - approximate range (clamped to 3° min, 90° max)
## @ maxSat, minSat - see MaskHS (not implemented yet)
## @ showmap  - if true, show active hue-to-luma map
## @ showargs - if true, Subtitle arguments for debugging
##
## @ raffriff42 Feb-2014
##
function SmoothHueMask(
\           clip c, int h, float d,
\           int "maxSat", int "minSat",
\           bool "showmap", bool "showargs")
{
    h = (h + 36000) % 360
    d = Min(Max(3, d), 90)
    showmap  = Default(showmap, false)
    showargs = Default(showargs, false)

    ## ** convert U,V to hue
    
    atd = "y 127 - x 127 - / atan 180 * pi / " ## arctan(V/U) in degrees

    tr0 = "x 127 == y 127 == & -1 "              ## undefined
    tr1 = "y 127 == x 127 > & 0 "                ##   0 degrees exactly (no arctan)
    tr2 = "x 127 == y 127 > & 90 "               ##  90   ''
    tr3 = "y 127 == x 127 < & 180 "              ## 180   ''
    tr4 = "x 127 == y 127 < & 270 "              ## 270   ''
    tr5 = "x 127 > y 127 < & " + atd + "360 + "  ## Quadrant 1
    tr6 = "x 127 > y 127 > & " + atd             ## Quadrant 2
    r78 = atd + "180 + "                         ## Quadrants 3, 4

    ## /mm/ == map mask string (translate hue to luma with offset)
    ## result hue(0)=luma(0); hue(180)=luma(180)±; hue(359)=luma(359)±; etc
    mm = tr0+tr1+tr2+tr3+tr4+tr5+tr6+r78+"? ? ? ? ? ? "

    ## /hc/ == "hue center", 180° from wrap/rollover; gray in final mask
    ## allowed values = (0|90|180|270); default = 180
    hc = (h>=315 || h<45)  ? 0
    \  : (h>=45  && h<135) ? 90
    \  : (h>=135 && h<225) ? 180
    \  : (h>=225 && h<315) ? 270
    \  : 180 ## not reached

    ## apply wrapped offset to put hue center at midrange
    ra = (hc==0)   ? "180 + "
    \  : (hc==90)  ?  "90 + "
    \  : (hc==270) ? "270 + "
    \  : " "
    mm = mm + ra + "360 % "

    ## translate hue to adjusted luma (0-359 scale)
    hl = (h>=315) ? h - 180
    \  : (h<45)   ? h + 180
    \  : (h>=45  && h<135) ? h + 90
    \  : (h>=135 && h<225) ? h
    \  : (h>=225 && h<315) ? h - 90
    \  : h ## not reached

    ## create soft hue mask;
    ## clamp to 0-359, scale to 0-255
    sm = (showmap)
    \  ? "359 > 359 " + mm + "? 256 * 360 / "
    \  : "359 > 359 " + mm + String(hl)
    \  + " - abs "+String(-230/d) + " * 360 + "
    \  + "? 256 * 360 / "

    mc = mt_lutxy(C.UtoY, C.VtoY, sm, U=1, V=1)
    \    .BilinearResize(c.Width, c.Height)

    return (showargs==false) ? mc
    \  : mc.Subtitle("SmoothHueMask: h="+String(h)
    \             +"; d="+String(d, "%1.0f"),
    \             align=8, size=mc.Height/20)
}

#############################################
### add labels to Histogram(mode="color")
##
## @ raffriff42 Feb-2014
##
function hist_color_labels(clip C, int "x", int "y")
{
    x = Default(x, C.Width-256)
    y = Default(y, 0)
    C
    Subtitle("B",    x=x+200, y=y+110, text_color=$9999ff)
    Subtitle("M",    x=x+180, y=y+200, text_color=$ff99ff)
    Subtitle("48°",  x=x+180, y=y+220, text_color=$ff99ff)
    Subtitle("R",    x=x+90,  y=y+210, text_color=$ff9999)
    Subtitle("102°", x=x+82,  y=y+230, text_color=$ff9999)
    Subtitle("Y",    x=x+40,  y=y+130, text_color=$cccc66)
    Subtitle("176°", x=x+35,  y=y+150, text_color=$cccc66)
    Subtitle("G",    x=x+50,  y=y+40,  text_color=$66dd66)
    Subtitle("230°", x=x+45,  y=y+60,  text_color=$66dd66)
    Subtitle("C",    x=x+145, y=y+35,  text_color=$66cccc)
    Subtitle("282°", x=x+140, y=y+55,  text_color=$66cccc)

    Subtitle("0°",   x=x+220, y=y+110, text_color=$9999ff)
    Subtitle("90°",  x=x+117, y=y+210, text_color=$cccccc)
    Subtitle("180°", x=x+15,  y=y+110, text_color=$cccccc)
    Subtitle("270°", x=x+115, y=y+10,  text_color=$cccccc)

    Subtitle("U+",   x=x+220, y=y+130, text_color=$cccccc)
    Subtitle("V+",   x=x+120, y=y+230, text_color=$cccccc)
    Subtitle("U-",   x=x+20,  y=y+90,  text_color=$cccccc)
    Subtitle("V-",   x=x+120, y=y+25,  text_color=$cccccc)
}

One reason for the complication is the "hue wrap-around" problem [EDIT which Gavino resolves more elegantly in his solution below]...

After getting a useful hue map, a difference is taken with the hue of interest. Here is a sample mask, with hue approximately centered on skin tones (hue=120°, ±15)

## ..............  h,  d,  hue, sat, neg,  v,   m
SmoothMaskedTweak(120, 30, 000, 1.0, 0>0, 1>0, 1>0)

Here is the test image, with skin tones reversed:

## ..............  h,  d,  hue, sat, neg,  v,   m
SmoothMaskedTweak(120, 30, 180, 1.5, 0>0, 1>0, 0>0)

And here it is with normal skin tones, and the color of the flowers (hue=172°, ±15) reversed:

## ..............  h,  d,  hue, sat, neg,  v,   m
SmoothMaskedTweak(172, 30, 180, 1.5, 0>0, 1>0, 0>0)

[Gavino]

Hi raffriff. Thanks for that.
I had already put together something similar myself, but didn't get round to tidying it up for publication as the OP seemed satisfied with what he had. Your post has prompted me to do so now.

I first created a Hue() function that returns an (infix) expression for the hue. This can be used to construct further expressions derived from the hue, eg to derive a mask for any arbitrary hue selection.

My HueMask() function is based on the MaskHS() interface with startHue and endHue, adding an interp parameter (as used by Tweak() for saturation), with the mask falling off to zero over a distance of interp degrees (>0) outside the selected hue range.
I got round the 'hue wrap-around problem' using some tricks with the abs function.
I convert input to YV24 to avoid any inaccuracy from chroma subsampling or alignment.

I also have a ShowHue() function that shows hue 0-360 as luma 0-180 - I preferred this range over 0-255 as you can read off the luma directly in AvsP and just double it to get the hue.

function atan2(string y, string x) {
# Returns an expression for atan2(y, x) (in range 0-360), corresponding to
#  (x == 0 ? (y > 0 ? 90 : (y < 0 ? 270 : 0)) : atan(y/x)*180/pi + (x < 0 ? 180 : (y < 0 ? 360 : 0)))
# Form of expression chosen to evaluate atan() once only.
  return "("+x+" == 0 ? ("+y+" > 0 ? 90 : ("+y+" < 0 ? 270 : 0)) : " + 
\               "atan(("+y+")/("+x+"))*180/pi + " +
\                   "("+x+" < 0 ? 180 : ("+y+" < 0 ? 360 : 0)))"
}

function Hue(string u, string v) {
# Returns an expression for hue (0-360) derived from U and V expressions
  return atan2(v+"-128", u+"-128")
}

function HueMask(clip c, float startHue, float endHue, float interp) {
  midHue = (startHue+endHue)/2.0
  midHue = startHue > endHue ? (midHue<180 ? midHue+180 : midHue-180) : midHue
  r = (endHue-startHue)/2.0  # radius of zone to be selected around midHue
  r = r < 0 ? r+180 : r
  g = 255.0/interp  # gradient of mask roll-off outside radius
  # reqd mask given by 255 + g*(r-d), where d is the absolute difference of hue from midHue (0-180)
  # d = abs(Hue-midHue) > 180 ? 360-abs(Hue-midHue) : abs(Hue-midHue)
  # To avoid evaluating Hue more than once, this is recast as d = 180-abs(abs(Hue-midHue)-180),
  # hence mask = 255 + g*(r-180+abs(abs(Hue("x", "y")-midHue)-180))
  # will be clamped to [0,255] by MaskTools, so get 255 between startHue and endHue, and 0 beyond interp zone
  expr = "255 + "+string(g)+"*("+string(r)+"-180+abs(abs("+Hue("x", "y")+"-"+string(midHue)+")-180))"

  c.ConvertToYV24()
  mt_lutxy(UtoY(), VToY(), mt_polish(expr), chroma="128")
}

function ShowHue(clip c) {
  c.ConvertToYV24()
  mt_lutxy(UtoY(), VToY(), mt_polish(Hue("x", "y")+"/2"), chroma="128")  
}

ColorBars()
orig = last
ShowHue()
# To see mask, need to convert with PC matrix
StackHorizontal(ConvertToRGB32(matrix="PC.601"), orig.ConvertToRGB32())

There are a couple of RPN oddities in your code:

Originally Posted by raffriff42

    mm = tr0+tr1+tr2+tr3+tr4+tr5+tr6+r78+"? ? ? ? ? ? "
...
    sm = (showmap)
    \  ? "359 > 359 " + mm + "? 256 * 360 / "
    \  : "359 > 359 " + mm + String(hl)
    \  + " - abs "+String(-230/d) + " * 360 + "
    \  + "? 256 * 360 / "

In the creation of mm, there is a missing '?' - there are 7 conditions but you only have 6 '?'s.
Later, two expressions both start with "359 >" which appears to have only one operand given for '>'.
I'm not sure what effect these have, as the result in MaskTools from ill-formed RPN is not defined. Your function seems to work, but there may be cases for which it does something unexpected or undesired.

Also, you use 127 for the neutral chroma point instead of the 128 used by Avisynth in determining hue.

Incidentally, where did you get your source images from? They look very useful for testing this sort of thing.

And I like your vectorscope labelling, which I have stolen for my utilities library.

[raffriff42]

Hi Gavino, yes "359 > " is a bug, but it always evaluates to false, so it has no effect. I don't know what I was thinking.

So throwing that out, we get six ?'s and then a final one at the end, which was the original intent, before I injected that nonsensical code at the last minute.

I need a course in RPN, which I have avoided all my life up until now.

Image source: it's called "DQ monitor" https://www.google.com/search?tbm=is...reference+file

I am pleased that you like my vector labels; steal away.

[Gavino]

Originally Posted by raffriff42

I need a course in RPN, which I have avoided all my life up until now.

It's very easy to make mistakes without realising it when constucting complex RPN expressions.
It doesn't help that MaskTools does no error checking (or at least fails to report ill-formed expressions) which can make mistakes hard to find.

So this time I decided to create the expressions as infix and use mt_polish() at the end. The MaskTools error checking is no better here, but at least it's easier for the script writer to get it right.
However, in doing so, I discovered a bug in mt_polish() that confused me for a while until I realised what was happening. So maybe I should have stuck with RPN.

[StainlessS]

Some really nice colour images contained in this PDF: http://www.comp.nus.edu.sg/~cs4243/lecture/colour.pdf
Need to extract as jpg, requires Acrobat (editor) to extract (or similar).

EDIT: eg


about 6 in total

[WorBry]

I came across this thread whilst researching available tools for targeted secondary color correction.

A few years back I put together a function SelSah, that applies Tweak through a luma range mask with high and low luma point 'roll-off' control - you may recall that you helped me with it, Gavino.

http://blog.niiyan.net/post/666717637/selsahv3

Actually, it's quite a useful tool and I still use it a fair bit in secondary color grading - not just for creating 'filmic looks' which was the original incentive. Using the Tweak hue and saturation range (start/End) parameters together with the luma range control, it is possible to target and isolate subjects with distinct and fairly uniform color characteristics (say a distracting brightly colored-saturated object) with a reasonable degree of precision. Rather more difficult, and finicky, is targeting subtle/diffuse tones, such as people skin, especially in 'real world' video, when the subject(s) is set in a natural background where base yellow tones are prevalent.

The SelSah function does include a Preview option that shows the original and adjusted chroma through the greyscale luma mask, but the faint chroma trace makes it difficult to visualize less-than-saturated subjects and one has to resort to exaggerating the chroma in a variety of ways (cranking up the saturation, desaturating and/or shifting the hue to some distinctive color) to help visualize the effects, or else crop down to particular area of interest - either way, rather tedious. I'll put up some examples when I have a mo.

What would be nice would be a tool that generates, and applies, a composite greyscale mask (ideally with intensity adjustment) from the set hue, saturation and luma ranges, with provision for controlling the 'softness' (roll-off) of the mask boundaries.

Seeing the Hue masks that raffriff42 and Gavino posted above, I'm sure such a function is quite do-able. Indeed, I tried applying raffriff42's SmoothMaskedTweak through a luma range mask and it works rather well for skin tones, especially with the approach taken for smooth roll-off from a target (centre) hue. What's lacking is the added refinement of a saturation range component.

Nothing new of course. All of the high end video editing and grading suites have far more sophisticated tools available for this type of thing...in RGB colorspace, that is.

Anyhow, I wonder if anyone would be interested in putting together such a function, if they haven't already. Something that could be easily used in AVSPMod with sliders. I'd have a go, but once again find myself limited by lack of knowledge of the theory and computations involved and my rather rudimentary experience in writing complex functions.

Edit: Or else, if someone could show me how to create greyscale saturation range mask (with boundary roll-off) using MaskTools2, I could have a go at creating the merged composite luma/hue/saturation mask.

Also Re:

Originally Posted by WorBry

I'll put up some examples when I have a mo.

An example of cranking up the sat and shifting the hue to help visualize the target skin tones - the "Avatar" approach. Probably could have tweaked it some more, but it illustrates the point.

[WorBry]

P.S. @raffriff42

I also like the Vectoscope with markers that you incorporated in SmoothMaskedTweak. I did something similar - lifted the graticule (with skin tone line and saturation targets) from a vectorscope in Premiere Pro (trial) and applied as a mask over an inverted (Color2 mode) vectoscope in Histogram. Pretty crude, but it does the job. Example:



In this example I used SelSah to target and shift the skin tones a tad:

SelSah_V3(Sat=0.98, Hue=-4, StartHue=80, EndHue=135, Luma_Low=60, Luma_High=220, minsat=0, maxsat=100,Preview=False)

Edit: minsat=10 gives a marginally better result for this shot

Edit2: Actually, an even "tighter" selection for the skin tones in that particular shot:

SelSah_V3(Sat=0.98, Hue=-4, StartHue=90, EndHue=129, Luma_Low=90, Luma_High=220, minsat=20, maxsat=40, Preview=False)

It would be nice if vectorscope markers could be incorporated as standard options in the native Histogram filter.

[WorBry]

Well, I've tried using the MaskHS mask together with a Luma range mask and I must say it does make it a lot easier to visually home in on the desired target. Assuming, as it would appear, that Tweak uses MaskHS internally, I guess I could use the combined masks to determine the optimal Hue, Saturation and Luma range values and then plug them into SelSah.

Still, I think a function that allows for smooth roll-off at the mask range boundaries would make this more refined.

[raffriff42]

Nice graticules you have there!

Originally Posted by WorBry

What's lacking is the added refinement of a saturation range component.

This could be done using the same techniques as used for hue selection. The math is not a problem, but I don't think you would get useful selectivity based on saturation, in the real world.

(*thinks*)

OK here is a function that masks the image based on Saturation. Merge it with Hue and/or Luma mask(s) as needed.

It uses this equation:
sat = Sqrt((u-127.5)^2 + (v-127.5)^2)

BTW extracting Saturation by this method looks better (IMHO) than channel splitting by HSV/HSL in GIMP etc.

HSV emphasizes anything that's green. HSL emphasizes anything that's bright.

##################################
### mask a range of source saturation; transform saturation->lightness
##
## @ maxSat, minSat - see MaskHS (default 255, 0)
## @ sensitivity - increase saturation->lightness in output (default 1.5)
##
function SatMask(clip c, int "maxSat", int "minSat", float "sensitivity")
{
    Assert(c.IsYV12 || c.IsYV24, 
    \   "SatMask: source must be YV12 or YV24")

    xlo   = Min(Max(    0, Default(minSat,   0)), 250) * 3.0 / 256
    xhi   = Min(Max(xlo+5, Default(maxSat, 255)), 255) * 3.0 / 256
    sens  = Default(sensitivity, 1.5)

    mask_sat = mt_lutxy(C.UtoY, C.VtoY, 
    \       yexpr=mt_polish(
    \           "((((x-127.5)^2)+((y-127.5)^2))^0.5)*"+String(2.0 * sens, "%0.3f")), 
    \       U=-128, V=-128)
    \    .BilinearResize(c.Width, c.Height)

    mask_lo = mask_sat.mt_lut(
    \       yexpr=mt_polish(
    \           "((-0.01 * x) + "+String(xlo)+") * 256"),
    \       u=-128, v=-128)
    
    mask_hi = mask_sat.mt_lut(
    \       yexpr=mt_polish(
    \           "1 - ((-0.01 * x) + "+String(xhi)+") * 256"),
    \       u=-128, v=-128)
    
    mask_mid = mt_lutxy(mask_lo, mask_hi, 
    \       yexpr=mt_polish(
    \           "-1 * (x + y) + 256"), 
    \       u=-128, v=-128)

    return (xlo<=0 && xhi>=(255*3.0/256)) ? mask_sat
    \ : mt_lutxy(mask_sat, mask_mid, 
    \       yexpr=mt_polish("min(x, y)"), 
    \       u=-128, v=-128)
}

P.S. The mask_lo/mask_hi logic was adapted from some code I wrote earlier. It's explained (poorly) here:
http://forum.doom9.org/showthread.ph...26#post1676326
It might be overkill. It's probably enough to manipulate SatMask's default output with Levels as needed, instead of using maxSat & minSat.

[WorBry]

Originally Posted by raffriff42

Nice graticules you have there!

That's what they all say

Originally Posted by raffriff42

This could be done using the same techniques as used for hue selection. The math is not a problem, but I don't think you would get useful selectivity based on saturation, in the real world.

Well, maybe I'm going a bit overboard on this soft roll off thing, at least as it applies to saturation, but closing down the saturation range, along with luma, can help to screen out like hue tones in the background, and preserve the color of more saturated objects (when that is desirable) as well as keeping white balance. Of course, it's always a judgement call as to what is more important.....even when using high end grading software:

https://www.youtube.com/watch?v=bcfWvRY3uTc

https://www.youtube.com/watch?v=iI1Ed5oEwqc

https://www.youtube.com/watch?v=XQx1ZJa_PIA From about 7.5 mins onwards.

That first Apple Color tutorial is a bit dated now, but this is basically the kind of control I'm aiming at, as far as is possible...in YV12 colorspace.

Anyhow, thanks a lot for the sat mask. I'll play around some more with the different mask options.

Edit:

Originally Posted by raffriff42

BTW extracting Saturation by this method looks better (IMHO) than channel splitting by HSV/HSL in GIMP etc.
.......
HSV emphasizes anything that's green. HSL emphasizes anything that's bright.

Interesting