Color banding and noise removal

[cretindesalpes]

Originally Posted by feisty2

are you planning on porting nnedi3 to high bit depth too?

No, other people took charge of it, I’ll leave it to them.

[Overdrive80]

One dude, how I can use below script with dither, for use high-bitdepth in his processing. Thanks.

ColorYUV( gain_y=-10, off_y=0, gamma_y=-8, cont_y=0,\
	  gain_u=4, off_u=0.0, gamma_U=-5, cont_u=0,\
	  gain_v=-4, off_v=0.0, gamma_v=-5, cont_v=0)

[cretindesalpes]

Overdrive80:

The code below should emulate ColorYUV in 16 bits:

Dither_convert_8_to_16 ()
Dither_lut16 (
\   yexpr=coloryuv_to_expr16 (-10, 0, -8, 0),
\   uexpr=coloryuv_to_expr16 (  4, 0,  0, 0),
\   vexpr=coloryuv_to_expr16 ( -4, 0,  0, 0),
\   y=3, u=3, v=3)
DitherPost ()

Function coloryuv_to_expr16 (float gain, float off, float gamma, float cont)
{
	pscale = 256.0
	vscale = 65536
	a = String ((gain + pscale) / pscale)
	c = String ((cont         ) / pscale)
	o = String (off)
	g = String (pscale / (gamma + pscale))
	v = String (vscale)
	h = String (vscale / 2)

	e = "x "+a+" * x "+h+" - "+c+" * + "+o+" + "+v+" / "+g+" ^ "+v+" *"

	return (e)
}

Set gamma_u and gamma_v to 0, they are ignored in the original ColorYUV.

[Overdrive80]

Thanks for your answer. ^^

[jackoneill]

Originally Posted by cretindesalpes

No, other people took charge of it, I’ll leave it to them.

Who did?

Uh, seems the line cretindesalpes was quoting didn't come through. It was about someone porting nnedi3 to 16 bit.

[Popwax]

I think patched dfttest-1.9.4 have a bug handling "plane" parameters from "nfile" or "nstring".
In "vi_src.width >> vi.GetPlaneWidthSubsampling (b);", maybe we should use "1<<b" instead of "b" here.

[Lenchik]

Is it possible to resize using a Sigmoidal Colorspace with Dither package?

[cretindesalpes]

Popwax:

Yes you’re right, there’s something like this to fix it. I’ll work on it when back home.

Lenchik:

Yes it’s possible, it’s just a LUT and its inverse to add. I will probably add sigmoid curves in a next release.

[Motenai Yoda]

Sorry, but this is right?

Function Dither_lut16_expr (string expr, bool gmsb)
{
# We use spaces to enclose " x" because we don't want operators
# like "max" or "exp" to have their "x" replaced…
repstr = (gmsb) ? " x 256 * y + " : " y 256 * x + "
expr = Dither_replace_string (" " + expr, " x", repstr)
part = Dither_expr_part (gmsb)
expr = expr + part

return (expr)
}

what's the meaning? doesn't use lsb as msb and viceversa?

[cretindesalpes]

Yes, it’s to cancel the swap between the LSB and MSB input clips in the mt_lutxy parameters when processing the LSB output. I don’t remember exactly why it was needed but I think it’s to simplify the handling of the y, u and v plane parameters.

[Motenai Yoda]

I wrote a Lutxy12 and a Lutxy16_8, can someone test them?

PHP Code:

###############################
##########  Yoda_Lutxy12  ##########
###############################

function Yoda_Lutxy12(clip src, clip src2, string "expr", string "yexpr",
\ string "uexpr", string "vexpr", int "y", int "u", int "v")
{   
    #needed coz if x is (n*256) -8 < x < n*256 it can't be rounded to n*256 with lsb only.
    src_quant = src.Yoda_Lutxy12_aux(y=y, u=u, v=v, up= true)
    src2_quant = src2.Yoda_Lutxy12_aux(y=y, u=u, v=v, up= false)

    s1_msb = src_quant.Dither_get_msb()
    s1_lsb = src_quant.Dither_get_lsb()
    s2_msb = src2_quant.Dither_get_msb()
    s2_lsb = src2_quant.Dither_get_lsb()

    ss_lsb = mt_adddiff(s1_lsb, s2_lsb, y=y, u=u, v=v)
    
    clip_msb = Yoda_Lutxy12_get(s1_msb, s2_msb, ss_lsb, expr,
    \ yexpr, uexpr, vexpr, y, u, v, True)

    clip_lsb = Yoda_Lutxy12_get(s1_msb, s2_msb, ss_lsb, expr,
    \ yexpr, uexpr, vexpr, y, u, v, False)
    
    stackvertical(clip_msb, clip_lsb)

    #handling chroma planes if y,u or v are 2 or 4
    (Defined (y)) ? (y == 2) ? MergeLuma(src) :
    \(y == 4) ? MergeLuma(src2) : last : last

    (Defined (u)) ? (Defined (v)) ? (u == 2 && v == 2) ? MergeChroma(src) :
    \(v == 2) ? ytouv(last.utoy(),src.vtoy()) :
    \(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
    \(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
    \(Defined (v) && v == 2) ? ytouv(last.utoy(),src.vtoy()) : last

    (Defined (u)) ? (Defined (v)) ? (u == 4 && v == 4) ? MergeChroma(src2) :
    \(v == 4) ? ytouv(last.utoy(),src2.vtoy()) :
    \(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
    \(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
    \(Defined (v) && v == 4) ? ytouv(last.utoy(),src2.vtoy()) : last
}


function Yoda_Lutxy12_aux(clip src, int "y", int "u", int "v", bool "up")
{
    expr = (up) ? "x 247 > 0 x 16 / round 16 * ?" : "x 247 > 0 x 16 / round 128 + ?"
    src_lsb=src.Dither_get_lsb()
    lsb_l=src_lsb.Mt_lut(expr, y=y , u=u, v=v)
    lsb_r=src_lsb.Mt_lut("x 247 > 129 128 ?", y=y , u=u, v=v)
    msb_l=src.Dither_get_msb().mt_adddiff(lsb_r, y=y , u=u, v=v)
    stackvertical(msb_l,lsb_l)
}

Function Yoda_Lutxy12_get (clip "s1_msb", clip "s2_msb", clip "ss_lsb",
\    string "expr", string "yexpr", string "uexpr", string "vexpr",
\    int "y", int "u", int "v", bool "gmsb")
{    
    expr  = Default (expr,  "x")
    yexpr = Default (yexpr, expr)
    uexpr = Default (uexpr, expr)
    vexpr = Default (vexpr, expr)
    gmsb = Default(gmsb, False)

    expr  = Yoda_lut12_expr (expr,  gmsb)
    yexpr = Yoda_lut12_expr (yexpr, gmsb)
    uexpr = Yoda_lut12_expr (uexpr, gmsb)
    vexpr = Yoda_lut12_expr (vexpr, gmsb)

    y = (Defined (y) && y < 0) ? (gmsb) ? y / 256 : y % 256 : y
    u = (Defined (u) && u < 0) ? (gmsb) ? u / 256 : u % 256 : u
    v = (Defined (v) && v < 0) ? (gmsb) ? v / 256 : v % 256 : v

    mt_lutxyz (s1_msb, s2_msb, ss_lsb, expr=expr, yexpr=yexpr,
    \ uexpr=uexpr, vexpr=vexpr, y=y, u=u, v=v)
}

Function Yoda_lut12_expr (string expr, bool gmsb)
{
    repstr1 = " x 256 * z z 16 % - + "
    repstr2 = " y 256 * z 16 % 16 * + " 
    
    expr = Dither_replace_string (" " + expr, " x", repstr1)
    expr = Dither_replace_string (" " + expr, " y", repstr2)
    part = Dither_expr_part (gmsb)
    expr = expr + part

    return (expr)
}

#################################
##########  Yoda_Lutxy16_8  ##########
#################################

function Yoda_Lutxy16_8(clip src, clip src2, string "expr", string "yexpr",
\ string "uexpr", string "vexpr", int "y", int "u", int "v")
{
    s_lsb = src.Dither_get_lsb()
    s_msb = src.Dither_get_msb()
    s2_c = src2.Dither_convert_8_to_16()
    
    clip_msb = Yoda_Lutxy16_8_get(s_msb, src2, s_lsb, expr,
    \ yexpr, uexpr, vexpr, y, u, v, True)
    clip_lsb = Yoda_Lutxy16_8_get(s_msb, src2, s_lsb, expr,
    \ yexpr, uexpr, vexpr, y, u, v, False)
    
    stackvertical(clip_msb, clip_lsb)

    #handling chroma planes if y,u or v are 2 or 4
    (Defined (y)) ? (y == 2) ? MergeLuma(src) :
    \(y == 4) ? MergeLuma(src2) : last : last

    (Defined (u)) ? (Defined (v)) ? (u == 2 && v == 2) ? MergeChroma(src) :
    \(v == 2) ? ytouv(last.utoy(),src.vtoy()) :
    \(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
    \(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
    \(Defined (v) && v == 2) ? ytouv(last.utoy(),src.vtoy()) : last

    (Defined (u)) ? (Defined (v)) ? (u == 4 && v == 4) ? MergeChroma(src2) :
    \(v == 4) ? ytouv(last.utoy(),src2.vtoy()) :
    \(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
    \(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
    \(Defined (v) && v == 4) ? ytouv(last.utoy(),src2.vtoy()) : last
} 

Function Yoda_Lutxy16_8_get (clip "s1_msb", clip "src2", clip "s1_lsb",
\    string "expr", string "yexpr", string "uexpr", string "vexpr",
\    int "y", int "u", int "v", bool "gmsb")
{    
    expr  = Default (expr,  "x")
    yexpr = Default (yexpr, expr)
    uexpr = Default (uexpr, expr)
    vexpr = Default (vexpr, expr)
    gmsb = Default(gmsb, False)

    expr  = Yoda_lut16_8_expr (expr,  gmsb)
    yexpr = Yoda_lut16_8_expr (yexpr, gmsb)
    uexpr = Yoda_lut16_8_expr (uexpr, gmsb)
    vexpr = Yoda_lut16_8_expr (vexpr, gmsb)

    y = (Defined (y) && y < 0) ? (gmsb) ? y / 256 : y % 256 : y
    u = (Defined (u) && u < 0) ? (gmsb) ? u / 256 : u % 256 : u
    v = (Defined (v) && v < 0) ? (gmsb) ? v / 256 : v % 256 : v

    mt_lutxyz (s1_msb, src2, s1_lsb, expr=expr, yexpr=yexpr,
    \ uexpr=uexpr, vexpr=vexpr, y=y, u=u, v=v)
    
}

Function Yoda_lut16_8_expr (string expr, bool gmsb)
{
    repstr1 = " x 256 * z + "
    repstr2 = " y 256 * "
    expr = Dither_replace_string (" " + expr, " x", repstr1)
    expr = Dither_replace_string (" " + expr, " y", repstr2)
    part = Dither_expr_part (gmsb)
    expr = expr + part

    return (expr)
} 


removed all ">>" "<<" stuff to make them faster.

[cretindesalpes]

Dither 1.26.0:

• Modded versions of dfttest and MVTools2 aren’t bundled with Dither anymore. Get them separately (at the same location).
• Added sigmoid curves to Dither_y_gamma_to_linear and Dither_y_linear_to_gamma.
• Added Dither_sigmoid_direct and Dither_sigmoid_inverse functions.
• Added the sigmoid parameter to Dither_srgb_display.
• Dither_y_gamma_to_linear and Dither_y_linear_to_gamma now process data contained in the chroma planes when u or v are set to 3, instead of just copying them.

— * —

Motenai Yoda:

Thanks. I’ll check it later.

[Sparktank]

Originally Posted by cretindesalpes

Dither 1.26.0

Originally Posted by cretindesalpes

dither-1.26.1

<strike>Is the update a minor thing? I hadn't even noticed it changed versions until I had to update my laptop (low priority udpates).</strike>

EDIT: Ah, the small change doesn't affect my normal work environment.

Here's what the changelog says in the doc for those who are interested:

v1.26.1, 2014.06.05

Fixed a wrong saturation in Dither_sub16 on the last pixels of a clip whose width is not mod-8.

[Motenai Yoda]

@cretindesalpes can u optimize luts by set uexpr and vexpr?

if expr is set then even with u=1, v=1 or chroma="copy", uexpr and vexpr will be calculated (at least with 06_taro's mod).

ie
yexpr = (Defined (y) && (y != 3)) ? "x" : yexpr
uexpr = (Defined (u) && (u == 3)) ? uexpr : "x"
vexpr = (Defined (v) && (v == 3)) ? vexpr : "x"

[colours]

Calculating uexpr and vexpr is effectively free since it only happens during initialisation unless you're using Dither_lut16/etc. in ScriptClip (or other runtime filtering scenarios).

In that case, what you should be doing is to figure out how to avoid runtime filtering instead of performing what would be a microoptimisation for normal people doing normal filtering.

Edit: Someone reminded me that tp7's Masktoolsv2 fork doesn't generate unnecessary LUTs, which may be more relevant than what I wrote above.

[Motenai Yoda]

Originally Posted by colours

Calculating uexpr and vexpr is effectively free since it only happens during initialisation unless you're using Dither_lut16/etc. in ScriptClip (or other runtime filtering scenarios).

yep but for some of my scripts it takes up to 30 mins to initialize (without any ScriptClip).

Originally Posted by colours

Edit: Someone reminded me that tp7's Masktoolsv2 fork doesn't generate unnecessary LUTs, which may be more relevant than what I wrote above.

I definitively will try it!
That guy is making a lot of interesting stuff.

[tormento]

Originally Posted by Motenai Yoda

I definitively will try it!
That guy is making a lot of interesting stuff.

Any compiled build around to test?

[Reel.Deel]

Originally Posted by tormento

Any compiled build around to test?

Check under the GitHub release section; also for future reference this is the discussion thread: http://forum.doom9.org/showthread.php?t=169832

[StainlessS]

Just thought I'de point out that FFTW library used by eg FFT3DFilter as fftw3.dll and dftTest as libfftw3f-3.dll (same dll, just different name)
has been updated twice since the one included in dither tools. Last update March 16 2014, v3.3.4. http://www.fftw.org/

[TheProfileth]

I would like to report a bug regarding Dither_median16 when I use dither_median16 if I set rt=0 and setting rx=1 and ry =1 then the output seems to be broken like it is all black except for about one line, however if I set rx and ry to something higher than 1 and leave rt set to 0 it outputs as expected, the same happens if you simply set rt=1 while rx=1 and ry=1. The weird thing about this behavior is that you can only exhibit it by using the default settings of the function which specifically say

Dither_median16 (
clip src,
int rx (1),
int ry (1),
int rt (0),
int ql (undefined),
int qh (undefined),
int y,
int u,
int v
)

I used this to repeatedly recreate the problem

Dither_convert_8_to_16
Dither_median16()
ditherpost(mode=-1)