Color banding and noise removal

[Dogway]

10 minutes after I release v1.9d? hmmmm.... ok hahahah. No really, thanks a lot, I will have a look at to what I can implement or need to change for v1.95d. : )

[pandy]

is there any way to reduce bit depth with dither to less than 8 bits?

Yes. Reduce the signal before dithering and amplify it afterwards.
bits = 4 # Valid range: 0 - 8
mul = String (Pow (2, 8 - bits))

Dither_convert_8_to_16()
Dither_lut16 (expr="x "+mul+" /")
DitherPost ()
mt_lut ("x "+mul+" *", y=3, u=3, v=3)

Thank You cretindesalpes, sorry for very late reply, it work almost OK.

[Lenchik]

How to apply Spline144Resize (http://forum.doom9.org/showthread.php?p=1356563#post1356563) analog with Dither_resize16 "impulse " notation?

[cretindesalpes]

You'd need to enter the 6-point spline polynomials into a math software and to make it calculate the coefficients with a reasonable oversampling. It would be probably painful and take longer than the few minutes I just spent to add the generic spline kernel in Dither_resize16. Thanks for the pointers, it will be delivered in the next release.

[Boulder]

I've been getting a lot of VirtualDub or x264pipe crashes with the new MVTools version. The crash occurs during the initialization of the script and it always points to fftw3.dll. The version supplied with Dither v1.13.3 seems to work, at least I couldn't make it crash by constant reloading of the script in VDub. I'm on Win7 64-bit with SEt's latest MT build of 32-bit Avisynth.

While fftw3.dll is the subject, do you happen which version of it should be used? There are many of them floating out there and apparently you have to rename a specific build to fftw3.dll since there's now 3 different dlls included in the latest FFTW release.

[cretindesalpes]

Please post your script.

[Boulder]

I'll send it to you in a PM, the function I use is very, very ugly spaghetti code... :)

[cretindesalpes]

I couldn't reproduce the crash (latest Avisynth 2.6 MT). I tried with various sources, everything is playing fine for me here. Have you tried to disable the MT modes? Anyway when the script is highly memory-intensive, sometimes VDub cannot reload it (F2) correctly. I think it tries to keep both versions until the second one is ready. Close the script first then open it again.

For FFTW, I have both the old version (fftw3.dll, 1627136 bytes, 2004-01-30) and the "new" ones (libfftw3-3.dll, libfftw3f-3.dll and libfftw3l-3.dll, 2009-07-19, probably not the most recent files). These versions are different, I think you should get a genuine file, not rename one of the newest files.

[Boulder]

OK, I'll try lowering the SetMemoryMax settings first and see if it works better then.

Regarding FFTW3.dll, I found this: http://forum.doom9.org/showthread.php?t=156028 , near the bottom of the first post. It seems that you can rename libfftw3f-3.dll to fftw3.dll if you want to upgrade. I don't know how MAnalyse uses fftw3 so it's not certain that there actually are any changes, but I'll see if anything weird happens anyway.

[Lenchik]

http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=179 and http://www.mediafire.com/file/8b6dhdat888i9do/SeparateResize_v1.4.2.7z
As ideas for Dither_resize16 from links above:
- setting separate algorithms for luma and chroma planes;
- some algorithms from madVR;
- some non-ringing algorithms maybe (i don't know if we can use Repair in 16bit without bugs of any kind).

[cybersharky]

DGSource("D:\magnum\title00.dgi",fieldop=0)
crop(0, 4, 0, -4)
LanczosResize(640,480)
tr = 6 # Temporal radius
super = MSuper (pel=2)
multi_vec = MAnalyse (super, multi=true, delta=tr)
MDegrainN (super, multi_vec, tr, thSAD=400, thSAD2=150)

Does temporal radius have anything to do with overlap?
Because I see in the examples overlap is used in MAnalyse, when using MDegrain2/3

[06_taro]

http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=179 and http://www.mediafire.com/file/8b6dhdat888i9do/SeparateResize_v1.4.2.7z
As ideas for Dither_resize16 from links above:
- setting separate algorithms for luma and chroma planes;
- some algorithms from madVR;
- some non-ringing algorithms maybe (i don't know if we can use Repair in 16bit without bugs of any kind).

An example for Spline64 luma kernel with SoftCubic75 chroma kernel(but no non-ringing algorithm, RemoveGrain16 and Repair16 is necessary for many scripts to be ported from 8-bit to 16-bit, and adding YV16/YV24 support is also interesting):
Dither_resize16(dest_width, dest_height, kernel="Spline64")
\ .MergeChroma(Dither_resize16(dest_width, dest_height, kernel="Bicubic", a1=0.75, a2=0.25))

[cretindesalpes]

cybersharky:

No, the temporal radius has nothing to do with the overlap. However, omitting the overlap (default is 0) may cause blocking-like artifacts.[/QUOTE]

Lenchik / 06_taro:

Thank you for the suggestions. Actually I prefer keeping the resizer function "simple", and use scripting for the non-linear parts and plane recombination. Anyway I will add a built-in function for colorspace conversions (including simple chroma subsampling conversions) probably in combination with resizing, because scripting these things is quite complicated and not very efficient.

In your example you could use the plane filters to avoid unnecessary computations that will be thrown out:
Dither_resize16(dest_width, dest_height, kernel="Spline64", u=1, v=1)
\ .MergeChroma(Dither_resize16(dest_width, dest_height, kernel="Bicubic", y=1, a1=0.75, a2=0.25))

Even without 16-bit RemoveGrain, it's possible to work in 8 bits for ringing cancellation and merge the modified pixels with the 16-bit results. This would be a good approximation for visible ringing. Pixel-sized artifacts (aliasing, ringing, halos…) generally don't need 16-bit processing to be fixed as they often result in large value changes.

[06_taro]

Agree with keeping the resizer "simple". And applying such a filter on 8-bit clip and adding the diff is usually enough in terms of visual effect. :) But if the aimed output clip is 16-bit, would it result in some artefacts in LSB, such as mosquito around edges for deringing? And although such artefacts in LSB shouldn't be noticeable after dithered to 8-bit, I'm worried if they may reduce the efficiency of compression.

[Lenchik]

Same questions as 06_taro's but for aiming output in 10 bit (http://forum.doom9.org/showpost.php?p=1519194&postcount=252) to 10 bit x264 build.

[cretindesalpes]

Agree with keeping the resizer "simple". And applying such a filter on 8-bit clip and adding the diff is usually enough in terms of visual effect. :) But if the aimed output clip is 16-bit, would it result in some artefacts in LSB, such as mosquito around edges for deringing? And although such artefacts in LSB shouldn't be noticeable after dithered to 8-bit, I'm worried if they may reduce the efficiency of compression.

These artifacts are small relative to the edges that generate ringing and will be most likely crushed by the compression. And artifacts caused by the low-bitdepth processing are much smaller than the visual artifacts caused by the method itself which is far from perfect, or than the noise level that comes with most lossy or real-life video source.

Below is a quick attempt (http://screenshotcomparison.com/comparison/118826). I adopted a slightly different approach than just adding the 8-bit difference.

# Gradient
amt_l = 0.25
amt_c = 0.125
BlankClip (length=1, width=640, height=480, pixel_type="YV12", color_yuv=$000000)
KillAudio ()
base = last
x = "x 0.5 - 32768 * "+String(amt_c)+" * 32768 + 0 65535 clip "
y = "y 0.5 - 32768 * "+String(amt_c)+" * 32768 + 0 65535 clip "
z = "x y + 1 - 32768 * "+String(amt_l)+" * 32768 + 0 65535 clip "
msb = base.mt_lutspa (mode="relative", yexpr=z+"8 >>u 255 &u", uexpr=x+"8 >>u 255 &u", vexpr=y+"8 >>u 255 &u", y=3, u=3, v=3)
lsb = base.mt_lutspa (mode="relative", yexpr=z+" 255 &u", uexpr=x+" 255 &u", vexpr=y+" 255 &u", y=3, u=3, v=3)

# Text
txt = msb
txt = txt.Subtitle ("This is a test.\n\n--\\||//@=#§", align=5, lsp=1, text_color=$E0E020, halo_color=$202020)
txt = txt.Subtitle ("\nTest with light ringing.", align=5, lsp=1, text_color=$848484, halo_color=$808080)
msk = mt_lutxy (msb, txt, "x y != 255 0 ?", y=3, u=3, v=3)
lsb = mt_merge (lsb, base, msk, y=3, u=3, v=3)
msb = txt

StackVertical (msb, lsb)

# Rings. Source: http://www.imagemagick.org/Usage/img_photos/rings_lg_orig.png
rings = ImageSource ("rings_lg_orig.png").Trim (0, -1)
rings = rings.ConvertToYV12 ().Crop (0, 0, 0, base.Height ()).Dither_convert_8_to_16 ()
StackHorizontal (last, rings)


# Resize
dw = Width() * 2
dh = Height()/2 * 2

main = Dither_resize16 (dw, dh, kernel="Spline64" )
nrng = Dither_resize16 (dw, dh, kernel="Gauss", a1=100)

m8 = main.DitherPost (mode=-1)
n8 = nrng.DitherPost (mode=-1)
f8 = m8.Repair (n8, 1)
k8 = mt_lutxy (m8, f8, "x y != 255 0 ?", y=3, u=3, v=3)
f16 = f8.Dither_convert_8_to_16 ()
f16 = nrng.Dither_limit_dif16 (f16, thr=0.5)
Dither_merge16_8 (main, f16, k8, y=3, u=3, v=3)


# Display
main.Subtitle ("Spline64") + last.Subtitle ("nrSpline64")
DitherPost (mode=6)
last
\ + mt_lut (expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128)
\ + k8.GreyScale ().Subtitle ("Ringmask")

[SilaSurfer]

Hey cretindesalpes. Since I'm using a lot of your functions from Dither package, I decided to incorporate Dither_resize16 in my general workflow. Now I want to produce the same output as ResampleHQ, meaning gamma aware resizing, and I wanted to check if I got it correctly:

Dither_convert_8_to_16 ()
Dither_y_gamma_to_linear()
Dither_resize16(w, h, kernel="blackman", taps=5) - I'm using Blackman as my resizer for Bluray sources where I resize down to 720p.
Dither_y_linear_to_gamma()
Ditherpost(mode=-1,ampo=0,ampn=0)

As you can see my sources are YV12. Is this ok?

[cretindesalpes]

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="709", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="709", mode=-1)
But unless you really have a reason, prefer mode=6 (or 0) to mode=-1.

[SilaSurfer]

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="709", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="709", mode=-1)
But unless you really have a reason, prefer mode=6 (or 0) to mode=-1.

Thanks cretindesalpes. About mode parameter I was just posting an example. I will incorporate this into my denoising and dithering workflow from your Dither tools. What about if I would want to downsize to SD, I would need to change Matrix parameter to 601?

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="601", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="601", mode=-1)

Like so?

[cretindesalpes]

Yes for Dither_convert_rgb_to_yuv(), but keep "709" in Dither_y_linear_to_gamma(). This is the same transfer characteristic anyway. I just should add an alias for convenient use.

[SilaSurfer]

Thanks, cretindesalpes

Do you have any convolutions examples for your resizer?

[cretindesalpes]

Dither 1.16.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added generic spline kernel to Dither_resize16.
Added the SMPTE 240M gamma curve and aliases for BT.601 and SMPTE 170M.
Fixed a bug related to chroma subsampling and multi-threading in Dither_merge16.
Bug fixed in MVTools.

——— Edit ———

SilaSurfer:

# Equivalent to mt_convolution (horizontal="1 5 -1 7 -1 5 1", vertical="1", u=3, v=3) for stack16 clips
w = Width ()
h = Height () / 2
Dither_resize16 (w, h, kernelh="impulse 1 5 -1 7 -1 5 1", fh=-1, center=false)

[Reel.Deel]

Thanks cretindesalpes!!

[SilaSurfer]

Dither 1.16.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added generic spline kernel to Dither_resize16.
Added the SMPTE 240M gamma curve and aliases for BT.601 and SMPTE 170M.
Fixed a bug related to chroma subsampling and multi-threading in Dither_merge16.
Bug fixed in MVTools.

——— Edit ———

SilaSurfer:

# Equivalent to mt_convolution (horizontal="1 5 -1 7 -1 5 1", vertical="1", u=3, v=3) for stack16 clips
w = Width ()
h = Height ()
Dither_resize16 (w, h, kernelh="impulse 1 5 -1 7 -1 5 1", fh=-1, center=false)

Thanks cretindesalpes for updated version and the example.

[SilaSurfer]

Dither_convert_8_to_16 ()
Dither_resize16 (1280, 720, kernel="impulse -1 6 -1",
\ fh=-1, fv=-1, cnorm=true, center=false)
DitherPost (mode=-1)

I also tried your example from the docs and the result is just beautiful. But I had to change it from this:

Dither_convert_8_to_16 ()
Dither_resize16 (Width (), Height () / 2, kernel="impulse -1 6 -1",
\ fh=-1, fv=-1, cnorm=true, center=false, y=3, u=2, v=2)
DitherPost ()

If I specify my own resolution, that is without /2, I get some color distortion, I had to remove y=3,u=2,v=2 parameters to normalize it. Is it save to use the way I posted it in the first example? Thanks for your patience.

[Boulder]

Out of interest: have you had any chance to look at the improvements made to MVTools v2 by the SVP team?

[cretindesalpes]

Not yet. But I'll have a look when possible.

[ajp_anton]

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="709", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="709", mode=-1)
But unless you really have a reason, prefer mode=6 (or 0) to mode=-1.Why go to RGB for resizing?

[cretindesalpes]

It’s not for resizing, it’s for gamma correction (to do things correctly).

[ajp_anton]

Isn't gamma_to_linear doing the gamma "correction"? Or is linear scale not well defined for the UV planes or something?

[Lenchik]

ajp_anton
You may find these links useful for understanding idea of linear light resize and gamma correction.
http://www.glennchan.info/articles/technical/chroma/chroma1.htm
http://archive2.avsforum.com/avs-vb/showthread.php?p=4811927&&#post4811927
http://www.4p8.com/eric.brasseur/gamma.html
http://www.imagemagick.org/Usage/resize/#resize_colorspace
http://forum.doom9.org/showthread.php?p=1279308#post1279308 (and posts nearby)

[ajp_anton]

Tested with both "earth_lights" and that monk image, and indeed I only get correct results in RGB. Maybe they are extreme examples, but then again, why are we resizing in linear light unless we want it perfect =).
Don't know exactly how subsampling should be handled though, didn't test it.

[06_taro]

cretindesalpes, the BT.709 coefficients matrix used in dither package is:
kr = 0.2125
kg = 0.7154
kb = 0.0721

That is the outdated version in early draft of ITU.709, and the latest one in ITU.709-5 is:
kr = 0.2126
kg = 0.7152
kb = 0.0722

Hope to correct it in the next release.

[cretindesalpes]

Thanks. I've taken the coefficients from the Avisynth documentation (http://avisynth.org/mediawiki/Color_conversions), without checking if they were right. It's fixed now, it will be available in the next release. I also corrected the Avisynth page.

[06_taro]

Another useful thing is introducing YCgCo matrix in Dither_Convert_YUV_TO_RGB/Dither_Convert_RGB_TO_YUV, as 8-bit RGB can be losslessly converted to 8-bit Y and 9-bit Cg/Co (full-range and YV24). Padding such YCgCo to 9/10-bit and encoding them with x264 has been possible now, and madVR can render YCgCo matrix correctly. But old YCgCo tools (jmac698's script (http://forum.doom9.org/showthread.php?p=1510465#post1510465) or xv's plugin (http://forum.doom9.org/showthread.php?p=1510268#post1510268)) can only converted 8-bit RGB to 8-bit YCgCo, which is still lossy. It would be great if Dither implement it.

[tormento]

cretindesalpes, have you noticed LLVM freeware compiler has added CUDA support?

I'd like to see some compiles to understand if it could help heavy scripts.

Thanks for any effort ;)

[SilaSurfer]

hello cretindesalpes. I was wondering something. I'm working with a source (bluray) which inherits a lot of "bad" grain. My goal is to remove it and redither the source, but I want to use just noise without bayer matrix in Ditherpost(ampo=0,ampn=?). The noise just adds that feeling to the movie. I was thinking "Mode=2" in DitherPost, correct? Or can I use Error Diffusion modes just with noise? I don't wanna use Grandfun3 :p. Thanks in advance.

[cretindesalpes]

Dither 1.17.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added the YCgCo matrix in RGB conversions.
More dithering settings exposed in RGB conversions.
Fixed a very small error in BT.709 coefficients.
Bug fixed in MVTools: artifacts related to overlap in MDegrainN with tr > 3 in multithreading mode.

SilaSurfer:

Error diffusion modes (6-8) would be right for this. Anyway if you don't want the ordered dithering with modes 0-5, only the noise, you can just set ampo=0 as you suggested. You can also generate the noise with Dither_add_grain16 before dithering with mode 6.

[jmartinr]

Dither 1.17.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added the YCgCo matrix in RGB conversions.


:thanks:

[SilaSurfer]

SilaSurfer:

Error diffusion modes (6-8) would be right for this. Anyway if you don't want the ordered dithering with modes 0-5, only the noise, you can just set ampo=0 as you suggested. You can also generate the noise with Dither_add_grain16 before dithering with mode 6.

Sorry cretindesalpes for my previous question. That moment I still felt effects of a birthday party which ended couple of hours before my writing, :rolleyes: so I posted a confusing question. I'm going to make a simple post. I always use error diffusion modes with Ditherpost, I just wondered if I can dither with those modes just using noise.

My Regular use:

...
DitherPost(mode=6,ampo=1.0-1.5, anpn=0.5-0.8) - my setup for many sources.


This case:

...
Ditherpost(mode=6, ampo=0, ampn=2) - will this be Ok, it shouldn't break anything right?

Again sorry for confusing post and taking already precious time from you. BTW Thanks for your work (especially Mvtools2), I've completely switched to your functions in my video encodings. Thank you for sharing your work with us here. Best regards

[cretindesalpes]

Ditherpost(mode=6, ampo=0, ampn=2)
It is valid but won't have any specific effect. In error diffusion modes, setting ampo to a value below 1 is currently ignored. Use mode=-1 if you don't want the noise specifically added by the error diffusion (it turns the correlated quantization error into a noise whose power is mainly located in high frequencies)

[SilaSurfer]

It is valid but won't have any specific effect. In error diffusion modes, setting ampo to a value below 1 is currently ignored. Use mode=-1 if you don't want the noise specifically added by the error diffusion (it turns the correlated quantization error into a noise whose power is mainly located in high frequencies)

Thanks for your answer. So Modes 0-5 can have ordered dithering deactivated setting ampo=0, while modes 6-8 just ignore it. Got it. I decided to go with my usual setting and I'm going to add some grain with Dither_add_grain16. Just one more question, your MVTools2 mod MdegraiNN is internally multithreaded right, like MVDegrainMulti? I get speed ups compared to standard MVTools2 from Fizick.

[Yellow_]

On the same subject as SilaSurfer, I've been using 06_taro's 16bit mod of GrainFactory3 instead of Dither's 16bit addgrain, so when it comes to DitherPost if I use mode=-1 to avoid additional noise, ampo doesn't matter for modes 6 - 8? Most natural dither at expense of bitrate would be choice of 6 - 8?

[cretindesalpes]

ampo is ignored in mode 6–8 for values < 1 only. When set to a value > 1, the diffused error is augmented from ampo-1, which actually amplifies the dithering pattern in a somewhat different way from just lowering the target bitdepth ; it's more progressive. But I don't know if it's really useful.

And yes, MDegrainN is multithreaded (as well as MSuper and MAnalyse, but partially for these ones), with the help of avstp.dll.

[GMJCZP]

Hi cretindesalpes:

When I use version 2.0.6.2 MVTools (I have not tested the latest version) I have problems with SetMTMode, I hang up VirtualDubMod and the error message tells me about a ntdll.dll file (bug report). I use Windows XP SP3 and the version 31/03/2012 2.6.0 SEt Avisynth MT. If I use MVTools 2.5.11.3 SetMTMode works well, but with version 2.0.6.2 I have to disable SetMTMode.

[the_weirdo]

@GMJCZP:

You may want to check the latest version (2012.05.06) of SEt's Avisynth 2.6 MT.
http://forum.doom9.org/showthread.php?p=1574708#post1574708
Not sure if that fix is related to your problem though.

[GMJCZP]

I did a test with the new versions of MVTools and Avisynth and seems to work best, however it would be good to check if there is a relationship with ntdll.dll and operation of MVTools.

[ajp_anton]

dither_add16, merge16, and maybe others complain about the clips' format being different. The only difference is their length, and is fixed with a trim.
Afaik, the regular mt functions just loop the last frame of the shortest clip. Can you fix that?

[cretindesalpes]

I don't think it has to be "fixed". Trying to combine clips of different length is often a sign that something is borked in the script or the source files, and sometimes it's something subtle. I'd rather have the problem detected as soon as possible, not after hours of encoding. And if mixing different lengths is desired, I would explicitly Trim or LengthenClip at the appropriate place. Frankly, I had too many headaches because of a lack of checks on the plug-in side. But what do others think about it?

[ajp_anton]

I've used this to merge a single frame to a longer clip, as well as merge multiple recordings from a single VHS tape. The latter is what I'm doing right now. It's easier to not care about their lengths, just sync, merge, and then worry about trimming it.
It's not a big deal though, it was just unexpected and spent half an hour trying to figure out why they had different "formats"...