Color banding and noise removal

[Dogway]

Is it possible to sharpen between dfttest and dither?

[cretindesalpes]

I'm having a bit of difficulty getting this to work properly in the absence of a denoiser.

I fear that calling dfttest with sigma=0 will do absolutely nothing excepted warming your room and wasting your time.

Ideally, your denoiser could be able to output 16-bit data. I haven't studied how TNLMeans works, but I wouldn't be surprised if its internal computations are done at a higher bit depth than its actual output.

Anyway, to deband a denoised clip, you can simply use the GradFun3 function. Just pass it your denoised clip and it will act like Gradfun2db. No need for additional DitherPost. But these functions just try to repair the harm done by the 8-bit quantification happening in the denoiser. If you don't want to use dfttest/MDegrain as main denoiser, restore a small bit of noise in your previously filtered clip and use dfttest with a low sigma (1.0 for example).

# Your favourite denoiser here
denoised = MyExcellentDenoiser (parameters)

# Restore a bit of noise
mt_lutxy (last, denoised, expr="x y < y 1 - x y > y 1 + y ? ?", y=3, u=3, v=3)

super = MSuper ()

# You could also reuse the vectors found in a previous processing stage
vf1 = super.MAnalyse (isb=false, delta=1, overlap=4, blksize=16)
vb1 = super.MAnalyse (isb=true, delta=1, overlap=4, blksize=16)

# Turns remaining noise into 16-bit gradients
cf1 = MCompensate (super, vf1, thSAD=200)
cb1 = MCompensate (super, vb1, thSAD=200)
Interleave (cf1, last, cb1)
dfttest (sigma=1.0, tbsize=3, lsb=true)
SelectEvery (3, 1)

# Additional smoothing pass
tmp = last
SmoothGrad (thr=0.20, radius= 8, stacked=true, ref=tmp)
SmoothGrad (thr=0.17, radius=12, stacked=true, ref=tmp)
SmoothGrad (thr=0.15, radius=16, stacked=true, ref=tmp)

# Back to 8 bits
DitherPost (prot=false, stacked=true)

EDIT : better use the denoised clip for the MAnalyse steps.


Is it possible to sharpen between dfttest and dither?

Sure, although I think it's better to use the sharpener afterwards, with a mask to avoid sharpening the remaining noise/grain and the dither patterns.

w = Width ()
h = Height ()
dfttest (sigma=1.0, tbsize=1, lsb=true)
msb = Crop (0, 0, w, h)
lsb = Crop (0, h, w, h)
msb_sharp = msb.Sharpen (0.5)
mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

[Heaud]

Is there a way to restore fine detail, any lost sharp edges, and an adjustable amount of noise back? Using dfttest to denoise can sometimes soften the hard edges on certain sigma values. Setting it too low causes ditherpost to do almost no dithering and setting it too high smoothes hard edges (animation) and fine detail is lost.

I noticed that QTGMC has a feature to restore lost noise/detail from the denoising stage using a script: # Add back any extracted noise, after final temporal smooth. This will appear as noise/grain in the output
# Average luma of FFT3DFilter extracted noise is 128.5, so deal with that too
addNoise2 = (NoiseRestore > 0.0) ? lossed2.mt_adddiff( deintNoise.mt_lut( "x 128.5 - " + string(NoiseRestore) + " * 128 +", U=1,V=1 ), U=2,V=2 ) : lossed2 Could that be used in junction to dfttest+ditherpost?

What about the lost edges, could a repair function fix that?

[-Vit-]

That's just the last line of the noise bypass. The bare principle ignoring a few steps is this:

source=last
clean = source.YourFavoriteDenoiser()
noise = mt_makediff( source, clean, U=3,V=3 )

# Calculate motion vectors on clean source ...

processed=clean.OtherProcessing() # QTGMC does it's deinterlacing here
#...

restored1 = processed.mt_lutxy( noise, "x y 128 - " + string(DetailRestore) + " * 128 +", U=3,V=3 )

smoothed = restored1.MoCoSmooth() # Some kind of motion-compensated temporal smooth

restored2 = smoothed.mt_lutxy( noise, "x y 128 - " + string(NoiseRestore) + " * 128 +", U=3,V=3 )


The first restoral tries to recover detail that was misrecognized as noise in the denoising step - it may survive a temporal smooth compensated by motion vectors calculated from the clean source. The second restoral adds in some measure of true noise as you wish.

This part of QTGMC is still work in progress.

[Dogway]

Im making up my mind to make this filter work without changing my workflow too much.

Normally my filtering methods go as follow:
input=last
temp=mdegrain...
...
denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
luma=lumamask(temp,denoised,1,40)
ContraSharpening(luma, input)
Im trying to put ditherpost into play here adding:
msb_sharp = msb.ContraSharpening(luma, input)
mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)
but a warning message warns me:
Repair: clips must be of equal type, line 160.
I think it has to be with comparing 8 bit against 16 bit data in the contrasharpening. Besides I have been reading the dither documentation and its unclear about the msb and lsb variables. It looks like when dfttest is inside a variable msb/lsb can't be called, which renders the masking and contrasharpening not being usable.
Maybe its not very flexible, but Im curious to know the improvements when encoding patterned dithering specially on anime content.

[cretindesalpes]

Modified dfttest or MDegrain with parameter lsb=true will always produce a double-height clip, with the pixel MSB stacked on top of the LSB. You have 2 solutions get back to a 8-bit clip :

– Extract the MSB with a Crop. This is the simplest solution, although the maximum rounding error is suboptimal. The operation is actually value_8bits = floor (value_16bits / 256.0), leading to an error in the 0.0 – 1.0 range. Use it only if you need to process the clip in the 8-bit land and substitute the result to the previous MSB before dithering. This will give you a better chance to maintain the MSB/LSB consistency.

– Just round the pixel values, as it would be done at the output of a regular denoiser without the 16-bit kludge. Just pass your clip to DitherPost (mode=-1, stacked=true, prot=false). Here, value_8bits = round (value_16bits / 256.0), leading to an error in the -0.5 – +0.5 range.

input=last
temp=mdegrain...
...
denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
rounded = denoised.DitherPost (mode=-1, stacked=true, prot=false)
luma=lumamask(temp,rounded,1,40)
...
msb_sharp = msb.ContraSharpening(luma, input)
mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

You can also contrasharpen the rounded clips, compute the difference between before and after using mt_makediff and add this difference back on the final result.

[Dogway]

I really would like to understand better what MSB and LSB are, documentation is very loose. As I understand it, MSB is the 16bit denoised output, while LSB is the denoised 8bit output, which are stacked one above the other (thus the double height). In this situation where we have only denoiser+ditherpost we use stacked=true, and ditherpost will automatically select (crop) to obtain the MSB and LSB. But if we do some working in between we need first to separate msb and lsb by cropping ourselves.
_______
| |
| MSB |
|_______|
| |
| LSB |
|_______|
What you propose is to go back to 8bits, do the stuff and then in ditherpost compare that and lsb, right?
Right now I have as follows:
w = Width ()
h = Height ()

temp=mdegrain

denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
lsb = denoised.Crop (0, h, w, h)
rounded = denoised.DitherPost (mode=-1, stacked=true, prot=false)
luma=lumamask(temp,rounded,1,40)

msb_sharp=ContraSharpening(luma, temp)

mask = DitherBuildMask (msb_sharp, luma)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

Is it correct? I edited out, as I think I figured out the working. Still I have a question, msb_sharp in ditherpost is already 8bit (done in rounded), is it ok?

I might try your last suggestion as well. Anyway the effect is too subtle for my source after testing with several parameters, looks more like banding prevention than dithering.

[cretindesalpes]

I really would like to understand better what MSB and LSB are

Here, MSB/LSB stands for Most/Least Significant Byte. The color depth of the filter output is 16 bits per component (total: 48 bits) but Avisynth can only handle 8-bit pixel values. Therefore we have to split each 16-bit pixel value into a pair of 8-bit pixels. It's exactly like separating digits of numbers from 0 to 99 into tens and units.

There are many ways to organize these pixels. We can store the MSB and LSB on separate frames, group them horizontally or vertically, or group all MSB and LSB and stack them. I choose the latter for the denoiser output, stacking MSB and LSB vertically. Below is an example of what it looks like :

http://img218.imageshack.us/img218/8765/msblsb.th.png (http://img218.imageshack.us/i/msblsb.png/)

Original               | Without dither (rounded) |          Dithered
http://img24.imageshack.us/img24/7480/originalci.th.png (http://img24.imageshack.us/i/originalci.png/) http://img222.imageshack.us/img222/5058/rounded.th.png (http://img222.imageshack.us/i/rounded.png/) http://img338.imageshack.us/img338/2183/dithered.th.png (http://img338.imageshack.us/i/dithered.png/)

The MSB part is very close to the simple 8-bit picture, but the full precision requires a combination of the LSB and the MSB.

What you propose is to go back to 8bits, do the stuff and then in ditherpost compare that and lsb, right?
Technically, it's not exactly that, but you got it. The LSB is not compared, it is added. To take the previous comparison again, it's like sharpening using the tens, and adding the units at the very end. On gradients, the sharpening shouldn't change anything, so the operation should be transparent regarding gradients. But we need to protect the detailed zones and their close neighbourhood.

Now, how to get the tens ? Let's suppose we have a pixel value of 28. If we take just the tens (the Crop method), we get a 2. If we round before to the closest ten (the other method), we get 3. The second method is indeed better, especially if we have to compare the clip with the original. But let's say we are in a gradient area, and the sharpener changes nothing. If we inject back the 3 as MSB, we got a 38, which is a (relatively) big error. Nothing should have changed. Therefore we have to work with differences (between before/after processing, or between before/after rounding, your choice).

Regarding this issue, my first attempt at modifying your script wasn't right. Here is another version based on your previous post:

w = Width ()
h = Height ()

temp=mdegrain

denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
msb = denoised.Crop (0, 0, w, h)
lsb = denoised.Crop (0, h, w, h)
rounded = denoised.DitherPost (mode=-1, stacked=true, prot=false)
luma=lumamask(temp,rounded,1,40)

contra = ContraSharpening(luma, temp)
contra_d = mt_makediff (contra, rounded, y=3, u=3, v=3)
msb_sharp = mt_adddiff (msb, contra_d, y=3, u=3, v=3)

mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

I supposed your lumamask function handles the chroma planes too.

[Dogway]

Sweet, now it works as supposed. I will do some testings with parameters and encodings tomorrow, but I like what I see. At last it turned to be your suggestion what worked out.
A bit offtopic but as I see you are used with anime, do you recommend me going on tbsize=1 over dfttestmc? even mocomp'ed it introduces some artifacting but well, just asking people's doings.

PD: I edited my previous post for easier reading.

[cretindesalpes]

A bit offtopic but as I see you are used with anime, do you recommend me going on tbsize=1 over dfttestmc? even mocomp'ed it introduces some artifacting but well, just asking people's doings.

The tbsize=1 was just for the sake of simplifying the example; I find it introduces too much blurring. I generally use a combination of MDegrain3 and MC dfttest with tbsize=5, both with a low thSAD (200 or even lower) and a prefiltered motion search clip. It's not perfect, but it's still a very good compromise and I rarely have ghosting problems. Then I finalize the gradient smoothing with a single SmoothGrad pass with thr=0.15 before DitherPost.

[Heaud]

Could you post the script that uses the combination MDegrain3 and MC dfttest? I would like to try it out.

[cretindesalpes]

I already posted here (http://forum.doom9.org/showthread.php?p=1390594#post1390594) an earlier version of this script. It is a bit rough and has to be finetuned (especially for the motion search clip), but the basic concept remains the same.

[Dogway]

It produces flatter color areas and preserves more details, compared to my gradfun workflow. Lossless compressed to FFV1 chunk weighted less (163Mb) than the gradfun method (189Mb) which indicates that it is more efficient fot bits allocation in h.264. Im using a very high bitrate for h.264 but (ordered) dither encodes nicely. Its perfect for 2D animation. Im gonna try your last suggestions, and methods to see if I can improve quality or optimization.

edit: It also deals nicely with the problem I was having with GradFun2DBmod (http://forum.doom9.org/showthread.php?p=1475973#post1475973) (ghosting)

[Heaud]

Dogway, would your ghosting problem be similar to mine (http://doom10.org/index.php?topic=919.0)? It's not the filter that is causing it for my source. I've tried many different ways and I noticed that it has been occuring in b-frames.

[LaTo]

edit: It also deals nicely with the problem I was having with GradFun2DBmod (http://forum.doom9.org/showthread.php?p=1475973#post1475973) (ghosting)

GradFun2DBmod can not create ghosting, it is because your source's filter is not accurate in temporal domain.

Set temp=-1 (disable temporal stabilization) in GF2DBmod is a good workaround.

[Dogway]

Without dither (http://i212.photobucket.com/albums/cc35/Dogway/Misc/no-dither.png)
GradFun2DBmod temp=-1 (http://i212.photobucket.com/albums/cc35/Dogway/Misc/gradfun.png)
Dither (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither.png)
Maybe ghosting is not the right word.

[SilaSurfer]

@cretindesalpes

Very nice! I have a two questions. This is a standard usage call for Dither?

h = Height ()
w = Width ()
dfttest (lsb=true) # Output contains stacked MSB-LSB on the same frame.
msb = last.Crop (0, 0, w, h)
lsb = last.Crop (0, h, w, h)
DitherPost (msb, lsb, prot=false)

and

Could one for example use only Dithering?

TiA

[Dogway]

@SilaSurfer: if thats all you are going to do after dfttest, then I think you can just use:
dfttest (lsb=true)
DitherPost (stacked=true, prot=false)

[cretindesalpes]

Maybe ghosting is not the right word.
Indeed, this is haloing, not ghosting. This issue with gradfun2db is known and has been described here (http://forum.doom9.org/showthread.php?p=1159205#post1159205). This is related to how the gf2db filter works. Gradfun2db averages a large area around the processed pixel (box filtering), because box filtering is very fast and its speed is almost constant whatever the size of the box. The resulting 16-bit pixel value is compared to the current pixel and replaces it if the difference is small enough (with a smooth curve controlled by thr). Then, the result is dithered to 8 bits using error diffusion.

Actually the box filter + threshold is intended to be a simplification of a bilateral filter. It should have rejected input values which are too far away from the processed pixel, but instead it rejects output values which are not very close from the current pixel. In most case this works pretty well, but sometimes it fails and generates large, subtle halos near gradient boundaries.

This is a standard usage call for Dither?
Yes for the purpose of debanding, but now you can write this even simpler:

dfttest (lsb=true)
DitherPost (prot=false, stacked=true)

Could one for example use only Dithering?
Of course, if you can generate a 16-bit input. The general purpose of dither is basically converting high bitdepth data to a lower bitdepth while preserving the perceptive quality. The most interesting uses I can think of are high quality levels and colorspace conversions. For debanding purpose, you'll need a filter first to generate the 16-bit gradients from a 8-bits picture. The exact kind of filter depends on the nature of your source. Noisy: use a noise filter operating on large areas (like dfttest). Already banded: use a gradient rebuilder (like SmoothGrad).

[SilaSurfer]

I'm confused here.:D

dfttest (lsb=true)
DitherPost (prot=false, stacked=true) #Debanding

What if I wanted to use Mdegrain also?

Sorry guys for the confusion, can't test this since encode is running here for the next 22 hours.

[cretindesalpes]

dfttest (lsb=true) #Denoising
DitherPost (prot=false, stacked=true) #Debending


Not exactly : dfttest = denoising + debanding (or banding prevention). DitherPost = turns the previous result into something watchable and somewhat resilient to lossy video compression.

MDegrain* (lsb=true) does the same job as dfttest, but only temporally. Therefore it cannot remove banding when it is already in the source, but won't introduce new banding.

[SilaSurfer]

Thanks cretindesalpes

So if I want to use DFTest

dfttest (lsb=true)
DitherPost (prot=false, stacked=true)

Mdegrain

Mdegrain (lsb=true)
DitherPost (prot=false, stacked=true)

[cretindesalpes]

Yes, just replace MDegrain with MDegrain1/MDegrain2/MDegrain3, and add at least the mandatory parameters required for the MDegrain* filters (superclip and motion vectors, see the MVTools2 documentation).

[mp3dom]

For what I've seen, debanding filters in general suffers on dark areas. For example a thick black line in a dark gray background once debanded have the risk to be completely lost. I'm currently interested in only debanding (not denoise+debanding) so I think I'd need to use SmoothGrad (am I right?). In that case, have SmoothGrad some routines to avoid this (seems) common problem in dark areas without lowering the debanding strength on all other parts of the image? (I've not tested it yet, so I'm just asking). Thanks.

[cretindesalpes]

In that case, have SmoothGrad some routines to avoid this (seems) common problem in dark areas without lowering the debanding strength on all other parts of the image?
No, SmoothGrad does just the basic debanding task and works just like gradfun2db, leaving the dithering for DitherPost. I even think about splitting the function again. You can protect the very dark shades this way: apply a curve to amplify the dynamic range of low luma (something like a gamma correction), SmoothGrad, then apply the inverse curve. Computation in the 16-bit land with 8-bit clips is a bit tricky, but it looks workable with the Masktools (mt_lutxy is your friend). You can also process the clip using two SmoothGrad with different settings, and use a luma mask to select the right parts.

However SmoothGrad was intended to be a postprocessor for the main filtering functions and therefore operates directly on a 16-bit clip, so you'll have to convert your clip to 16 bits. Just stack it on the top of a BlankClip with color_yuv=$000000. GradFun3 is a more or less sophisticated wrapper around SmoothGrad + DitherPost that would replace gradfun2db, but I'm not totally happy with its current implementation. I may rework it later.

[Dogway]

I found out that you dont need the quotes when using the filter string in ditherpre. But I wanted to note that despite it works wonders when it works, in ocassions it can smooth too much the results, more than dithering banding. I can tell from the last source Im testing with, it causes the lineart to blur. I can post comparisons.
What I thought was to use no dither (mode=-1) for lineart, and dither for the rest, but the effect is the same and running 2 instances (one with ditherpre, and one without) is a bit of CPU waste, moreover when Im using tnlmeans.

The code would come as follows:

line=src.MT("tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6,sse=false)",2,2)
flat=Dither2Pre(line,stacked=true)

Would this run tnlmeans twice, or will it just use the buffered image for dither2pre?

Also I have discovered some big ghosting problems (not halo-like like gradfun) in fast motion scenes. This is strange because I didnt think "Dither" had temporal qualities.

[cretindesalpes]

I found out that you dont need the quotes when using the filter string in ditherpre.
If you don't put the double quotes, Avisynth will interpret the filter as a command to process last and will use the result as input (first parameter) for DitherPre. And because you probably multiplied the filter strength by 2 or 4 depending on the DitherPre function you use, you'll get maximum blur here. Moreover, the clip will be filtered a second time with the default filter. This is clearly not what you want.

Would this run tnlmeans twice, or will it just use the buffered image for dither2pre?
This will run tnlmeans once, and the internal default filter (a vanilla dfttest) four times. This code should work better:
Dither2Pre (flt="tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6,sse=false)", stacked=true)I don't know exactly which function you should wrap with MT() for maximum performances.

Also I have discovered some big ghosting problems (not halo-like like gradfun) in fast motion scenes. This is strange because I didnt think "Dither" had temporal qualities.
The ghosting comes from dfttest used in DitherPre as default filter. Dfttest is not protected against ghosting, therefore you have to MCompensate/Interleave the previous and next frames first, and SelectEvery the result (see the dfttestMC function). Regarding motion compensation, it is advised to do it outside the DitherPre functions. Avoid using dfttestMC (or other motion-aware filters) in the filter string, because the motion estimation may fail on the material actually submitted to the filter.

[Dogway]

Thank you. In this case tnlmeans works best with MT. I had to use three quotes, which is what I did first, but Dither weakens the denoise effect so I thought it wasn't working at all or something. Then I got fooled with the "clip" parameter I guess.
That explains all the issues I was having, so hopefuly I dont need any edgemask workaround anymore.

Now what Im trying to do is some resizing in between, the code I have is as follows, but Im unsure if I can resize msb and lsb without ditherposting them first.

den=Dither2Pre(flt="""MT("tnlmeans(settings)",2,2)""",stacked=true)
msb=den.Crop(0,0,w,h)
lsb=den.Crop(0,h,w,h)
rounded=den.SmoothGrad (stacked=true).DitherPost(mode=-1,stacked=true,prot=false)
contraV=rounded.splinaa(720,480).LSFmod()
contra_d=mt_makediff (contraV,rounded.spline36resize(720,480),y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb.spline36resize(720,480),contra_d,y=3,u=3,v=3)
mask=DitherBuildMask(msb_sharp,msb.spline36resize(720,480))
DitherPost(msb_sharp,lsb.spline36resize(720,480),prot=false,mask=mask)

I omitted the parameters but I realised how they need to be tweaked to obtain the same denoising as without using dither. In help, you say "amplitude-relative parameters must be doubled, and powers quadrupled" (for Dither1Pre which Im testing with), yet I dont get the values:
tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6)
tnlmeans(ax=20,ay=20,az=4,sx=4,sy=4,bx=1,by=1,h=2.4)
Are you sure filter behaviour is maintained?

[cretindesalpes]

Now what Im trying to do is some resizing in between, the code I have is as follows, but Im unsure if I can resize msb and lsb without ditherposting them first.
Resizing a 16-bit picture is possible but prone to (light) artifacts. You should resize the lsb with BilinearResize in order to minimize the overshoots. Also, use prot=true in DitherPost. Anyway, I recommend to do 16-bit operations only on the final resolution. Resizing 16-bit data would not be an issue if we had 16-bit resizers... PhrostByte's ResampleHQ could do the job but it seems he's not interested in developing true 16-bit input/output.

In help, you say "amplitude-relative parameters must be doubled, and powers quadrupled" (for Dither1Pre which Im testing with), yet I dont get the values:
tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6)
tnlmeans(ax=20,ay=20,az=4,sx=4,sy=4,bx=1,by=1,h=2.4)
Are you sure filter behaviour is maintained?
I don't know much TNLmeans so I can't give an definitive opinion. Anyway from what I read in the doc, ax, ay and az are spatial or temporal parameters, so they shouldn't be modified. Only a and h are related to pixel values, and I guess both must be multiplied by 2 in this case.

[Dogway]

uhmm As you said doubling only a and h seemed to work almost identical, I then applied your suggestions to ditherpost. Only after I realised of some artifacts so-to-speak, which I linked to the fact of resizing in 16bits. So I run a few tests up to a simple ditherpre-ditherpost, and still there, so maybe there's need to tweak more tnlmeans parameters, or they are wrong....

These are the two shots for comparison, the first one running only tnlmeans and dithering with gradfun, the second one with this workaround with dither. I want you look at the wall, grass loses some detail too.

old-workflow (http://i212.photobucket.com/albums/cc35/Dogway/Misc/no-dither-1.png)
dither-workflow (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither-1.png)

edit: the issue could be in ditherpost as well, although I tried several modes(-1,0,6,7) and didn't see any improve.
edit2:about ResampleHQ it would be nice in the meantime, bear in mind even yv12 dither isn't implemented. Its a bit pity to develop a 16b processing filter to benefit from wider bitdepth, and then crush it down to 8 back again.

[cretindesalpes]

The artifacts don't come from DitherPost, the denoising stage is clearly the cause here... Try to keep a or h at its original value, or even both? I don't know exactly how these parameters affect the algorithm.

[cretindesalpes]

New Dither version (1.6), some important changes :
Masktools 2 a48 is now the minimum requirement because some Dither functions use the new bitwise operators. I don't know how to automatically detect the Masktools version, so be sure to update your Masktools before installing this Dither version.
Renamed the rem parameters into clsb in DitherPost and SmoothGrad functions
Changed the default for the stacked (now true) and prot (now false) parameters.
The modified MVTools2 have been synchronised with the latest official release (2.5.11.2 beta)
Added various utility functions, mainly Dither_lut8, Dither_lut16, Dither_add16 and Dither_merge16.
Some documentation improvements

[Heaud]

Great to see an update!

I have a question on the new utility functions. Would dither_convert_8_to_16 be able to replace the blank clip + stackvertical inside of gradfun3? Is there any other ways to improve gradfun3? I added a custom smoothgrad since I did not like the results of the orignal smoothgrad inside of the script (it was causing weird banding halos).

[cretindesalpes]

I have a question on the new utility functions. Would dither_convert_8_to_16 be able to replace the blank clip + stackvertical inside of gradfun3?
Yes, that's exactly what it does. But it's just a convenient way to write it in a script, it gives the same result and won't improve anything in GradFun3.

[Dogway]

I did a few dozen tests, tweaking sx,sy,bx,by,a and h, even removing MT(). None of them sort the artifact, furthermore I tried dither2pre to "gain" more accuracy but what I got were more artifacts, check the lineart. (h is quadrupled according.)

old-workflow (http://i212.photobucket.com/albums/cc35/Dogway/Misc/no-dither-1.png)
dither1pre (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither-1.png)
dither2pre (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither2pre.png)

Im gonna test dither 1.6 now so I can tell you. edit: same results

[cretindesalpes]

Thank you very much for testing and helping me to improve the functions. I'm going to rewrite Dither1Pre to make the "slices" overlap, you'll tell me if it works better when it's ready.

[TheProfosist]

to me "Dither+SmoothGrad" seems like the best option. i would like to use it but cant determine from the help file what exactly you used/did. if it helps i usually denoise using fft3dgpu/fft3dfilter and sharpen using LSFmod before debanding.

[HymmexyLeadia]

im using a condensor mic to record samples, and theres a bit of noise in the background, i use audacity, but the noise removal algorithm isnt very good.
is there any software that does this better? thanks.
_________________
God rest ye, little children; let nothing you affright, For Jesus Christ, your Saviour, was born this happy night; Along the hills of Galilee the white flocks sleeping lay, When Christ, the Child of Nazareth, was born on Christmas day.
http://www.chung-sim.com/omron.3g3jv.html

[cretindesalpes]

Dither upgraded to v1.7 (http://ldesoras.free.fr/src/dither-1.7.zip). Changes:

Slightly improved Dither1Pre and Dither2Pre functions, added the overlap and tvopt parameters.
People complained about GradFun3 being too strong and washing fine shades away, so I changed the default parameter values for a better match with the gradfun2db default settings. thr is now set to 0.35 and radius to 12.
_____________________________________________________________________

to me "Dither+SmoothGrad" seems like the best option. i would like to use it but cant determine from the help file what exactly you used/did. if it helps i usually denoise using fft3dgpu/fft3dfilter and sharpen using LSFmod before debanding.See: http://forum.doom9.org/showthread.php?p=1390594#post1390594
but replace DitherPost (mode=2) with SmoothGrad (radius=16, thr=0.25, elast=2).DitherPost ()

[coolguy89]

Hi,
I tried your example above, but, the resolution changed from 720x480 to 720x240. Did I do something wrong?

[cretindesalpes]

You probably have a DitherPost() too many. Also make sure you have the latest Dither version. A few default parameters have changed recently and I updated the script in the post #8 accordingly.

[coolguy89]

This is what I use

FFT3DGpu (Sigma=1)
DitherPost (mode=2)

Function zzz_denoise (clip src, float "sigma", int "thr", bool "mask", int "sad")
{
sigma = Default (sigma, 16)
thr = Default (thr, 5)
mask = Default (mask, False)
sad = Default (sad, 200)

w = src.Width ()
h = src.Height ()

# Motion analysis
super = MSuper (src)
super_a = MSuper (src.TTempSmooth ().RemoveGrain (12))

fwd_vect_3 = super_a.MAnalyse (isb=false, delta=3, overlap=4)
fwd_vect_2 = super_a.MAnalyse (isb=false, delta=2, overlap=4)
fwd_vect_1 = super_a.MAnalyse (isb=false, delta=1, overlap=4)
bck_vect_1 = super_a.MAnalyse (isb=true, delta=1, overlap=4)
bck_vect_2 = super_a.MAnalyse (isb=true, delta=2, overlap=4)
bck_vect_3 = super_a.MAnalyse (isb=true, delta=3, overlap=4)

fwd_comp_2 = src.MCompensate (super, fwd_vect_2, thSAD=sad)
fwd_comp_1 = src.MCompensate (super, fwd_vect_1, thSAD=sad)
bck_comp_1 = src.MCompensate (super, bck_vect_1, thSAD=sad)
bck_comp_2 = src.MCompensate (super, bck_vect_2, thSAD=sad)

# Spatio-temporal denoising using modified dfttest
c_dft = Interleave (fwd_comp_2, fwd_comp_1, src, bck_comp_1, bck_comp_2)
c_dft = c_dft.dfttest (sigma=sigma, lsb=true) # Double height
c_dft = c_dft.SelectEvery (5, 2)

# Temporal-only denoising using modified MDegrain
c_deg = src.MDegrain3 (super, bck_vect_1, fwd_vect_1, bck_vect_2, fwd_vect_2, bck_vect_3, fwd_vect_3, thSAD=sad, lsb=true) # Double height

# Spatio-temporal denoising smoothes too much the details,
# therefore we use pure temporal denoising on edges or detailed areas.
edge_src = c_deg.Crop (0, 0, w, h)
edge_mask = edge_src.mt_edge (mode="prewitt", thY1=thr, thY2=thr)
edge_mask = edge_mask.mt_expand ()
edge_mask = StackVertical (edge_mask, edge_mask) # Double height
c_hyb = mt_merge (c_dft, c_deg, edge_mask, luma=true, y=3, u=3, v=3)

return (mask ? edge_mask.GreyScale () : c_hyb)
}

I'm using the latest Dither btw.

[cretindesalpes]

Use zzz_denoise(). This function does almost all the job, with SmoothGrad. DitherPost only converts the 16-bit result into a regular 8-bit dithered clip.
zzz_denoise()
SmoothGrad (radius=16, thr=0.25, elast=2)
DitherPost ()

If you want to keep FFT3DGpu as denoising filter, just do :
FFT3DGpu (Sigma=1)
GradFun3 ()or
Dither2Pre (flt="FFT3DGpu(Sigma=4)")
SmoothGrad (radius=16, thr=0.25, elast=2)
DitherPost ()

[coolguy89]

It works! Thanks :).

One more question, if I only want to deband, I only need the GradFun3, right? Or is there any other way in the dither packages that can improve debanding process.

[cretindesalpes]

Yes, use GradFun3 to deband only.

[coolguy89]

Thank you for your support and your awesome script. ;)

[Dogway]

cretin, the error I was having was caused by MT inside dither1pre. I had to change splitvertical to true also to make it aware dither was changing the size.
On a note, before this I tried stacked=false and it gave me temporal artifacts. Finally after an afternoon battle it seems this code is what it works, no problems for encoding either.
MT("""Dither1Pre(flt="tnlmeans()",stacked=true)""",2,2,true)
DitherPost(stacked=true,prot=false)
You could also add it to documentation for reference.

[SSH4]

cretindesalpes can you check your mod16 of Mvtools?
Looks like all (latest mvtools-2.5.11.2-mod16a.zip and previous) version have some bug with tsadc. tsadc always == tsad.
you can check it if you made big tsad (about 600 or more) and try low tsadc. UV channel will be bad :(
on original mvtools-2.5.11.2 all work well.

thanx!

[cretindesalpes]

Indeed there was a bug, thanks for the report. I'll release a fixed version soon.

[cretindesalpes]

Dither 1.7.1 (http://ldesoras.free.fr/src/dither-1.7.1.zip) update. Changes:
Dither.avsi doesn't use Max() anymore, so it can now run on Avisynth 2.5.7 or earlier.
Bugfix in modified MVTools2: the thSADC argument was ignored in MDegrain1/2/3.