Color banding and noise removal

[osgZach]

Is it possible to deband a source and also denoise it?

Should I denoise first and then deband?

Using GradFun2DB for instance, it won't get rid of it but it will take the banding down a notch, however if I were to there apply something like LSFMod, or in general some other noise reducers / it will undo the bebanding( as it seems it just adds grain to break it up), and then the banding will come back with even more clarity and well defined borders.

So it ends up looking like an ugly gradient, for lack of a better term, with those wavey line borders and all that fun stuff.

What also puzzles me, is that even though I am doing no color mode conversions, etc to even create this banding (its present on the DVD source) when I watch either the DVD or a straight unfiltered encode of the same episode, you can almost not even tell it is there on the TV screen.

Why is that?

Any suggestions on some strong, but safe settings for GradFun2DB, or other x64 compiled filters for this (degrainmedian, etc?) would be preferred. I'll provide a source section once a test encode I am running is done, if its needed. But this was more of a general question on effective settings for some filters versus a plea for someone to do the work for me.

[OvejaNegra]

After ALOT of battle agains grain VS Banding i can tell you this for sure:
You eat the grain, you ll get banding.
Some sources are encoded without dithering and have banding.
Some sources have no grain but are encoded with dithering to avoid banding (dithering in the end is noise like grain for you)

My actual choices (after MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANY teest with animes and grainy sources)

Opt A: Do a strong denoising using FFT3d or moderate denoising using vaguedenoiser or anything like that -> good compression (not too high bit rate needed, not too big file) -> banding -> enable deband filter on FFDSHOW duting playback and it works excellent.

Opt B: Calm the grain using some temporal grain compressor like TTEMPSMOTH (my favorite) or this:
http://forum.doom9.org/showthread.php?t=139766
or this:
http://forum.doom9.org/showthread.php?t=130611&highlight=grain+optimizer
that will give you a more stable grain
then -> encode using high bit rate (1800 kbps or more) using grain retention settings on X264 or qpel + MPEG matrix on XVID -> Bigger file -> no banding -> looks very nice to me (for anime i'm ok sometimes with the anime preset on x264, i don need the grain preset)


IF your source have banding -> subtle denoiser (there is ALWAYS some noise on DVDs) -> encode normally -> enable deband on ffdshow during playback.

Or

Encode using gradfun or THIS: GradFun2DBmod.v1.3.avsi (google) Your source will look grainy again but it wont have banding. Of course you need high bit rate for this.


But i can tell you this for sure >
DVD sources have grain or dithering (i cant speak for HD yet, ask other person) so Degrain or denoise + no banding is impossible.

Yes on the TV (a common TV) you can see the banding even if is an encode or the original DVD with banding.
When you are on your monitor maybe bacause a bigger luma range you can see the banding. If you have a flat display it will look worst.

It's up to you, keep the grain or remove it and leave the banding. If your DVD has banding, well use some of the solutions i tested.

Good luck


EDIT:
Read this:
http://forum.doom9.org/showthread.php?t=118255&highlight=grain+optimizer

[cretindesalpes]

(Original reply replaced by this Dither presentation.)


Download

>>>> dither-1.28.1.zip <<<< (https://ldesoras.fr/src/avs/dither-1.28.1.zip)
Main scripts and plug-ins from the Dither package, including avstp.dll.
>>>> dfttest 1.9.4 <<<< (https://ldesoras.fr/src/avs/dfttest-1.9.4.zip)
Mod16 version of dfttest.
>>>> MVTools 2.6.1.0 <<<< (https://ldesoras.fr/src/avs/mvtools-2.6.1.0.zip)
Mod16 version of MVTools 2
Masktools 2 a48 (http://manao4.free.fr/) or above is required. Previous Masktools versions will silently fail and cause the greenish screen of death. Please update your Masktools before installing the Dither script.

All these plug-ins include the -Vit- threading fixes.


What is it?

Dither is a package including several scripts and avisynth plug-ins. Some of them are modification of existing ones (dfttest and MVTools 2). I primarily wrote the Dither tools to fight color banding (http://en.wikipedia.org/wiki/Color_banding) during denoising in an unusual way. But Dither can perform various operations :

Recover the high bitdepth data possibly contained in a noisy clip. This can be seen as an "inverse-dithering" operation. In computer graphics, people usually use a gaussian filter to turn halftones into shades of grey. We'll use denoisers here.
Dither a high bitdepth picture into a standard YV12. The picture may have been obtained with the inverse-dithering operation, the combination could be called "transdithering".
Smooth fine gradients to remove colorbanding, resulting in a high bitdepth picture which can be dithered later.
Performing basic operations (masking, curves...) on high bitdepth pictures, as they cannot be manipulated safely with conventional avisynth filters.


Why should I use it?

Because denoising and lossy compression can easily make colorbanding appear. By transdithering, it is possible to recover original gradients better than using a filter + GradFun2DB(mod). Moreover, classic error-diffusion dithering algorithms require a high bitrate during encoding, otherwise color banding appears again. Dither offers an ordered dithering (http://en.wikipedia.org/wiki/Ordered_dithering) (Bayer matrix), which is more resilient to data compression.


How can I use it?

The simplest way to transdither a relatively clean source, using the modified dfttest:
dfttest (sigma=2, lsb=true)
DitherPost ()

You can add a SmoothGrad() call between both statements to remove colorbanding present in the source. And because dfttest is a spatio-temporal filter, it would be better to also use motion compensation (http://forum.doom9.org/showthread.php?t=147676) to increase filter accuracy and prevent ghosting.

For an already denoised source, use
GradFun3 ()

GradFun3 is intended to replace GradFun2DB(mod). Actually it works on the same principle and does almost the same things, but it uses ordered dithering by default, which is better for compressibility.

[osgZach]

Interesting solution you have there.

As for this particular source, I decided to just let it be as it was too much confusion on my part. Its worth noting I don't watch much on my PC, I encode for playback on my WDTV Live on a large screen (sadly I don't know its size haha) HDTV. Compressibility isn't much of a concern as x264 under CRF gives close to DVD quality results in the upper range of sub-1000kbps (on the TV). But I'll probably come back to this for future encodes I know will be pretty noisy as they were poor transfers to DVD.

[Terka]

could you please post same samples?

[osgZach]

Clip #1 (http://www.oldskoolgames.com/videos/clip1.demuxed.m2v) This is not an example clip, I was wondering however if this is an example of Halo artifacts? (the edges around the students portable game thing)

Clip #2 (http://www.oldskoolgames.com/videos/clip2.demuxed.m2v) - This is the color banding (I hope) clip. It's 26MB and spans several scenes which each have varying degrees of banding. I have never been able to successfully filter them, or reduce noise without greatly enhancing the visibility of the bands.

These come from Death Note, Episode 1, R1 DVD

[osgZach]

The only thing I've had the patients to sit around and check was Dither1Pre ().Dither1Post ()


Which did make a minor difference but I think mostly did some denoising.. Under x32 using some of the other examples (which I honestly copied and pasted, I can't even begin to understand them or how they work) I got some pretty nice results that pretty much made it look about as good as watching it on my TV or converting the file to RGB. Although banding was still there (big reduction though)..

It would be really nice if someone could port the modified DFTTEST linked to above, to x64 so I can try this out under JoshyD's MT_x64 build. Maybe some other people would find it useful too.

I recently purchased Last Exile when I saw it at Best Buy.. A really nice show, and very good quality for the most part. However I was disheartened to find the same horrible color banding issues when I popped it into the drive and took a quick peek at it.. Very dismayed :(

Seems to be a rather ugly trend in the industry to not push these shows out the door properly... They have to know this is a huge problem :(

[cretindesalpes]

I think the banding you have comes from the final YV12 to RGB conversion. I tried to process your second clip with the following script :

FFmpegSource2 ("clip2.demuxed.m2v")
TFM ()
TDecimate (mode=1)

zzz_denoise ()
DitherPost (mode=2)

Function zzz_denoise (clip src, float "sigma", int "thr", bool "mask", int "sad")
{
sigma = Default (sigma, 16)
thr = Default (thr, 5)
mask = Default (mask, False)
sad = Default (sad, 200)

w = src.Width ()
h = src.Height ()

# Motion analysis
super = MSuper (src)
super_a = MSuper (src.TTempSmooth ().RemoveGrain (12))

fwd_vect_3 = super_a.MAnalyse (isb=false, delta=3, overlap=4)
fwd_vect_2 = super_a.MAnalyse (isb=false, delta=2, overlap=4)
fwd_vect_1 = super_a.MAnalyse (isb=false, delta=1, overlap=4)
bck_vect_1 = super_a.MAnalyse (isb=true, delta=1, overlap=4)
bck_vect_2 = super_a.MAnalyse (isb=true, delta=2, overlap=4)
bck_vect_3 = super_a.MAnalyse (isb=true, delta=3, overlap=4)

fwd_comp_2 = src.MCompensate (super, fwd_vect_2, thSAD=sad)
fwd_comp_1 = src.MCompensate (super, fwd_vect_1, thSAD=sad)
bck_comp_1 = src.MCompensate (super, bck_vect_1, thSAD=sad)
bck_comp_2 = src.MCompensate (super, bck_vect_2, thSAD=sad)

# Spatio-temporal denoising using modified dfttest
c_dft = Interleave (fwd_comp_2, fwd_comp_1, src, bck_comp_1, bck_comp_2)
c_dft = c_dft.dfttest (sigma=sigma, lsb=true) # Double height
c_dft = c_dft.SelectEvery (5, 2)

# Temporal-only denoising using modified MDegrain
c_deg = src.MDegrain3 (super, bck_vect_1, fwd_vect_1, bck_vect_2, fwd_vect_2, bck_vect_3, fwd_vect_3, thSAD=sad, lsb=true) # Double height

# Spatio-temporal denoising smoothes too much the details,
# therefore we use pure temporal denoising on edges or detailed areas.
edge_src = c_deg.Crop (0, 0, w, h)
edge_mask = edge_src.mt_edge (mode="prewitt", thY1=thr, thY2=thr)
edge_mask = edge_mask.mt_expand ()
edge_mask = StackVertical (edge_mask, edge_mask) # Double height
c_hyb = mt_merge (c_dft, c_deg, edge_mask, luma=true, y=3, u=3, v=3)

return (mask ? edge_mask.GreyScale () : c_hyb)
}

(This script also requires a modified version of MDegrain3. I added a link to my previous post (http://forum.doom9.org/showthread.php?p=1386559#post1386559))

I've got pretty good results regarding denoising and debanding. However, depending on which software I play the processed clip, bands may appear or not, and obviously come from the player's YV12->RGB converter. But I don't know if it's only a matter of codec or if the video card's driver is involved too.

An example :

Original, noisy clip :
http://img24.imageshack.us/img24/2337/noisy.th.png (http://img24.imageshack.us/img24/2337/noisy.png)

Denoised, with bands :
http://img576.imageshack.us/img576/815/denoised.th.png (http://img576.imageshack.us/img576/815/denoised.png)
On this picture, one can see clearly the YV12->RGB conversion problem. Every six bands, there is a bigger jump caused by a step of 1 unit in luma values converted to a step of 2 units in the RGB colorspace instead of 1.

Denoised and dithered :
http://img46.imageshack.us/img46/2183/dithered.th.png (http://img46.imageshack.us/img46/2183/dithered.png)

For the record, GradFun2DBMod :
http://img356.imageshack.us/img356/8567/gf2dbm.th.png (http://img356.imageshack.us/img356/8567/gf2dbm.png)

To show the effect, a magnified part of the picture above (bottom left), "HSV stretched" (with GIMP) :
http://img63.imageshack.us/img63/7898/magnifiedandstretched.png
Original / Denoised
GradFun2DBMod / Dithered

[osgZach]

There shouldn't be ANY YV12 - > RGB conversion going on. I work on my clips in Vdub, so unless its auto color mode detection is borked up and doesn't work, what I am getting -should- be YV12 ? Or is it gonna be RGB at viewing stage regardless, just by nature of PC displays?

My encoding path is off the DVD to a HuffYV12 (ffdshow) file. Not sure if that would have any impact on it though. I will say doing a color conversion to RGB seems to clean it up rather nicely all by itself. So I'm not sure why even at the viewing stage, an RGB conversion would look so horrible.

That being said. I appreciate your efforts. I'm not sure if I have tried Gradfun2db MOD, but I bet I at least played around with the original.. Your GF2DBMod image looks very nice for certain. I may have to install Set's 2.5.8 MT x32 version of Avisynth I guess so I can at least experiment at reasonable speeds.

Would anything during the DGindex process affect this too? i.e there is PC / TV modes for clamping some color stuff IIRC. I don't recall which I used but probably default to PC..

[OvejaNegra]

osgZach:
I havent tested your samples, but there are animes with banding, i dont know why but it happens.
GradFun2DBMod should put grain on those banded areas, yes you need a higgher bit rate or x264 will eat the grain again.

By the way (maybe you dont know this):
VDUB will convert yoyr source to RGB if you use full processing mode and the codec will re-convert to YV12 during encoding (in case you are encoding with xvid / divx / etc on vdub).

Go to the options menu / preferences -> output color depth: is on fastest 16 bit? because if it is, THATS the cause of your banding.
you should put 24 bit or match display depth :)

Maybe you know this but maybe not, but you mention you are using VDUB and maybe you should check this.

And, if you eat your grain with FFTxxx or vaguedenoiser or any good spatial filter, well you always will have banding.
The mpeg2 encoders have dithering for make the color conversions (like CCE, i dont know the modern versions, but the old SP has it).
Try a temporal filter to calm the grain on the static surfaces like ttempsmotth (the only one who gave me a good looking Kanokon).
I can send you samples if you like it, there is one scene on episode one that you can use to test your anti-banding strategy and find the best filter to your needs :)

Good luck
Please check VDUB settings

[osgZach]

My Vdub should be set to "use output settings". Actually not quite sure what that means, but I definitely take it off the 16-bit junk every time I setup a fresh install, etc.. As for my processing mode, I never use full. I use fast recompress, so it pass right through without being touched, except by Avisynth. So in my mind its VOB - > FFDShow (in this case FFDshow's HuffYV12), which I then send through x264 via an AVS script after I am done working on a file.

I do have "Enable dithering for 16-bit displays" turned off as well.. Not sure if that would impact anything. But at any rate.. I probably need a break from all this stuff..

[OvejaNegra]

Output color depth:
"Selects the default color precision for video display -- 24-bit looks better, but 16-bit is generally faster. This does not affect file output"
Enable 16-bit dithering:
"If enabled, 24-bit images are dithered when displaying in 16-bit to reduce banding, at the cost of a little speed. This does not affect file output"


If you set that to "Fastest 16 bit" the videos looks like if your display to 16 bits color. They band a lot. If you enable dithering, well you'll see grain/noise/dither that is not part of the original source.

I always use "Match display depth"

you could install xvid and disable raw video support from the VFW and the DSHOW filter, so xvid will handle YV12 content, to see if that helps. (The divx pack also have a YV12 decoder).

[cretindesalpes]

dither.avsi updated to v1.1, see the original post (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

[Floatingshed]

I get "the named argument clip was passed more than once to blankclip". Line 755, 413, 250.
Please help....

[foxyshadis]

Replace your version of the script with the one in the zip file. Yours might be modified or corrupted.

[dlnm]

Is it possible to use other denoisers instead of MDegrain or dfttest?

[cretindesalpes]

Is it possible to use other denoisers instead of MDegrain or dfttest?Yes. Use Dither1Pre() or Dither2Pre() to generate a 16-bit denoised clip (actually it will have only 9 or 10 significant bits).

Usage :Dither2Pre ("YourFavoriteDenoisingFilter (param=1234)")
DitherPost ()
If your filter has signal amplitude dependent parameters (thresholds, etc.), multiply them by 2 or 4 for Dither1Pre and ~2Pre. The drawback of this function is that the processing time will also be multiplied by 2 or 4.

[andrew_seul]

dither.avsi updated to v1.1, see the original post (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

There's any chances that you can port those plugins you modifed to 64 bit mode (including NoiseGenerator also)? I need a little speed on my encodings thats why Im asking :)

[Jawed]

Just chain stuff together :devil:

e.g. I use this to kill blocking artefacts:



function BlockKiller(clip source)
{
y=source.greyscale()
y=y.ChannelBlockKiller()
u=source.utoy()
u=u.ChannelBlockKiller()
v=source.vtoy()
v=v.ChannelBlockKiller()

return ytouv(u, v, y)
}

function ChannelBlockKiller(clip source)
{
source
""" spline36resize(source.width*16,source.height*16)
gradfunkmirror()
gradfunkmirror()
gradfunkmirror()
spline36resize(source.width*8,source.height*8)
gradfunkmirror()
gradfunkmirror()
gradfunkmirror() """
spline36resize(source.width*4,source.height*4)
gradfunkmirror()
gradfunkmirror()
gradfunkmirror()
spline36resize(source.width*2,source.height*2)
gradfunkmirror()
gradfunkmirror()
gradfunkmirror()
spline36resize(source.width,source.height)
gradfunkmirror()
gradfunkmirror()
gradfunkmirror()

return last
}

notice, in the function ChannelBlockKiller, that the *16 and *8 sections are commented out with the """. As posted, that code only uses *4 and *2 sizes, before returning the picture to its original size.

Basically you can adjust the maximum size, e.g. make it larger for smaller frames by un-commenting the *8 and *16 sections. The basic concept here is that any resizing algorithm creates a gradient on all edges (banding and non-banding). The gradient will be "smooth" but "regular". So then apply gradfunkmirror() to dither that gradient. Then resize down. Resizing down averages the dither.

The quantity of steps and the count of gradfunkmirror() calls per step adjust the result. You adjust for blockiness versus softness. (I'm still experimenting with this function - which is why there are no funky parameters, as I haven't decided which parameters are important and how to graduate the effect.)

In general you can simply use 2 or more instances of gradfunkmirror(), one after the other, to control most banding.

Also, I always use gradfunkmirror() on playback (the DeBand filter in FFDShow).

Jawed

[Gavino]

notice, in the function ChannelBlockKiller, that the *16 and *8 sections are commented out with the """.
An innovative use of quotes.
Why not use a real comment /* ..... */ which can span multiple lines.

[Jawed]

An innovative use of quotes.
Why not use a real comment /* ..... */ which can span multiple lines.
Not on my system.

If Avisynth allows that, now, then :cool:

I'm using 2.57 tsp MT version 5, built March 1 2007 - it works for the things I do - so I have no interest in spending time debugging anything newer.

[Floatingshed]

Replace your version of the script with the one in the zip file. Yours might be modified or corrupted.

That is the version I'm using...

[cretindesalpes]

The dither package (http://forum.doom9.org/showthread.php?p=1386559#post1386559) has been updated to version 1.3.

[tormento]

Do you plan to release a x64 aware version?

[cretindesalpes]

I don't know. I will release soon an additional function to smooth the low gradients in the 16-bit world, and a derived function roughly equivalent to gradfun2db(). Then, I'll try to port the whole thing to x64, but only if I can obtain the source code for the x64 ports of dfttest and MVTools2. Currently, I have no news from JoshyD on this topic.

[tormento]

However, thanks ;)

[tormento]

I am trying to use your package but I keep on getting VirtualDub crashing when opening the file. Please, tell me a simple, stupid denoise script you get working. =P

[cretindesalpes]

I am trying to use your package but I keep on getting VirtualDub crashing when opening the file.

"Crashing" is very vague and I cannot do anything with only this bit of information. Please copy the crash error message and the script you tried to open.

Please, tell me a simple, stupid denoise script you get working. =P

This is taken from the dither.avsi documentation:

h = Height ()
w = Width ()
dfttest (lsb=true)
msb = last.Crop (0, 0, w, h)
lsb = last.Crop (0, h, w, h)
DitherPost (msb, lsb)

[tormento]

Please copy the crash error message and the script you tried to open.
I always use the word crash when I have nothing to report, i.e. the error is so severe the program simply hangs or outputs a stupid meaningless hex error.
This is taken from the dither.avsi documentation:
Mm.. the script I tried to wrote was even simpler but took in account both pre and post. Wait a couple of hours, I'll arrive at office and try again, now I'm on a sluggish train..

[tormento]

Ok, I had some spare time to play with dither but I can't see any difference between the results of these 2 scripts:

super = MSuper(pel=2, sharp=1)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=8, blksize=16)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=8, blksize=16)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=8, blksize=16)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=8, blksize=16)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=8, blksize=16)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=8, blksize=16)

MDegrain3(super, bv1, fv1, bv2, fv2, bv3, fv3, thSAD=400)


h = Height ()
w = Width ()

super = MSuper(pel=2, sharp=1)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=8, blksize=16)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=8, blksize=16)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=8, blksize=16)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=8, blksize=16)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=8, blksize=16)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=8, blksize=16)

MDegrain3(super, bv1, fv1, bv2, fv2, bv3, fv3, thSAD=400, lsb=true)

msb = last.Crop (0, 0, w, h)
lsb = last.Crop (0, h, w, h)
DitherPost (msb, lsb)


Am I doing something wrong?

P.S: The purpose is to degrain without too much banding.

[cretindesalpes]

Am I doing something wrong
This looks correct. Anyway, the result will really depend on your source. If the noise you try to remove has nothing to do with dithering of a high-bitdepth master or is not related to film grain, and if your source already has banding, the MDegrain won't fix anything.

You'll have probably more success with a spatio-temporal denoiser, dfttest for instance. I generally use a combination of both MDegrain for the detailed areas, and dfttest for low gradient and flat areas. See my post #8 on this thread.

[tormento]

Well, a lsb enabled version of fft3dfilter and fft3dgpu (perhaps this is more difficult) would be nice ;)

[james99]

It increases the FPS of progressive Blu ray video from 24fps to 48fps during my test
I do believe it is
Interleave (base, rem) that is doubling the fps in the Dither2Pre function.

can be fixed using
assumefps(24) to change the fps back to 24.

[cretindesalpes]

It increases the FPS of progressive Blu ray video from 24fps to 48fps during my test

You're right. But put the AssumeFps after the DitherPost call. This will be fixed in the next release.

[cretindesalpes]

I released a new version of the Dither tools (1.4) (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

I added SmoothGrad, a smoothing function to get the gradients clean where it low frequency noise or color-banding steps remain after denoising. From this, I built GradFun3, a function intended to smooth sources containing banding and already denoised. This is very similar to GradFun2db.

Here are some examples, comparing several denoising+debanding methods:

http://img835.imageshack.us/img835/6113/comparisonoriginal.th.png (http://img835.imageshack.us/img835/6113/comparisonoriginal.png) http://img818.imageshack.us/img818/1936/comparisonfiltered.th.png (http://img818.imageshack.us/img818/1936/comparisonfiltered.png)
Original / Filtered
http://img38.imageshack.us/img38/206/comparisongradfun2db.th.png (http://img38.imageshack.us/img38/206/comparisongradfun2db.png) http://img411.imageshack.us/img411/6465/comparisongradfun3.th.png (http://img411.imageshack.us/img411/6465/comparisongradfun3.png)
GradFun2db / GradFun3
http://img31.imageshack.us/img31/5200/comparisondither.th.png (http://img31.imageshack.us/img31/5200/comparisondither.png) http://img809.imageshack.us/img809/7963/comparisondithersmoothg.th.png (http://img809.imageshack.us/img809/7963/comparisondithersmoothg.png)
Dither / Dither+SmoothGrad

From the previous pictures, I magnified and amplified several zones in order to show the differences more clearly:
http://img4.imageshack.us/img4/3425/comp1v.th.png (http://img4.imageshack.us/img4/3425/comp1v.png) http://img295.imageshack.us/img295/4491/comp2.th.png (http://img295.imageshack.us/img295/4491/comp2.png) http://img38.imageshack.us/img38/4233/comp3.th.png (http://img38.imageshack.us/img38/4233/comp3.png)
Still the same order:
Original / Filtered
GradFun2db / GradFun3
Dither / Dither+SmoothGrad

One can see that both GradFun function perform poorly near the edges. This is because they are based on massive box filtering and have to stop as soon as the box overlaps an area of a different color. Ideally, these function would use a large bilateral filter, but this would be really overkill and much much slower.

[Floatingshed]

I'm struggling to understand how to use this. Some examples don't use dither2pre others do.
This is what I've been trying, every odd frame is all green noise, and the framerate has doubled:


LoadPlugin("D:\Avisynth scripts\best_dither\dither.dll")
Import("D:\Avisynth scripts\dither.avs")
LoadPlugin("D:\Avisynth scripts\dfttest-1.8mod16a\dfttest.dll")
LoadPlugin("D:\Avisynth scripts\mvtools-2.5.10.1-mod16a\mvtools2.dll")

Dither2pre()

h = Height ()
w = Width ()

super = MSuper(pel=2, sharp=1)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=8, blksize=16)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=8, blksize=16)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=8, blksize=16)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=8, blksize=16)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=8, blksize=16)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=8, blksize=16)

MDegrain3(super, bv1, fv1, bv2, fv2, bv3, fv3, thSAD=400, lsb=true)

msb = last.Crop (0, 0, w, h)
lsb = last.Crop (0, h, w, h)
DitherPost (msb, lsb)

[cretindesalpes]

Either use Dither1Pre/Dither2Pre, either the modified MDegrain or dfttest, but not both. The only purpose of DitherXPre is to give the possibility to use a specific denoiser which hasn't been modified to output 16 bit data. For example :

# (Put your source here)
Dither2Pre (flt="TTempSmoothF(lthresh=4*4,cthresh=5*4).frfun7(Lambda=1.1*4)")
DitherPost ()

Without Dither2Pre, your script becomes:

# (Import scripts, plug-ins and input clip here)

h = Height ()
w = Width ()

super = MSuper(pel=2, sharp=1)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=8, blksize=16)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=8, blksize=16)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=8, blksize=16)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=8, blksize=16)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=8, blksize=16)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=8, blksize=16)

MDegrain3(super, bv1, fv1, bv2, fv2, bv3, fv3, thSAD=400, lsb=true)

msb = last.Crop (0, 0, w, h)
lsb = last.Crop (0, h, w, h)
DitherPost (msb, lsb)

With the latest version, the three last lines can just be replaced with DitherPost (stacked=true) and you don't need to collect w and h.

[Floatingshed]

Now its making sense and working. Many thanks.

[Dark Shikari]

From the previous pictures, I magnified and amplified several zones in order to show the differences more clearly:
http://img4.imageshack.us/img4/3425/comp1v.th.png (http://img4.imageshack.us/img4/3425/comp1v.png) http://img295.imageshack.us/img295/4491/comp2.th.png (http://img295.imageshack.us/img295/4491/comp2.png) http://img38.imageshack.us/img38/4233/comp3.th.png (http://img38.imageshack.us/img38/4233/comp3.png)
Still the same order:
Original / Filtered
GradFun2db / GradFun3
Dither / Dither+SmoothGrad

One can see that both GradFun function perform poorly near the edges. This is because they are based on massive box filtering and have to stop as soon as the box overlaps an area of a different color. Ideally, these function would use a large bilateral filter, but this would be really overkill and much much slower.Every single one there except gradfun looks utterly terrible. Why? Because they use ordered dither. Use error diffusion, for the sake of your eyes.

[cretindesalpes]

Every single one there except gradfun looks utterly terrible. Why? Because they use ordered dither. Use error diffusion, for the sake of your eyes.

This topic is about noise removal and color banding, not converting a 16M color picture in 256 colors. In actual use, you should never really see the patterns you quoted. I HSV-stretched these excerpts of the previous pictures in GIMP just to show what the plug-in does without having to push the contrast & gamma of your monitor. Also, keep in mind that these patterns have a temporal component.

I used ordered dither instead of a 3D-version of the Floyd-Steinberg algorithm to give a chance to the dither to resist better to the video compression.

Here [dead link, sorry] (http://comparescreenshots.slicx.com/comparison/82872) is a comparison of a standard dither from a high-quality picture (direct bluray rip) compressed @ 6 Mbps and the same movie, re-dithered and compressed with the same settings.

[stpdrgstr]

I get "there is no function named "Max"" when trying to use Gradfun3().

[Didée]

min() and max() functions were added in Avisynth 2.5.8.

Seems you're still using 2.5.6 or 2.5.7. ;)

[stpdrgstr]

So that's why. I was using the 2.5.7 mt version I could find at the avisynth wiki because one of the 2.5.8 versions gave me weird errors with assumefps. What 2.5.8 mt do you recommend?

EDIT: I just noticed there's only one 2.5.8 supporting MT, but it's 64 bits. Isn't there a 32 bits version?

[Didée]

Sure there is (http://forum.doom9.org/showthread.php?t=148117).

[stpdrgstr]

That's weird, I can't see that and other threads listed.

I went with 2.6 at the end, but thanks anyways.

[cretindesalpes]

Dither (http://forum.doom9.org/showthread.php?p=1386559#post1386559) updated to version 1.5. Added several error diffusion algorithms (however ordered dithering still looks better after video compression) and fixed the randomness of the internal noise generator.

[sneaker_ger]

I wanted to try dither out but I always get the error "dfftest: unable to load libfftw3f-3.dll". I'm using AviSynth 2.5.8 (MT) on Windows 7 64bit with a Core i7. I have copied libfftw3f-3.dll to my plugin directory. Any idea?

[Usedocne]

libfftw3f-3.dll goes in the Windows\SysWOW64 folder.

[sneaker_ger]

Thank you, works now.

[Fullmetal Encoder]

I'm having a bit of difficulty getting this to work properly in the absence of a denoiser. I prefer to use TNLMeans to denoise and I have a clip which has been denoised to my satisfaction. From the sample images posted in this thread it appears that the debanding methods that DitherPost employs are superior to GradFun2DBMod. Plus, I would like to see how I can use the mask tool to preserve detail beyond the level I have been able to accomplish with GradFun2DBMod. How do I go about using/testing DitherPost on a previously denoised clip? While I would like to have 16-bit depth output with TNLMeans I'm afraid that the sun would go supernova before Dither2Pre finished processsing the video. Here is what I have attempted:

SetMTMode(2,8)
SetMemoryMax(1400)
AVISource("clip.avi")
dfttest(sigma=0,sigma2=0,lsb=true)
msb = last.Crop (0, 0, 720, 480)
lsb = last.Crop (0, 480, 720, 480)
DitherPost (msb, mode=0, lsb, prot=false)

Since I didn't want DFTTest to be doing any extra denoising my idea was to deactivate any of it's processing and still have the filter output an msb and lsb, which, as I understand it, are absolutely necessary for DitherPost. I also tried setting y,u,v to "false" in dfttest() and using various other modes for DitherPost but in all cases this script either produces nothing (displays the source clip without any apparent processing) or slows the script down (dfttest appears to slow down navigation in VirtualDub but no dithering is taking place). There really isn't any point in doing extra processing with a very slow DFTTest just to get rid of banding. This is somewhat of a hack, however, as invoking DFTTest at all leaves me not knowing whether it may be slowing down processing even if I use sigma=0 so ideally I'd like to be able to use DitherPost by itself. While I have been able to get DitherPost to add dithering by omitting DFTTest and msb and lsb the results are very ugly with very visible spots and band.

BTW, I was the one who created a thread earlier asking about the possibility of sending only portions of a frame through to a filter based on a luma threshold or other criteria. What I was thinking of using that to accomplish appears to be almost exactly what you have done with DitherPost and BuildDitherMask, which is to allow high quality dithering while protecting more detailed areas of the image.

Thanks in advance for any help you can provide.

[Dogway]

Is it possible to sharpen between dfttest and dither?

[cretindesalpes]

I'm having a bit of difficulty getting this to work properly in the absence of a denoiser.

I fear that calling dfttest with sigma=0 will do absolutely nothing excepted warming your room and wasting your time.

Ideally, your denoiser could be able to output 16-bit data. I haven't studied how TNLMeans works, but I wouldn't be surprised if its internal computations are done at a higher bit depth than its actual output.

Anyway, to deband a denoised clip, you can simply use the GradFun3 function. Just pass it your denoised clip and it will act like Gradfun2db. No need for additional DitherPost. But these functions just try to repair the harm done by the 8-bit quantification happening in the denoiser. If you don't want to use dfttest/MDegrain as main denoiser, restore a small bit of noise in your previously filtered clip and use dfttest with a low sigma (1.0 for example).

# Your favourite denoiser here
denoised = MyExcellentDenoiser (parameters)

# Restore a bit of noise
mt_lutxy (last, denoised, expr="x y < y 1 - x y > y 1 + y ? ?", y=3, u=3, v=3)

super = MSuper ()

# You could also reuse the vectors found in a previous processing stage
vf1 = super.MAnalyse (isb=false, delta=1, overlap=4, blksize=16)
vb1 = super.MAnalyse (isb=true, delta=1, overlap=4, blksize=16)

# Turns remaining noise into 16-bit gradients
cf1 = MCompensate (super, vf1, thSAD=200)
cb1 = MCompensate (super, vb1, thSAD=200)
Interleave (cf1, last, cb1)
dfttest (sigma=1.0, tbsize=3, lsb=true)
SelectEvery (3, 1)

# Additional smoothing pass
tmp = last
SmoothGrad (thr=0.20, radius= 8, stacked=true, ref=tmp)
SmoothGrad (thr=0.17, radius=12, stacked=true, ref=tmp)
SmoothGrad (thr=0.15, radius=16, stacked=true, ref=tmp)

# Back to 8 bits
DitherPost (prot=false, stacked=true)

EDIT : better use the denoised clip for the MAnalyse steps.


Is it possible to sharpen between dfttest and dither?

Sure, although I think it's better to use the sharpener afterwards, with a mask to avoid sharpening the remaining noise/grain and the dither patterns.

w = Width ()
h = Height ()
dfttest (sigma=1.0, tbsize=1, lsb=true)
msb = Crop (0, 0, w, h)
lsb = Crop (0, h, w, h)
msb_sharp = msb.Sharpen (0.5)
mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

[Heaud]

Is there a way to restore fine detail, any lost sharp edges, and an adjustable amount of noise back? Using dfttest to denoise can sometimes soften the hard edges on certain sigma values. Setting it too low causes ditherpost to do almost no dithering and setting it too high smoothes hard edges (animation) and fine detail is lost.

I noticed that QTGMC has a feature to restore lost noise/detail from the denoising stage using a script: # Add back any extracted noise, after final temporal smooth. This will appear as noise/grain in the output
# Average luma of FFT3DFilter extracted noise is 128.5, so deal with that too
addNoise2 = (NoiseRestore > 0.0) ? lossed2.mt_adddiff( deintNoise.mt_lut( "x 128.5 - " + string(NoiseRestore) + " * 128 +", U=1,V=1 ), U=2,V=2 ) : lossed2 Could that be used in junction to dfttest+ditherpost?

What about the lost edges, could a repair function fix that?

[-Vit-]

That's just the last line of the noise bypass. The bare principle ignoring a few steps is this:

source=last
clean = source.YourFavoriteDenoiser()
noise = mt_makediff( source, clean, U=3,V=3 )

# Calculate motion vectors on clean source ...

processed=clean.OtherProcessing() # QTGMC does it's deinterlacing here
#...

restored1 = processed.mt_lutxy( noise, "x y 128 - " + string(DetailRestore) + " * 128 +", U=3,V=3 )

smoothed = restored1.MoCoSmooth() # Some kind of motion-compensated temporal smooth

restored2 = smoothed.mt_lutxy( noise, "x y 128 - " + string(NoiseRestore) + " * 128 +", U=3,V=3 )


The first restoral tries to recover detail that was misrecognized as noise in the denoising step - it may survive a temporal smooth compensated by motion vectors calculated from the clean source. The second restoral adds in some measure of true noise as you wish.

This part of QTGMC is still work in progress.

[Dogway]

Im making up my mind to make this filter work without changing my workflow too much.

Normally my filtering methods go as follow:
input=last
temp=mdegrain...
...
denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
luma=lumamask(temp,denoised,1,40)
ContraSharpening(luma, input)
Im trying to put ditherpost into play here adding:
msb_sharp = msb.ContraSharpening(luma, input)
mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)
but a warning message warns me:
Repair: clips must be of equal type, line 160.
I think it has to be with comparing 8 bit against 16 bit data in the contrasharpening. Besides I have been reading the dither documentation and its unclear about the msb and lsb variables. It looks like when dfttest is inside a variable msb/lsb can't be called, which renders the masking and contrasharpening not being usable.
Maybe its not very flexible, but Im curious to know the improvements when encoding patterned dithering specially on anime content.

[cretindesalpes]

Modified dfttest or MDegrain with parameter lsb=true will always produce a double-height clip, with the pixel MSB stacked on top of the LSB. You have 2 solutions get back to a 8-bit clip :

– Extract the MSB with a Crop. This is the simplest solution, although the maximum rounding error is suboptimal. The operation is actually value_8bits = floor (value_16bits / 256.0), leading to an error in the 0.0 – 1.0 range. Use it only if you need to process the clip in the 8-bit land and substitute the result to the previous MSB before dithering. This will give you a better chance to maintain the MSB/LSB consistency.

– Just round the pixel values, as it would be done at the output of a regular denoiser without the 16-bit kludge. Just pass your clip to DitherPost (mode=-1, stacked=true, prot=false). Here, value_8bits = round (value_16bits / 256.0), leading to an error in the -0.5 – +0.5 range.

input=last
temp=mdegrain...
...
denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
rounded = denoised.DitherPost (mode=-1, stacked=true, prot=false)
luma=lumamask(temp,rounded,1,40)
...
msb_sharp = msb.ContraSharpening(luma, input)
mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

You can also contrasharpen the rounded clips, compute the difference between before and after using mt_makediff and add this difference back on the final result.

[Dogway]

I really would like to understand better what MSB and LSB are, documentation is very loose. As I understand it, MSB is the 16bit denoised output, while LSB is the denoised 8bit output, which are stacked one above the other (thus the double height). In this situation where we have only denoiser+ditherpost we use stacked=true, and ditherpost will automatically select (crop) to obtain the MSB and LSB. But if we do some working in between we need first to separate msb and lsb by cropping ourselves.
_______
| |
| MSB |
|_______|
| |
| LSB |
|_______|
What you propose is to go back to 8bits, do the stuff and then in ditherpost compare that and lsb, right?
Right now I have as follows:
w = Width ()
h = Height ()

temp=mdegrain

denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
lsb = denoised.Crop (0, h, w, h)
rounded = denoised.DitherPost (mode=-1, stacked=true, prot=false)
luma=lumamask(temp,rounded,1,40)

msb_sharp=ContraSharpening(luma, temp)

mask = DitherBuildMask (msb_sharp, luma)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

Is it correct? I edited out, as I think I figured out the working. Still I have a question, msb_sharp in ditherpost is already 8bit (done in rounded), is it ok?

I might try your last suggestion as well. Anyway the effect is too subtle for my source after testing with several parameters, looks more like banding prevention than dithering.

[cretindesalpes]

I really would like to understand better what MSB and LSB are

Here, MSB/LSB stands for Most/Least Significant Byte. The color depth of the filter output is 16 bits per component (total: 48 bits) but Avisynth can only handle 8-bit pixel values. Therefore we have to split each 16-bit pixel value into a pair of 8-bit pixels. It's exactly like separating digits of numbers from 0 to 99 into tens and units.

There are many ways to organize these pixels. We can store the MSB and LSB on separate frames, group them horizontally or vertically, or group all MSB and LSB and stack them. I choose the latter for the denoiser output, stacking MSB and LSB vertically. Below is an example of what it looks like :

http://img218.imageshack.us/img218/8765/msblsb.th.png (http://img218.imageshack.us/i/msblsb.png/)

Original               | Without dither (rounded) |          Dithered
http://img24.imageshack.us/img24/7480/originalci.th.png (http://img24.imageshack.us/i/originalci.png/) http://img222.imageshack.us/img222/5058/rounded.th.png (http://img222.imageshack.us/i/rounded.png/) http://img338.imageshack.us/img338/2183/dithered.th.png (http://img338.imageshack.us/i/dithered.png/)

The MSB part is very close to the simple 8-bit picture, but the full precision requires a combination of the LSB and the MSB.

What you propose is to go back to 8bits, do the stuff and then in ditherpost compare that and lsb, right?
Technically, it's not exactly that, but you got it. The LSB is not compared, it is added. To take the previous comparison again, it's like sharpening using the tens, and adding the units at the very end. On gradients, the sharpening shouldn't change anything, so the operation should be transparent regarding gradients. But we need to protect the detailed zones and their close neighbourhood.

Now, how to get the tens ? Let's suppose we have a pixel value of 28. If we take just the tens (the Crop method), we get a 2. If we round before to the closest ten (the other method), we get 3. The second method is indeed better, especially if we have to compare the clip with the original. But let's say we are in a gradient area, and the sharpener changes nothing. If we inject back the 3 as MSB, we got a 38, which is a (relatively) big error. Nothing should have changed. Therefore we have to work with differences (between before/after processing, or between before/after rounding, your choice).

Regarding this issue, my first attempt at modifying your script wasn't right. Here is another version based on your previous post:

w = Width ()
h = Height ()

temp=mdegrain

denoised = temp.dfttest(sigma=20, tbsize=1, lsb=true)
msb = denoised.Crop (0, 0, w, h)
lsb = denoised.Crop (0, h, w, h)
rounded = denoised.DitherPost (mode=-1, stacked=true, prot=false)
luma=lumamask(temp,rounded,1,40)

contra = ContraSharpening(luma, temp)
contra_d = mt_makediff (contra, rounded, y=3, u=3, v=3)
msb_sharp = mt_adddiff (msb, contra_d, y=3, u=3, v=3)

mask = DitherBuildMask (msb_sharp, msb)
DitherPost (msb_sharp, lsb, prot=false, mask=mask)

I supposed your lumamask function handles the chroma planes too.

[Dogway]

Sweet, now it works as supposed. I will do some testings with parameters and encodings tomorrow, but I like what I see. At last it turned to be your suggestion what worked out.
A bit offtopic but as I see you are used with anime, do you recommend me going on tbsize=1 over dfttestmc? even mocomp'ed it introduces some artifacting but well, just asking people's doings.

PD: I edited my previous post for easier reading.

[cretindesalpes]

A bit offtopic but as I see you are used with anime, do you recommend me going on tbsize=1 over dfttestmc? even mocomp'ed it introduces some artifacting but well, just asking people's doings.

The tbsize=1 was just for the sake of simplifying the example; I find it introduces too much blurring. I generally use a combination of MDegrain3 and MC dfttest with tbsize=5, both with a low thSAD (200 or even lower) and a prefiltered motion search clip. It's not perfect, but it's still a very good compromise and I rarely have ghosting problems. Then I finalize the gradient smoothing with a single SmoothGrad pass with thr=0.15 before DitherPost.

[Heaud]

Could you post the script that uses the combination MDegrain3 and MC dfttest? I would like to try it out.

[cretindesalpes]

I already posted here (http://forum.doom9.org/showthread.php?p=1390594#post1390594) an earlier version of this script. It is a bit rough and has to be finetuned (especially for the motion search clip), but the basic concept remains the same.

[Dogway]

It produces flatter color areas and preserves more details, compared to my gradfun workflow. Lossless compressed to FFV1 chunk weighted less (163Mb) than the gradfun method (189Mb) which indicates that it is more efficient fot bits allocation in h.264. Im using a very high bitrate for h.264 but (ordered) dither encodes nicely. Its perfect for 2D animation. Im gonna try your last suggestions, and methods to see if I can improve quality or optimization.

edit: It also deals nicely with the problem I was having with GradFun2DBmod (http://forum.doom9.org/showthread.php?p=1475973#post1475973) (ghosting)

[Heaud]

Dogway, would your ghosting problem be similar to mine (http://doom10.org/index.php?topic=919.0)? It's not the filter that is causing it for my source. I've tried many different ways and I noticed that it has been occuring in b-frames.

[LaTo]

edit: It also deals nicely with the problem I was having with GradFun2DBmod (http://forum.doom9.org/showthread.php?p=1475973#post1475973) (ghosting)

GradFun2DBmod can not create ghosting, it is because your source's filter is not accurate in temporal domain.

Set temp=-1 (disable temporal stabilization) in GF2DBmod is a good workaround.

[Dogway]

Without dither (http://i212.photobucket.com/albums/cc35/Dogway/Misc/no-dither.png)
GradFun2DBmod temp=-1 (http://i212.photobucket.com/albums/cc35/Dogway/Misc/gradfun.png)
Dither (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither.png)
Maybe ghosting is not the right word.

[SilaSurfer]

@cretindesalpes

Very nice! I have a two questions. This is a standard usage call for Dither?

h = Height ()
w = Width ()
dfttest (lsb=true) # Output contains stacked MSB-LSB on the same frame.
msb = last.Crop (0, 0, w, h)
lsb = last.Crop (0, h, w, h)
DitherPost (msb, lsb, prot=false)

and

Could one for example use only Dithering?

TiA

[Dogway]

@SilaSurfer: if thats all you are going to do after dfttest, then I think you can just use:
dfttest (lsb=true)
DitherPost (stacked=true, prot=false)

[cretindesalpes]

Maybe ghosting is not the right word.
Indeed, this is haloing, not ghosting. This issue with gradfun2db is known and has been described here (http://forum.doom9.org/showthread.php?p=1159205#post1159205). This is related to how the gf2db filter works. Gradfun2db averages a large area around the processed pixel (box filtering), because box filtering is very fast and its speed is almost constant whatever the size of the box. The resulting 16-bit pixel value is compared to the current pixel and replaces it if the difference is small enough (with a smooth curve controlled by thr). Then, the result is dithered to 8 bits using error diffusion.

Actually the box filter + threshold is intended to be a simplification of a bilateral filter. It should have rejected input values which are too far away from the processed pixel, but instead it rejects output values which are not very close from the current pixel. In most case this works pretty well, but sometimes it fails and generates large, subtle halos near gradient boundaries.

This is a standard usage call for Dither?
Yes for the purpose of debanding, but now you can write this even simpler:

dfttest (lsb=true)
DitherPost (prot=false, stacked=true)

Could one for example use only Dithering?
Of course, if you can generate a 16-bit input. The general purpose of dither is basically converting high bitdepth data to a lower bitdepth while preserving the perceptive quality. The most interesting uses I can think of are high quality levels and colorspace conversions. For debanding purpose, you'll need a filter first to generate the 16-bit gradients from a 8-bits picture. The exact kind of filter depends on the nature of your source. Noisy: use a noise filter operating on large areas (like dfttest). Already banded: use a gradient rebuilder (like SmoothGrad).

[SilaSurfer]

I'm confused here.:D

dfttest (lsb=true)
DitherPost (prot=false, stacked=true) #Debanding

What if I wanted to use Mdegrain also?

Sorry guys for the confusion, can't test this since encode is running here for the next 22 hours.

[cretindesalpes]

dfttest (lsb=true) #Denoising
DitherPost (prot=false, stacked=true) #Debending


Not exactly : dfttest = denoising + debanding (or banding prevention). DitherPost = turns the previous result into something watchable and somewhat resilient to lossy video compression.

MDegrain* (lsb=true) does the same job as dfttest, but only temporally. Therefore it cannot remove banding when it is already in the source, but won't introduce new banding.

[SilaSurfer]

Thanks cretindesalpes

So if I want to use DFTest

dfttest (lsb=true)
DitherPost (prot=false, stacked=true)

Mdegrain

Mdegrain (lsb=true)
DitherPost (prot=false, stacked=true)

[cretindesalpes]

Yes, just replace MDegrain with MDegrain1/MDegrain2/MDegrain3, and add at least the mandatory parameters required for the MDegrain* filters (superclip and motion vectors, see the MVTools2 documentation).

[mp3dom]

For what I've seen, debanding filters in general suffers on dark areas. For example a thick black line in a dark gray background once debanded have the risk to be completely lost. I'm currently interested in only debanding (not denoise+debanding) so I think I'd need to use SmoothGrad (am I right?). In that case, have SmoothGrad some routines to avoid this (seems) common problem in dark areas without lowering the debanding strength on all other parts of the image? (I've not tested it yet, so I'm just asking). Thanks.

[cretindesalpes]

In that case, have SmoothGrad some routines to avoid this (seems) common problem in dark areas without lowering the debanding strength on all other parts of the image?
No, SmoothGrad does just the basic debanding task and works just like gradfun2db, leaving the dithering for DitherPost. I even think about splitting the function again. You can protect the very dark shades this way: apply a curve to amplify the dynamic range of low luma (something like a gamma correction), SmoothGrad, then apply the inverse curve. Computation in the 16-bit land with 8-bit clips is a bit tricky, but it looks workable with the Masktools (mt_lutxy is your friend). You can also process the clip using two SmoothGrad with different settings, and use a luma mask to select the right parts.

However SmoothGrad was intended to be a postprocessor for the main filtering functions and therefore operates directly on a 16-bit clip, so you'll have to convert your clip to 16 bits. Just stack it on the top of a BlankClip with color_yuv=$000000. GradFun3 is a more or less sophisticated wrapper around SmoothGrad + DitherPost that would replace gradfun2db, but I'm not totally happy with its current implementation. I may rework it later.

[Dogway]

I found out that you dont need the quotes when using the filter string in ditherpre. But I wanted to note that despite it works wonders when it works, in ocassions it can smooth too much the results, more than dithering banding. I can tell from the last source Im testing with, it causes the lineart to blur. I can post comparisons.
What I thought was to use no dither (mode=-1) for lineart, and dither for the rest, but the effect is the same and running 2 instances (one with ditherpre, and one without) is a bit of CPU waste, moreover when Im using tnlmeans.

The code would come as follows:

line=src.MT("tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6,sse=false)",2,2)
flat=Dither2Pre(line,stacked=true)

Would this run tnlmeans twice, or will it just use the buffered image for dither2pre?

Also I have discovered some big ghosting problems (not halo-like like gradfun) in fast motion scenes. This is strange because I didnt think "Dither" had temporal qualities.

[cretindesalpes]

I found out that you dont need the quotes when using the filter string in ditherpre.
If you don't put the double quotes, Avisynth will interpret the filter as a command to process last and will use the result as input (first parameter) for DitherPre. And because you probably multiplied the filter strength by 2 or 4 depending on the DitherPre function you use, you'll get maximum blur here. Moreover, the clip will be filtered a second time with the default filter. This is clearly not what you want.

Would this run tnlmeans twice, or will it just use the buffered image for dither2pre?
This will run tnlmeans once, and the internal default filter (a vanilla dfttest) four times. This code should work better:
Dither2Pre (flt="tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6,sse=false)", stacked=true)I don't know exactly which function you should wrap with MT() for maximum performances.

Also I have discovered some big ghosting problems (not halo-like like gradfun) in fast motion scenes. This is strange because I didnt think "Dither" had temporal qualities.
The ghosting comes from dfttest used in DitherPre as default filter. Dfttest is not protected against ghosting, therefore you have to MCompensate/Interleave the previous and next frames first, and SelectEvery the result (see the dfttestMC function). Regarding motion compensation, it is advised to do it outside the DitherPre functions. Avoid using dfttestMC (or other motion-aware filters) in the filter string, because the motion estimation may fail on the material actually submitted to the filter.

[Dogway]

Thank you. In this case tnlmeans works best with MT. I had to use three quotes, which is what I did first, but Dither weakens the denoise effect so I thought it wasn't working at all or something. Then I got fooled with the "clip" parameter I guess.
That explains all the issues I was having, so hopefuly I dont need any edgemask workaround anymore.

Now what Im trying to do is some resizing in between, the code I have is as follows, but Im unsure if I can resize msb and lsb without ditherposting them first.

den=Dither2Pre(flt="""MT("tnlmeans(settings)",2,2)""",stacked=true)
msb=den.Crop(0,0,w,h)
lsb=den.Crop(0,h,w,h)
rounded=den.SmoothGrad (stacked=true).DitherPost(mode=-1,stacked=true,prot=false)
contraV=rounded.splinaa(720,480).LSFmod()
contra_d=mt_makediff (contraV,rounded.spline36resize(720,480),y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb.spline36resize(720,480),contra_d,y=3,u=3,v=3)
mask=DitherBuildMask(msb_sharp,msb.spline36resize(720,480))
DitherPost(msb_sharp,lsb.spline36resize(720,480),prot=false,mask=mask)

I omitted the parameters but I realised how they need to be tweaked to obtain the same denoising as without using dither. In help, you say "amplitude-relative parameters must be doubled, and powers quadrupled" (for Dither1Pre which Im testing with), yet I dont get the values:
tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6)
tnlmeans(ax=20,ay=20,az=4,sx=4,sy=4,bx=1,by=1,h=2.4)
Are you sure filter behaviour is maintained?

[cretindesalpes]

Now what Im trying to do is some resizing in between, the code I have is as follows, but Im unsure if I can resize msb and lsb without ditherposting them first.
Resizing a 16-bit picture is possible but prone to (light) artifacts. You should resize the lsb with BilinearResize in order to minimize the overshoots. Also, use prot=true in DitherPost. Anyway, I recommend to do 16-bit operations only on the final resolution. Resizing 16-bit data would not be an issue if we had 16-bit resizers... PhrostByte's ResampleHQ could do the job but it seems he's not interested in developing true 16-bit input/output.

In help, you say "amplitude-relative parameters must be doubled, and powers quadrupled" (for Dither1Pre which Im testing with), yet I dont get the values:
tnlmeans(ax=10,ay=10,az=2,sx=2,sy=2,bx=1,by=1,h=0.6)
tnlmeans(ax=20,ay=20,az=4,sx=4,sy=4,bx=1,by=1,h=2.4)
Are you sure filter behaviour is maintained?
I don't know much TNLmeans so I can't give an definitive opinion. Anyway from what I read in the doc, ax, ay and az are spatial or temporal parameters, so they shouldn't be modified. Only a and h are related to pixel values, and I guess both must be multiplied by 2 in this case.

[Dogway]

uhmm As you said doubling only a and h seemed to work almost identical, I then applied your suggestions to ditherpost. Only after I realised of some artifacts so-to-speak, which I linked to the fact of resizing in 16bits. So I run a few tests up to a simple ditherpre-ditherpost, and still there, so maybe there's need to tweak more tnlmeans parameters, or they are wrong....

These are the two shots for comparison, the first one running only tnlmeans and dithering with gradfun, the second one with this workaround with dither. I want you look at the wall, grass loses some detail too.

old-workflow (http://i212.photobucket.com/albums/cc35/Dogway/Misc/no-dither-1.png)
dither-workflow (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither-1.png)

edit: the issue could be in ditherpost as well, although I tried several modes(-1,0,6,7) and didn't see any improve.
edit2:about ResampleHQ it would be nice in the meantime, bear in mind even yv12 dither isn't implemented. Its a bit pity to develop a 16b processing filter to benefit from wider bitdepth, and then crush it down to 8 back again.

[cretindesalpes]

The artifacts don't come from DitherPost, the denoising stage is clearly the cause here... Try to keep a or h at its original value, or even both? I don't know exactly how these parameters affect the algorithm.

[cretindesalpes]

New Dither version (1.6), some important changes :
Masktools 2 a48 is now the minimum requirement because some Dither functions use the new bitwise operators. I don't know how to automatically detect the Masktools version, so be sure to update your Masktools before installing this Dither version.
Renamed the rem parameters into clsb in DitherPost and SmoothGrad functions
Changed the default for the stacked (now true) and prot (now false) parameters.
The modified MVTools2 have been synchronised with the latest official release (2.5.11.2 beta)
Added various utility functions, mainly Dither_lut8, Dither_lut16, Dither_add16 and Dither_merge16.
Some documentation improvements

[Heaud]

Great to see an update!

I have a question on the new utility functions. Would dither_convert_8_to_16 be able to replace the blank clip + stackvertical inside of gradfun3? Is there any other ways to improve gradfun3? I added a custom smoothgrad since I did not like the results of the orignal smoothgrad inside of the script (it was causing weird banding halos).

[cretindesalpes]

I have a question on the new utility functions. Would dither_convert_8_to_16 be able to replace the blank clip + stackvertical inside of gradfun3?
Yes, that's exactly what it does. But it's just a convenient way to write it in a script, it gives the same result and won't improve anything in GradFun3.

[Dogway]

I did a few dozen tests, tweaking sx,sy,bx,by,a and h, even removing MT(). None of them sort the artifact, furthermore I tried dither2pre to "gain" more accuracy but what I got were more artifacts, check the lineart. (h is quadrupled according.)

old-workflow (http://i212.photobucket.com/albums/cc35/Dogway/Misc/no-dither-1.png)
dither1pre (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither-1.png)
dither2pre (http://i212.photobucket.com/albums/cc35/Dogway/Misc/dither2pre.png)

Im gonna test dither 1.6 now so I can tell you. edit: same results

[cretindesalpes]

Thank you very much for testing and helping me to improve the functions. I'm going to rewrite Dither1Pre to make the "slices" overlap, you'll tell me if it works better when it's ready.

[TheProfosist]

to me "Dither+SmoothGrad" seems like the best option. i would like to use it but cant determine from the help file what exactly you used/did. if it helps i usually denoise using fft3dgpu/fft3dfilter and sharpen using LSFmod before debanding.

[HymmexyLeadia]

im using a condensor mic to record samples, and theres a bit of noise in the background, i use audacity, but the noise removal algorithm isnt very good.
is there any software that does this better? thanks.
_________________
God rest ye, little children; let nothing you affright, For Jesus Christ, your Saviour, was born this happy night; Along the hills of Galilee the white flocks sleeping lay, When Christ, the Child of Nazareth, was born on Christmas day.
http://www.chung-sim.com/omron.3g3jv.html

[cretindesalpes]

Dither upgraded to v1.7 (http://ldesoras.free.fr/src/dither-1.7.zip). Changes:

Slightly improved Dither1Pre and Dither2Pre functions, added the overlap and tvopt parameters.
People complained about GradFun3 being too strong and washing fine shades away, so I changed the default parameter values for a better match with the gradfun2db default settings. thr is now set to 0.35 and radius to 12.
_____________________________________________________________________

to me "Dither+SmoothGrad" seems like the best option. i would like to use it but cant determine from the help file what exactly you used/did. if it helps i usually denoise using fft3dgpu/fft3dfilter and sharpen using LSFmod before debanding.See: http://forum.doom9.org/showthread.php?p=1390594#post1390594
but replace DitherPost (mode=2) with SmoothGrad (radius=16, thr=0.25, elast=2).DitherPost ()

[coolguy89]

Hi,
I tried your example above, but, the resolution changed from 720x480 to 720x240. Did I do something wrong?

[cretindesalpes]

You probably have a DitherPost() too many. Also make sure you have the latest Dither version. A few default parameters have changed recently and I updated the script in the post #8 accordingly.

[coolguy89]

This is what I use

FFT3DGpu (Sigma=1)
DitherPost (mode=2)

Function zzz_denoise (clip src, float "sigma", int "thr", bool "mask", int "sad")
{
sigma = Default (sigma, 16)
thr = Default (thr, 5)
mask = Default (mask, False)
sad = Default (sad, 200)

w = src.Width ()
h = src.Height ()

# Motion analysis
super = MSuper (src)
super_a = MSuper (src.TTempSmooth ().RemoveGrain (12))

fwd_vect_3 = super_a.MAnalyse (isb=false, delta=3, overlap=4)
fwd_vect_2 = super_a.MAnalyse (isb=false, delta=2, overlap=4)
fwd_vect_1 = super_a.MAnalyse (isb=false, delta=1, overlap=4)
bck_vect_1 = super_a.MAnalyse (isb=true, delta=1, overlap=4)
bck_vect_2 = super_a.MAnalyse (isb=true, delta=2, overlap=4)
bck_vect_3 = super_a.MAnalyse (isb=true, delta=3, overlap=4)

fwd_comp_2 = src.MCompensate (super, fwd_vect_2, thSAD=sad)
fwd_comp_1 = src.MCompensate (super, fwd_vect_1, thSAD=sad)
bck_comp_1 = src.MCompensate (super, bck_vect_1, thSAD=sad)
bck_comp_2 = src.MCompensate (super, bck_vect_2, thSAD=sad)

# Spatio-temporal denoising using modified dfttest
c_dft = Interleave (fwd_comp_2, fwd_comp_1, src, bck_comp_1, bck_comp_2)
c_dft = c_dft.dfttest (sigma=sigma, lsb=true) # Double height
c_dft = c_dft.SelectEvery (5, 2)

# Temporal-only denoising using modified MDegrain
c_deg = src.MDegrain3 (super, bck_vect_1, fwd_vect_1, bck_vect_2, fwd_vect_2, bck_vect_3, fwd_vect_3, thSAD=sad, lsb=true) # Double height

# Spatio-temporal denoising smoothes too much the details,
# therefore we use pure temporal denoising on edges or detailed areas.
edge_src = c_deg.Crop (0, 0, w, h)
edge_mask = edge_src.mt_edge (mode="prewitt", thY1=thr, thY2=thr)
edge_mask = edge_mask.mt_expand ()
edge_mask = StackVertical (edge_mask, edge_mask) # Double height
c_hyb = mt_merge (c_dft, c_deg, edge_mask, luma=true, y=3, u=3, v=3)

return (mask ? edge_mask.GreyScale () : c_hyb)
}

I'm using the latest Dither btw.

[cretindesalpes]

Use zzz_denoise(). This function does almost all the job, with SmoothGrad. DitherPost only converts the 16-bit result into a regular 8-bit dithered clip.
zzz_denoise()
SmoothGrad (radius=16, thr=0.25, elast=2)
DitherPost ()

If you want to keep FFT3DGpu as denoising filter, just do :
FFT3DGpu (Sigma=1)
GradFun3 ()or
Dither2Pre (flt="FFT3DGpu(Sigma=4)")
SmoothGrad (radius=16, thr=0.25, elast=2)
DitherPost ()

[coolguy89]

It works! Thanks :).

One more question, if I only want to deband, I only need the GradFun3, right? Or is there any other way in the dither packages that can improve debanding process.

[cretindesalpes]

Yes, use GradFun3 to deband only.

[coolguy89]

Thank you for your support and your awesome script. ;)

[Dogway]

cretin, the error I was having was caused by MT inside dither1pre. I had to change splitvertical to true also to make it aware dither was changing the size.
On a note, before this I tried stacked=false and it gave me temporal artifacts. Finally after an afternoon battle it seems this code is what it works, no problems for encoding either.
MT("""Dither1Pre(flt="tnlmeans()",stacked=true)""",2,2,true)
DitherPost(stacked=true,prot=false)
You could also add it to documentation for reference.

[SSH4]

cretindesalpes can you check your mod16 of Mvtools?
Looks like all (latest mvtools-2.5.11.2-mod16a.zip and previous) version have some bug with tsadc. tsadc always == tsad.
you can check it if you made big tsad (about 600 or more) and try low tsadc. UV channel will be bad :(
on original mvtools-2.5.11.2 all work well.

thanx!

[cretindesalpes]

Indeed there was a bug, thanks for the report. I'll release a fixed version soon.

[cretindesalpes]

Dither 1.7.1 (http://ldesoras.free.fr/src/dither-1.7.1.zip) update. Changes:
Dither.avsi doesn't use Max() anymore, so it can now run on Avisynth 2.5.7 or earlier.
Bugfix in modified MVTools2: the thSADC argument was ignored in MDegrain1/2/3.

[Dogway]

Thanks for the new version, I really liked Dither_lut*8() functions, but I wondered whether Dither_lut*16() would be possible, seems more logical having also Dither_convert_8_to_16 in the pack.

[cretindesalpes]

but I wondered whether Dither_lut*16() would be possible
It's possible, but not implemented as lookup tables, and it would be much slower than the mt_lut*. Also, this is not a trivial task.

But wait, there is a new dither (http://ldesoras.free.fr/src/dither-1.8.zip) version today:
Dfttest has been modified to make it accept 16-bit input clips (v1.8 mod16b). It's now possible to apply a curve on the input and the inverse curve to the output to protect dark shades for example, without resolution loss or risk of color banding.
Made available the two internal components of the SmoothGrad function: Dither_box_filter16 and Dither_limit_dif16.
Added Dither_bilateral16, a bilateral filter giving cleaner results than the box filter for gradient smoothing.
Improved GradFun3: Better detail/edge masking with adjustable thresholds and optional mask display for debugging purpose.
Added two smoothing modes using the modified dfttest and the new bilateral filter. The latter is slow as hell and should be considered as experimental.
Added a Dither_crop16 function to properly crop stacked 16-bit clips.
Added a luma parameter to Dither_merge16_8 and fixed a slight chroma bug.

Next, I'm planning to write a 16-bit resizer. This could be used to do high bitdepth colorspace conversions.

[Dogway]

ooh shhh**. You are the man! so 16-bit resizer and colorspace conversion, then by being linear, gamma isn't an issue here, right? That's better news than expected if done right. I hope to use your spline36 then next time. Although lately I discovered MultiSWAR (http://forum.doom9.org/showthread.php?t=112526)resizer by *.mp4 guy which works wonders for animated content.

Also Im going to test your Dither_bilateral16 function. Although it is only space dimensional I think I couldn't afford more through dither. I really try to refrain using dfttest as much for animation, not only it produces temporal artifacts/ghosting but it also blurs lineart or details, and use instead a combination of a good temporal with a bilateral, like ttempsmooth+tnlmeans.

[Heaud]

Tried out that smode 2 and it indeed is slow as all hell. Visually, it's slightly better than smode 1 and both new modes are way much better than the old method. Looks like I no longer have to do a 5 pass smoothgrad any more!

[SilaSurfer]

Although lately I discovered MultiSWAR (http://forum.doom9.org/showthread.php?t=112526)resizer by *.mp4 guy which works wonders for animated content.

For upsizing I presume?

Nice work cretindesalpes!;):p

[Dogway]

For upsizing I presume?
Ye, didn't try downsizing tho. In my pack I included a modified version for awarpsharp2 and masktools2, but Id like to remove/replace more dependencies of old or I-dont-use-much plugins.

I tested Dither_bilateral16. It works ok a bit too raw maybe, I will try to use some combination of edge/luma/difference mask. Also the get_msb/lsb function saves any speed or is just for convenience?

[cretindesalpes]

I tested Dither_bilateral16. It works ok a bit too raw maybe, I will try to use some combination of edge/luma/difference mask.
Indeed, this function alone hasn't any protection. Its main target is blurring everything to smooth the gradients, but in a smarter way than the box filter do (taking only close colors into account). I recommend adding a first protection with Dither_limit_dif16(). Then, use a mask on the result. This is how it is implemented in GradFun3(). I have ideas to optimize the bilateral filter, but unfortunately I wouldn't expect significant speed gains.

Also the get_msb/lsb function saves any speed or is just for convenience?
It is just for convenience. Well, it's just a Crop(), so it would be difficult to write something faster!

[ESPARO]

I have a problem when I use GradFun3 ()

when i used it no error appears but the video turn to " Green" screen

i install Masktools 2 a48 before I install the filters.. any feedback?

i also re install the avisynth .. but no different ..

[cretindesalpes]

Could you please post your script? Which Avisynth version do you use ?

[ESPARO]

this is my script :
LoadPlugin("C:\Program Files (x86)\dgmpgdec158\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins\ColorMatrix.dll")
DGDecode_mpeg2source("C:\Users\Luffy\Desktop\Ao-No\onepiece-498.D2V", info=3)
tfm(D2V="C:\Users\Luffy\Desktop\Ao-No\onepiece-498.D2V")
trim(4625,9110)+trim(11220,34018)+trim(40612,54817)
gradfun3()

and I use AVIsynth 2.5.8 .. AND I tried 2.5.7 too
and there is no news just green clip ..

I tried it at many script but no different and

DitherPost and the others is work fine with me ..!
only gradfun3 () is not working fine.

[cretindesalpes]

I think you still have an old Masktools version installed somewhere and autoloaded by Avisynth. This is the only way I could reproduce this green screen. Search on your disk mt_masktools.dll, mt_masktools-25.dll and possibly mt_masktools-26.dll, remove the culprit(s), then install the latest version.

[ESPARO]

Thx a lot .. that's works with me ^_^

[cretindesalpes]

Dither 1.8.1 (http://ldesoras.free.fr/src/dither-1.8.1.zip):

Optimised Dither_bilateral16 with SSE(2) and introduced a subsampling parameter. This gives more than a 150-fold speed increase with the default settings at the cost of a minor quality loss, making the function usable now.
Still in Dither_bilateral16, fixed a bug related to the use of a ref clip.
Added a troubleshooting section to the documentation.

[SilaSurfer]

Example of your functions from user guide that came in the package


o = last
Dither2Pre ("TTempSmooth (lthresh=16, cthresh=20, lmdiff=8, cmdiff=12, scthresh=48)", overlap=0, tvopt=true)
ref = last
mask = DitherPost (mode=-1).mt_edge (mode="min/max", thY1=0, thY2=255, thC1=0, thC2=255)
mask = mask.mt_binarize (1, u=3, v=3).mt_expand (u=3, v=3).mt_expand (u=3, v=3)
mask = mask.RemoveGrain (20).RemoveGrain (20)
Dither_bilateral16 (radius=16, thr=1.5, wmin=1, flat=0.75)
Dither_bilateral16 (radius=8, thr=2.5, wmin=1, flat=0.33, ref=ref)
Dither_limit_dif16 (ref, thr=0.3)
DitherPost ()
mt_merge (last, o, mask, u=3, v=3)

Could Ttempsmooth be replaced with modified dfttest so Ditherpre wouldn't be needed :rolleyes:

o = last
Dfttest(settings) #16bit output
ref = last
mask = DitherPost (mode=-1).mt_edge (mode="min/max", thY1=0, thY2=255, thC1=0, thC2=255)
mask = mask.mt_binarize (1, u=3, v=3).mt_expand (u=3, v=3).mt_expand (u=3, v=3)
mask = mask.RemoveGrain (20).RemoveGrain (20)
Dither_bilateral16 (radius=16, thr=1.5, wmin=1, flat=0.75)
Dither_bilateral16 (radius=8, thr=2.5, wmin=1, flat=0.33, ref=ref)
Dither_limit_dif16 (ref, thr=0.3)
DitherPost ()
mt_merge (last, o, mask, u=3, v=3)

Nice work on optimizations.

[cretindesalpes]

Yes, of course you can, but then you should add all the motion compensation code if tbsize is not set to 1, otherwise you'll get ghosting.

[SilaSurfer]

I think your Mdegrain1/2/3 16bit mod instead of dfttest would be a better idea.

[Dogway]

hey cretin, Im still testing and learning your tools. I just tested smoothcurve16 and everything's ok, but in this certain case Im doing the color correction before denoising. I use it like this:
Dither_convert_8_to_16()
smoothcurve16(ycurve="string")
dfttest(lsb_in=true,lsb=true)
#Dither1Pre("dfttest()",stacked=true)
DitherPost(stacked=true,prot=false,mode=6)
You see a commented line, this is me trying to reproduce the same behaviour with dither1pre, but it doubles the height of my already doubled(msb+lsb) input. The lsb_in parameter is something I guessed myself because I couldn't find any reference in the help.

I learned finally how to use smoothgrad but what differentiates it from gradfun3? Can I use them together? I tested gradfun3 yesterday and I remember performing better if added at the very end of the script (after ditherpost) rather than added through the masking workaround, normally right after my sharpening stage. I can elaborate more on this later on.

I also realised I had some kind of banding or dither alteration when doing a resize using spline36. I attach a sample and my script just in case something is wrong, but it could easily be the resizer being 8bit. With prot=true I was having more severe banding than this
raw=last
filtered=MT("""Dither1Pre(flt="tnlmeans(settings)",stacked=true)""",2,2,true)
msb=filtered.dither_get_msb
lsb=filtered.dither_get_lsb
filtered=filtered.DitherPost(mode=-1,stacked=true,prot=false)
msb_sharp=Contrasharpening(filtered,raw)
msb_sharp=mt_makediff (msb_sharp,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3).Spline36Resize(720,480)
msb=msb.Spline36Resize(720,480)
mask=DitherBuildMask(msb_sharp,msb)
SmoothGrad(msb_sharp,lsb.BilinearResize(720,480),radius=16, thr=0.25, elast=2).DitherPost(prot=false,mask=mask)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/artifact.png

[SilaSurfer]

Hello cretindesalpes.

I'm would like to use this workflow for a perticular movie:

spatial = MinBlur(1)
temporal = spatial.FluxsmoothT()
mixed = temporal.merge(spatial,0.251)
Fft=mixed.sbr()

pre = fft.MSuper(pel=2, sharp=1)
src_super = o.MSuper(pel=2, sharp=2)
backward_vec2 = MAnalyse(pre, isb = true, delta = 2, overlap=4,truemotion=false)
backward_vec1 = MAnalyse(pre, isb = true, delta = 1, overlap=4,truemotion=false)
forward_vec1 = MAnalyse(pre, isb = false, delta = 1, overlap=4,truemotion=false)
forward_vec2 = MAnalyse(pre, isb = false, delta = 2, overlap=4,truemotion=false)
Nr1=o.MDegrain2(lsb=true, src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2thSAD=300,thscd1=300,thscd2=90).contra(o)


x1=subtract(Nr1.deen("a2d",3,4,0),Nr1)
subtract(Nr1,x1)

DitherPost()

I would like to have contrasharpening, and deen enhancer between Mdegrain2 and Ditherpost with using masks and I just cant get it working. I would appreciate your help if it can be done.

[Dogway]

@SilaSurfer: try this if it works
filtered=o.MDegrain2(lsb=true, src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2thSAD=300,thscd1=300,thscd2=90)
msb=filtered.dither_get_msb
lsb=filtered.dither_get_lsb
filtered=filtered.DitherPost(mode=-1,stacked=true,prot=false)
con=Contra(filtered,o)
x1=subtract(con.deen("a2d",3,4,0),con)
msb_sharp=subtract(con,x1)
msb_sharp=mt_makediff (msb_sharp,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3)
mask=DitherBuildMask(msb_sharp,msb)
DitherPost(msb_sharp,lsb,prot=true,mask=mask)

[SilaSurfer]

@SilaSurfer: try this if it works
filtered=o.MDegrain2(lsb=true, src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2thSAD=300,thscd1=300,thscd2=90)
msb=filtered.dither_get_msb
lsb=filtered.dither_get_lsb
filtered=filtered.DitherPost(mode=-1,stacked=true,prot=false)
x1=subtract(filtered.deen("a2d",3,4,0),filtered)
msb_sharp=Contra(filtered,o).subtract(filtered,x1)
msb_sharp=mt_makediff (msb_sharp,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3)
mask=DitherBuildMask(msb_sharp,msb)
DitherPost(msb_sharp,lsb,prot=true,mask=mask)

Thanks for the effort Dogway, can't try it right know (encoding), but I did notice something about contrasharpening function. I have a modified version.

taken from my example above:

o=last
spatial = o.MinBlur(1)
temporal = spatial.FluxsmoothT()
mixed = temporal.merge(spatial,0.251)
Fft=mixed.sbr()

pre = fft.MSuper(pel=2, sharp=1)
src_super = o.MSuper(pel=2, sharp=2)
backward_vec2 = MAnalyse(pre, isb = true, delta = 2, overlap=4,truemotion=false)
backward_vec1 = MAnalyse(pre, isb = true, delta = 1, overlap=4,truemotion=false)
forward_vec1 = MAnalyse(pre, isb = false, delta = 1, overlap=4,truemotion=false)
forward_vec2 = MAnalyse(pre, isb = false, delta = 2, overlap=4,truemotion=false)
Nr1=o.MDegrain2(lsb=true, src_super, backward_vec1,forward_vec1,backward_vec2,forward_vec2thSAD=300,thscd1=300,thscd2=90).contra(o)

You can use it in chain reaction and only specify a source in this case (o). You don't need to input filtered source. It was modified by Didée in thread where I found it. Here is the function itself:

function contra (clip denoised, clip original)
{
s = denoised.minblur(1,1) # Damp down remaining spots of the denoised clip.
allD = mt_makediff(original,denoised) # The difference achieved by the denoising.
ssD = mt_makediff(s,s.removegrain(11,-1)) # The difference of a simple kernel blur.
ssDD = ssD.repair(allD,1) # Limit the difference to the max of what the denoising removed locally.
ssDD = SSDD.mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?") # abs(diff) after limiting may not be bigger than before.

denoised.mt_adddiff(ssDD,U=2,V=2) # Apply the limited difference. (Sharpening is just inverse blurring.)

return( last )
}

Thanks for your script I will try it as soon as I can.

[Dogway]

But here this is a variable chain so you need to specify. If you dont you are implying "last", which is the latest filter used outside the variable chain. I use this all the time with the same contra function and it works, but just give it a try and tell me. Once you get the grasp of dither mechanics it becomes easy (and fun!)

[SilaSurfer]

Yeah, can't wait! Gracias Dogway

[mandarinka]

Is there a way to apply gradfun3 but keep the final result in 16bits (the stacked MSB/LSB format), for 10bit x264 encoding? I tried to remove the ditherpost line from the function, but that didn't go so well (I don't understand scripting, sadly...)

I would like to do this: 8bit source with banding -> debanded 16bit -> 10bit h264.

[mp3dom]

Hi cretindesalpes, amazing work and I found the result of Gradfun3 more pleasant now than gradfun2db. Actually I need to do a dithering on a YUY2 clip and I'm getting errors if I don't convert it to YV12 (the script is asking for a "isYV16()" function that doesn't exists in AVS 2.58 and is only available on 2.60). I still would prefer to dither natively on YUY2 because the encoder accepts YUY2 and I would like to avoid colorspace conversions. I've tried to convert the clip to planar (with SSE2Tools available with RemoveGrain package) but the script always ask for that function. Is there a way to 'fix' it? Thank you very much!

[tormento]

Any x64 compatible code? Please ;)

[LaTo]

Hi cretindesalpes, amazing work and I found the result of Gradfun3 more pleasant now than gradfun2db. Actually I need to do a dithering on a YUY2 clip and I'm getting errors if I don't convert it to YV12 (the script is asking for a "isYV16()" function that doesn't exists in AVS 2.58 and is only available on 2.60). I still would prefer to dither natively on YUY2 because the encoder accepts YUY2 and I would like to avoid colorspace conversions. I've tried to convert the clip to planar (with SSE2Tools available with RemoveGrain package) but the script always ask for that function. Is there a way to 'fix' it? Thank you very much!

YUY2 <-> YV16 conversion is lossless... It's the same colorspace but YV16 is planar.

[mp3dom]

Yes, but under AVS 2.58 (which I'm currently use) there's no function named "isYV16()". There are other functions too that are called to check the source colorspace: IsYV12, IsYV16, IsYV24, IsYV411, IsY8. With AviSynth 2.58 seems that only YV12 source can be processed so I'm basically asking if it's possible to add AviSynth 2.58 support for planar YUY2. Thanks.

[cretindesalpes]

You see a commented line, this is me trying to reproduce the same behaviour with dither1pre, but it doubles the height of my already doubled(msb+lsb) input. The lsb_in parameter is something I guessed myself because I couldn't find any reference in the help.
DitehrXPre () is for 8 to 16 bit conversion with filtering. Here, your clip has already a 16 bit depth, no need for these functions. Your version with lsb_in is right. This option is documented is in the dfttest-1.8-mod16b.zip readme file.

I learned finally how to use smoothgrad but what differentiates it from gradfun3?
SmoothGrad is a 100% 16-bit (in and out) building block. GradFun3 is a no-brainer function based on SmoothGrad, and is to be called on 8-bit clips like gradfun2db(mod). Don't call it on 16-bit clips, it would give unexpected results. Actually GF3 calls internally SmoothGrad multiple times, with different radius and thresholds.

I also realised I had some kind of banding or dither alteration when doing a resize using spline36.
Your script looks correct. Unfortunately, using prot=true doesn't solve all the problems related to resizing. There are no real solution for clean resizing in the 8-bit world. That's why I'm currently working on a 16-bit resizer, which will do the job correctly, without the prot=true hack.

Is there a way to apply gradfun3 but keep the final result in 16bits (the stacked MSB/LSB format), for 10bit x264 encoding? I tried to remove the ditherpost line from the function, but that didn't go so well (I don't understand scripting, sadly...)
Removing everything after the "# Dithering" line in GradFun3 should output regular stacked 16 bits data. Then if you want to output it to x264-10bit, put just after the call the code snippet I wrote in the SmoothAdjust thread (http://forum.doom9.org/showpost.php?p=1504502&postcount=216). Anyway, I don't know if there are players which can play correctly 10-bit AVC streams. Recent ffmpeg builds can decode them, but the extended accuracy is available only for yuv420p16 output...

Actually I need to do a dithering on a YUY2 clip and I'm getting errors
Indeed, the functions are not designed for non-planar data. I should investigate how Interleaved2Planar() works as it is not well documented on a technical POV, and find if it's really interesting to implement, because mt_merge(luma=true) doesn't work with YUY2.

In the meantime, you can use this workaround:
y = ConvertToYV12 ()
u = UToY ().ConvertToYV12 ()
v = VToY ().ConvertToYV12 ()
YToUV (u, v, StackVertical (y, y))

GradFun3 (mask=0)

y = Crop (0, 0, 0, Height () / 2).ConvertToYUY2 ()
u = UToY ().ConvertToYUY2 ()
v = VToY ().ConvertToYUY2 ()
YToUV (u, v, y)
Theses conversions are lossless. However in the converted clip, luma and chroma planes are not spatially synchronised. Everything mixing them should be avoided, like the GradFun3's mask functionality using mt_merge(luma=true). If you really want masking, you can still modify the script this way:

ca = YToUV (u, v, StackVertical (y, y))
cb = YToUV (u, v, y.PointResize (Width (), Height () * 2))

Interleave (ca, cb)
GradFun3 ()
MergeChroma (SelectEven (), SelectOdd ())

(BTW note to myself: make the mask threshold vary according to the main threshold, because increasing only the latter is completely useless to combat serious banding).

Any x64 compatible code? Please ;)
Not in my priority list, unless the source code for x64 dfttest and MVTools2 suddenly becomes available. Unfortunately JoshyD hasn't connected to this forum for more than a year.

[mandarinka]

Removing everything after the "# Dithering" line in GradFun3 should output regular stacked 16 bits data. Then if you want to output it to x264-10bit, put just after the call the code snippet I wrote in the SmoothAdjust thread (http://forum.doom9.org/showpost.php?p=1504502&postcount=216). Anyway, I don't know if there are players which can play correctly 10-bit AVC streams. Recent ffmpeg builds can decode them, but the extended accuracy is available only for yuv420p16 output...

Thanks! I'll give it a try.

[SilaSurfer]

Hello everyone. Dogway I just tried your example that you posted and it works. For another source I swapped Mdegrain with dfttest like this:

filtered=o.dfttest(lsb=true,sigma=1.5, sbsize=48,tbsize=3,sosize=16,U=false,V=false)
msb=filtered.dither_get_msb
lsb=filtered.dither_get_lsb
filtered=filtered.DitherPost(mode=-1,stacked=true,prot=false)
con=Contra(filtered,o)
x1=subtract(con.deen("a2d",3,4,0),con)
msb_sharp=subtract(con,x1)
msb_sharp=mt_makediff (msb_sharp,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3)
mask=DitherBuildMask(msb_sharp,msb)
DitherPost(msb_sharp,lsb,prot=true,mask=mask)

but I have a little trouble again :p, adding Smoothgrad in there to do some debanding which was covered by grain that was removed. Appreciate any help.

[cretindesalpes]

Just replace your last line with the following code:
SmoothGrad (msb_sharp, clsb=lsb)
DitherPost (prot=true, mask=mask)

[SilaSurfer]

cretindesalpes Thank you very much, BTW how is that 16-bit resizer coming along?

[mp3dom]

Thanks cretindesalpes!

[SilaSurfer]

cretindesalpes does Smoothgrad remove details since it's operation is smoothing gredients to eliminate banding?

[cretindesalpes]

does Smoothgrad remove details since it's operation is smoothing gredients to eliminate banding? It depends on the thr value, but with the default settings the damages will be minimal. Anyway, I would recommend to use a mask (see the GradFun3 implementation).
Thank you very much, BTW how is that 16-bit resizer coming along?
The resizer is ready and working but I just installed Avisynth 2.6 on another computer and I'm trying (struggling?) to make a single dll work on both 2.5.x and 2.6.0 while supporting correctly the new planar colorspaces.

[SilaSurfer]

Thanks for your response. I had another idea but I need your opinion once more.;)

Taken from my example above, just removing deen:

o=last
filtered=o.dfttest(lsb=true,sigma=1.5, sbsize=48,tbsize=3,sosize=16,U=false,V=false)
msb=filtered.dither_get_msb
lsb=filtered.dither_get_lsb
filtered=filtered.DitherPost(mode=-1,stacked=true,prot=false)
con=Contra(filtered,o)
msb_sharp=mt_makediff (con,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3)
mask=DitherBuildMask(msb_sharp,msb)
SmoothGrad (thr=0.15,msb_sharp, clsb=lsb)
DitherPost (prot=true, mask=mask)

to use Smoothgrad after dfttest like this so any detail loss will be returned with contrasharpening

o=last
filtered=o.dfttest(lsb=true,sigma=1.5, sbsize=48,tbsize=3,sosize=16,U=false,V=false)-16-bit output.SmoothGrad (thr=0.15, clsb=lsb)
msb=filtered.dither_get_msb
lsb=filtered.dither_get_lsb
filtered=filtered.DitherPost(mode=-1,stacked=true,prot=false)
con=Contra(filtered,o)
msb_sharp=mt_makediff (con,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3)
mask=DitherBuildMask(msb_sharp,msb)
DitherPost(msb_sharp,lsb,prot=true,mask=mask)

Didn't test this since I'm at work, just wanted your opinion.Thank you.

PS:
Can't wait for that resizer :p

[cretindesalpes]

Yes it should work (without the typos and the clsb parameter, useless here because the dfttest output is stack16). But be careful, the contrasharpening might restore the banding as well. You'll have to check with your source.

[cretindesalpes]

Dither 1.9 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) update:

The threshold of the detail mask in GradFun3() now automatically adjusts to the debanding threshold. This helps removing serious color banding, when band boundaries are masked because they are detected as details. Also, the radius for smode 0 has been internally reduced to make the value reflect more accurately the actual range in pixels.
Added an interlaced mode for DitherPost().
Added Dither_resize16(), a 16-bit resizer which can also work as a chained horizontal and vertical convolution engine.
Added y, u and v parameters to almost all the dither.dll functions. Their behaviour is the same as in masktools2.
Added Dither_y_gamma_to_linear() and Dither_y_linear_to_gamma(), for gamma <-> linear conversions in 16 bits.
Added Dither_convey_yuv4xxp16_on_yvxx() and Dither_convey_rgb48_on_yv12(), functions to export clips with 16 bits per component when Avisynth's output is read as rawvideo via a pipe.
Added Dither_convert_yuv_to_rgb() and Dither_convert_rgb_to_yuv() to convert 8 or 16 bits YUV to RGB and back with dithering. It's also possible to output RGB48 conveyed on YV12 via a rawvideo pipe.
Added Dither_max_dif16() to get from two clips the pixels having the maximum difference with a third reference clip.
Fixed an error message about wrong colorspace.
Bugfix: removed artifacts on flat pure white areas (luma == 255) when using error diffusion dithering (modes 6–8).
Non-YV12 planar Y(UV) colorspaces should now work correctly on Avisynth 2.6.0. I haven't tested all the combinations with all the functions, so please report me any defect!
All these new things work correctly but are not well optimised. Consequently, they are insanely slow and resource hungry, especially the colorspace conversion function which would deserve a true C++/asm implementation instead of a scripted hack using the convolver. Currently, consider this more as a proof of concept and base for experiments rather than full-fledged production tools. Moreover, the stack16 format is nice and convenient for previewing, but horrendous for programming.

[Dogway]

Wow, you are really the game changer. Avisynth to the next level, thanks for all, its gonna take me a few days to assimilate.
The 16b resizer also fixed the banding problem I had in my other post (http://forum.doom9.org/showthread.php?p=1504967#post1504967)

I use it like this (+adding prot=true at the end)
filtered=MT("""Dither1Pre(flt="tnlmeans(settings)",stacked=true)""",2,2,true).Dither_resize16(720,480,kernel="spline36")
msb=filtered.dither_get_msb
lsb=filtered.dither_get_lsb
filtered=filtered.DitherPost(mode=-1,stacked=true,prot=false)
msb_sharp=Contrasharpening(filtered,raw.Spline36Resize(720,480))
...
Is this ok? it makes sense, but just wondering, I can't resize after sharpening right? not at least in 16b

edit: In the mean time, Im gonna try to port contrasharpening to 16b if its possible.

[cretindesalpes]

Is this ok? it makes sense, but just wondering, I can't resize after sharpening right? not at least in 16b
The code looks OK but I'm not sure to understand your question. Do you mean you want to put the resizing code after having obtained msb_sharp ? I think the safest way to do this is merging a 16 bit conversion of the 8-bit sharp clip with the denoised 16 bit clip, and resizing the result. Use a mask keeping only the pixels modified by the (contra)sharpener.

[Dogway]

I think the safest way to do this is merging a 16 bit conversion of the 8-bit sharp clip with the denoised 16 bit clip, and resizing the result. Use a mask keeping only the pixels modified by the (contra)sharpener.
This?
filtered1=MT("""Dither1Pre(flt="tnlmeans(settings)",stacked=true)""",2,2,true)
msb=filtered1.Crop(0,0,706,478)
lsb=filtered1.Crop(0,478,706,478)
filtered=filtered1.DitherPost(mode=-1,stacked=true,prot=false)
msb_sharp=Contrasharpening(filtered,raw)

msb_sharp=mt_makediff (msb_sharp,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3)
to16=msb_sharp.Dither_convert_8_to_16()

mask=DitherBuildMask(to16,filtered1)
Dither_merge16(filtered1,to16,mask).Dither_resize16(720,480,kernel="spline36")
SmoothGrad(radius=16, thr=0.25, elast=2).DitherPost()

# SmoothGrad(msb_sharp,lsb,radius=16, thr=0.25, elast=2).DitherPost(prot=true,mask=mask)

I dont know if the last line is correct, the commented line was what I was using before.

[SilaSurfer]

cretindesalpes Thanks for the new package. Could your resizer be modified to output 8 bit like Gradfun3 for example? The reason is that I denoise after downsize rather then denoising 1080p source which is very slow with about any filter.

[Dogway]

@SilaSurfer: Simply use DitherPost(). Dither is a bit of a group of tools and functions spread around. You need to join pieces like a puzzle for "user friendly" usage. That's why I just made a function for myself for resizing, its optimized for rgb input, but you have parameters to change them.

function LinearResize(clip a, int "w", int "h", bool "rgb", string "kernel", bool "linear", bool "TVrange",string "matrix"){

w=default(w, a.width)
h=default(h, a.height)
kernel = default(kernel, "spline36")
rgba = default(rgb, true)
linear = default(linear, true)
TV = default(TVrange, false)
m = default(matrix, "601")
a
rgba ? Dither_convert_rgb_to_yuv(lsb=true,matrix=m,tv_range=tv,output="yv24") : Dither_convert_8_to_16()
linear ? Dither_y_gamma_to_linear(tv_range_in=tv,tv_range_out=tv) : last
Dither_resize16(w,h,kernel=kernel)
linear ? Dither_y_linear_to_gamma(tv_range_in=tv,tv_range_out=tv) : last
rgba ? Dither_convert_yuv_to_rgb(lsb_in=true,matrix=m,tv_range=tv,ampn=0.5,mode=6) : DitherPost}


Implemented matrix conversion and automatizations depending on input, not sure if it works flawlessly, but I did my tests...
Linear=true doesn't work when downsizing from HD to SD, it is to take advantage of the high resolution input for color conversion, it is that or linear resizing which I still consider experimental.

function LinearResize(clip a, int "w", int "h", string "kernel", bool "linear", bool "TVrange", string "matrix", \
string "matrixOut", float "fh", float "fv", bool "rgb", bool "lsb_in", bool "lsb_out"){

w = default(w, a.width)
h = default(h, a.height)
kernel = default(kernel, "spline36")
linear = default(linear, false)
rgba = default(rgb, a.isrgb ? true : false)
TV = default(TVrange,rgba ? false : true)
m1 = default(matrix, rgba || a.width<1280 ? "601" : "709")
m2 = default(matrixOut, Defined(matrix) ? matrix : ( rgba || w<1280 ? "601" : "709"))
fh = default(fh, 1.0) # Shrink (>1.0 Sharpen) or expand (<1.0 Blur) the Kernel
fv = default(fv, fh) # idem but vertically
lsb_in = default(lsb_in, false)
lsb_out = default(lsb_out, false)

function mmod2(clip a){
bh = 2 - ((a.Width() -1)%2 + 1)
bv = 2 - ((a.Height()-1)%2 + 1)
a.crop(bh/2,bv/2,-bh,-bv)}

size = a.width>=1280 && w<1280 ? true : false
mtrx = m1!=m2 ? true : false
if1 = mtrx && size ? true : false
if2 = linear && (!size || rgba) ? true : false

if1 ? Assert(linear==false, "HD to SD downsize requires linear=false") : nop
rgba ? Assert(mtrx==false, "You can't change matrix in RGB") : nop

!lsb_in && rgba ? a.mmod2 : a
!lsb_in ? (rgba ? Interleave(showred("yv12"), showgreen("yv12"), showblue("yv12")).Dither_convert_8_to_16 \
: (if1 ? Dither_convert_8_to_16.Dither_convert_yuv_to_rgb(lsb_in=true,matrix=m1,tv_range=tv,output="rgb48y") \
: Dither_convert_8_to_16)) \
: (if1 ? Dither_convert_yuv_to_rgb(lsb_in=true,matrix=m1,tv_range=tv,output="rgb48y") : last)

if2 ? Dither_y_gamma_to_linear(tv_range_in=tv,tv_range_out=false) : last
Dither_resize16(w%2+w,h%2+h,kernel=kernel,fh=fh,fv=fv,u=rgba?1:3,v=rgba?1:3)
if2 ? Dither_y_linear_to_gamma(tv_range_in=false,tv_range_out=tv) : last


!lsb_out && rgba ? Ditherpost(ampn=0.5,mode=6,u=1,v=1) : last
mtrx && !size ? Dither_convert_yuv_to_rgb(lsb_in=true,matrix=m1,tv_range=tv,output="rgb48y") : last
r = mtrx || rgba ? SelectEvery (3, 0) : nop
g = mtrx || rgba ? SelectEvery (3, 1) : nop
b = mtrx || rgba ? SelectEvery (3, 2) : nop

!lsb_out ? (rgba ? Mergergb(r,g,b) \
:(mtrx ? Dither_convert_rgb_to_yuv(r,g,b,lsb=false,matrix=m2,tv_range=tv) : Ditherpost)) \
: (mtrx ? Dither_convert_rgb_to_yuv(r,g,b,lsb=true ,matrix=m2,tv_range=tv) : last)}

[SilaSurfer]

Yeah I'm bombarding you guys with question. :( Work takes a lot of my time. Thanks Dogway for your presentation. I decided to spend some time with these great tools in Dither package and learn. Thanks again

[cretindesalpes]

msb_sharp=Contrasharpening(filtered,raw)

msb_sharp=mt_makediff (msb_sharp,filtered,y=3,u=3,v=3)
msb_sharp=mt_adddiff(msb,msb_sharp,y=3,u=3,v=3)
to16=msb_sharp.Dither_convert_8_to_16()

mask=DitherBuildMask(to16,filtered1)

This would more likely become:

sharp8 = Contrasharpening (filtered, raw)
mask = DitherBuildMask (sharp8, filtered)
to16 = sharp8.Dither_convert_8_to_16()

but the mask is 8 bit here, use Dither_merge16_8() instead of Dither_merge16.

edit: also what is the most accurate dither algorithm, similar to gradfun2db. Is for images.
The most accurate dithering algorithm are the error diffusion ones (6 to 8). Mode=6 is a classic serpentine Floyd-Steinberg and should give good results. It's a good idea to add a slight bit of noise before (ampn=0.5) to avoid artifacts related to this kind of algorithm. Anyway, I don't think gradfun2db dither can really be considered as good! But it does its job well and very quickly.

http://img225.imageshack.us/img225/6927/dither.png
Top: gradfun2db,
bottom: serpentine Floyd-Steinberg + noise

[Dogway]

yes, sorry cretin, I read up a few pages back and help and found out by myself mode=6. :angry:

Im gonna test the code.
edit:
This then?
filtered1=MT("""Dither1Pre(flt="tnlmeans(settings)",stacked=true)""",2,2,true)
msb=filtered1.Crop(0,0,706,478)
lsb=filtered1.Crop(0,478,706,478)
filtered=filtered1.DitherPost(mode=-1)

sharp8 = Contrasharpening (filtered, raw)
mask=DitherBuildMask (sharp8, filtered)
to16=sharp8.Dither_convert_8_to_16()

Dither_merge16(filtered1,to16,stackvertical(mask,mask)).Dither_resize16(720,480,kernel="spline36")
SmoothGrad(radius=16, thr=0.25, elast=2).DitherPost()

I dont know why, but this shifts my colors a bit.
edit2: No problem, it is because Im taking the ditherpost version instead of msb, so Im doing the right thing.

edit3: Ive been doing some more tests and Dither_convert_yuv_to_rgb is likely to be bugged, well, it just crashes when input is not mod4

[mandarinka]

While testing smoothtweak, I encountered a color hue shift when using this (http://forum.doom9.org/showthread.php?p=1504502#post1504502) code to import 16bit video into x264cli via avs2yuv. I have no idea where this minor difference comes from (it could be due to difference in x264's conversion between 16>10 and 8>10; it could be due to x264 expecting slightly different format of the 16bit raw data?). In any case, I thought I better post this in case it is some real/fixable issue.

I'm not sure if the tint appears in the 8bit encodes too, but it is perceivable to me when I compare the 10bit encodes (2 screens from each - most obvious on the hair in #1 and the door curtain in #2):
10bit encode using 16bit input via avs2yuv: #1 (http://img137.imageshack.us/img137/9174/000000011016bitavs2yuv.png) #2 (http://img715.imageshack.us/img715/9970/000000021016bitavs2yuv.png)
10bit encode using avs2yuv normaly (8bit yv12 all along): #1 (http://img89.imageshack.us/img89/9108/00000001108bitavs2yuv.png) #2 (http://img830.imageshack.us/img830/9984/00000002108bitavs2yuv.png)


---------------------------------
What I did & 8bit encode screens:

encoding from 16bit avs2yuv input:

AVS:
smoothtweak(contrast=0.84)
Dither_convert_8_to_16()

Interleave (Dither_get_lsb (), Dither_get_msb ())
TurnRight ()
AssumeFieldBased ().AssumeTFF ()
Weave ()
TurnLeft ()
CLI:
08bit: avs2yuv -raw XXXX-16bit.avs -o - | x264-1995.exe - --demuxer raw --input-depth 16 --input-res 704x480 --fps 23.976 --preset ultrafast --qp 0 -o XXXX.mkv
10bit: avs2yuv -raw XXXX-16bit.avs -o - | x264-1995-10.exe - --demuxer raw --input-depth 16 --input-res 704x480 --fps 23.976 --preset ultrafast --qp 0 -o XXXX_10.mkv

Results:
10bit encode #1 (http://img137.imageshack.us/img137/9174/000000011016bitavs2yuv.png) 10bit encode #2 (http://img715.imageshack.us/img715/9970/000000021016bitavs2yuv.png)
8bit encode #1 (http://img101.imageshack.us/img101/9924/00000001816bitavs2yuv.png) 8bit encode #2 (http://img864.imageshack.us/img864/9234/00000002816bitavs2yuv.png)


encoding from 8bit avs2yuv input:

AVS:
smoothtweak(contrast=0.84)
CLI:
08bit: avs2yuv -raw XXXX.avs -o - | x264-1995.exe - --demuxer raw --input-depth 8 --input-res 704x480 --fps 23.976 --preset ultrafast --qp 0 -o XXXX.mkv
10bit: avs2yuv -raw XXXX.avs -o - | x264-1995-10.exe - --demuxer raw --input-depth 8 --input-res 704x480 --fps 23.976 --preset ultrafast --qp 0 -o XXXX_10.mkv


Results:
10bit encode #1 (http://img89.imageshack.us/img89/9108/00000001108bitavs2yuv.png) 10bit encode #2 (http://img830.imageshack.us/img830/9984/00000002108bitavs2yuv.png)
8bit encode #1 (http://img29.imageshack.us/img29/1655/0000000188bitavs2yuv.png) 8bit encode #2 (http://img7.imageshack.us/img7/1516/0000000288bitavs2yuv.png)


The screenshots were taken with mplayer2 build by lachs0r (http://mplayer2.srsfckn.biz/):
mplayer2-high-bit-depth-20110609.exe -vo png XXXX.mkv

P.S.
Here are (for reference) the same screens using smoothtweak16 (no ditherpost) and 16bit avs2yuv output as in case 1: #1 (http://img828.imageshack.us/img828/5058/000000011016in.png) #2 (http://img821.imageshack.us/img821/1589/000000021016in.png)

[Yellow_]

Hi, I'm trying a simple script, import 8bit h264AVC, upto 16bit and out as 10bit lossless h264 using avs2yuv.

But can't get past the error, dfttest: unable to load libfftw3f-3.dll

I've downloaded it from the fftw site and using the 3.2.2.pl1 32bit version as all other stuff is 32bit, although I'm using Win7 64bit.

Read dither.html and have mvtools2, masktools alpha48, removegrain beta etc but still can't get it to work. :-(

Any ideas? Apart from usual stuff I have these lines for above.

dfttest (lsb=true)
Dither_convey_yuv4xxp16_on_yvxx()

[Yellow_]

Ok sorted, had to put libfftw3-3.dll etc in Windows/SysWOW64 :devil:

[Dogway]

A few notes on testing with my wrapped resizer.
Dither_convert_rgb_to_yuv: Fails on big inputs (greater than 1500px or so for any dimension)
Dither_convert_yuv_to_rgb: Fails on big inputs. Fails on not mod4
Linear also occasionaly makes some strong black outline/ringing, almost the same as ResampleHQ, I wouldn't had taken it as a bug if Archimedes didnt state (http://forum.doom9.org/showthread.php?p=1492573#post1492573) otherwise.

[cretindesalpes]

Dither_convert_rgb_to_yuv: Fails on big inputs (greater than 1500px or so for any dimension)
Dither_convert_yuv_to_rgb: Fails on big inputs. Fails on not mod4
Yes I know. Actually it comes from the resizer/convolver wanting to allocate huge temporary data buffers. I'll rework the buffering system to something more efficient for a future update. In the meantime, a working solution would be splitting the picture in tiles or bands, interleaving, converting, deinterleaving and stacking them back to reduce the size pressure on the convolver.

Linear also occasionaly makes some strong black outline/ringing
This is normal. Because linear dark levels have their range reduced relative to gamma corrected levels, ringing caused by bright pixels on dark ones are greatly emphasized and produce super-black artifacts (and very bright secondary ringing). To avoid them, you should use a kernel with very low or no negative lobes at all. This is a drawback of linear light operations.

[Yellow_]

I'm getting very odd discoloration in my first attempt at going 8bit h264AVC to 10bit h264.

http://blendervse.files.wordpress.com/2011/06/8-10-10bit_h264.jpg

I'm using this:

FFmpegSource2("clip.MOV")
Dither_convert_8_to_16 ()
Dither_convey_yuv4xxp16_on_yvxx()

Then avs2yuv:

avs2yuv -raw dither16test.avs -o - | x264-64-10bit --demuxer raw --input-depth 16 --input-res 1920x1088 --fps 25 --fullrange on --output dither16testout.mp4

The source looks like this:

http://blendervse.files.wordpress.com/2011/06/orig8bitmov.jpg

I'm using ffmpeg to decompress the 10bit h264 and with other 10bit sources there is no discolouration so don't think this is an issue with reading the file correctly.

Also if I add --preset ultrafast --qp 0 for lossless h264 encoding (10bit) all I get is a solid mid grey output. Tried both 32 & 64bit current (1995) 10bit x264 builds from http://x264.nl/

[cretindesalpes]

Correct 10-bit H264 decoding is quite fresh in libav/FFmpeg. Get a recent FFmpeg build (http://ffmpeg.zeranoe.com/builds/) and your color problems will be gone.

[Yellow_]

Great, ffplay shows correct output from latest git ffmpeg. Many thanks indeed. :-)

[Yellow_]

Is it possible to use Dither_convert_yuv_to_rgb and then avs2yuv to pipe to 16bit image sequences to a graphics library like Imagemagick (Q16) or Openexr?

[cretindesalpes]

Using output="rgb48yv12", it should work with any program capable of reading raw RGB48 streams. I haven't tried with Imagemagick, but it looks like it can do it.

[Yellow_]

Using output="rgb48yv12", it should work with any program capable of reading raw RGB48 streams. I haven't tried with Imagemagick, but it looks like it can do it.

Ok, I'll pursue that.

**EDIT** removed queries about x264 colormatrix for HD BT601. Plenty of info here if I'd search first. :-)

[Yellow_]

Using output="rgb48yv12", it should work with any program capable of reading raw RGB48 streams. I haven't tried with Imagemagick, but it looks like it can do it.

I'm not familiar with IM so want to be sure I'm feeding it the 'correct' format out of avisynth so, does this script look correct:

LoadPlugins...

FFmpegSource2("MVI_8068.MOV")
Dither_convert_8_to_16 ()
Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", mode=6, output="rgb48yv12", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)

Dither_convey_rgb48_on_yv12(r,g,b)

My source files are HD Canon DSLR h264AVC, which is BT601 color matrix and full range luma.

[cretindesalpes]

Mode=6 will be ignored because you're not dithering to 8 bits, and given your script, you would have to use output="rgb48y" to get the three RGB components on interleaved luma planes.

But you can do it in a simpler way:
FFmpegSource2 ("MVI_8068.MOV")
Dither_convert_yuv_to_rgb (matrix="601", tv_range=false, cplace="MPEG2",
\ chromak="bicubic", output="rgb48yv12")

[Yellow_]

Mode=6 will be ignored because you're not dithering to 8 bits, and given your script, you would have to use output="rgb48y" to get the three RGB components on interleaved luma planes.

But you can do it in a simpler way:
FFmpegSource2 ("MVI_8068.MOV")
Dither_convert_yuv_to_rgb (matrix="601", tv_range=false, cplace="MPEG2",
\ chromak="bicubic", output="rgb48yv12")

Thanks for the clarification, so I need nothing more than the above, no Dither_convey_rgb48_on_yv12() needed?

I'm making progress with Imagemagick for writing image sequences, recieved help from the IM forum, all sounds encouraging. :-)

**EDIT**

Success, using the above with avs2yuv and IM (Q16) I can export 16bit image sequences to .tif, .exr and .dpx. :-)

Many thanks

[Yellow_]

Is it possible to visualise and export just the additional information generated / extracted from the video source using dither over and above what is normally seen at 8bit playback?

[cretindesalpes]

Success, using the above with avs2yuv and IM (Q16) I can export 16bit image sequences to .tif, .exr and .dpx. :-)
Excellent! Would you like to publish the command line and other specific instructions so I could add them to the documentation?


Is it possible to visualise and export just the additional information generated / extracted from the video source using dither over and above what is normally seen at 8bit playback?
Call export_rgb48() instead of visu() if you want to export RGB48 data.
FFmpegSource2 ("MVI_8068.MOV")
p = Dither_convert_yuv_to_rgb (matrix="601", tv_range=false, cplace="MPEG2",
\ chromak="bicubic", output="rgb48y")
q = ConvertToRGB (matrix="PC.601")
q = Interleave (q.ShowRed ("YV12"), q.ShowGreen ("YV12"), q.ShowBlue ("YV12"))

dm = mt_makediff (p.Dither_get_msb (), q)
StackVertical (dm, p.Dither_get_lsb ())

# Amplification
Dither_lut16 (expr="x 32768 - 64 * 32768 +", y=3, u=1, v=1)

visu ()

Function visu (clip src)
{
src.DitherPost (mode=6, y=3, u=1, v=1)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
MergeRGB (r, g, b)
}

Function export_rgb48 (clip src)
{
src
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convey_rgb48_on_yv12 (r, g, b)
}

[Yellow_]

Excellent! Would you like to publish the command line and other specific instructions so I could add them to the documentation?

This is what I'm using for h264 from my Canon DSLR which is 1920x1088. Thanks to help from the IM community at www.imagemagick.org/discourse-server.

avs2yuv.exe -raw dithertest.avs -o - | convert.exe -depth 16 -size 1920x1088 rgb:- dithertest_%d.tif

Cheers for the exportrgb48() & visu() info. :-)

A sample 16bit video frame: (12.5mb) http://www.yellowspace.webspace.virginmedia.com/test_0.tif

And the export from visualise for same frame: (12.5mb) http://www.yellowspace.webspace.virginmedia.com/visualise_0.tif

Should it really look like that? :-)

[cretindesalpes]

Thanks for the command line.

Should it really look like that? :-)
Yeah, unless I misunderstood your request, it's the amplified difference between 16 and 8-bit conversions. You can comment the x64 amplification line in the script, but it will look grey everywhere.

[Yellow_]

Thanks for the command line.


Yeah, unless I misunderstood your request, it's the amplified difference between 16 and 8-bit conversions. You can comment the x64 amplification line in the script, but it will look grey everywhere.

No that's great, thank you for the functions. I don't quite know what I expected to see. :-)

[mandarinka]

I noticed that there's the possibility to do colormatrix conversion in 16bits now, thanks a lot!

Which chroma kernel would you suggest for converting colormatrix (709->601) using Dither_convert_yuv_to_rgb + Dither_convert_rgb_to_yuv?

My script:
Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="601", lsb=true)


At first I was thinking about using point scaling because it is kind of reversible, but then I realised that's probably silly because of the rgb step :(

BTW, is this route somewhat compromised by the double conversion, or is that not an issue thanks to the 16bit processing, meaning that this workflow is close to optimal quality?
(Please forgive me for my newby questions, I don't know much about how exactly the colormatrices influence the conversion to rgb, or which yuv channels are influenced by colormatrix conversion, etc...)

[cretindesalpes]

Which chroma kernel would you suggest for converting colormatrix (709->601) using Dither_convert_yuv_to_rgb + Dither_convert_rgb_to_yuv?
You should use something that does not ring too much to avoid false colors. A generic "bicubic" (Mitchell-Netravali) or "spline16" should be fine. You can even try "bilinear", but it may be a bit blurry.

BTW, is this route somewhat compromised by the double conversion, or is that not an issue thanks to the 16bit processing, meaning that this workflow is close to optimal quality?
The rounding errors shouldn't be a problem here thanks to the high bitdepth, however spatial shape and bandwidth losses are the main concerns here: the chroma is resampled twice, for up/downscaling to 4:4:4 and sub-pixel shifting. A direct route could use only one resampling per plane, which would be better. I'll probably include a generic 16-bit colormatrix in a future Dither release. Anyway we already have something of high quality right now with the YUV->RGB->YUV conversions.

[mandarinka]

I see, thanks for the info.
BTW, I think the Dither suite would deserve its own thread, it's not even mentioned in the title...
I really like how this made high bitdepth processing possible:thanks:

[Yellow_]

In searching to find information to read on the methods used and similar methods of the dither plugin, stacked msb/lsb etc I came across this which I found interesting and thought I'd post a link to:

http://www.banterle.com/francesco/publications/download/EGstar2009.pdf

Specifically 3.0 LDR to HDR Expansion on page 4.

Comments?

[SilaSurfer]

The most accurate dithering algorithm are the error diffusion ones (6 to 8). Mode=6 is a classic serpentine Floyd-Steinberg and should give good results. It's a good idea to add a slight bit of noise before (ampn=0.5) to avoid artifacts related to this kind of algorithm. Anyway, I don't think gradfun2db dither can really be considered as good! But it does its job well and very quickly.

Hi cretindesalpes. I'm interested in those three dithering algorithms especially in:

mode 7: Stucki error diffusion + noise. Looks sharp and preserve light edges and details well.

Do you recommend using some noise generation through ampn in this case too and what about apmo setting? I'm intersested very much in this. Thanks

Edit:

Well after doing some testing

ampo - is paramater for incresing amplitude of that fancy ;) ordered dithering from your function

ampn- paramater for using dithering + noise, but raising it increases the amount of noise aka similar to gradfun2db.

Anyway I like ampn setting set to 0, ampo setting from 2~3. It plays a lot better with compressibility compared using noise like from Gradfundb. :rolleyes:

[TheProfileth]

Hmmm I don't know why but every time I use gradfun3 (since version 1.5) all it does is output the luma (like I get the generic green from removing chroma) if I leave the mask on. If I set the mask to 0 then this does not happen, I am sort of thinking it has something to do with the introduction of the dither_removegrain but I don't really know. Either way could someone propose a solution to this problem.

[mandarinka]

That seems to happen if you have masktools conflicts.

Assuming you use plugin autoload directory: get rid of masktools.dll (that is masktools v1... or was it mt_masktools.dll?), and make sure you only have installed that Masktools2 a48 version that cretindesalpes links. If you run on avisynth 2.5.x, use the mt_masktools-25.dll file, if you are on avisynth 2.6, use mt_masktools-26.dll. Having both or the wrong compile (xxx-26/xxx-25) seems to give the green screen.

[cretindesalpes]

ampn- paramater for using dithering + noise, but raising it increases the amount of noise aka similar to gradfun2db.
Actually we have two different uses here:

If you dither for direct display or further 8-bit processing, you need the maximum visual quality. Error diffusion algorithms perform much better than simple ordered dithering. Adding noise before can be interesting to add a temporal component to the dithering, and to homogenize flat and gradient parts.
If you dither for efficient video compression, prefer the ordered dithering (Bayer matrix) without noise.
Note that ampo and ampn parameters have an effect on all the dithering modes, ordered and error diffusion. Also, the noise is always added before dithering, not after (like in GradFun2DBmod). This allows it to be part of the noise shaping.

@TheProfileth:

As mandarinka said, it's likely a problem of Masktools version.

[SilaSurfer]

Thanks cretindesalpes for your explenation.

[pereant]

Thanks so much for the dither package - this works really really great and it appears to be very benefiting in terms of compressibility. as far as i understand it, it prevents from strong visible blocking artefacts. in other words they are still there, but not significant. is this right so far?

[cretindesalpes]

Dither 1.9.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released:
Improved memory usage and CPU load for the functions based on the 16-bit resizer.
Simplified a bit the GradFun3() detail masking.
Fixed a few glitches.
Next, I'm planning to add native multithread support to the most CPU-intensive functions. Currently, the Dither functions support all the MT modes, but these aren't available in recent 2.6 Avisynth versions.

as far as i understand it, it prevents from strong visible blocking artefacts.
The target of gradient smoothing is not strong blocking. Strong blocking artifacts are addressed by deblocking and deringing filters. Actually, banding artifacts are not really blocking; the main problem appears in the final colorspace. Of course, too high quantization during compression will cause color banding, the light dithering noise being filtering out. However dithering noise is hard to compress, and if you spend all your bits in the dither, you'll end up with a poorly encoded picture. By changing the dither from noise to repetitive patterns (ordered dithering), you make the dither more predictable, thus much more compressible, at a minor visual cost.

[Dogway]

Thank you! just before my encode! I wanted to request 2 new options for dither if it's possible.
One is implement nnedi3 for resize. I think I can do some masking add difference et al but do you think is possible to do it natively?
The other is when encoding anamorphic, to have also anamorphic dither.

edit: I just noticed that when using 4 taps Dither_resize16 crashes, so spline64, lanczos 4 taps, etc.

[Yellow_]

Many Thanks.

[pereant]

Lovin' it :)

http://imgur.com/a/Oc7Pu

But i guess i won't use Dither seriously until i can effort a new pc. It would take me a week to run this script. Core2Duo T7200 atm. :(

[cretindesalpes]

But i guess i won't use Dither seriously until i can effort a new pc. It would take me a week to run this script. Core2Duo T7200 atm. :(
If you just did Dither2Pre ().DitherPost () as quoted in your screenshots, no wonder it's slow. IIRC the default Dither2Pre() creates something like 7 instances of dfttest! You can achieve the same result just with dfttest (lsb=true).DitherPost (), but much faster.

An if your goal is denoising without creating colorbanding, you'll probably get better visual results by using only the previous line with a carefully tuned motion compensation.

One is implement nnedi3 for resize. I think I can do some masking add difference et al but do you think is possible to do it natively?
I haven't studied the nnedi3 code in detail, but I have the feeling that the masking solution will be simpler! One could just modify nnedi3 to make it output the mask of the pixels processed with the neural networks.

The other is when encoding anamorphic, to have also anamorphic dither.
Why would you want this one? For maximum efficiency, the dither must be related to the pixel grid, not to the aspect ratio. It's not intended to be actually seen.

I just noticed that when using 4 taps Dither_resize16 crashes, so spline64, lanczos 4 taps, etc.
Interesting. I'll check this issue.

Despite this my LinearResize function works pretty much ok, one just to be aware that it must not be optimal for rgb sources, since you will need mod2 input and at least avisynth 2.6.0 to avoid chroma subsampling. Let it be what it is, focused on video sources the same as Dither originally is.
You can bypass the YUV conversion by using ShowRed ("YV12"), ShowGreen and ShowBlue, resizing the luma channel of these 3 clips and merging the components back with MergeRGB().

[pereant]

If you just did Dither2Pre ().DitherPost () as quoted in your screenshots, no wonder it's slow. IIRC the default Dither2Pre() creates something like 7 instances of dfttest! You can achieve the same result just with dfttest (lsb=true).DitherPost (), but much faster.

An if your goal is denoising without creating colorbanding, you'll probably get better visual results by using only the previous line with a carefully tuned motion compensation.

Sorry for my ignorance. I just had a look what dfttest really does/is. It looks like as if it makes the use of the fft3dfilter in my script rather unnecessary i guess. Have to play around w/ it - Thanks!

[cretindesalpes]

Dither 1.9.2 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed crashes and bugs in Dither_resize16().

[Yellow_]

Hi

I was having some odd resizing problems with 1.9.1, the image was getting split over two frames, noise in one, image in other, some green frames and was going to post my queries but you provided 1.9.2 before I got time to comment. :-)

But still having problems with 1.9.2.

First, should this work uprezzing a 1920x1088:

Dither_convert_8_to_16 ()

Dither_resize16 (3480, 2176)

Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", output="rgb48yv12")

With the above writing out to imagemagick (and specifying the same image resolution) with avs2yuv I get pairs of images for each frame of video, so processing 6 frames of video gives me 12 images, the first image of each pair is 'normal', the second image of each pair is the grey and green noise.

On the CLI Dither tells me: script.avs: 6960x8704, 25fps, 6 frames

[cretindesalpes]

Just add the parameter lsb_in=true to Dither_convert_yuv_to_rgb(). Actually, the functions cannot know the kind of data (8 or 16 bits) they are fed with, so it's important to tell them when multiple cases are allowed.

[Yellow_]

Just add the parameter lsb_in=true to Dither_convert_yuv_to_rgb(). Actually, the functions cannot know the kind of data (8 or 16 bits) they are fed with, so it's important to tell them when multiple cases are allowed.

That's great thanks, also seem to notice quite a speed up. Excellent.

[TheElix]

Hey, cretindesalpes, your script is magic. http://screenshotcomparison.com/comparison/64373
I'm total noob in AviSynth, but I was suggested to use these lines:
SetMTMode(3)
ffdShow_Source()
SetMTMode(2)
Dither_convert_8_to_16 ()
SmoothGrad(thr=1.0, radius=8)
DitherPost ()
It works wonders on such sources as above. But I was wondering if there's any room for improvement here.

[mp3dom]

It works wonders on such sources as above. But I was wondering if there's any room for improvement here.

There's something wrong, I think you should lower the values of smoothgrad or use Gradfun3 with mask because you have a lot of noise around the edges.

[cretindesalpes]

It works wonders on such sources as above. But I was wondering if there's any room for improvement here.

It's hard to give advices just on a single picture. The thr parameter is high and efficient for removing the dirt on the flat areas, but I fear it will affect details too much. And as mp3dom said, it makes halos around the edges (but masking is almost useless for this kind of artifacts). Using a bilateral filter for the main task will improve things a lot, albeit being slower.

FFImageSource ("Source_screenshot_7849102365.jpg")

Dither_convert_8_to_16 ()
o16 = last
r = 8
Dither_bilateral16 (radius=r, flat=0.75, subspl=8)
Dither_limit_dif16 (last, o16, thr=0.7)
SmoothGrad (thr=0.15, radius=r*2)
DitherPost ()

Or you can just use a denoiser with a high spatial range like dfttest. As a side effects, it will greatly reduce the blocking and smooth the edges, but will blur the picture a little bit.

FFImageSource ("Source_screenshot_7849102365.jpg")

o = last
r = 8
dfttest (sigma=r, tbsize=1, sbsize=r*2, sosize=r*3/2, lsb=true)
f = DitherPost (mode=-1)
d = ContraSharpening_dif (f, o)
DitherPost ()
mt_adddiff (last, d, y=3, u=2, v=2)

Function ContraSharpening_dif(clip denoised, clip original)
{
# contra-sharpening: sharpen the denoised clip, but don't add more to
# any pixel than what was removed previously.
# script function from Didee from the VERY GRAINY thread
# Damp down remaining spots of the denoised clip.
s = denoised.minblur(1,1)
# The difference achieved by the denoising.
allD = mt_makediff(original,denoised)
# The difference of a simple kernel blur.
ssD = mt_makediff(s,s.removegrain(11,-1))
# Limit the difference to the max of what the denoising removed locally.
ssDD = ssD.repair(allD,1)
# abs(diff) after limiting may not be bigger than before.
ssDD = SSDD.mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?")
# Apply the limited difference. (Sharpening is just inverse blurring.)
# denoised.mt_adddiff(ssDD,U=2,V=2)
return (ssDD)
}

[TheElix]

With the last script I have this error http://rghost.ru/14201971/image.png
As for the first one, it gives me a slideshow... Strange, even though CPU usage isn't rising above 65%.
As for the quality, here's a comparison between my old script and the new one: http://screenshotcomparison.com/comparison/65664
I have to add that haloing is barely noticable here on my PDP. You may say it's plasma panel's own dithering that prevents to see those 'details' or you may say my panel is good at masking those artifacts :) But yes, I do see haloing in my old variant now. But these new settings are either too much stress on my system or there's something in the them that's making video stutter.

[cretindesalpes]

With the last script I have this error http://rghost.ru/14201971/image.png
Install the modified dfttest version included in the Dither archive.

As for the first one, it gives me a slideshow...
Yeah, I said it's slower (bilateral filters are very slow). Definitely not for realtime.

[TheElix]

I failed to mention I was looking for a real-time version... Could you suggest a setting that's between the two versions that have been compared?

[cretindesalpes]

For realtime, just stick with your SmoothGrad script, dfttest or bilateral-based processing will be overkill. You can lower the thr parameter to something like 0.5-0.7 to reduce the halos, but the original banding artifacts will become more visible. You'll have to find the right compromise according to your taste.

[Rumbah]

I'm using MCTemporalDenoise(settings="very high", edgeclean=true) to denoise a clip of mine. It works great but introduces some banding.
If I use enhance=true it dithers the output with gradfun2dbmod if I understand it correctly (I guess as the last step). It looks good but encoding with x264 uses a lot of bandwidth.
So I thought of using dither for postprocessing with gradfun3. Can I directly use gradfun3 as a substitute for gradfun2db in gradfun2dbmod? Or can you convert the settings from one to the other to get a similar result?
I wasn't able to achieve a similar result by playing with the settings so I'd be grateful for some help.

[cretindesalpes]

You can replace GradFun2DBmod with GradFun3, not just GradFun2DB. The GradFun3 defaults are similar to GF2DBm excepted it doesn't generate additional grain and processes luma and chroma the same way. You'll find below a conversion table for the parameters of equivalent role:
GradFun2DBmod Conversion GradFun3
------------- ---------- --------
thr * 0.3 thr
radius = mask
range = thr_det

Anyway, you can still replace GradFun2DB with GradFun3 in GradFun2DBmod. Use the following settings:
LUM = RDY.GradFun3(thr=thr *0.3, mask=0).Crop(...)
CHR = RDY.GradFun3(thr=thrC*0.3, mask=0).Crop(...)
These lines appear three times in the GF2DBm code, with different Crop() values, make sure you don't forget one or mix the Crop().

[Rumbah]

Thanks for your help, I'll try to "mod" MCTemporalDenoise to use dither instead of gradfun2db.

I hope it gets me some compression gain while still looking good (I don't know if differing between chroma and luma has a visible impact).

[Rumbah]

I just tested it and it's much more compressible now although there's a little more banding.

Just to be sure that I got it right here are the changes (in bold):
\ : (GPU==false&&fixband==true ) ? i.FFT3Dfilter(sigma=sigma*0.8,sigma2=sigma*0.6,sigma3=sigma*0.4,sigma4=sigma*0.2,bw=bwbh,bh=bwbh,ow=owoh,oh=owoh,plane=chroma?4:0,bt=bt,ncpu=ncpu).gradfun2db(thr=1.0+sigma*0.05)
\ : i.FFT3Dgpu(sigma=sigma*0.8,sigma2=sigma*0.6,sigma3=sigma*0.4,sigma4=sigma*0.2,bw=bwbh,bh=bwbh,ow=owoh,oh=owoh,plane=chroma?4:0,bt=bt,precision=precision).gradfun2db(thr=1.0+sigma*0.05)

\ : (GPU==false&&fixband==true ) ? i.FFT3Dfilter(sigma=sigma*0.8,sigma2=sigma*0.6,sigma3=sigma*0.4,sigma4=sigma*0.2,bw=bwbh,bh=bwbh,ow=owoh,oh=owoh,plane=chroma?4:0,bt=bt,ncpu=ncpu).gradfun3(thr=(1.0+sigma*0.05)*0.3, mask=0)
\ : i.FFT3Dgpu(sigma=sigma*0.8,sigma2=sigma*0.6,sigma3=sigma*0.4,sigma4=sigma*0.2,bw=bwbh,bh=bwbh,ow=owoh,oh=owoh,plane=chroma?4:0,bt=bt,precision=precision).gradfun3(thr=(1.0+sigma*0.05)*0.3, mask=0)
\ : mt_makediff(smL,mt_makediff(smL,smP,U=chr31,V=chr31).gradfun2DB(thr=1.0+post*0.5),U=chr31,V=chr31)

\ : mt_makediff(smL,mt_makediff(smL,smP,U=chr31,V=chr31).gradfun3(thr=(1.0+post*0.5)*0.3, mask=0),U=chr31,V=chr31)
GFc = (enhance==true) ? TTc.GradFun2DBmod(thr=GFthr,thrC=chroma?GFthr:1.0,mode=0,str=AGstr,strC=0.0,temp=temp,adapt=bias,mask=false,show=false)

GFc = (enhance==true) ? TTc.GradFun3(thr=GFthr*0.3,mask=0)
I wasn't sure about the complex gradfun2dbmod call, perhaps I did something wrong.

Changing gradfun2dbmod to use gradfun3 didn't result in the desired effect of better compressibility as it seems that it adds noise in addition to gradfun2db.

[cretindesalpes]

Looks OK to me. If you feel you have more banding than before, increase GFthr, or the 0.3 multiplier in the modified parts so it matches your taste.

EDIT: only the last part (enhance==true) needs to be changed. You can leave the other GradFun2DB, or add mode=6 to these first GradFun3 calls.

[Rumbah]

Thanks again.
I only changed the enhanced==true line and it worked as intended.

I noticed some strong haloing so I started tinkering with the values again and noticed that using
GradFun3(thr=1.8,mask=0,smode=2,radius=16)crashed dither.
GradFun3(thr=1.8,mask=0,smode=2)works.
(I need some strong debanding and there aren't many details as it's an 16x upscale ;) )

EDIT: I also noticed that I mistook the halo artifacts for banding so with smode=2 the quality is much better and the high threshold isn't needed.

EDIT2: After some testing I found that using a radius>13 crashes the above example.

[Disabled]

Did you try other ordered dithering matrices like those generated by the void and cluster algorithm (very bottom (http://caca.zoy.org/wiki/libcaca/study/2))?
In my tests they showed a similar compressibility to other ordered dithering algorithms and looked superior to bayer dithering.

[cretindesalpes]

EDIT2: After some testing I found that using a radius>13 crashes the above example.
Oh right, thanks for the feedback.

Dither 1.9.3 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released:


Fixed crashes in Dither_bilateral16() occuring with subsampling using more than 32 reference pixels. This affected GradFun3(smode=2) with radii greater than 13 and default subsampling settings.
Fixed a mod4 check in Dither_convert_yuv_to_rgb().


Did you try other ordered dithering matrices like those generated by the void and cluster algorithm (very bottom (http://caca.zoy.org/wiki/libcaca/study/2))?
In my tests they showed a similar compressibility to other ordered dithering algorithms and looked superior to bayer dithering.
I was afraid of the compressibility so I implemented only the Bayer matrix, but if you say the V&C algorithm has a good compressibility too, I'll probably add it later.

[mswaino2]

Im liking the results from your script, I just dislike how GradFun3(smode=1) is so slow. I went from a 4 hour pass on a movie to a 12 hour pass. Is this supposed to be like that? because regular GradFun3() runs MUCH faster it just doesnt give quite as good of results as smode1.

[cretindesalpes]

I just dislike how GradFun3(smode=1) is so slow. I went from a 4 hour pass on a movie to a 12 hour pass. Is this supposed to be like that?
:eek: smode=1 is indeed slower than smode=0, but not that slower. Something must be wrong. Are you sure your figures are right? Anyway, if you do a 2-pass encoding, render first to a lossless file then encode it, you'll save one avisynth pass.

[mswaino2]

:eek: smode=1 is indeed slower than smode=0, but not that slower. Something must be wrong. Are you sure your figures are right? Anyway, if you do a 2-pass encoding, render first to a lossless file then encode it, you'll save one avisynth pass.

I dunno, maybe its just Im using it in the script wrong or something. Could it be because Im using a denoiser and sharpener along with it?

I been doing a 2 pass encode with this for a script


LoadPlugin("C:\Program Files\megui\tools\dgindex\DGDecode.dll")

Import("C:\Program Files\AviSynth 2.5\plugins\LimitedSharpenFaster.avsi")

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-25.dll")
Import("C:\Program Files\AviSynth 2.5\plugins\dither.avsi")
Import("C:\Program Files\AviSynth 2.5\plugins\GradFun2DBmod.avsi")
Import ("C:\Program Files\AviSynth 2.5\plugins\mt_xxpand_multi.avsi")
DGDecode_mpeg2source("C:\Users\Owner\Desktop\Case Closed - The Last Wizard Of The Century\VTS_06_1.d2v", info=3)
LoadPlugin("C:\Program Files\Megui\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
#deinterlace
crop( 0, 0, -2, 0)
LanczosResize(852,480) # Lanczos (Sharp)
LoadPlugin("C:\Program Files\megui\tools\avisynth_plugin\UnDot.dll")
Undot() # Minimal Noise
deen()
LimitedSharpenFaster(ss_x=2.0, ss_y=2.0, strength=150, overshoot=0, undershoot=0, soft=0, edgemode=0)
GradFun3(smode=1)
__film = last
__t0 = __film.trim(0, 142520)
__t0



the only thing I have changed is the script and its taking a very long time for some reason. I was getting 11-12 FPS now Im at 3 FPS after adding it.

[Dogway]

cretindesalpes: Is it possible to add lsb_in feature in mvtools2. It would be very useful for certain sources with blocking+noise, where I would run a spatial dfttest pass then mdegrain. I read that normally spatial denoisers should go before temporal.

[mandarinka]

You can use a prefilter for the motion vector search (which I assume is completely fine to do in 8bit), to partially achieve that.


Something like this I think (hopefuly I got it right).
prefilt = FFT3Dgpu(sigma=2,bw=32,bh=32,bt=3,ow=16,oh=16,plane=4)

superfilt = prefilt.MSuper(pel=2, sharp=1)
super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(superfilt, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(superfilt, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(superfilt, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(superfilt, isb = false, delta = 2, overlap=4)

MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=300,lsb=true)

[Dogway]

Hello mandarinka. Thanks for the input, I meant to denoise, not feed a better suited (prefiltered) clip for motion estimation. I know that lsb_in in mvtools is going to be quite annoying for cretindesalpes to make since he would probably need to adapt all the functions (manalyse,mmask,mcompensate,etc) for lsb_in in mdegrain. But I think it makes sense because spatial+temporal denoisers normally operate internally first spatial then temporally (dfttest,tnlmeans,etc)

By the way I found probably a bug in Dither_resize16 when processing a .png file. I tested and the error comes from the fh and fv parameters when resize dimensions are small:
This is my code, pitty I couldn't take advantage of high bitdepth rgb to yuv conversion : P
converttoyv12(matrix="PC.601")
MT("""Dither1Pre(flt="tnlmeans()",stacked=true)""",2,2,true)
dither_resize16(224,316,kernel="spline64",fh=1.1,fv=1.1)
Dither_convert_yuv_to_rgb(lsb_in=true,tv_range=false,output="rgb32",mode=6,ampn=0.5)

[cretindesalpes]

I know that lsb_in in mvtools is going to be quite annoying for cretindesalpes
Yes it would... but I could just leave the analysis part in 8 bits and focus on MSuper/MDegrain. Still, it's a lot of work, so I don't plan to do this in the near future.

By the way I found probably a bug in Dither_resize16 when processing a .png file.
I can't reproduce it here, everything looks OK. What is the bug, exactly? What are the dimensions of the original picture?

[Dogway]

Still, it's a lot of work, so I don't plan to do this in the near future.
No problem, I knew it was. You hard work on Dither is already impressive for a one man.
I can't reproduce it here, everything looks OK. What is the bug, exactly? What are the dimensions of the original picture?
Dimensions are 610x1114. I think its relevant because the other images work.

[cretindesalpes]

Dimensions are 610x1114. I think its relevant because the other images work.
OK I spotted the problem, a tile size calculation error again, resulting in a buffer overflow.

Fixed (I hope so) in Dither 1.9.4 (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

[TheProfosist]

:eek: smode=1 is indeed slower than smode=0, but not that slower. Something must be wrong. Are you sure your figures are right? Anyway, if you do a 2-pass encoding, render first to a lossless file then encode it, you'll save one avisynth pass.

im getting crazy slower speed with smode=1 as well

smode=0 1.87FPS
smode=1 0.12FPS
this is with 1080p

[cretindesalpes]

im getting crazy slower speed with smode=1 as well

Weird.

I ran a few speed tests with 1280x720 progressive input, in single-threading and multi-threading modes. I added a naked dfttest() because it's the core of GradFun3(smode=1) and I would suspect this is the one slowing down the whole thing. Also added GradFun2DBMod for reference.

#SetMTMode (5, 4)
FFVideoSource ("random 720p source")
#SetMTMode (2)
Trim (0, -1000)

# Select one of these lines
#NOP ()
#GradFun3 (smode=0, mask=0)
#GradFun3 (smode=0)
#GradFun3 (smode=1, mask=0)
#GradFun3 (smode=1)
#GradFun3 (smode=2, mask=0)
#GradFun3 (smode=2)
#dfttest (sigma=35, tbsize=1, sbsize=36, sosize=27, lsb=true)
#GradFun2DBmod ()

PointResize (32, 32)
# Encoded with: x264 --preset ultrafast --output NUL "speed.avs"

Results (Phenom II X4 965 3.4 GHz):

| Single-threaded | Multi-threaded x4
| fps CPU | fps CPU
-----------------------------+-------------------+------------------
NOP () | 453.93 33 % | 444.44 33 %
GradFun3 (smode=0, mask=0) | 30.72 27 % | 89.63 90 %
GradFun3 (smode=0) | 7.38 25 % | 26.74 99 %
GradFun3 (smode=1, mask=0) | 7.96 78 % | 9.53 97 %
GradFun3 (smode=1) | 4.38 55 % | 7.65 97 %
GradFun3 (smode=2, mask=0) | 10.03 26 % | 32.80 98 %
GradFun3 (smode=2) | 4.93 25 % | 17.25 99 %
dfttest (...) | 8.87 85 % | 10.11 98 %
GradFun2DBmod () | 5.19 25 % | 18.95 96 %

So, GradFun3(smode=1) is 3.5 times slower than GradFun3(smode=0) in 4x MT mode, and just 1.6 times slower in single-threaded mode.

I'll check later if I get very different figures for 1080p input.

[Dogway]

Another error: 1434x1476 dimensions for Dither_resize16 resizing to half. Hope you get this kind of problems sorted, sorry I can't help more.

[cretindesalpes]

Another error: 1434x1476 dimensions for Dither_resize16 resizing to half. Hope you get this kind of problems sorted, sorry I can't help more.
I can't reproduce it. Is 1434x1476 the input size? Halving 1434 gives 717 which is odd. Could you please post the script?

[Yellow_]

I'm using Dither to go from 8bit h264AVC to 16bit OpenEXR image sequences but really those should be linear not gamma encoded, the compositing application I'm importing them into assumes linear. So does the Dither_y_gamma_to_linear function serve this purpose?

Is it only necessary to linearize the luma, more accurate and technically correct to do it that way before the conversion to RGB, rather than a typical 0.45 reverse gamma on all channels in RGB data after conversion?

Also as the source was encoded with a BT709 transfer curve not sRGB 2.2, to undo that ie: linearize YCbCr, I should be assuming something like the reciprocal of 2.35? If I understand correctly this helps prevent compressing shadow detail that can occur applying 0.45 to BT709 source?

[Dogway]

This is the code I'm testing with:
Interleave(showred("yv12"), showgreen("yv12"), showblue("yv12")).Dither_convert_8_to_16
Dither_resize16(1434/2,1476/2,kernel="spline36",y=3,u=1,v=1)

The error is very ugly, I thought Dither_resize did some kind of rounding. Anyway in the LinearResize function I changed target dimensions to (w%2+w,h%2+h). Nice to know it wasn't THAT error again.

As a last question, Im forcing to proces Dither_y_gamma_to_linear in PC range to have more data to work with. Do you think this is more detrimental than beneficial?

[cretindesalpes]

Dither 1.9.5 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released:

Bug fixed in Dither_resize16(), displaying green bars when SSE2 optimisations are disabled. Better check of stack16 clip dimensions.

As a last question, Im forcing to proces Dither_y_gamma_to_linear in PC range to have more data to work with. Do you think this is more detrimental than beneficial?
TV range is all about keeping a headroom for the ringing causing by some filters (resizers, sharpeners…), so with PC range this headroom is lost. But this is not a big difference, and the bottom headroom is cleared by the gamma/linear conversion anyway.

the compositing application I'm importing them into assumes linear. So does the Dither_y_gamma_to_linear function serve this purpose?
Yes.

Is it only necessary to linearize the luma, more accurate and technically correct to do it that way before the conversion to RGB, rather than a typical 0.45 reverse gamma on all channels in RGB data after conversion?
GammaYUV -> linearYUV -> linearRGB will give you wrong colors. Inverse colorspace conversions should be done in the same gamma/linearity as the forward conversion. For gamma YUV to linear RGB conversion you can use:

Dither_convert_yuv_to_rgb (output="rgb48y")
Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false)
Dither_convey_rgb48_on_yv12 (
\ SelectEvery (3, 0),
\ SelectEvery (3, 1),
\ SelectEvery (3, 2) )

Also as the source was encoded with a BT709 transfer curve not sRGB 2.2, to undo that ie: linearize YCbCr, I should be assuming something like the reciprocal of 2.35? If I understand correctly this helps prevent compressing shadow detail that can occur applying 0.45 to BT709 source?
You're right, it appears that BT709 doesn't use the same transfer curve as sRGB, the slope at 0 in sRGB is much steeper than in BT709. If you want to linearize the R'G'B' components converted from the BT709 Y'CbCr, apply on each one : R = R'/4.5 if R' < 0.081 or R = ((R' + 0.099) / 1.099) ^ (1/0.45) if R' ≥ 0.081. The formula are the same as in sRGB, but with different values. I will add the BT709 mode to Dither conversion functions in a future release.

[SilaSurfer]

Hey cretindesalpes. Just wanted to stop by and say outstanding work. Finished off an encode using your ordered dithering in my filter chain. Awasome results. Thanks again for all of your hard work.

[Dogway]

hey I just needed to use addborders inside stacked 16b. Thought on sharing the function (primitive but useful). Thanks for the new version too!Function Dither_addborders16 (clip src, int "left", int "top",
\ int "right", int "bottom")
{
left = Default (left, 0)
top = Default (top, 0)
right = Default (right, 0)
bottom = Default (bottom, 0)

src
msb = crop(0,0,width,height/2).addborders (left, top, right, bottom)
lsb = crop(0,height/2,width,height/2).addborders (left, top, right, bottom)

StackVertical (msb, lsb)}

btw Im also looking forward the v&c implementation

[mandarinka]

Hey cretindesalpes. Just wanted to stop by and say outstanding work. Finished off an encode using your ordered dithering in my filter chain. Awasome results. Thanks again for all of your hard work.

Indeed. Dither brought quite some fresh momentum into the abilities of avisynth, single-handedly. Thanks from me too.

[Alek93j]

Function zzz_denoise (clip src, float "sigma", int "thr", bool "mask", int "sad")
{
sigma = Default (sigma, 16)
thr = Default (thr, 5)
mask = Default (mask, False)
sad = Default (sad, 200)

w = src.Width ()
h = src.Height ()

# Motion analysis
super = MSuper (src)
super_a = MSuper (src.TTempSmooth ().RemoveGrain (12))

fwd_vect_3 = super_a.MAnalyse (isb=false, delta=3, overlap=4)
fwd_vect_2 = super_a.MAnalyse (isb=false, delta=2, overlap=4)
fwd_vect_1 = super_a.MAnalyse (isb=false, delta=1, overlap=4)
bck_vect_1 = super_a.MAnalyse (isb=true, delta=1, overlap=4)
bck_vect_2 = super_a.MAnalyse (isb=true, delta=2, overlap=4)
bck_vect_3 = super_a.MAnalyse (isb=true, delta=3, overlap=4)

fwd_comp_2 = src.MCompensate (super, fwd_vect_2, thSAD=sad)
fwd_comp_1 = src.MCompensate (super, fwd_vect_1, thSAD=sad)
bck_comp_1 = src.MCompensate (super, bck_vect_1, thSAD=sad)
bck_comp_2 = src.MCompensate (super, bck_vect_2, thSAD=sad)

# Spatio-temporal denoising using modified dfttest
c_dft = Interleave (fwd_comp_2, fwd_comp_1, src, bck_comp_1, bck_comp_2)
c_dft = c_dft.dfttest (sigma=sigma, lsb=true) # Double height
c_dft = c_dft.SelectEvery (5, 2)

# Temporal-only denoising using modified MDegrain
c_deg = src.MDegrain3 (super, bck_vect_1, fwd_vect_1, bck_vect_2, fwd_vect_2, bck_vect_3, fwd_vect_3, thSAD=sad, lsb=true) # Double height

# Spatio-temporal denoising smoothes too much the details,
# therefore we use pure temporal denoising on edges or detailed areas.
edge_src = c_deg.Crop (0, 0, w, h)
edge_mask = edge_src.mt_edge (mode="prewitt", thY1=thr, thY2=thr)
edge_mask = edge_mask.mt_expand ()
edge_mask = StackVertical (edge_mask, edge_mask) # Double height
c_hyb = mt_merge (c_dft, c_deg, edge_mask, luma=true, y=3, u=3, v=3)

return (mask ? edge_mask.GreyScale () : c_hyb)
}

Maybe i'm gonna say something stupid, but this line:
super = MSuper (src)
It's used only by MCompensate and MDegrain3, which don't need to use levels=0 (if I understood right), so, it would be:
super = MSuper (src,levels=1)
to get the same result with a little increase of the velocity?

[mswaino2]

im getting crazy slower speed with smode=1 as well

smode=0 1.87FPS
smode=1 0.12FPS
this is with 1080p


Seems I wasnt the only one who had this problem. I guess Ill just have to wait it out and hope it gets faster/fixed in time.

[cretindesalpes]

so, it would be:
super = MSuper (src,levels=1)
to get the same result with a little increase of the velocity?
Yes, you're right. No need for all levels in MDegrain and MCompensate.

Seems I wasnt the only one who had this problem. I guess Ill just have to wait it out and hope it gets faster/fixed in time.
Have you run the test in post #211 (http://forum.doom9.org/showthread.php?p=1514203#post1514203)? How does it compare to my results? I don't know where your problem comes from, so I need a bit more information.

hey I just needed to use addborders inside stacked 16b. Thought on sharing the function (primitive but useful).
[...]
btw Im also looking forward the v&c implementation
Thank you. I added it to the next Dither release, but with the LSB part being a true 0.
What do you mean "v&c"?


SilaSurfer & mandarinka: thank you for your kind words.

[Dogway]

What do you mean "v&c"?


Void & Cluster algo (http://forum.doom9.org/showthread.php?p=1513101#post1513101). I also recall reading something related to it a few months ago, but I can't talk since I have no idea on dither algorithms. Did you test it?

[cretindesalpes]

OK. No, I haven't tested it yet.

[Dogway]

I had some time to look at making work nnedi with dither. I also wanted to contrasharp after dfttest and before nnedi so I mixed both workarounds. Can you confirm it is correct?
I also added a mask for nnedi because it was introducing some banding/artifacts at flat/gradient areas...

This is ultimately going to be processed by smdegrain(lsb=true) so I dither it down to mode=6. Is this ok, or should I use mode=-1?
raw=last

predf=dfttest(sigma=10,tbsize=1,lsb=true)
post=predf.ditherpost(mode=-1)
contr=Contrasharpening(post,raw).LSFmod(defaults="slow",strength=30,edgemode=1,soothe=true,ss_x=1.0,ss_y=1.0)

spl=contr.Spline36Resize(640,360)
nn=contr.nnedi3_rpow2(rfactor=2,cshift="spline64resize",nns=4,qual=2,pscrn=4)
hop=mt_edge (mode="prewitt", thY1=10, thY2=30).mt_expand.BilinearResize(640,360)
nn=mt_merge(spl,nn,hop, luma=true, y=3)

mask=DitherBuildMask(contr, post)
sharped=Dither_merge16_8(predf,contr.Dither_convert_8_to_16,mask).Dither_resize16(640,360,kernel="spline36")

mask2=DitherBuildMask(nn,spl)
Dither_merge16_8(sharped,nn.Dither_convert_8_to_16,mask2)

DitherPost(mode=6,prot=true)

[SilaSurfer]

Error diffusion modes (6,7,8) are awasome when using high bitrates, otherwise mode (0) 8-bit ordered dither + noise (Bayer matrix) accept that I don't use ampn setting but rather ampo (~0.5) when using higher compression.

[SSH4]

v&c looks great but "This technique is covered by Ulichney’s U.S. patent 5535020 and the specific implementation we showed is partly covered by Epson’s U.S. patent 6088512."
But I hope you don't care about this. :)

[atra dies]

I'm testing some images from here:
http://www.4p8.com/eric.brasseur/gamma.html


ImageReader("C:\gamma_colors.jpg")

Dither_convert_rgb_to_yuv(matrix="601",tv_range=false,lsb=false)
Dither_convert_8_to_16()
Dither_y_gamma_to_linear(false,false)
Dither_resize16(128,192)
Dither_y_linear_to_gamma(false,false)
DitherPost(mode=-1)

What am I doing wrong? The page claims they come out wrong but the one at the bottom says the scaling software rules. Also the NASA big lights from space image comes out with the lights less yellow than the original.

I modified the linear to gamma function to process chroma but that didn't change anything. Is chroma on the power scale too? Is this plugin built just for yv12 or should it work for rgb the same way? It could be something I don't know about because I haven't read much about it.

[cretindesalpes]

Can you confirm it is correct?
Yes at the first glance it looks correct, but this becomes a bit complicated to follow... Anyway, I would have extracted the "hop" edge mask from the nnedi result rather than from the original picture, although I don't know if it makes a significant difference.

This is ultimately going to be processed by smdegrain(lsb=true) so I dither it down to mode=6. Is this ok, or should I use mode=-1?
Definitely not mode=-1, you'd lose all the benefits of this complex script. Mode 6 looks like the most appropriate dither if you're going to filter the clip again, because you need the maximum quality at this intermediate stage and you don't want the motion estimation to be tricked by the regular patterns of an ordered dithering.

What am I doing wrong? The page claims they come out wrong but the one at the bottom says the scaling software rules. Also the NASA big lights from space image comes out with the lights less yellow than the original.
This is the same problem as with the colorspace conversions mentioned a few posts above. Ideally, for a color sRGB picture, linear/gamma scaling should be done on the RGB components, not on the converted Y signal only. This modified piece of code should work as expected:
ImageReader ("gamma_colors.jpg")

Interleave (ShowRed ("YV12"), ShowGreen ("YV12"), ShowBlue ("YV12"))
Dither_convert_8_to_16()
Dither_y_gamma_to_linear(false,false) # u=1, v=1
Dither_resize16(128, 192, u=1, v=1)
Dither_y_linear_to_gamma(false,false) # u=1, v=1
DitherPost(mode=-1, u=1, v=1)
MergeRGB (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2))

I modified the linear to gamma function to process chroma but that didn't change anything. Is chroma on the power scale too?
Chroma depends on something on a power scale. The correct conversion is related to the initial RGB->YUV matrix. Keeping the chroma untouched is technically wrong, but the formula to fix it is complicated!

[wOxxOm]

How about an option to have a static ampn noise? e.g. a negative value = static, or better an additional parameter like ampn_temp_avg=0..100 (similar to GrainFactory3) with -1 for a completely static noise.

[pandy]

is there any way to reduce bit depth with dither to less than 8 bits? (like 5 or 4 for embedded applications - quite frequently only 4 or 5 bits depth per component is supported)?

[cretindesalpes]

How about an option to have a static ampn noise? e.g. a negative value = static, or better an additional parameter like ampn_temp_avg=0..100 (similar to GrainFactory3) with -1 for a completely static noise.
You'll have static noise in the next Dither release. For a more fancy noise, I think it's better to provide a noise clip produced by an external source. I might implement that too.

is there any way to reduce bit depth with dither to less than 8 bits?
Yes. Reduce the signal before dithering and amplify it afterwards.
bits = 4 # Valid range: 0 - 8
mul = String (Pow (2, 8 - bits))

Dither_convert_8_to_16()
Dither_lut16 (expr="x "+mul+" /")
DitherPost ()
mt_lut ("x "+mul+" *", y=3, u=3, v=3)

[SilaSurfer]

Hello guys

cretindesalpes what would be better in your opinion. I'm doing some heavyweigth filtering Mvtools style on 1080p Bluray source that is going to be encoded to x264. But my processor wont handle that kind of stress (time to get I7 I guess :p). I was thinking of doing a lossless pass and then going to x264 but should I use your dithering before the lossless pass or later on lossless source before feeding it to x264? Thanks in advance

[cretindesalpes]

should I use your dithering before the lossless pass or later on lossless source before feeding it to x264? Thanks in advance

I think it doesn't matter much. DitherPost() is quite light on resource, compared with 1080p MC filtering. But if the output of your first lossless pass has a 16-bit depth, the generated file will be huge (compression of the LSB part is inefficient).

[SilaSurfer]

But if the output of your first lossless pass has a 16-bit depth, the generated file will be huge (compression of the LSB part is inefficient).

No I was thinking of staying in 8-bits and then on lossless as source doing:

Dither_convert_8_to_16 ()
Smoothgrad if needed
DitherPost()

[atra dies]

Seems I wasnt the only one who had this problem. I guess Ill just have to wait it out and hope it gets faster/fixed in time.

I am getting super slow speeds with smode 1 and 2 (I think 2 is supposed to be the highest quality, so I would try that one instead) but then I tried lowering the radius from 13 to 1, got an error saying valid values are 2-64, wouldn't accept anything below 6, then it works at twice the speed so I would try lowering the radius and seeing if you get acceptable results.

I am using it to smooth horrible compression artifacts, bands of blocks for flat animation colors and nothing removes it so completely as gradfun3 .66 strength and I tried a lot of filters and pp settings. For non-animation I have found that smoothd works wonders at removing lots of noise but blurs the picture, set adaptive mode to 1. Seems promising if someone used that principle for another filter let me know. (Edit: sorry, should have tried mdegrain2, it is great but slow and one of the mod16 filters.)

Now I have to figure out where or if levels or regular denoisers can be done on linear gamma for most accurate results?

[Dogway]

Definitely not mode=-1, you'd lose all the benefits of this complex script. Mode 6 looks like the most appropriate dither if you're going to filter the clip again, because you need the maximum quality at this intermediate stage and you don't want the motion estimation to be tricked by the regular patterns of an ordered dithering.

I kept wondering... smdegrain, actually any mdegrain that uses motion vectors is going to denoise only on static, or semistatic areas. This way moving areas are going to keep the floyd & steingberg dithering which consumes lots of bitrate. Do you think manalyse is really affected by the ordered dithering in the case I used mode=0 over mode=0?
Anyway this is about the easy way. I guess I can do something like, process motion analysis over mode=6 dummy, and degrain over mode=0, implement this inside the smdegrain function as a "fake" lsb_in.


In my last post I also wanted to tell you about something I observed. I didn't fully test so I was unsure to comment, but maybe you may want to check the fh and fv parameters of Dither_resize16 for next version. I think that in some occasions I couldn't get sharper results, maybe with small sources, downsizing or upsizing, can't remember. Can I assume >1.0 = sharper is always true?

@atra dies: Lately I discovered how effective dfttest can be at deblocking, this is a known fact, but the tricky thing I found is that deblocking works more effective when tbsize=1 (no temporal). try dfttest(tbsize=1,sigma=10,lsb=true).ditherpost. Im very surprised (using it every day). It blurs a little but this is unavoidable, although you can (contra)sharpen afterwards. sigma=10 looks like a fixed value, you won't likely deblock more with higher values.

[Heaud]

How about an option to have a static ampn noise? e.g. a negative value = static, or better an additional parameter like ampn_temp_avg=0..100 (similar to GrainFactory3) with -1 for a completely static noise.

For now you can achieve static noise on the post dithered content with addgrainc(constant=true). Set the strength to 0.8 as a good starting point and then adjust to taste.

[wOxxOm]

For now you can achieve static noise on the post dithered content with addgrainc(constant=true). Set the strength to 0.8 as a good starting point and then adjust to taste.
Actually for some types of content (and for most cartoons similar to anime) this is undesirable since the constant grain over the whole frame will be perceived as physical dust on a display device, visible everywhere except very bright areas during any panorama shift or character movement.

Adding of grain should be performed only in areas altered by DitherPost, and proportionally of course, so that there wouldn't be any abrupt grain density changes. Currently I do so by using a motion-mask, so that moving areas get moving grain applied and still areas get constant grain, with the final result applied only to areas altered by GradFun3.

[SilaSurfer]

Adding of grain should be performed only in areas altered by DitherPost, and proportionally of course, so that there wouldn't be any abrupt grain density changes. Currently I do so by using a motion-mask, so that moving areas get moving grain applied and still areas get constant grain, with the final result applied only to areas altered by GradFun3.

wOxxOm could you post a sample of this script? I'm interested of trying it out.

[wOxxOm]

well it might need some tweaking to make it usable for your type of content, but even in this state it might be handy to conceal dither pattern of GradFun3 in default mode=0. Requires variableblur plugin (http://avisynth.org/mediawiki/Variableblur).

#your GradFun3 settings
vGF3=GradFun3(0.5,8,2,1,thr_det=3,thr_edg=20,dthr=0.05,debug=0,smode=1)

#build motion mask
blk=8
vMA=msuper(rfilter=1,hpad=blk,vpad=blk).manalyse(blksize=blk)
vSC=MSCDetection(vMA,thscd1=300).mt_invert()
vMotionMask=mmask(vMA,ml=10,kind=0).mt_lut(y=0,w=blk).mt_lut(y=0,h=blk).mt_lut(y=0,offx=width-blk,w=blk).mt_lut(y=0,offy=height-blk)
vMotionMask=vMotionMask.mt_logic(vMotionMask.trim(1,0),"or").mt_logic(vSC,"and").mt_logic(vSC.trim(1,0),"and")
vMotionMask=vMotionMask.bilinearresize(ceil(width/blk/2)*2,ceil(height/blk/2)*2).mt_binarize(16)
vMotionMask=vMotionMask.removegrain(4).mt_expand().removegrain(11).bicubicresize(width,height).binomialblur(200,u=1,v=1)

#create motion-adaptive grain
vGray=mt_lut(y=-128,u=-128,v=-128)
vGrain=vGray.addgrainC(1,0,0,0,-1,true).mt_merge(vGray.addgrainC(1,0,0,0,-1,false),vMotionMask,true)

#grain will be added to dark areas only
vGrainMask=mt_lut("255 x 32 - 2 << 255 / 2 ^ 255 * -")
#contract mask down to areas touched by GF3 only, apply extreme blur to feather edges of the mask
vGrainMask=vGrainMask.mt_logic(mt_lutxy(vGF3,last,"x y - abs 8 << ").mt_expand().binomialblur(200,u=1,v=1),"min")

#apply grain to GF3
vGF3.mt_merge( vGF3.mt_adddiff(vGrain,u=2,v=2), vGrainMask,true )

[SilaSurfer]

Thanks man. ;)

[upyzl]

Excuse me

Does dither.dll & dither.avsi support x64 now?
It's pretty tools and I want to use it for 10-bit x264 encoding

[atra dies]

I am using it to smooth horrible compression artifacts, bands of blocks for flat animation colors and nothing removes it so completely as gradfun3 .66 strength and I tried a lot of filters and pp settings. For non-animation I have found that smoothd works wonders at removing lots of noise but blurs the picture, set adaptive mode to 1. Seems promising if someone used that principle for another filter let me know. (Edit: sorry, should have tried mdegrain2, it is great but slow and one of the mod16 filters.)


Make that .41 strength and default radius (defaults are good). Took a better look at it with histogram "luma". I would like to apply only to Y. Is there a command to show Y or U or V separately in avisynth?

The live video problem was 16mm grain. I would use ttempsmoothf but I see banding in the motion parts (in 8bit land). I tried all kinds of denoisers but I won't accept softening instead of grain. I usually deblock live stuff with cpu=6 or cpu2 on just the luma but this also leads to banding and what would I use on that? haha

[Dogway]

In tone of my last post.
I haven't done any tests still but Im sketching some theories to put in practice later and see the results.
edit: updated with results. This is against ffv1 lossless codec, Im unsure if ordered dithering is rather targeted to h.264 encodings but results are more or less the expected.

EX01
ditherpost(mode=6)
+
mdegrain in mode=0
theory = consumes lots of bitrates because of mode=6 left overs
result = to my surprise this one was the best in compressibility o_O!
clip = 48.37Mb (http://www.mediafire.com/?x49ww3sggh7mx62)

EX02
ditherpost(mode=0)
+
mdegrain in mode=0
theory = motion estimation is(could be) altered
result = This was the worst for compressibility. It showed heavy overlapping ordered dithering
clip = 50.49Mb (http://www.mediafire.com/?hdj8djbd8kj6dq2)

EX03
ditherpost(mode=0)
+
motion estimation of ditherpost(mode=6) and mdegrain in mode=0
theory = ideal but probably overlapping ordered dithering, plus denoising "ordered dither" which is not recommended.
result = Looks like motion estimation has some effect when mode=0, so this compresses better than EX02. Actually results or more towards EX01 than EX02 !O_o
clip = 49.08Mb (http://www.mediafire.com/?bssfx4opk4tkuuv)

EX04
ditherpost(mode=0)
+
mdegrain in mode=0 over mode=6 dummy
+
Ditherbuildmask workaround
theory = paired with motion estimation over the mode=6 dummy this could work.
result = The thinking behind stays true but the code is a bit different, see the code box below.
clip = 48.4Mb (http://www.mediafire.com/?0vbtb8bp42pmpp1)


o=dfttest(tbsize=1,sigma=10,lsb=true)
la2=o.DitherPost(mode=6)
la1=o.DitherPost(mode=-1)

super = la2.MSuper(pel=2, sharp=2) #hpad=0,vpad=0, para +velocidad
b3vec = super.MAnalyse(isb = true, delta = 3, overlap=2, blksize=8, search=4)
b2vec = super.MAnalyse(isb = true, delta = 2, overlap=2, blksize=8, search=4)
b1vec = super.MAnalyse(isb = true, delta = 1, overlap=2, blksize=8, search=4)
f1vec = super.MAnalyse(isb = false, delta = 1, overlap=2, blksize=8, search=4)
f2vec = super.MAnalyse(isb = false, delta = 2, overlap=2, blksize=8, search=4)
f3vec = super.MAnalyse(isb = false, delta = 3, overlap=2, blksize=8, search=4)

ms=la1.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)
ms2=la2.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)

msk=DitherBuildMask(la1,ms.ditherpost(mode=-1))
Dither_merge16_8(o,ms2,msk)

DitherPost(mode=0)

Conclusion: EX04 could possibly be better than EX01 but differences are too small to be judged fairly, plus processing time would be longer and compressibility reduced. So Im more or less like in the beginning, in wonder. : /

I decided to spatially analyse the dither, checking with the used masks:
http://img833.imageshack.us/img833/8471/mdegrainmask.th.png (http://img833.imageshack.us/img833/8471/mdegrainmask.png)http://img508.imageshack.us/img508/1300/ex04gammeddfttestmdegra.th.png (http://img508.imageshack.us/img508/1300/ex04gammeddfttestmdegra.png)http://img202.imageshack.us/img202/8324/ex04gammed.th.png (http://img202.imageshack.us/img202/8324/ex04gammed.png)http://img839.imageshack.us/img839/1351/ex01gammed.th.png (http://img839.imageshack.us/img839/1351/ex01gammed.png)

What I see: This would probably not be the best example as there is little motion (showed by the mdegrain mask). The relevant "mask" to check out is the one in 2nd picture, the enclosed regions belongs to the pure dfttest parts, where the rest is mdegrain. Here you can see, specially on the cloak how error diffusion changes to ordered dithering, making it ideal for encoding.

[06_taro]

Excuse me

Does dither.dll & dither.avsi support x64 now?
It's pretty tools and I want to use it for 10-bit x264 encoding

you can find the answer here (http://forum.doom9.org/showthread.php?p=1432866#post1432866) and here (http://forum.doom9.org/showthread.php?p=1505297#post1505297).

[SSH4]

I thought about upscaling some source after dfttest 16bit mod. So i thought that will be great dither 16 bit data after upscaling, this must not upscale ditherings.
I was play a little bit with 16 bit MSB/LSB data and upscale with nnedi3. And found that not good idea for this moment. LSB part have aliased edges when value drop from 255 to 0. and nnedi and most of other scalers smooth them, so in result after ditherpost() we have worst result :(
Is there any another way for simulate 16 bit data in avisinth but without MSB/LSB and sharp edges in LSB?
My brain not work and i cant imagine anything %)

[cretindesalpes]

You can resize stacked 16-bit MSB/LSB data with Dither_resize16(). Probably not as sharp as nnedi3, but it works.
It's also possible to combine 8-bit nnedi3 with Dither_resize16(). For example:

# 16 bit input

nw = 1280 # new width
nh = 960 # new height

upnn8 = DitherPost (mode=-1)
upnn8 = upnn8.nnedi3_rpow2 (rfactor=2, fwidth=nw, fheight=nh, cshift="Spline36Resize")
upnn16 = upnn8.Dither_convert_8_to_16 ()

Dither_resize16 (nw, nh, kernel="bicubic")

last.Dither_limit_dif16 (upnn16, thr=1.0, elast=2.0)
DitherPost ()

[TheProfosist]

right now im trying to use:

Dither2Pre (flt="FFT3DGpu(Sigma=4)")
SmoothGrad (radius=16, thr=0.25, elast=2)
DitherPost ()

but it it throws a error that asks me to debug:
http://img.photobucket.com/albums/v519/TheProfosist/doom9/Unexpectederrorencountered_2011-08-11_12-11-21.png

i would like to use dither+smoothgrad with the current plugins as best as possible. current sample of the filters i normally use in a script.
#RemoveGrain()
#RemoveGrainHD()

#crop(0,0,-0,-0,align=true)

#FFT3dGPU(sigma=0.2, precision=2)
#FFT3DFilter(sigma=0.2, ncpu=4)

#EdgeCleaner()

#DAA()
#MAA()

#LSFmod(strength=20, preblur="OFF", ss_x=2.0, defaults="slow")

#DeHalo_alpha(darkstr=1.0, brightstr=1.0, ss=2.0)

#GradFun2DBmod()

i would like to keep that order if possible (not required)
i would like the script to be able to output 16bit for 10bit x264 but also be possible to output 8bit for 8bit x264 (should be 1 line of code difference DitherPost()?)

Since i dont think anyone would want to work on the whole script i plan to work on getting one filter at a time as i need them.

For now i just need Dither+SmoothGrad to output 8bit&16bit to 8bit&10bit x264 respectively.

[cretindesalpes]

right now im trying to use:
Dither2Pre (flt="FFT3DGpu(Sigma=4)")

IIRC FFT3DGpu can be run only once at a time. Dither2Pre instantiates many of them, so you cannot use it here. You can try to use the classic FFT3DFilter instead, but it will be very slow. A much a better alternative would be using dfttest(lsb=true) (no need for Dither2Pre).

i would like the script to be able to output 16bit for 10bit x264 but also be possible to output 8bit for 8bit x264 (should be 1 line of code difference DitherPost()?)
For 8/8 bits, use DitherPost() as usual. For 16/10 bits, replace DitherPost() with Dither_convey_yuv4xxp16_on_yvxx(). No further filtering will be possible past this line, so this should be the last one of the script. Then encode with the appropriate command line (see the Dither documentation).

Edit:

A more general way to insert 8-bit processing between 16-bit filters is the following:
dfttest (lsb=true) # or whatever generating stack16 data

# Insert 16-bit filters here

s16 = last
DitherPost (mode=-1)

# Insert 8-bit filters here

Dither_convert_8_to_16 ()
s16.Dither_limit_dif16 (last, thr=1.0, elast=2.0)

# Insert 16-bits filters again...

# Finally
DitherPost () # or Dither_convey_yuv4xxp16_on_yvxx ()

To work correctly, the 8-bit filters must leave the low-gradient or flat area intact, i.e. touch only the edges, details or their immediate surrounding. It won't work with level/curve things (use SmoothAdjust in 16 bits instead).

Set thr and elast according to taste. You can also use masking instead of Dither_limit_dif16(), but this is a bit more work.

[TheProfosist]

is there a way to dither when going to 16bit i.e.: Dither_convey_yuv4xxp16_on_yvxx () ?

[cretindesalpes]

is there a way to dither when going to 16bit i.e.: Dither_convey_yuv4xxp16_on_yvxx () ?

This is a planned feature. In the meantime… here is a quick and dirty hack:

Edit: rewritten, and made a function of it (but it's still a suboptimal hack)

# n = target bitdepth (8–14)
# outn = true : output is scaled to the specified bitdepth
# = false: output is kept in 16 bits, the lowest bits are set to 0 after dithering.
# other parameters: transmitted to DitherPost().
#
# Example:
#
# .avs:
# ...
# GradFun3(lsb=true) # Stack16 clip
# fslg_dither_to_n_bits (10, outn=true)
# Dither_convey_yuv4xxp16_on_yvxx ()
#
# .bat:
# avs2yuv -raw script.avs -o - | x264-10bit --demuxer raw --input-depth 10 --input-res 1280x720 --fps 24000/1001 --output video.mkv -

Function fslg_dither_to_n_bits (clip o, int n, bool "outn", int "mode", int "ampo", int "ampn", bool "dyn")
{
outn = Default (outn, false)

o
msb = Dither_get_msb ()
lsb = Dither_get_lsb ()
hr = 2

m = String (Round (Pow (2, 16 - hr - n)) - 1)
l = msb.mt_lut ("x "+m+" &u", y=3, u=3, v=3)
h = msb.mt_lut ("x 255 "+m+" - &u", y=3, u=3, v=3)

s1 = String (n - 8)
ofs = String (Round (Pow (2, 7 - hr)))
a = StackVertical (l, lsb)
a = a.Dither_lut16 ("x "+s1+" <<u "+ofs+" 256 * +", y=3, u=3, v=3)

b = a.DitherPost (mode=mode, ampo=ampo, ampn=ampn, dyn=dyn)

(outn)
\ ? fslg_dither_to_n_bits_out_n (b, h, n, hr, s1, ofs, m)
\ : fslg_dither_to_n_bits_out_16 (b, h, n, hr, s1, ofs)
}

Function fslg_dither_to_n_bits_out_16 (clip b, clip h, int n, int hr, string s1, string ofs)
{
s2 = String (16 - n)
ofs2 = String (Round (Pow (2, 15 - hr - n)))
b_msb = b.mt_lut ("x "+s1+" >>u 255 &u "+ofs2+" - 128 +", y=3, u=3, v=3)
b_lsb = b.mt_lut ("x "+ofs+" - "+s2+" <<u 255 &u", y=3, u=3, v=3)

c = mt_adddiff (h, b_msb, y=3, u=3, v=3)

StackVertical (c, b_lsb)
}

Function fslg_dither_to_n_bits_out_n (clip b, clip h, int n, int hr, string s1, string ofs, string m)
{
ofs2 = String (Round (Pow (2, 7 - (n-8) - hr)))
b2 = b.mt_lut ("x "+s1+" >>u "+ofs2+" - 128 + 255 "+m+" - &u", y=3, u=3, v=3)
h2 = mt_adddiff (h, b2, y=3, u=3, v=3)
msb = h2.mt_lut ("x "+String(16-n)+" >>u", y=3, u=3, v=3)

m2 = String (Round (Pow (2, 8 - hr)) - 1)
lsb = mt_lutxy (h2, b, "x "+s1+" <<u 255 "+m2+" - &u y "+ofs+" - "+m2+" &u +", y=3, u=3, v=3)

StackVertical (msb, lsb)
}

Note: by sending 10-bit content to x264 (instead of 8- or 16-), you make sure that it will encode the exact values instead of scaling and dithering them again. Thus, you can encode safely with a vanilla 10-bit x264 instead of having to use a patched build.

Note 2: scaling issue mentioned in the previous note is now solved in the official x264 build (from r2164).

[Yellow_]

Dither 1.9.5 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released:

Bug fixed in Dither_resize16(), displaying green bars when SSE2 optimisations are disabled. Better check of stack16 clip dimensions.


TV range is all about keeping a headroom for the ringing causing by some filters (resizers, sharpeners…), so with PC range this headroom is lost. But this is not a big difference, and the bottom headroom is cleared by the gamma/linear conversion anyway.


Yes.


GammaYUV -> linearYUV -> linearRGB will give you wrong colors. Inverse colorspace conversions should be done in the same gamma/linearity as the forward conversion. For gamma YUV to linear RGB conversion you can use:

Dither_convert_yuv_to_rgb (output="rgb48y")
Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false)
Dither_convey_rgb48_on_yv12 (
\ SelectEvery (3, 0),
\ SelectEvery (3, 1),
\ SelectEvery (3, 2) )


You're right, it appears that BT709 doesn't use the same transfer curve as sRGB, the slope at 0 in sRGB is much steeper than in BT709.

If you want to linearize the R'G'B' components converted from the BT709 Y'CbCr, apply on each one : R = R'/4.5 if R' < 0.081 or R = ((R' + 0.099) / 1.099) ^ (1/0.45) if R' ≥ 0.081.The formula are the same as in sRGB, but with different values. I will add the BT709 mode to Dither conversion functions in a future release.

Could you clarify, do you mean with Imagemagick via avs2yuv as this will be 48bit RGB or can this be done with the stacked in avisynth?

[Zero1]

I downloaded and placed the dither package in the plugins directory and all the 16 bit stuff seems to be working fine, however I can not get gradfun3 to work (in normal YV12). I simply just get a green screen. I've deleted a bunch of plugins in an attempt to troubleshoot and this is what I'm left with:

15/05/2011 21:14 192,512 dfttest.dll
21/12/2008 22:46 26,624 DirectShowSource.dll
20/07/2011 22:03 44,994 dither.avsi
21/07/2011 08:18 233,472 dither.dll
21/06/2010 11:12 1,527,650 libfftw3f-3.dll
31/12/2010 18:13 966,144 mt_masktools-25.dll
31/12/2010 18:12 966,656 mt_masktools-26.dll
19/03/2006 20:56 430,080 mt_masktools.dll
11/09/2010 14:39 2,787 mt_xxpand_multi.avsi
20/01/2005 21:58 212,992 MVTools.dll
11/05/2011 22:09 753,664 mvtools2.dll
01/08/2005 00:08 65,536 RemoveGrainS.dll

I'm using avisynth 2.5.8. Any ideas?

[wOxxOm]

remove mt_masktools-25.dll, mt_masktools.dll, MVTools.dll and make sure that mt_masktools-26.dll is alpha48

[cretindesalpes]

Could you clarify, do you mean with Imagemagick via avs2yuv as this will be 48bit RGB or can this be done with the stacked in avisynth?
I meant this can be done with avisynth. It's just a few coefficients to change in Dither_y_gamma_to_linear and Dither_y_linear_to_gamma.

remove mt_masktools-25.dll, mt_masktools.dll, MVTools.dll and make sure that mt_masktools-26.dll is alpha48
Zero1 wrote he was using 2.5.8, so it's the other way round: keep mt_masktools-25 (alpha 48) and remove mt_masktools-26.dll

[Zero1]

Thanks for the info. I think I have inadvertantly found out what is going wrong. I tried to run dfttest() earlier and it returns and error saying it cannot load libfftw3f-3.dll.
When I moved it to another directory and used loadplugin, it tells me that it is not an Avisynth 2.5 pluginr?!

Edit
I googled and found that I needed to put that in my syswow64 dir, so now dfttest works. Also after deleting the numerous masktools dlls, gradfun3 appears to work too. Much thanks guys, these are really going to come in handy.

[Yellow_]

I meant this can be done with avisynth. It's just a few coefficients to change in Dither_y_gamma_to_linear and Dither_y_linear_to_gamma.

I've looked through dither.avsi at the above functions and quite simply I'm lost, so thanks anyway, think I'll just look out for the BT709 mode in some future version of Dither and find an alternative route for the linearizing bit in the meantime.

[MADAJ]

Hi,

I fed an 8-bit source to x264-10bit, and I converted the 8-bit to 16 bit (using dither scripts) then fed it to x264-10bit.
However color wise, I noticed that the 1st encoding is fine and looks like the 8-bit source, while the later encoding is overlayed with greenish color.
So which one is correct?

[cretindesalpes]

Is this related with this issue (http://forum.doom9.org/showthread.php?p=1517620#post1517620)? But the difference should be hardly noticeable. If you really have a green tint, first check your masktools version (must be alpha 48 or later), then post your script and encoding command.

[atra dies]

How do you do the pfclip? I want to use blur() but any other examples are welcome. I finally saw this in the dither doc and it looks pretty good. It says invalid arguments to function blur. I also tried it x 4 and as a variable.

o = last
Dither2Pre ("TTempSmoothf(maxr=2,lthresh=40,cthresh=40,strength=2,scthresh=32,pfclip=last.blur())",
overlap=0, tvopt=true)

[MADAJ]

Is this related with this issue (http://forum.doom9.org/showthread.php?p=1517620#post1517620)? But the difference should be hardly noticeable. If you really have a green tint, first check your masktools version (must be alpha 48 or later), then post your script and encoding command.

Yeah, that issue. I have the mentioned masktools version.

Script

gradfun3 #modified version where (result = last) instead of DitherPost()

Interleave (Dither_get_lsb (), Dither_get_msb ())
TurnRight ()
AssumeFieldBased ().AssumeTFF ()
Weave ()
TurnLeft ()


encoding command

avs2yuv -raw "script.avs" -o - | x264-10bit --OTHER_SETTINGS --demuxer raw --input-depth 16 --input-res 1280x720 --fps 24000/1001 --output "out.mkv" -


I just wanted to compare the two scenarios (8bit-10bit and 8bit->16bit-10bit) and see which one gives better result, but I can't judge because of the green tint in the second case.
Theoretically, which one do you think is better?

I hope the bug get solved soon.

[cretindesalpes]

How do you do the pfclip? I want to use blur() but any other examples are welcome.
Technically, it's possible to use a wrapper function around TTempSmooth and Blur, but I tried and it gave massive artifacts. This is one of the Dither1/2Pre() limitations.

but I can't judge because of the green tint in the second case.
Theoretically, which one do you think is better?
If your 8-bit source is untouched (no use of GradFun3), both methods are (should be) equivalent. If you use GradFun3, direct 16->10 bits encoding should give better results than 16->8->10 via DitherPost().

[atra dies]

Nevermind, I'm going with mdegrain sharp=2 if anything (and maybe pel=4).

I assume the output of dither 8 to 16 is supposed to have a plain green LSB(should be black?)?

Can 8 bit filters be used on msb and lsb separately (I remember you said something about processing luma separately) or is the lsb too sensitive (you mentioned protecting it with unsharp mask)? Or is it better to sharpen outside in 8 bit cause of artifacts. I'm thinking of limitedsharpenfaster here.


By the way for anyone wandering in and you don't know like me a few seconds ago, this is how you generate 16 bit with mdegrain, add lsb=true at the end. Does this process it in 16 bits or just output 16 bits? <-May be a stupid question like most of them.


super = MSuper(pel=4, sharp=2)
backward_vec2 = MAnalyse(super, blksize=4,isb = true, delta = 1, overlap=0,temporal=true)
forward_vec2 = MAnalyse(super, blksize=4,isb = false, delta = 1, overlap=0,temporal=true)
MDegrain1(super, backward_vec2,forward_vec2,thSAD=400,lsb=true)

or

super = MSuper(pel=4, sharp=2)
backward_vec2 = MAnalyse(super, isb = true, blksize=4,delta = 2, overlap=0,temporal=true)
backward_vec1 = MAnalyse(super, isb = true, blksize=4,delta = 1, overlap=0,temporal=true)
forward_vec1 = MAnalyse(super, isb = false, blksize=4,delta = 1, overlap=0,temporal=true)
forward_vec2 = MAnalyse(super, isb = false, blksize=4,delta = 2, overlap=0,temporal=true)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400,lsb=true)

I tried all the block sizes and overlaps and temporal = true. This filter is driving me nuts but removes a lot of noise without blurring it like FFT. Still seems soft and motion looks a little blurred but not as bad as ttempsmooth. Still high thSAD settings only cause artifacts beyond 400 instead of removing more noise.

Can ditherpost or some other step convert to yuy2 or is it fine for avisynth to do that. I thought I saw that there was a bug with that command or was that just for interlaced true?

Also I'm seeking opinions on whether users remove 16mm grain mpeg2 blocking or leave as is since it is considered resolution by some.

[cretindesalpes]

I assume the output of dither 8 to 16 is supposed to have a plain green LSB(should be black?)?
Exactly.

Can 8 bit filters be used on msb and lsb separately (I remember you said something about processing luma separately) or is the lsb too sensitive (you mentioned protecting it with unsharp mask)?
It's possible in some cases but not recommended. If you want to mix 8 and 16 bit processing, see post #250 (http://forum.doom9.org/showpost.php?p=1518953&postcount=250).

this is how you generate 16 bit with mdegrain, add lsb=true at the end. Does this process it in 16 bits or just output 16 bits?
Input is 8 bits, output 16 bits and the internal processing is done in 32 bits.

Can ditherpost or some other step convert to yuy2 or is it fine for avisynth to do that.
You can do it with ConvertToYUY2(), but the conversion will probably cancel the dithering benefits on the chroma plane (luma will stay fine).[/QUOTE]


・ ・ ・ ・・・————◇◆◇————・・・ ・ ・ ・

Dither 1.10.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released:
Added Dither_addborders16.
Added the BT 709 curve to Dither_y_gamma_to_linear and Dither_y_linear_to_gamma, as well as the plane selectors.
Added the possibility to have static noise in DitherPost.
Added Dither_add_grain16, generating stabilized noise.
Added a dif parameter to Dither_add16.
Added the lsb and lsb_in parameters to GradFun3 to deal with 16-bit data. The function also accepts smaller radii than before.

[Yellow_]

cretindesalpes, excellent update. :-)

I have a bit of a query regarding spiking luma using Dither_y_gamma_to_linear function and conversion to RGB. :-)

I have two test video's, ones a h264 AVC off a Canon HDSLR and the other off a HDV camera both sources converted with Dither 1.10 and piped through avs2yuv to ImageMagick Q16 hdri build. Writing 16bit tifs & exr's.

Using a 709 curve with the h264AVC (BT601 Color Matrix, full range luma, 709 transfer curve) gives me spiked luma histogram, using the srgb curve doesn't.

But with the HDV video source (BT709 Color Matrix, full range luma, BT709 transfer curve) whether I use srgb or 709 curve both give smooth luma histogram and definite variation in 'exposure' which I anticipated.

The basis of the scripts is:

Dither_convert_8_to_16 ()

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")

Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="srgb")

Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )

Obviously I'm changing the matrix="" to suit colour matrix of source and adjusting pixel dims to avoid any scaling ie: 1920x1088 and 1440x1080.

What could be causing the spiking luma with the h264? I thought the problem lay with Imagemagick colorspace handling but that appears to be ok, considering HDV source works fine.

Thanks again for a great update.

[cretindesalpes]

Using a 709 curve with the h264AVC (BT601 Color Matrix, full range luma, 709 transfer curve) gives me spiked luma histogram, using the srgb curve doesn't.

But with the HDV video source (BT709 Color Matrix, full range luma, BT709 transfer curve) whether I use srgb or 709 curve both give smooth luma histogram and definite variation in 'exposure' which I anticipated.
Do you have source samples showing these spiked histograms? How does look the spikes? 8 to 16 bit conversions always give large histogram holes, just because we spread only 256 discrete values into 65536 slots. Some additional slots are filled with overshoot, ringing, bluring or other artifacts caused by the chroma resampling, but probably not enough to cover the whole 16-bit histogram continuously.

[Yellow_]

Do you have source samples showing these spiked histograms?

I'll post a link to some examples once I get back to my machine later.

I wonder why I'm seeing this spikes when using rec709 inverse curve with h264 but not HDV, could this be more to do with compression type HDV vs h264, I don't see spikes with HDV so could this be luma deadzone quantisation in the h264 maybe?

The h264 bitrate is about 44Mbps compared to 20Mbps for the HDV, both consumer 4:2:0.

[SSH4]

Added the possibility to have static noise in DitherPost.

Great! But i think one thing missed - seed like in AddgrainC that really neded for static noise. (Imagine encoding per chapters...)

[cretindesalpes]

In static mode, the noise will remain the same and depends only on the frame size. The seed of random number generator is reset at the same value at the beginning of each plane. Therefore setting the seed explicitly is not needed.

[SilaSurfer]

Added Dither_add_grain16, generating stabilized noise.


Nice cretindesalpes to see you are working on your functions. Dither_add_Grain 16 - is that to be used for banding prevention or is it for usage to add grain to the whole frame?

[Yellow_]

I'll post a link to some examples once I get back to my machine later.

I wonder why I'm seeing this spikes when using rec709 inverse curve with h264 but not HDV, could this be more to do with compression type HDV vs h264, I don't see spikes with HDV so could this be luma deadzone quantisation in the h264 maybe?

The h264 bitrate is about 44Mbps compared to 20Mbps for the HDV, both consumer 4:2:0.

Scrub this, I see spiking in the HDV's as well at closer inspection.

Once I apply gamma correction (rec709 or srgb) after compositing (32bit float), histogram smooths again. :-)

I guess some sort of levels smoothing would happen using other Dither functions, a denoiser or such?

[SSH4]

cretindesalpes, is it possible made 1bit dithered image from gray with any of your functions from dither.dll?

small correction. what the fastest way for made this.

[cretindesalpes]

Dither_add_Grain 16 - is that to be used for banding prevention or is it for usage to add grain to the whole frame?
There is no specific purpose, it was mainly a wOxxOm request. You can use it to add visible grain or to replace the internal noise generator in DitherPost(). It was much easier to write a separate function than changing DitherPost().

cretindesalpes, is it possible made 1bit dithered image from gray with any of your functions from dither.dll?
small correction. what the fastest way for made this.
From an RGB picture in grey shades:
ShowRed ("YV12") # Or ConvertToYV12(matrix="PC.709")
StackVertical (last.BlankClip (color_yuv=0), last)
DitherPost (mode=7, u=1, v=1)
mt_binarize (threshold=0, u=-128, v=-128)Output will be full range. For YV12 TV-range, replace the mt_binarize with mt_lut ("x 219 * 16 +", u=-128, v=-128).

EDIT:
For much more accurate results, dither in linear light by replacing the StackVertical() with:
Dither_convert_8_to_16 ()
Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="srgb", u=1, v=1)
Dither_lut16 ("x 256 /", u=1, v=1)

[SSH4]

Thanx! I thought about StackVertical-DitherPost-mt_binarize(or mt_lut) too.

Oh my god, i missed this - Dither_lut8, Dither_lutxy8, Dither_lutxyz8, Dither_lut16! :)

cretindesalpes may be possible made full 16bit (MSB/LSB) mod of masktools?

[Dogway]

Hello cretindesalpes, Im having problems with a source. Im trying to convert yuv to rgb32 skipping chroma planes all along, (output is greyscale). The code Im more confident with is the next but I tried many variations with no success:
SmoothGrad (thr=0.25, radius=16, stacked=true,u=1,v=1)

Dither_convert_yuv_to_rgb(lsb_in=true,tv_range=true,output="rgb48y")
ditherpost(mode=6,ampn=0.5,y=3,u=-128,v=-128)
MergeRGB(SelectEvery(3,0),SelectEvery(3,1),SelectEvery(3,2))

Main problems are first smoothgrad, there is no way to skip chroma and get correct results(screen is green), next is Dither_convert_yuv_to_rgb, I miss the u and v parameters, the output in this case is not 100% greyscale, you can find slight variations like 127,128,128, etc. Here ditherpost doesn't solve anything since mergergb creates these variations afterwards...

[xackphair]

I am getting a ghosting problem that I do not remember seeing before. I am just using the basic

dfttest (lsb=true)
DitherPost ()

I will post images when my lossless is done. However, if you have an idea of what is going on, please let me know. Thanks!

[cretindesalpes]

I am getting a ghosting problem that I do not remember seeing before.
It's most likely a dfttest related problem. Dfttest is not protected against ghosting, therefore you have to MCompensate/Interleave the previous and next frames first, and SelectEvery the result. Check the Dither documentation, there is an example using the MCompensate functions. You can also check dfttestMC (http://forum.doom9.org/showthread.php?t=147676) for a more complex example.


may be possible made full 16bit (MSB/LSB) mod of masktools?
I don't think so. It's a huge amount of work. Moreover, straight multi-dimension 16-bit look-up tables wouldn't work because their size would be squared (hundreds of terabytes for a single lut_xyz16).


Main problems are first smoothgrad, there is no way to skip chroma and get correct results(screen is green)
This is normal. u=1 and v=1 means "don't care about the chroma planes", and the output can be any value, including invalid combinations with the luma plane. Green screen most likely means: chroma is all 0, and the luma could be anything. Just add a GreyScale() afterwards if you want to check the luma (but don't process a stack16 clip after a GreyScale, the chroma LSB will be wrong).
next is Dither_convert_yuv_to_rgb, I miss the u and v parameters, the output in this case is not 100% greyscale, you can find slight variations like 127,128,128, etc. Here ditherpost doesn't solve anything since mergergb creates these variations afterwards...
The RGB conversion has no means to skip planes, however you are processing here the trashed chroma channels, so the conversion is definitely wrong. But if you want greyscale output, there is no need an RGB conversion. Just copy the Y channel directly to the R, G and B channels. You may also have to extend it to full range.
SmoothGrad (thr=0.25, radius=16, stacked=true,u=1,v=1)
Dither_lut16 ("x 4096 - 219 / 255 *", u=1, v=1) # TV range -> Full range
ditherpost(mode=6,ampn=0.5,y=3,u=1,v=1)
MergeRGB (last, last, last)

[Dogway]

Yeah, that was the problem, I couldn't use greyscale to LSB, besides LSB also stores chroma modification values. I also tried with u and v = -128 but output was also green. Also tried copying luma to the r,g,b channels but I missed the tv range thing, so THANKS a lot, this is something I already asked in the smoothadjust thread (equation of range conversion).

Im also interested in your opinion on my previous post (http://forum.doom9.org/showthread.php?p=1518483#post1518483)
The main problem was floyd dithering on non mdegrain'ed areas (where there is motion) EX01. I came up with the EX04 workaround so ordered dithering is always used, does the logic work or you see any flaws in the code? I have in sight modifying my smdegrain mod upon your observations...

[cretindesalpes]

I also tried with u and v = -128 but output was also green.
If the output clip is 16 bits, filling values have to be 16 bits too, so it's u and v = -128*256

Im also interested in your opinion on my previous post (http://forum.doom9.org/showthread.php?p=1518483#post1518483)
The main problem was floyd dithering on non mdegrain'ed areas (where there is motion) EX01. I came up with the EX04 workaround so ordered dithering is always used, does the logic work or you see any flaws in the code? I have in sight modifying my smdegrain mod upon your observations...
Oh I haven't realized you updated your post. I don't know if the MDegrain difference is reliable enough to build the mask. Intuitively, I would have used a combination of the MMask() SAD results, but your solution looks fine too. Maybe it's possible to get rid of the second MDegrain and compare ms2.DitherPost(mode=-1) to la2? But I suppose you probably had good reasons to use a second MDegrain. Anyway, it seems that differences between EX01 and EX04 are minor.

[Dogway]

Thanks for the input, normally I try to elaborate from your hints on dither usage, but mmask seems more reasonable, Im not sure which will be more correct or faster but my thinking was more towards "not all areas that can be mdegrain'ed (masked) are definitively going to be degrained"
What I was thinking was; mode=6 is nice to feed for further degraining/denoising, but not as left over, so what wasn't degrained over mode=6, then use the mode=0

I experimented with another source in a fast panning frame:

RAW
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_raw-2.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/raw-2.png)

with smdegrain:
20.3s (encoding time)
11.08Mb (http://www.mediafire.com/?7f0vnbykm2s8o2b)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_sm.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/sm.png)
dfttest(tbsize=1,sigma=10,lsb=true)
ditherpost(mode=6)

smdegrain(tr=3,thSAD=400,pel=2,sharp=2,lsb=true,overlap=4)

with the EX04 code
29.7s (encoding time)
11.07mb (http://www.mediafire.com/?36t8wcrj4uqh4w8)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_ex04.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/ex04.png)
o=dfttest(tbsize=1,sigma=10,lsb=true)
la2=o.DitherPost(mode=6)
la1=o.DitherPost(mode=-1)
over=4
super = la2.MSuper(pel=2, sharp=2)
b3vec = super.MAnalyse(isb = true, delta = 3, overlap=over, blksize=8, search=4)
b2vec = super.MAnalyse(isb = true, delta = 2, overlap=over, blksize=8, search=4)
b1vec = super.MAnalyse(isb = true, delta = 1, overlap=over, blksize=8, search=4)
f1vec = super.MAnalyse(isb = false, delta = 1, overlap=over, blksize=8, search=4)
f2vec = super.MAnalyse(isb = false, delta = 2, overlap=over, blksize=8, search=4)
f3vec = super.MAnalyse(isb = false, delta = 3, overlap=over, blksize=8, search=4)

ms=la1.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)
ms2=la2.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)

msk=DitherBuildMask(la1,ms.ditherpost(mode=-1))
Dither_merge16_8(o,ms2,msk)

DitherPost(mode=0)

with the mmask workaround:
Not sure if the mask matches that used in EX04...
8.9s (encoding time)
11.05mb (http://www.mediafire.com/?mk1iujah2d9a5mb)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_mmask3.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/mmask3.png)
o=dfttest(tbsize=1,sigma=10,lsb=true)
la2=o.DitherPost(mode=6)

gam=1.0
SCD1=400
SCD2=150
over=4
super = la2.MSuper(pel=2, sharp=2)
b3v = super.MAnalyse(isb = true, delta = 3, overlap=over, blksize=8, search=4)
b2v = super.MAnalyse(isb = true, delta = 2, overlap=over, blksize=8, search=4)
b1v = super.MAnalyse(isb = true, delta = 1, overlap=over, blksize=8, search=4)
f1v = super.MAnalyse(isb = false, delta = 1, overlap=over, blksize=8, search=4)
f2v = super.MAnalyse(isb = false, delta = 2, overlap=over, blksize=8, search=4)
f3v = super.MAnalyse(isb = false, delta = 3, overlap=over, blksize=8, search=4)
SADbv3 = MMask(b3v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADbv2 = MMask(b2v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADbv1 = MMask(b1v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADfv1 = MMask(f1v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADfv2 = MMask(f2v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)
SADfv3 = MMask(f3v,kind=1,gamma=gam,thSCD1=SCD1,thSCD2=SCD2)

SAD6 = average(SADbv3,1./6,SADbv2,1./6,SADbv1,1./6,SADfv1,1./6,SADfv2,1./6,SADfv3,1./6)
ms2=la2.MDegrain3(super, b1v, f1v, b2v, f2v , b3v, f3v, thSAD=400, limit=255, limitc=255,lsb=true)

Dither_merge16_8(ms2,o,sad6)

DitherPost(mode=0)

with the EX04 revised, getting rid of one mdegrain:
27.8s (encoding time)
11.01mb (http://www.mediafire.com/?6myzolcdkf3brr8)
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_ex04r.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/ex04r.png)
o=dfttest(tbsize=1,sigma=10,lsb=true)
la2=o.DitherPost(mode=6)

over=4
super = la2.MSuper(pel=2, sharp=2)
b3vec = super.MAnalyse(isb = true, delta = 3, overlap=over, blksize=8, search=4)
b2vec = super.MAnalyse(isb = true, delta = 2, overlap=over, blksize=8, search=4)
b1vec = super.MAnalyse(isb = true, delta = 1, overlap=over, blksize=8, search=4)
f1vec = super.MAnalyse(isb = false, delta = 1, overlap=over, blksize=8, search=4)
f2vec = super.MAnalyse(isb = false, delta = 2, overlap=over, blksize=8, search=4)
f3vec = super.MAnalyse(isb = false, delta = 3, overlap=over, blksize=8, search=4)

ms2=la2.MDegrain3(super, b1vec, f1vec, b2vec, f2vec , b3vec, f3vec, thSAD=400, limit=255, limitc=255,lsb=true)

msk=DitherBuildMask(la2,ms2.ditherpost(mode=-1))
Dither_merge16_8(o,ms2,msk)

DitherPost(mode=0)

I don't understand how the mmask workaround is so much faster than smdegrain. I don't know if it is a reliable test since the internal mask used in smdegrain and EX04 could be different. The EX04r with one mdegrain workaround compressed the best, I liked the idea but I was just unsure if comparing areas of mode=6 of dfttest and mode=-1 of mdegrain was legit.

The funny thing is here, I overlayed the mask from mmask and the mask from the difference of dfttest and mdegrain of EX04r. One would think that there is going to be 0 changes in the areas enclosed in the mmask mask (because its where there is more movement), but as you see in the DitherBuildMask, changes are everywhere.
EX04r mask .....................EX04 mask
http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_mask-1.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/mask-1.png)http://i212.photobucket.com/albums/cc35/Dogway/Misc/th_maskex04.png (http://i212.photobucket.com/albums/cc35/Dogway/Misc/maskex04.png)

Sorry for the bulky post

[Dogway]

Ok I will implement finally the EX04r code, it compresses the best (tested on another anime source) and is not the slowest. The thinking has logic, and ssim compared to ex04 is the same, and probably better than smdegrain running after ditherpost(mode=6)

The only thing I still want to test is using the mmask workaround. But it's difficult to know what mask produces manalyse internally. When I try to match with the exact same settings the mmask code with the smdegrain (excluding dfttest), I get totally different results. Probably this is more a question for the mvtools thread, and even this workaround has its flaws, like passing mode=6 through static areas finally not denoised by mdegrain.

I encourage anyone who wants to test EX04r (or my 1.5d version of smdegrain below) over (old) smdegrain code to see if it compresses better under different sources and codecs.
I have run some encoding tests and ffv1 and h.264 (CRF) take less space than not using the EX04r code.

Here it is smdegrain 1.5d (http://www.mediafire.com/?4928ukpuvj85jgz) with "fake" lsb_in option. Despite compressing better in ffv1 and in same CRF in h.264, quality is slightly dropped in h.264 whether it is CRF or 2 pass, so I still consider this option experimental.

lsb_in=true against using ditherpost(mode=6) before smdegrain with lsb_in=false.

[ajp_anton]

How do I get 16-bit 4:4:4 video into x264?
(Dithered) 8-bit YV24 works directly, and 16-bit YV12 works via avs2yuv, but 16-bit YV24 gives weird results...

[cretindesalpes]

I experimented with another source in a fast panning frame
From your posted pictures, I'd say that the MMask version looks a bit more faithful to original than the others. There is a couple of fine shading that get blurred (mainly the line between the bottom of the robot and its shadow, under the sunlight halo) in the SMDegrain/EX04 versions.

How do I get 16-bit 4:4:4 video into x264?
(Dithered) 8-bit YV24 works directly, and 16-bit YV12 works via avs2yuv, but 16-bit YV24 gives weird results...
I couldn't make it work neither. However I don't think avs2yuv supports YV24 output, you should try avs2pipe26mod instead. But I've got this from x264 r2074:
resize [warning]: converting from yuv444p16le to yuv420p16le
[swscaler @ 003e53e0] full chroma interpolation for destination format 'yuv420p16le' not yet implemented

———


Updated Dither (http://forum.doom9.org/showthread.php?p=1386559#post1386559) to v1.11.0. Actually this is more a MVTools update:

MDegrainN added.
MAnalyse: Added the "multi" mode for MDegrainN.
MAnalyse: Documented the negative delta values and fixed some functions accordingly.
MFlowInter: Fixed the YUY2 planar mode.


With MDegrainN, filtering should be simpler, a bit like the MVDegrainMulti in Josey Wells' version:
tr = 6 # Temporal radius
super = MSuper ()
vmulti = MAnalyse (super, multi=true, delta=tr)
MDegrainN (super, vmulti, tr, thSAD=400, thSAD2=150, lsb=true)
DitherPost ()
I added a SAD threshold bell curve (with thSAD2) to allow the filter to be more selective with the furthest reference frames.

You can also extract single vectors from the multi-vect:
vb1 = vmulti.SelectEvery (tr * 2, 0)
vf1 = vmulti.SelectEvery (tr * 2, 1)
vb2 = vmulti.SelectEvery (tr * 2, 2)
vf2 = vmulti.SelectEvery (tr * 2, 3)
MDegrain2 (super, vb1, vf1, vb2, vf2)

I had to change more things than expected in the MVTools code to implement the multi-vector mode so I hope I didn't break anything…

[Dogway]

Thanks for the feedback!

I see the no-blurring thing more as a fault of the mmask example than an achievement. All in all what I expect is a result very near to the smdegrain example, but with some ordered dithering in a few places instead of the error diffusion. This only difference doesn't make areas to "sharpen/no-blur" as in the mmask ex. I think the same blurring could happen by just running a simple degrain script, one using the vectors from manalyse and another using mmask to mask out the mdegrain. I need to know how to match these two procedures with identical results. edit: probably Id need to be using kind=0 instead of 1 in mmask(), not sure.

I will have a look at your mvtools mod update for using it in smdegrain : P


It has a small issue I think, when I render frame in avspmod the image gets full of artifacts, they get away if I frameskip forth and back.
Also is there a way I can use MRecalculate with the vmulti vectors?

[cretindesalpes]

Indeed, and it's not restricted to AvsPmod. It's a serious issue IMHO. Don't use this version until I fix the bug! Sorry for the trouble.

[cretindesalpes]

Fixed, hopefully (Dither 1.11.1)

[Dogway]

Thanks! looks ok.
mrecalculate is not modified for vmulti, right?

[cretindesalpes]

Yes. I'll modify it later.

[Dogway]

ah, ok. No prob! I just have smdegrain v1.1d with your updates waiting in the drawer for when its ready

[cretindesalpes]

Dither 1.12.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) is an MVTools update again:
MRecalculate can now process multi vector clips (don't forget to set the new tr parameter).
Added the -Vit-'s fix that could improve the multithreading stability.

[Yellow_]

cretindesalpes, excellent update. :-)

I have a bit of a query regarding spiking luma using Dither_y_gamma_to_linear function and conversion to RGB. :-)

I have two test video's, ones a h264 AVC off a Canon HDSLR and the other off a HDV camera both sources converted with Dither 1.10 and piped through avs2yuv to ImageMagick Q16 hdri build. Writing 16bit tifs & exr's.

Using a 709 curve with the h264AVC (BT601 Color Matrix, full range luma, 709 transfer curve) gives me spiked luma histogram, using the srgb curve doesn't.

But with the HDV video source (BT709 Color Matrix, full range luma, BT709 transfer curve) whether I use srgb or 709 curve both give smooth luma histogram and definite variation in 'exposure' which I anticipated.

The basis of the scripts is:

Quote:
Dither_convert_8_to_16 ()

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")

Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="srgb")

Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )
Obviously I'm changing the matrix="" to suit colour matrix of source and adjusting pixel dims to avoid any scaling ie: 1920x1088 and 1440x1080.

What could be causing the spiking luma with the h264? I thought the problem lay with Imagemagick colorspace handling but that appears to be ok, considering HDV source works fine.

Thanks again for a great update.

Do you have source samples showing these spiked histograms? How does look the spikes? 8 to 16 bit conversions always give large histogram holes, just because we spread only 256 discrete values into 65536 slots. Some additional slots are filled with overshoot, ringing, bluring or other artifacts caused by the chroma resampling, but probably not enough to cover the whole 16-bit histogram continuously.

I realise now why I'm seeing spiked histogram with linearising BT601 matrix HD video, where as BT709 Matrix is smooth histogram, BT601 & 709 luma levels are slightly different, so assume Dithers linearising is 709 only. Any chance of adding a 601 linearising option at some point, if you agree that could be the issue.

[cretindesalpes]

BT.601 and BT.709 use the same formula to correct the gamma of the RGB signals, so you can use curve="709". I could add a "601" curve, but it will just map to the same formula.

[Yellow_]

Ok, thanks, I'll try again to find what might be causing the spikes, only does it when using BT601 matrix going to RGB. :-(

[SilaSurfer]

cretindesalpes Thanks for the update. Looking forward on using MdegrainN. One request if I may, would it be possible to add to your function Dither_Add_Grain_16 to be able to add 3 layers of grain like what one would get with Grainfactory3 from Didée? (hope he's well) Thanks.

[Davjd]

Hello Dear Doom9 users,
Is-it possible to input YUY2 4:2:2 (ConvertToYV16 ?)
and output with RGB48 for ImageMagick ? if yes, how ?
I try with

y = ConvertToYV12 ()
u = UToY ().ConvertToYV12 ()
v = VToY ().ConvertToYV12 ()
YToUV (u, v, StackVertical (y, y))

GradFun3 (mask=0)

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false,
cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")


But it's only YV12 which is 4:2:0 ?

I'm sorry but I try to understand how everything works but I'm still lacking knowledge about all this color things.

Yellow_, how do you input your canon footage ?

Thank you.

[Yellow_]

@DavJd

FFmpegSource2("MVI_0000.MOV")

Dither_convert_8_to_16 ()

Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")

#Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="709")

Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )


I've hashed out the linearising line, in case you don't want that, if you're writing to EXR then should linearise first. It's linearised based on inverse of 709 curve so output is Scene Linear.

The above is 16bit, ie to 10bit x264 or 16bit image sequences via IM. You'll need to dither back to 8bit if you want 8bit output.

Then use Avs2yuv or probably better to use Avs2pipemod to either x264 or IM like this from Dither.html that comes with the plugin. ;-)

avs2yuv -raw "script.avs" -o - | x264-10bit --demuxer raw --input-depth 16 --input-res 1280x720 --fps 24 --output "out.mp4" - [Delete the - at the end]

avs2yuv -raw "script.avs" -o - | convert -depth 16 -size 1280x720 rgb:- "out-%06d.tif"

[cretindesalpes]

You could use YV16 on avisynth 2.6 instead of YUY2. But for whatever reason, passing YV16 to Dither_convert_yuv_to_rgb doesn't work. Before I fix this issue, you can convert to YV24 instead:
ConvertToYV24 ()

GradFun3 (mask=0, lsb=true)

Dither_convert_yuv_to_rgb(matrix="709", tv_range=false,
\ cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")
GradFun3 outputs 8 bit, you must specify manually that you want 16 bit output.

Edit:

Bug found. dither.avsi, lines 245 and 254, replace && with ||.

[ajp_anton]

I couldn't make it work neither. However I don't think avs2yuv supports YV24 output, you should try avs2pipe26mod instead. But I've got this from x264 r2074:
resize [warning]: converting from yuv444p16le to yuv420p16le
[swscaler @ 003e53e0] full chroma interpolation for destination format 'yuv420p16le' not yet implementedI get that too, and the output is still messed up.
So there's no way to encode "real" 10-bit 4:4:4 videos?

edit:
Nevermind. Of course I figured it out just after I posted that. You have to add "--input-csp i444" to x264. "--input-csp yv24" looks like the chroma channels are swapped or something, but i444 works fine.

[cretindesalpes]

No it doesn't work. It looks OK, but the 4:4:4 colorspace is converted to 4:2:0. Or did you use a specific option to force the encoding in 4:4:4 colorspace? Anyway it should be possible to do it, this sample (http://www.cccp-project.net/beta/test_files/H.264-444-10bit-sample.mkv) looks like it has been encoded with x264.

[ajp_anton]

No, it definitely works. The extra chroma resolution is clearly visible on video game footage, and even more so when loading the encoded video into Avisynth and comparing the chroma planes directly.
Maybe you forgot the "--output-csp i444", which is always needed for 4:4:4 encoding?

edit:
So the full command line would be
avs2pipe26mod -rawvideo script.avs | x264-10b - --demuxer raw --input-res wxh --input-depth 16 --input-csp i444 --output-csp i444 --fps x --crf x -o o.mkv

[cretindesalpes]

Oh you're right, I only had the --input-csp. Thanks for the tip! I'll add it to the documentation.

[Yellow_]

I'm getting weird ghosting exporting 16bit to 10bit x264 with 1.12.0 that with the same script worked fine on 1.10.0 using the following:

LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll")
LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools_26.dll")

FFmpegSource2("MVI_8068.MOV")

trim(0,10)

Dither_convert_8_to_16 ()

Dither_convey_yuv4xxp16_on_yvxx()

Most of the image is intact but there is a semi transparent overlap starting from the left which travels across the frame as it plays, just seems to affect chroma and only certain colors, lots of grey appearing too.

I'll try to upload a image frame later.

[cretindesalpes]

I'm getting weird ghosting exporting 16bit to 10bit x264 with 1.12.0 that with the same script worked fine on 1.10.0
This is weird, because dither.avsi hasn't changed from 1.10.0 to 1.12.0. Are you there wasn't any problem with 1.10.0?

One request if I may, would it be possible to add to your function Dither_Add_Grain_16 to be able to add 3 layers of grain like what one would get with Grainfactory3 from Didée? (hope he's well) Thanks.
I might consider this later, but it's currently not in my priority task list.

[Yellow_]

Ok, my bad I left the -raw out of the avs2yuv command line. so dumb, sorry for confusion. :-(

[SilaSurfer]

I might consider this later, but it's currently not in my priority task list.

Thanks cretindesalpes. No preasure. Keep on the good work. ;)

[Dogway]

Several questions in Dither_convert_yuv_to_rgb could you implement rgb24? This in case it saves something because I never use alpha.
Also it's strange, when I convert yuv red color AFAIK (81, 90, 240) to RGB in TV range I get 238,14,14, is it ok?
Do you still have planned v&c algorithm, Im curious to know how it compares to bayer dithering

[cretindesalpes]

Several questions in Dither_convert_yuv_to_rgb could you implement rgb24?
OK.

Also it's strange, when I convert yuv red color AFAIK (81, 90, 240) to RGB in TV range I get 238,14,14, is it ok?
Are you sure ?

BlankClip (pixel_type="YV12", color_yuv=(81*256+90)*256+240)
Dither_convert_yuv_to_rgb (tv_range=true)

gives me the expected values around (255, 0, 0), and I obtain (238, 14, 14) with tv_range=false. Have you this problem with a specific setting?

Do you still have planned v&c algorithm, Im curious to know how it compares to bayer dithering
I haven't made any progress on this topic. I'm currently trying to multithread internally the MAnalyse/MRecalculate part of the MVTools because this is a major speed bottleneck for lots of scripts, but understanding how all this works is a quite complex and time-consuming task.

[Dogway]

No, its ok, it just looked like wanting to be 240,16,16. Im converting a bunch of frames for manual editing and wanted to be sure, but I will probably use tvrange true so no range conversion is done.


No problem on v&c, at your pace. Now that you are on mvtools2 it could be interesting if you could have a look on how to make it more flexible, like
mdegrain not relying on vectors/mask, so one could mix spatial denoiser and mdegrain through the mvtools motion mask, you can see my discussion in the mvtools thread. This could also enable a more orthodox way for "fake" lsb_in in mdegrain, and play a role on the masktools pipeline.
Multithreading the motion search stage is going to be a big leap, you might already know the SVP group are on the GPU direction with this, but I prefer your tools because they work rock steady. No need to hurry.

[Yellow_]

Is there anything that can be done to improve memory usage when using Dither_convey_yuv4xxp16_on_yvxx() -> AVS2yuv -> Imagemagick written 16bit image sequences? On larger video files the process fails, with out of memory errors by trying to do the whole video in memory I guess before writing out of IM. :-(

LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\ffms2.dll")
LoadPlugin("c:\Program Files (x86)\AviSynth 2.5\plugins\mt_masktools_26.dll")

FFmpegSource2("MVI_8068.MOV")

Dither_convert_8_to_16 ()

Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")
Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="709")
Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )



Although I could probably batch write scripts to trim() so many frames into individual .avs files and batch convert, is there a more efficient way. Reduced bit depth to 12bit? But memory would just run out later in the process?

On another query, I see in the Dither.html that blending frames is possible, so is it possible to use Dither tools to do typical blend modes like overlay, multiply etc in linear light at 16bit depth? I'm interested in working this with new high bit depth tools in Avisynth now. http://forum.doom9.org/showpost.php?p=617523&postcount=219 & here: http://www.dcs.gla.ac.uk/~jhw/filmy/, not so much simulating film grain, don't see much point in that unless mixing computer generated imagery with film scans, but the build up of layers.

[Dogway]

The interlaced parameter in ditherpost is for interlaced inputs or (separated)fields input? I wonder because it would be unpractical to work 16 bit in real interlaced material.
Also I have been wondering about the motion search in x264, isn't it fooled by the ordered dither as in mvtools2? or does it use another approach for it.

At last Im trying to optimize smdegrain depending on processed planes. But there are some behaviours I don't get to grasp.

Shouldn't this be true?


dfttest(lsb=true,U=true,v=true)
DitherPost(mode=-1,u=2,v=2)
==
dfttest(lsb=true,U=false,v=false)
DitherPost(mode=-1,u=3,v=3)

I also would think that in the first case UV output would be the MSB part of the clip which is 8 bit, but that also doesn't apply if I compare it to :

dfttest(lsb=true,U=false,v=false)
DitherPost(mode=-1,u=2,v=2)
dfttest(Y=false)

[cretindesalpes]

Is there anything that can be done to improve memory usage when using Dither_convey_yuv4xxp16_on_yvxx() -> AVS2yuv -> Imagemagick
If I understand correctly your description of the problem, data accumulate in the pipe until Imagemagick decides to process them, probably after the pipe is closed at the AVS2yuv end. I don't know if replacing the pipe with an intermediate file would work, but you can try.

On another query, I see in the Dither.html that blending frames is possible, so is it possible to use Dither tools to do typical blend modes like overlay, multiply etc in linear light at 16bit depth?
It's possible to blend 8-bit pictures into a 16-bit frame using the masktools. However blending 16-bit pictures is another story, mainly because we lack functions that can combine two or three 16-bit clips.

The interlaced parameter in ditherpost is for interlaced inputs or (separated)fields input?
For interlaced input. It differs from progressive input only when using the error diffusion modes.

Also I have been wondering about the motion search in x264, isn't it fooled by the ordered dither as in mvtools2? or does it use another approach for it.
Sure but this is not a problem here if motion estimation is inaccurate in nearly flat areas. x264 isn't a temporal denoiser, it doesn't need to perfectly overlay fine details or textures. It has to take advantage of the data redundancy, and this is what an ordered dither can offer.

dfttest(lsb=true,U=true,v=true)
DitherPost(mode=-1,u=2,v=2)
==
dfttest(lsb=true,U=false,v=false)
DitherPost(mode=-1,u=3,v=3)
No, because in the first case, the U and V plane have been filtered by dfttest. Whatever the conversion to 8 bits done afterward, you'll never obtain the input back. Using u=2, v=2 in DitherPost will keep only the MSB (truncation) instead of rounding, like cropping the top half of a stack16 frame.

[Dogway]

For interlaced input. It differs from progressive input only when using the error diffusion modes.
I see, although I don't know when would it apply since interlaced inputs are commonly filtered after separated fields, in any case I rarely use error diffusion.


Using u=2, v=2 in DitherPost will keep only the MSB (truncation) instead of rounding, like cropping the top half of a stack16 frame.
Yes that makes sense, as I explained afterwards it would more likely keep the msb part when using u=2 v=2, but I compared it also to filtering chroma after ditherpost in 8 bits and it isn't the same so the question is whether the MSB part is like the result of filtering in 8 bit or it is something different.

dfttest(lsb=true,U=true,v=true)
DitherPost(mode=-1,u=2,v=2)
==
dfttest(lsb=true,U=false,v=false)
DitherPost(mode=-1,u=2,v=2)
dfttest(Y=false)

[Yellow_]

cretindesalpes, thanks. Investigating further I've found that Imagemagick has a function called -limit, which can be set to restrict how much memory IM can use before it has to write temp files to disk. Using this on the CLI:

identify -list resource

Gives info on what resources IM thinks is available. Then setting -limit memory some GiB and -limit map some GiB functions on the avs2yuv command line, to levels within those resources keeps memory errors away. :-)

[TheProfosist]

DitherPost dithers? a 16bit clip to 8bit. Then what does GradFun3 dither? What im trying to ask is how is using SmoothGrad+DithePost different from using GradFun3?

[AzraelNewtype]

DitherPost dithers? a 16bit clip to 8bit. Then what does GradFun3 dither? What im trying to ask is how is using SmoothGrad+DithePost different from using GradFun3?

DitherPost turns stacked 16bit, usually created by using a denoiser that is designed to take 8bit input and yield that special output format, into dithered single height 8bit. GradFun3 takes your 8bit video and dithers it to attempt to deband source that's already been filtered, in much the same way GradFun2 and its modifications did before it. Which you should use depends on your workflow.

[TheProfosist]

DitherPost turns stacked 16bit, usually created by using a denoiser that is designed to take 8bit input and yield that special output format, into dithered single height 8bit. GradFun3 takes your 8bit video and dithers it to attempt to deband source that's already been filtered, in much the same way GradFun2 and its modifications did before it. Which you should use depends on your workflow.

will DitherPost help with debanding at all? I know that SmoothGrad should by smoothing the gradients.

[Dogway]

TheProfosist: Gradfun3 basically is the same as:

Dither_convert_8_to_16()
put a low pass denoiser here...
ditherpost

with some masks, etc. Cretindesalpes explained it at some point in the thread.

Ditherpost just takes whatever 16 bit processing clip and converts it to 8 bit with the consequent dither so you don't see banding. So ditherpost in nature doesn't deband but rather prevents banding when doing filtering (among other things).

For example GradFun3 with smode=3 is the same code as above but with a bunch of smoothgrad() lines in between with varying settings. So I guess smoothgrad is also some kind of a very lowpass denoiser.

If you never filter in 16 bit, use gradfun3 to deband. If instead you filter in 16 bit (denoisers, resizers, etc) then before ditherpost add a smoothgrad() line, and I think you are good to go, I do this oftenly.

[TheProfosist]

If instead you filter in 16 bit (denoisers, resizers, etc) then before ditherpost add a smoothgrad() line, and I think you are good to go, I do this oftenly.
This is what I have been doing. Thanks for the info.

[SSH4]

This is a planned feature. In the meantime… here is a quick and dirty hack:

# Assume stack16 clip here

n = 10 # Final bitdepth (range: 8 to 14)
msb = Dither_get_msb ()
lsb = Dither_get_lsb ()
hr = 2
m = String (Pow (2, 16 - hr - n) - 1)
l = msb.mt_lut ("x "+m+" &u", y=3, u=3, v=3)
h = msb.mt_lut ("x 255 "+m+" - &u", y=3, u=3, v=3)

s1 = String (n - 8)
ofs = String (Pow (2, 7 - hr))
a = StackVertical (l, lsb)
a = a.Dither_lut16 ("x "+s1+" <<u "+ofs+" 256 * +", y=3, u=3, v=3)

b = a.DitherPost ()

s2 = String (16 - n)
ofs2 = String (Pow (2, 15 - hr - n))
b_msb = b.mt_lut ("x "+s1+" >>u 255 &u "+ofs2+" - 128 +", y=3, u=3, v=3)
b_lsb = b.mt_lut ("x "+ofs+" - "+s2+" <<u 255 &u", y=3, u=3, v=3)

c = mt_adddiff (h, b_msb, y=3, u=3, v=3)
StackVertical (c, b_lsb)

Dither_convey_yuv4xxp16_on_yvxx ()

how about this kind dither LSB from 8bit to 2bit (16bit to 10bit)


# Assume stack16 clip here

a=Dither_get_lsb()
b=Dither_get_msb()
a.mt_lut("x 6 >>",u=3,v=3)
DitherPost (last,a,mode=6, u=3, v=3)
mt_lut("x 6 <<",u=3,v=3)



And why ampn not work on mode 2-5 ?

And what can give better encoding result (quality/size on same CRF) on x264-10bit encode:
• dithered (ordered or floid steinberg + static noise) 8-bit source;
• 16-bit (MSB/LSB)
• 10-bit dithered (probably with static noise too) from 16-bit
?

[cretindesalpes]

how about this kind dither LSB from 8bit to 2bit (16bit to 10bit)
Yes it works too, but I think you omitted a StackVertical() at the end. Actually I developed the more complex solution to ensure the quantized values can spread across the 2-bit levels, if required (but it's not perefect). Extreme example (http://screenshotcomparison.com/comparison/88852) using ampo=4. Also, significant improvement can be done by shifting the LSB before dithering. Example (http://screenshotcomparison.com/comparison/88862) showing the amplified difference between a 16-bit gradient directly dithered to 8 bits and the same gradient using the 10-bit intermediate. Here is a fixed version:
a=Dither_get_lsb()
b=Dither_get_msb()
c1 = a.mt_lut("x 6 >>",u=3,v=3)
c2 = a.mt_lut ("x 2 << 255 &u", u=3, v=3)
DitherPost (c1,c2,mode=6, u=3, v=3)
mt_lut("x 6 <<",u=3,v=3)
StackVertical (b, last)

and why ampn not work on mode 2-5 ?
Because the modes 2–5 use a piece of code different from mode 0–1 and modes 6–8, and I was too lazy to add the noise stuff to it. Moreover, I now consider these modes obsolete. The patterns for values +1/4 and +3/4 look less resilient to compression than ordered dithering, although they may be perceptually a bit better (but this is questionable). I might remove them later to simplify further code maintenance, if nobody complains.

Edit:

And what can give better encoding result (quality/size on same CRF) on x264-10bit encode:
• dithered (ordered or floid steinberg + static noise) 8-bit source;
• 16-bit (MSB/LSB)
• 10-bit dithered (probably with static noise too) from 16-bit
?
I haven't a firm answer to this question, one should conduct tests with different types of content. I would think noise/grain of very low level could play a key role between choosing dithering to 8 or 10/16 bits. Compression size drops drastically when the picture looks almost stable and clean after dithering, with distinct patterns on flat areas, compared to more obvious grain.

[SSH4]

nice.


#put here any source with banding :)

n=6 # bit dithering
o=String(8 - n) r=String(n)

a=last
d=a.mt_lut("x " + o + " >>",u=3,v=3)
a.mt_lut("x " + r + " << 255 &u", u=3, v=3)

b=DitherPost (d,last,mode=6,ampn=1,staticnoise=true, u=3, v=3).mt_lut("x " + o + " <<",u=3,v=3)

Interleave(a.Subtitle("orig"),b.Subtitle("dither"))

[SSH4]

oops. why src_left not work right for me (on all version dither.dll)?

http://i30.fastpic.ru/big/2011/1023/65/_2b4cda3e71e3d176768bf917f8406465.png?noht=1

Dither_resize16 (1920,1080,src_left=9,src_width=-7,kernel="bicubic",a1=0,a2=0.33)

while BicubicResize(1920,1080,0,0.33,src_left=9,src_width=-7) work right on already dithered source.

[cretindesalpes]

You're right, I spotted the bug a few days ago and it is now fixed, along with some other bugs related to Dither_resize16 and extreme horizontal/vertical resizing ratios. I need to do a bit more testing. I'll make a new release as soon as possible.

[SSH4]

May i be a beta tester? :) One my project stalled because i can't out for this moment 1080p in 16/10bit.

And here i made some tests with 16/10/8 bit output and encoding in 10-bit x264:
BD source VC1 "opening" filtered with dfttest (16bit mod) with motion compensation.

Three output:
8 bit (DitherPost(mode=6,ampn=1,staticnoise=false)
10 bit (MSB+LSB dithered with DitherPost(mode=6,ampn=1,staticnoise=false)
16 bit LSB+MSB
encoded with hq settings with 2-pass in 5000K/s and 3000K/s

16 bit 5Mb/s - http://www.mediafire.com/?3bxu032qsbcyxdx
10bit 5Mb/s - http://www.mediafire.com/?oodq8xyrz6brmcj
8 bit 5Mb/s - http://www.mediafire.com/?62jlpy7m3gclbtf
16 bit 3Mb/s - http://www.mediafire.com/?1sj5iavs85xryux
10 bit 3 Mb/s - http://www.mediafire.com/?cf9l942zmc1ecaz
8 bit 3 Mb/s - http://www.mediafire.com/?yo3m33c5mrf7m8o

And dumn i can't understand what look better. 8bit sometime looks better because grain a little bit more visible than 10Bit or 16. Sometime 10 or 16 looks better than 8 :(

[Keiyakusha]

SSH4
Just a random thought. Where is the source we can compare these results to? Also are you sure you not throwing away extra information in 10-16 bit files? For example you can't open them in avisynth like any other 8bit ones. What decoders you use - also matters.
EDIT: also it doesn't sounds right that in 8 bits grain looks better, preserving grain and dither is the point of 10 bit.

[SSH4]

hmm, you want me upload luseless HUFFYUV LSB/MSB avi in >10Gb??

avs2yuv rawpipe to x264-10bit
avs2yuv raw dither_convey_... MSB/LSB 16bit pipe to x264 raw input 16bit
and as we talk later in http://forum.doom9.org/showthread.php?p=1533008#post1533008 10bit (2bit dithered LSB) and MSB/LSB 16bit pipe to x264 raw input 16bit....

Just check results. what result look better than other, not what result better or more like original.
I understand that 16 bit without dither vs 10 bit (2-3bit dithered) or 8 bit (3-4 bit dithered) not the same sources. and probably we must use different psy-rd, tree and other settings for compare results...

But anyway. with identical encode settings in same bitrate what file looks better?
Because if i try encode in CRF 10/16 bit source beat 8bit (4-5Mb/s vs 8-10Mb/s)

[Keiyakusha]

hmm, you want me upload luseless HUFFYUV LSB/MSB avi in >10Gb??
Eh, no. I was referring to unprocessed VC1 stream. Or better to say small part of it. I just realized that you didn't tell how exactly you filtered stuff so I won't be able to replicate it anyway. But whatever.
By the way I misunderstood that by 8bit you only mean 8bit source but output from x264 is still 10 bit.
To me 16 and 10 bits looks better than 8 bits. 8 bits has more artifacts, that's all. Between 10 and 16 bits my choice is 10, because 16 is an overkill, none of them is better they just different.
EDIT: however, personally I use different deband/dither so I'm not sure which one (10 or 16) bits consumes less resources in your case. If that's 16bit one, then my choice is 16bit.
BTW, I'm talking about 5mbps samples, all 3mbps looks not good to even compare.

[SSH4]

I think you understand why x264-10 bit can made better result on 8bit stream (specially on gradients) than 8bit x264...

This is not about filtering, this question about 8/10/16 bit
So my test is compare similar source but in 8/10/16 bit (3-4bit/2-3bit/0 bit dither) and encode it in 10bit x264 only.

ok, 10bit looks more better for me too. on 8bit sometime looks like grain can help protect a little bit more textures than in 10/16bit.
But mostly clean source better in 10/16 bit because it made less artifacts.
But... clean source "CLI" ^___^ looks not real.
So my choice use 10bit with add small grain in MSB for more realistic result. And i hope i can test it on my Mushishi "remaster" :)

[Keiyakusha]

I think you understand why x264-10 bit can made better result on 8bit stream (specially on gradients) than 8bit x264...
Sure.
Yet again, If you filter stuff in 16 bit, I don't see reason to go to 8 as you seems to be doing in (8 bit (DitherPost(mode=6,ampn=1,staticnoise=false)).
But... clean source "CLI" ^___^ looks not real.
I wonder if it was really meant to look real :Р

[cretindesalpes]

Dither 1.12.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed a bug with Dither_resize16, occuring with positive values for src_left or src_top.
Fixed other issues in Dither_resize16, related to huge differences between vertical and horizontal resizing ratios. It's not perfect yet, but still much better than before.
Dither_convert_yuv_to_rgb now supports RGB24 output colorspace.
Simplified Dither_crop16 for pure horizontal cropping.

[SSH4]

cretindesalpes Thank you!

Sure.
Yet again, If you filter stuff in 16 bit, I don't see reason to go to 8 as you seems to be doing in (8 bit (DitherPost(mode=6,ampn=1,staticnoise=false)).


Yep, i think you right.

But:
• x264 just clip 6 bites from 16 bit, without any dither.
• 8bit data because of rounding in the encoding process take 10bit values.
• Most of sources i ever seen have or need some grain.

So i try find best workflow for me.
Original style avisynth 8bit, or avialable with dither.dll 16bit LSB/MSB, but more difficult to implement.

[Dogway]

is it possible to add u and v to dither_merge16_8?
Also is there a way to extract a plane of a 16 bit clip? I suspect this involves using luts functions...

[cretindesalpes]

Yes, I'll probably rewrite this function as a native plugin, and then I will add the y, u and v plane selectors.

But what do you mean "extract a plane"?

[Dogway]

The idea came from the Color Vibrance thread (http://forum.doom9.org/showthread.php?t=162882)
Vtoy(),Utoy() and YToUV(). I didn't know how to do that in 16 bit. There might be more uses for these functions than I can think now, maybe it's not important, but I felt a bit limited when I thought about it. Same for mergechroma and mergeluma.


edit:
To put an example:

16 bit input
...
o=ditherpost(mode=6)
a=Dither_lutxy8(o,o.SwapUV(),"x 128 - 2 ^ y...",y=2,u=3,v=3)
...
16 bit output (but luma comes from a ditherpost stage)


So now I need to mix in 16 bit the luma previous to the ditherpost stage with the chroma of the dither_lutxy8().
My initial thoughts are:

Dither_merge16(last,a,mt_lut("65535",u=-65535,v=-65535),y=2,u=4,v=4)

Then I guess this can become sort of a mergeluma, mergechroma workaround for 16bit.

Greyscale or how to get Y plane:
dither_lut16("x",y=2,u=-128,v=-128)

Ditherpost if applied in this case dithers chroma too despite being flat, so one gets 129 values in UV planes.

About UtoY, VtoY, YtoUV, I have no idea, nor I know if they would be of any use in 16 bit. I'm just thinking loud I guess, so sorry for that...

[cretindesalpes]

You can use all these functions (UToY, VToY, MergeXXX, …) with 16 bit clips. Of course, for functions with multiple clip arguments, all clips must be of the same bitdepth.

[redfordxx]

Hi I am little bit confused with so many parameters with the Dither_resize16, so I ask plainly.
What should I do, when I need to do on the stacked 16bit clip exactly same thing as following does on 8bit:
mt_convolution(horizontal=h,vertical=v,total=t,...)Thank you

[cretindesalpes]

Currently, there is no direct mapping between mt_convolution and Dither_resize16. You'll have to scale the kernel coefficients manually (dividing them by t, if required). Also, it's not possible to use a different kernel for vertical and horizontal directions, so you'll have to do it in 2 passes:
Dither_resize16 (Width(), Height()/2, kernel="impulse "+h, fh=-1, center=false)
Dither_resize16 (Width(), Height()/2, kernel="impulse "+v, fv=-1, center=false)
I'll add some parameters in a next release to make Dither_resize16 more compatible with mt_convolution.

[vampiredom]

One simple thing I have done is to use FFT3dFilter, with fairly strong sigma, then blend that with the original clip to a certain percentage, like:

orig = last
filt = FFT3DFilter(sigma=2)
Merge(filt, orig, weight=0.25)

This seems to do a fairly good job of reducing visible noise while maintaining detail and avoiding banding (depending on the source, of course).

[redfordxx]

One simple thing I have done is to use FFT3dFilter, with fairly strong sigma, then blend that with the original clip to a certain percentage, like:My personal experience:
dfttest keeps more detail than dctfilter

[redfordxx]

I made a function for convolutions, so I am sharing it here. function Convolve(clip c, val hor, val ver, float "tot", bool "lsb", int "hctr", int "vctr", int "y", int "u", int "v", bool "info")
{
#Performs flat convolution of given dimensions...v0.1 by redfordxx
#needed Dither 1.12
# issues: Dither_resize16 applies kernel in reverse
# Dither_resize16 normalizes kernel, so tot parameter is ignored in 16bits
#Usage parameters:
#hor, ver arguments...dimensions of the convolution or kernels
#tot...divider, negative values are relative to sum of elements, so for 3x3 convo and tot=-2 the resul is tot=18, tot=0 is automatic. Doesnot work in 16bits
#lsb...is input stacked 16bit?
#hctr,vctr...centerpoints of the convolution string, by default in the middle...ignored if given hor and ver are kernels
#info...shows convolution strings on screen
y=default(y,3)
u=default(u,3)
v=default(v,3)
htext = IsString(hor) ? hor : ""
vtext = IsString(ver) ? ver : ""
hi = IsString(hor) ? 1 : int(hor)
vi = IsString(ver) ? 1 : int(ver)
lsb=default(lsb,false)
info=default(info,false)
horcenter=default(hctr,(hi+1)/2)
vercenter=default(vctr,(vi+1)/2)
sum=hi*vi
tot=default(tot,0.0)
tot = (tot>0) ? tot
\ : (tot<0) ? -sum*tot
\ : 0
hcoef = 1
vcoef = 1
uh = " "+ string(hcoef)
uv = " "+ string(vcoef)
uhl =strlen(uh)
uvl =strlen(uv)
flt=((int(tot)!=tot)&&(lsb==false)) ? ".0" : ""
shl=(hi-2*horcenter+1)*(lsb ? -1 : 1 ) #Dither applies convolution in reverse
svl=(vi-2*vercenter+1)*(lsb ? -1 : 1 )
strh=uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh+uh
strv=uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv+uv
str0=" 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0"
hh=leftstr(str0,max(2*shl,0))+leftstr(strh,uhl*hi)+flt+leftstr(str0,max(-2*shl,0))
vv=leftstr(str0,max(2*svl,0))+leftstr(strv,uvl*vi)+flt+leftstr(str0,max(-2*svl,0))
hh=(htext=="") ? hh : htext
vv=(vtext=="") ? vv : vtext
hvsame=(hh==vv) # when hor and vertical is same, do 16bit in one pass
c
(!lsb && tot==0) ? mt_convolution(horizontal=hh,vertical=vv,y=y,u=u,v=v) : nop
(!lsb && tot>0) ? mt_convolution(horizontal=hh,vertical=vv,total=tot,y=y,u=u,v=v) : nop
(lsb && hh!="1" && hvsame) ? Dither_resize16 (Width(), Height()/2, kernel="impulse "+hh, fh=-1, fv=-1, center=false) : nop
(lsb && hh!="1" && !hvsame) ? Dither_resize16 (Width(), Height()/2, kernel="impulse "+hh, fh=-1, center=false) : nop
(lsb && vv!="1" && !hvsame) ? Dither_resize16 (Width(), Height()/2, kernel="impulse "+vv, fv=-1, center=false) : nop
info ? Subtitle(" h=("+hh+") v=("+vv+")") : nop
info ? Subtitle(" 16bit="+string(lsb)+" total=("+string(tot)+") size=("+string(hor)+","+string(ver)+") center=("+string(horcenter)+","+string(vercenter)+") ",y=24) : nop
return last
}
I hope it is understandable what it does, so won't comment more, if you wanna know just play with parameters with info=true, see what is the difference eg on the following
convolve(5, 4, info=true)
convolve(5, 4, tot=-2, lsb=false, hctr=-3, vctr=1, info=true)
convolve(5, 4, tot=0, lsb=true, hctr=-3, vctr=1, info=true)
convolve("1 0 -1 0 1", 4, tot=0, lsb=false, hctr=-3, vctr=1, info=true)

There are two issues introduced by dither, see them in the comments
@cretindesalpes: I wonder whether these are intents or bugs
Now I believe that it was main reason why I was confused with using Dither_resize16

[vampiredom]

dfttest keeps more detail than dctfilter

Ah, interesting. I am certainly not that well-versed in all the available denoisers for AviSynth. I will have to check it out. My comment was not really intended to suggest that FFT was the best way, but rather the concept of blending the denoised output with the original video in small degrees as a method of reducing visible banding.

[cretindesalpes]

Dither_resize16 applies kernel in reverse
Dither_resize16 normalizes kernel, so tot parameter is ignored in 16bits
[...]
I wonder whether these are intents or bugs
Not really a bug nor an intent. I implemented the convolution in the simplest possible way, and didn't care about the kernel traversal sign, because resizing kernels are symmetric. I'll fix these issues in the next release.

[ajp_anton]

Avisynth open failure:
Evaluate: operands of '==' and '!=' must be comparable
(dither.avsi, line 850)
(dither.avsi, line 814)
I tried to set u=3,v=3 in dither_lut8.
In the function dither_lut8_lsb, line 850, it checks if u or v are defined, but if they are, it then tries to check what y is. But if y is not defined, it fails. You need to set y=default(y,3) somewhere.
Don't know if there are other places with this error.

Also, a feature request...
Would it be possible to convert into any colorspace by defining the matrix yourself?

[cretindesalpes]

Dither (http://forum.doom9.org/showthread.php?p=1386559#post1386559) updated to v1.13.0:

Added kernelh, kernelv, totalh and totalv to Dither_resize16. Please note that the custom convolution coefficients are now read in the correct order.
Fixed an error with the plane selectors in the Dither_lut*8 functions.
Updated MVTools to v2.5.13.1: MAnalyse, MRecalculate: added an SATD approximation for blksize > 16 and dct >= 5. Previously, the SATD calculation was silently bypassed and always returned a null SAD value. Actually this is more a mixed SAD-SATD calculation, because the SATD for large blocks is implemented as a sum of the SATD of smaller blocks, instead as a SAD of the Hadamard transform of the whole block. It was already the case for blksize=8 or above.
Functions now check that the pel setting is the same in the superclip and the motion vectors, instead of crashing if not.


Would it be possible to convert into any colorspace by defining the matrix yourself?
Do you have a specific use in mind? I could add other standard matrices, if required. Or give control over the kr, kg and kr parameters. But defining the 9 matrix coefficients + 3 constants would be a bit more complex.

[ajp_anton]

Well I was thinking of YCgCo, but then thought there might be others so a more general solution might be possible.

[Dogway]

I'd like to ask if currently the staticnoise option of ditherpost works like the wOxxOm workflow (http://forum.doom9.org/showpost.php?p=1517049&postcount=241), or instead noise is spread evenly in the image. Ideally one would want noise only where ditherpost is dithering more aggresively.

I also noticed gradfun3 needs the staticnoise option. It could be a RAM problem but gradfun for smode=2 at least feels very unstable I usually have crashes, raising ampo over 1 also makes it unstable but I'm not sure which one is the trigger. My tests are in 16bit pipeline. I'm tackling fine banding here.
GradFun3(0.6,14,lsb_in=true,smode=2,lsb=true)
ditherpost(mode=0,ampn=0.5,staticnoise=true,ampo=1.2)

BTW I'd like to know if it's ok to use ampn with ampo at the same time when using ordered dither as in the example.

Another idea I had is to use floyd-steingberg on detailed areas to shoot for accuracy, but ordered dither on low gradients, if it leaves banding then use the staticnoise as mentioned. The drawback is having to use 2 ditherposts. I don't know if mixing 2 different algos harms too much.

At last to confirm something I didnt get clear:
(interlaced source)
separatefields
dfttest(lsb=true,tbsize=1)
ditherpost(mode=6,interlaced=true)
weave
Is this ok?

[cretindesalpes]

I'd like to ask if currently the staticnoise option of ditherpost works like the wOxxOm workflow (http://forum.doom9.org/showpost.php?p=1517049&postcount=241), or instead noise is spread evenly in the image. Ideally one would want noise only where ditherpost is dithering more aggresively.
No, currently the noise (static or not) is global. If the mask is active, only the non-edge area will be debanded and dithered, whatever the noise settings. The function is very basic actually.

I also noticed gradfun3 needs the staticnoise option. It could be a RAM problem but gradfun for smode=2 at least feels very unstable I usually have crashes, raising ampo over 1 also makes it unstable but I'm not sure which one is the trigger. My tests are in 16bit pipeline. I'm tackling fine banding here.
GradFun3(0.6,14,lsb_in=true,smode=2,lsb=true)
ditherpost(mode=0,ampn=0.5,staticnoise=true,ampo=1.2)
I couldn't reproduce any crash here. Could you post the full script? What is the size of your input clip? Which Avisynth version do you run?

BTW I'd like to know if it's ok to use ampn with ampo at the same time when using ordered dither as in the example.
Yes, the noise will just add to the dither pattern before bit reduction.

Another idea I had is to use floyd-steingberg on detailed areas to shoot for accuracy, but ordered dither on low gradients, if it leaves banding then use the staticnoise as mentioned. The drawback is having to use 2 ditherposts. I don't know if mixing 2 different algos harms too much.
There shouldn't any problem with this.

At last to confirm something I didnt get clear:
(interlaced source)
separatefields
dfttest(lsb=true,tbsize=1)
ditherpost(mode=6,interlaced=true)
weave
Is this ok?
Noting that tbsize=1, yes it should work but Weave() before DitherPost(), that's how it is intended to work in interlaced mode.

[cretindesalpes]

The Dither package (http://forum.doom9.org/showthread.php?p=1386559#post1386559) has been updated to v1.13.1:
Fixed height checks in DitherPost with interlaced content.
Updated dfttest to v1.9: added the quiet parameter to deactivate the filter spectrum output when sigma is specified with sstring/ssx/ssy/sst.
Updated MVTools to v2.5.14.0: Added MStoreVect, MRestoreVect and MScaleVect, from the Vit's MVExtras plugin. Function names have been changed to avoid collisions with MVExtras.

[Nevilne]

This is very sweet!
Seeing as vector scaling is being developed again, can I ask if this is possible:

Manalyze on half size clip with pel=1
Mflow/blur/whatever on full clip with pel 2 or 4

I've tried tinkering with adjustSubPel and other settings but there seems to be no way,
and using pelsearch=0 + blank pel clip is not very clean.

Also, vector scaling doesn't work with divide>0.
I didn't want to bug Vit with this but since you've included vector scaling it would be great if you could look into it :o

[-Vit-]

Manalyze on half size clip with pel=1
Mflow/blur/whatever on full clip with pel 2 or 4
That's the wrong way round. The idea behind adjustSubPel is that vectors measured in half-pixels (pel=2) on a half-size clip can be left unchanged and treated as measured in full pixels (pel=1) on a full size clip. Only the pel setting in the vectors changes. I was experimenting with this before I had to stop avisynth development for more important things. My initial observations were that it worked, but did not provide any advantage over scaling the vectors and leaving the pel alone.

Maybe you had a different idea...?

Also, vector scaling doesn't work with divide>0.
Yes, this is true and should be added to the documentation. I'd had enough after deciphering the internal layout/format of standard multilevel blocks/padding. I guess I'll add that final (rarely used) layer at some stage.

[Dogway]

I had a look to your workaround (http://forum.doom9.org/showthread.php?p=1505297#post1505297)to make yuy2 colorspace sources be processed by ditherpost, but I'm having a hard time to understand how it works.

Do you apply that code to an 8 bit interleaved yuy2 ?

I'm testing by comparing the next to its yv12 counterpart:

(interleaved 8 bit yuy2 source)

y = ConvertToYV12 ()
u = UToY ().ConvertToYV12 ()
v = VToY ().ConvertToYV12 ()
YToUV (u, v, StackVertical (y, y))

dfttest(lsb=true)
ditherpost(mode=0)

y = Crop (0, 0, 0, Height () / 2).ConvertToYUY2 ()
u = UToY ().ConvertToYUY2 ()
v = VToY ().ConvertToYUY2 ()
YToUV (u, v, y)

But there's something different in the chroma planes so I wouldn't call that lossless, maybe I'm doing something wrong?

[cretindesalpes]

Yes this code applies to standard YUY2. It works by copying the chroma planes from the YUY2 clip to the YV12. Therefore the overall height must be doubled, so I have to stack two original luma planes. The filtering will work only if each plane is processed separately because chroma and luma are not synchronized. Fortunately, this is the case with dfttest and DitherPost.

If you compare the result with a YV12 source, it's normal you get something different on the chroma planes: regular YV12 to YUY2 conversions are not lossless because of the chroma placement and the interpolation method. If you bypass the dfttest/DitherPost lines in your code, you should obtain the exact source clip.

[Dogway]

I almost finished implementing it to my script but there's something that doesn't work.
separatefields
interleaved2planar
super_search=MSuper(planar=true)
bv2=super_search.MAnalyse(isb = true, delta = 2)
fv2=super_search.MAnalyse(isb = false, delta = 2)
MDegrain1(super_search, bv2, fv2, thSAD=300,planar=true,lsb=true)

It only happens when I use separatefields with lsb=true. Even setting assumeframebased doesn't fix it and always throws an error. This is the base for interlaced YUY2 denoising but I don't know what else to try.

[cretindesalpes]

Indeed, there was a bug with planar=true and lsb=true. Fixed in Dither 1.13.2 (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

[Stephen R. Savage]

cretindesalpes,

Where are these custom versions of dfttest and mvtools you use coming from? I can't find any information about a dfttest 1.9 or mvtools 2.5.14. Can you provide a link to the original versions of these filters or are they solely your creation?

[Dogway]

It works like charm now! I forgot to ask, do you think if by adding a blankclip to the below stacked luma can save processing for ditherpost?
YToUV (u, v, StackVertical (y, blank_clip))

[cretindesalpes]

Where are these custom versions of dfttest and mvtools you use coming from? I can't find any information about a dfttest 1.9 or mvtools 2.5.14. Can you provide a link to the original versions of these filters or are they solely your creation?
Yes, these are my own updated versions of these tools. It seems they are no longer updated by their authors (excepted occasional bugfixes for mvtools) so I took the liberty to continue their development. I hope it doesn't hurt anyone.

do you think if by adding a blankclip to the below stacked luma can save processing for ditherpost?
No, I don't think it will change anything.

[mastrboy]

Do the modified MVtools in the dither package contain the threading fixes from the QTGMC modded plugins package by -Vit- ?

Little annoying that there are multiple forks of mvtools2 out there now, the official one, one from the dither package, one from QTGMC and one from www.svp-team.com :(

[cretindesalpes]

Yes, this version has the QTGMC threading fixes.

[mastrboy]

Thanks for answering so fast and that's good to hear :)

(You should probably update your post: http://forum.doom9.org/showthread.php?p=1386559#post1386559 so anyone else looking for the same info don't have to browse 18 pages...)

[-Vit-]

Little annoying that there are multiple forks of mvtools2 out there now, the official one, one from the dither package, one from QTGMC and one from www.svp-team.com :(
I'll remove my version from the QTGMC thread and use cretindesalpes' version for the modded plugins shortly. One down...

[mastrboy]

Good idea. A second idea to narrow it down further would be if both of your changes would be merged with the official release... Fizick seems to be the only "active" developer according to the changelog at least, hopefully he would read this post and accept your changes for 2.5.11.4.. :P

[SSH4]

cretindesalpes Is it possible add to dfttest (or may be to mdegrain) option like sstring(ssx/y/z) for change sigma depending on luma? I know different ways for simulate this (use luma as mask and merging filtered and original not the best one) but all of them require two passes or more of dfttest or mdeg :(

[Dogway]

Could you confirm one thing?. Motion vectors are very different when you take into account chroma between yv12 and yuy2 (planar) sources.
Since chroma is subsampled in yv12 I guess it is upsampled before or after calculating its motion vectors to comply to luma motion vectors.
Does this happen according to chroma in YUY2 sources taking into account the way chroma is stored in this format (when planar=true)?

And since I'm here, would you recommend mod16 inputs for more correct SAD calculation?

[cretindesalpes]

Since chroma is subsampled in yv12 I guess it is upsampled before or after calculating its motion vectors to comply to luma motion vectors.
All SAD calculations are scaled depending on the plane subsampling. For example, if you specify blksize=16, luma SAD part will be calculated on 16x16 blocks, 4:2:2 chroma on 8x16 blocks and 4:2:0 chroma on 8x8 blocks.

Motion vectors are very different when you take into account chroma between yv12 and yuy2 (planar) sources.
How different? Do you think there is a bug related to YUY2?

And since I'm here, would you recommend mod16 inputs for more correct SAD calculation?
I don't think using mod16 (or more likely a mod-overlap size) does any significant difference, because motion estimation is always funny on the frame borders.

Is it possible add to dfttest (or may be to mdegrain) option like sstring(ssx/y/z) for change sigma depending on luma?
It looks possible at the first glance; I made a note in my to do list. An additional improvement would be using the pixel values of a second clip as sigma multipliers. For MDegrain, you can obtain an equivalent effect by changing the luma curve of the search clip. What I usually do: prepare the search clip using dfttest(lsb=true) with basic MC, Dither_lut16() to apply a gamma-like curve and make it full range, convert it back to 8 bit using DitherPost(mode=-1) and feed MSuper (for the final MAnalyse only, not the actual filtering) with it. I think you could do a similar approximation with dfttest by applying a curve on the input clip luma (result in 16 bits to keep accuracy), filter with lsb=true, lsb_in=true then by applying the inverse function. Obviously, it would affect only the luma plane, the chroma plane filtering would be kept unchanged whatever the corresponding luma value.

Fizick seems to be the only "active" developer according to the changelog at least, hopefully he would read this post and accept your changes for 2.5.11.4.. :P
He wasn't interested when I asked him.

[Dogway]

I'm not very sure that's why I ask you if you could have a look. On a simple temporal radius of 1 there is this such difference (in the images below). If I set chroma false both output are 100% identical. Maybe this is just a matter of chroma being in different formats but difference catched my attention enough to not mention here.
planar=true
chroma=true
plane=0

planar ? interleaved2planar : last
super_search=MSuper(planar=planar,chroma=chroma?true:(plane==0?false:true))
bv2=super_search.MAnalyse(isb = true, delta = 1,chroma=chroma)
fv2=super_search.MAnalyse(isb = false, delta = 1,chroma=chroma)
MDegrain1(super_search, bv2, fv2, thSAD=300,planar=planar,lsb=false,plane=plane)
planar ? planar2interleaved : last
http://img849.imageshack.us/img849/6548/yv12.th.png (http://img849.imageshack.us/img849/6548/yv12.png)http://img190.imageshack.us/img190/6876/yuy2.th.png (http://img190.imageshack.us/img190/6876/yuy2.png)

edit:
...Dither_lut16() to apply a gamma-like curve and make it full range...
Very good idea! Do you also degamma? I'm not sure if this is preferred.

It also would be quite useful to add an interlaced parameter to manalyse for when multi=true, so interleaved delta order becomes ...-6,-4,-2, 2, 4, 6...

In another matter, I realised some time ago that in order to create motion vectors you don't need thSAD, this is only required for MDegrain (and well mrecalculate...), maybe the chroma smearing could be reduced by playing with plane and thSAD parameters running 2 instances of MDegrain but using the same vectors... since you know the internals of mvtools2 could you confirm this?

[Yellow_]

Is it possible to blend with float values and if so with values greater than 0.0 to 1.0 with Dither tools?

If I have two frames of video (YV12) of the same scene but with two exposures say one is 1 EV greater than the other, ie: twice as bright. Is it possible to blend them into a 16bit linear light frame with brightness values representing 0.0 - 2.0 and so on for wider EV spacing.

[Stephen R. Savage]

Is it possible to blend with float values and if so with values greater than 0.0 to 1.0 with Dither tools?

If I have two frames of video (YV12) of the same scene but with two exposures say one is 1 EV greater than the other, ie: twice as bright. Is it possible to blend them into a 16bit linear light frame with brightness values representing 0.0 - 2.0 and so on for wider EV spacing.

Pixels are just numbers; you can assume them to have whatever intervals you want. For example, there's nothing stopping you from averaging two values on the interval of [0.0 - 1.0] and calling the new value [0.0 - 2.0]. That said, you obviously can't get a display to show values greater than 100% brightness.

Pseudo-code:

a = Source1.ConvertTo16Bits().GammaToLinear()
b = Source2.ConvertTo16Bits().GammaToLinear()
Average(a,b)
MoreCode()


These values are nominally on the interval [0.0 - 1.0], but since they are linear values, there's no difference from a [0.0 - 2.0] interval with 15-bit precision.

[Yellow_]

Pixels are just numbers; you can assume them to have whatever intervals you want. For example, there's nothing stopping you from averaging two values on the interval of [0.0 - 1.0] and calling the new value [0.0 - 2.0]. That said, you obviously can't get a display to show values greater than 100% brightness.

Pseudo-code:

a = Source1.ConvertTo16Bits().GammaToLinear()
b = Source2.ConvertTo16Bits().GammaToLinear()
Average(a,b)
MoreCode()




Ok, I've found the AverageM.dll and discussion here:

http://forum.doom9.org/showthread.php?p=726930#post726930

These values are nominally on the interval [0.0 - 1.0], but since they are linear values, there's no difference from a [0.0 - 2.0] interval with 15-bit precision.

Ok, :-) I'm working within Dither Tools stacked 16bit functions and blending like this:

Blending 8-bit pictures in linear light:

# Blending amount for the first clip
bl = 0.75
bls1 = String ( bl)
bls2 = String (1 - bl)

# 8-bit clips converted to linear 16-bit full range (gamma undone)
ug = " 16 - 0 max 1.41624 / 2.2 ^ "

# Redo the gamma, result in 16 bits YUV
rg = " 0.454545 ^ 362.5585 * 4096 +"

# Blend
Dither_lutxy8 (src1, src2,
\ expr ="x " + bls1 + " * y " + bls2 + " * + 256 *",
\ yexpr="x" + ug + bls1 + " * y" + ug + bls2 + " * +" + rg,
\ y=3, u=3, v=3)



Then I'm using Avs2yuv (really should be using Avs2pipemod) to export raw 48bit rgb to imagemagick like this:

Dither_convert_yuv_to_rgb (lsb_in=true, output="rgb48y")
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convey_rgb48_on_yv12 (r, g, b)

And writing 16bit OpenEXR's from IM for each exposure, but as EXR supports HDR I would like to merge the two exposures from each pair frames into each OpenEXR image for Tonemapping externally rather than export 8bit tiffs, jpg's or pngs and then manually set the EV spacing for them in a HDR app like Photomatrix or psftools to export HDR formats for tonemapping. Too many steps for video. :-)

But I'm unsure how to 'position' the what were 'normalised' video frames shot at various EV's (8bit 0 - 255 -> Dither Tools to 16bit), for piping the raw rgb data into IM with EV spacing they were shot at. ie: There are options in camera to shoot at 1 EV ie twice bright, up to 5EV.

Sorry if it sounds convoluted or plain daft. :-) Not sure if Average is the tool in this scenario?

[mandarinka]

For MDegrain, you can obtain an equivalent effect by changing the luma curve of the search clip. What I usually do: prepare the search clip using dfttest(lsb=true) with basic MC, Dither_lut16() to apply a gamma-like curve and make it full range, convert it back to 8 bit using DitherPost(mode=-1) and feed MSuper (for the final MAnalyse only, not the actual filtering) with it.

Could you post a code snippet for that, if it isn't a problem?
Mainly the Dither_lut16() part, I'd love to see the formula you use... I've been interested in luma-sensitive mdegraining for quite some time, but my own solution is crude at best (weighting together two clips...)

[cretindesalpes]

Mandarinka:

# 16-bit TV-range -> full range with extra curvature
# s0 = new slope at 0
# c = length of the curved part, > 0, no specific unit
#
# Matlab code:
# c = 1/16;
# s0 = 2;
# x = linspace (0, 1, 10001);
# y1 = (x * 219 + 16) / 256; % Scale [0 ; 1]
# t = ((y1*256 - 16) / 219);
# k = (s0-1)*c;
# y2 = ((1+c) - (1+c)*c ./ (t + c)) * k + t * (1 - k);
# plot (x, y1, x, y2); grid on;
Function fslg_remap_luma_search_clip (clip src, float "s0", float "c")
{
s0 = Default (s0, 2.0)
c = Default (c, 1.0/16)

k = (s0 - 1) * c
t = "x 4096 - 56064 / 0 1 clip"
e = String(k)+" "+String(1+c)+" "+String((1+c)*c)+" "+t+" "+String(c)
\ +" + / - * "+t+" 1 "+String(k)+" - * + 65536 *"

src.Dither_lut16 (
\ yexpr=e,
\ expr="x 32768 - 32768 * 28672 / 32768 +",
\ y=3, u=3, v=3
\ )
}

[mandarinka]

Thanks!

[Dogway]

if I set s0 to 1 the result is the same as if I did ((x-16)/219)*255 so s0 is to revert source gamma or just to make the luma tv2pc range process gamma aware? Wouldn't be 2.2 more appropiate? and what is c, kind of a multiplier?

[cretindesalpes]

No, this is not a gamma function, and actually it makes the data going even further from the linear-light scale. The purpose is to amplify the scale of the dark parts relative the the other parts. So it acts like a luma-based multiplier on the SAD. With a slope s0 = 2.0, thSAD for the darkest parts is implicitly divided by 2. The c parameter indicates the length of the transition between the dark and normal SAD.

Here are some graphs to show how the curve looks for different s0 and c values:

http://img20.imageshack.us/img20/3832/s0c.th.png (http://img20.imageshack.us/img20/3832/s0c.png)

Blue = original luma, green = mapped luma

Note that for high c or s0 values, the brightest parts have also their thSAD modified (in the other direction), because the slope of the curve near 1 tends to flatten.

[Dogway]

Very explicative! so you just supply the knots for curve "free playing" (while mapping tv to pc)
So I assume it can undo gamma? possibly with s0=2.2,c=1.0?

By doing that you miss the advantage of tweaking the darks in the same filter, although I always thought that to ungamma was sufficient for manalyse to not underperform in dark parts. YET human eyes are very sensitive to low luma levels and thus films and such have a notable bias for obscure scenes so it really might be necessary to have an special care in these parts.

I would have thought something ideal as:
tv to pc->ungamma->dark enhacement

Possibly is to ask too much but having an equal thSAD weighting through the whole luma range as a base for dark enhacement isn't such a bad idea.
Thanks for the functions everyday is Xmas day : P

[cretindesalpes]

Very explicative! so you just supply the knots for curve "free playing" (while mapping tv to pc)
So I assume it can undo gamma? possibly with s0=2.2,c=1.0?
As I said it cannot undo gamma, it's not a "power" function, moreover the general curve is inverted (so it's more like additional gamma). It is intended to be applied to the motion search clip only, not the real clip, so after motion estimation and SAD calculation, the dark parts of the real clip will be less filtered (similar to a thSAD reduction) than the bright ones.

—

On a completely unrelated note, I updated this script (http://forum.doom9.org/showthread.php?p=1519194#post1519194). It's helpful to dither a 16-bit clip to 10 bits and make x264-10bit encode the correct values, without requiring a patched build.

[Yellow_]

On a completely unrelated note, I updated this script. It's helpful to dither a 16-bit clip to 10 bits and make x264-10bit encode the correct values, without requiring a patched build.

Could you just clarify a little more? I'm a bit confused. :-)

Are you saying 16bit from previous version(s) of Dither tools to 10bit vanilla x264 encoder results is slightly greenish output?

And as such it was necessary to do some hack if not using a patched x264 encoder build?

But that previous hack is now not needed due to a new function in Dither , which is less of a hack but still function needs to be added to our scripts?

That when going from Dither tools 16bit to patched 10bit build nothing is needed?

[sneaker_ger]

Currently the 10 bit vanilla builds of x264 will convert 10 bit input to 16 bit and then dither down to 10 bit again. Some patched builds (like the newest from JEEB or Taro's) can skip this unnecessary, potentially harmful/unwanted, conversion.
I guess that script uses the inverse function of vanilla x264's 10 bit -> 16 bit -> 10 bit conversion, so that the conversion can do no harm and you can get the same result with a vanilla build as you would get with one of the patched builds. I don't know where you got the "greenish" idea from?

[cretindesalpes]

@Yellow_:

Yes vanilla x264 now scales and dither 16-bit input. However I don't know when it started to do this. I think the hacked x264 builds were mostly concerned about the 8-bit issue which was there from the beginning. Anyway the 16-bit path in the JEEB or 06_taro builds should also be free of unnecessary scaling.

@sneaker_ger:

Actually the 10->16->10 bit conversion is neutral. It's probably a (minor) waste of processing power, but it seems to work as expected. See the discussion here (http://x264.fushizen.eu/?p=393#comment-5113).

[Yellow_]

sneaker_ger the greenish tint was from the thread cretindesalpes pointed to via #377.

Thanks to you both for the explanation.

[sneaker_ger]

@sneaker_ger:

Actually the 10->16->10 bit conversion is neutral. It's probably a (minor) waste of processing power, but it seems to work as expected. See the discussion here (http://x264.fushizen.eu/?p=393#comment-5113).

Ah, I remember reading that in the changelog, but I didn't understand it at that time. So what's the point of using that script (aside from lowering processing power a bit)? Or was 10 bit output not supported at all prior?

[cretindesalpes]

My previous script had 10-bit dithering with 16-bit output, based on the assumption that the 6 lowest bits would just be dropped or rounded to the nearest value when entering x264.

[Boulder]

Does MScaleVect work with the vector clip created by using multi=true in MAnalyse? The docs don't say anything about that, but I assume that it works as if the source clip was a regular clip containing one set of vectors.

[mp3dom]

I'm asking just to avoid further problems and to be sure...
If a progressive 16bit YV16 stacked clip needs to be converted to a 16bit YV12 stacked clip, can I safely use a plain ConvertToYV12() or do I need to pay attention to something (mainly the LSB part)?
Thanks

[cretindesalpes]

Does MScaleVect work with the vector clip created by using multi=true in MAnalyse?
Hmm… I'm checking the code, and it might work, but there are things that must be fixed to be safe. Actually the flip parameter will not work at all.

If a progressive 16bit YV16 stacked clip needs to be converted to a 16bit YV12 stacked clip, can I safely use a plain ConvertToYV12() or do I need to pay attention to something (mainly the LSB part)?
Your guess is right, it will not work out of the box. The chroma planes will be resampled, so they definitely have to be processed in 16 bits. I'll probably submit soon a new function to convert between the YUV modes.

[Yellow_]

In the bit depth increase with Dither tools is it possible to 'extract' xvYCC extended gamut from suitable sources to 16bit RGB. As explained here: http://sutlib2.sut.ac.th/sut_contents/H95009/DATA/5637_2.PDF

[Overdrive80]

Hi, is possible to use MT with this filter??? I tried but show error that says; try vertical.....

[Stephen R. Savage]

Hi, is possible to use MT with this filter??? I tried but show error that says; try vertical.....

Internally, before each operation, Dither has to unpack a stacked frame into the interleaved version. Therefore, you need to use the "splitvertical" option in MT that processes the left+right sides of the frames simultaneously instead of the top+bottom. This is a terrible design decision in Dither and results in probably 10-20% performance loss shuffling bits around...

[mp3dom]

Your guess is right, it will not work out of the box. The chroma planes will be resampled, so they definitely have to be processed in 16 bits. I'll probably submit soon a new function to convert between the YUV modes.

Thanks for the explanation. On the contrary, may I assume that I can safely convert YV16 to YUY2 back and forth without too much trouble? They should be equal (apart the former being planar and the latter interleaved) and lossless.

[Overdrive80]

Internally, before each operation, Dither has to unpack a stacked frame into the interleaved version. Therefore, you need to use the "splitvertical" option in MT that processes the left+right sides of the frames simultaneously instead of the top+bottom. This is a terrible design decision in Dither and results in probably 10-20% performance loss shuffling bits around...

Thanks for your answer. It's a shame to lose that speed of process.

@Cretindealpes

With this script
dfttest(tbsize=1,sigma=10,lsb=true)

LSharpAAF(aatype="supaa") #With any aa success the same

gradfun3(smode=1,lsb_in=true,mode=0,ampo=1.2)

I get this in bottom image:

http://s17.postimage.org/yxeg24im7/snapshot.png

[cretindesalpes]

Boulder:

Dither 1.13.3 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released.
I think I fixed the MScaleVect + multi issue but I couldn't test all the possible cases. Send feedback if something goes wrong.

Yellow:

Sounds complicated but doable. It would require a custom source plug-in.

Stephen R. Savage:

The actual reason is that MT() requires a constant clip height for horizontal division, and a constant clip width for vertical division. Anyway, the Dither tools have been designed with heavy filtering in mind (dfttest, motion compensation, bilateral filtering, etc). The overhead caused by the stack16 format is probably several orders of magnitude lower than the load of the whole filter chain, so I don't really care. Clearly not perfect from the programmer's point of view, but much more user-friendly than a straightforward interleaved format.

mp3dom:

Yes, the 16-bit conversion between YV16 and YUY2 is lossless. Regarding your previous request, the YV16-YV12 converter is a bit complicated to design (like the YUV-RGB conversion was) and I haven't much time to spend on it. Unfortunately it won't be ready "soon", contrarily to what I said previously.

Overdrive80:

The result of your dfttest command is a stack16 clip. If you display it, you'll find that the bottom part of the picture is full of colored garbage. So if you try to apply an anti-aliasing filter on it, both parts of the picture will interfere. Moreover, there is a risk of messing up the pixel MSB/LSB consistency. Try to apply the AA filter in the 8-bit land, before dfttest or after GradFun3. If you really want to apply the AA filter on the stack16 clip, see post #250 (http://forum.doom9.org/showpost.php?p=1518953&postcount=250), but it looks overkill to me just for one filter, I wouldn't recommend that in this situation.

[Boulder]

Boulder:

Dither 1.13.3 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) released.
I think I fixed the MScaleVect + multi issue but I couldn't test all the possible cases. Send feedback if something goes wrong.Thanks, I'll test it as soon as I have the time.

[Overdrive80]

@Cretindealpes

Thanks, i will try after gradfun3. ^^

[cretindesalpes]

For unpatched 10-bit x264 builds, straight 16-bit input will cause errors, unless you have scaled the original data in the inverse way x264 does. For 8-bit input, I don't know if things have changed or not (likely not).

Dither_convert_8_to_16 and other modified filters just shift bits to produce 16 bit output (an extension of the Rec 709 method to convert between 8- and 10-bit data), which is not the way x264 works. Therefore, to encode 10-bit video, you'll have to provide x264 with 10-bit data to be sure it won't rescale them. So the fslg_dither_to_n_bits script (or any equivalent) is required if you want to preserve the exact data scale. If you don't care about pre-dithering to 10 bits, the script becomes much simpler:
Dither_lut16 ("x 64 / 0.5 + 1023 min", y=3, u=3, v=3)
Dither_convey_yuv4xxp16_on_yvxx ()

[cretindesalpes]

If I understand correctly the x264 conversion code, the maximum value of the input scale (65535) is mapped onto the maximum value of the output scale (1023). So the error is roughly proportional to the value and remains below 1 quantum of the lowest bitdepth, in absolute value. That is -0.1 % for 10 bits.

[Keiyakusha]

It is not directly related to this thread but does anyone knows a way to open 16bit image sequence (png or tif) in avisynth for further processing? I imagine it should be opened as stacked YV24. (YV12 is a no go, I don't want subsampled chroma, upscaling image 2x with point resize before import is a no go too - they'll be too big, I don't have space or time for this.). So far with hacked ffms2 I only managed to open one image (not sequence) as stacked yv12. But PNG showed some corruption (tif is fine) and anyway it was far from what I need...

[atra dies]

Pardon my inexperience but I have several important questions:

What operations are more correctly done in linear scale light:
downsizing I assume, denoising (mdegrain)?, smoothlevels 16bit (altering y levels)? I remember someone said anything that averages pixels in another thread. I put mdegrain between a linear/gamma conversion and it looked OK although I did catch a blurring in some hay during a fast camera pan.

And when using bilinear to downsize, because every other filter rings or has a dark line around edges, is the lowpass an accurate calculation obtaining the highest sustainable frequency without aliasing or is it a conservative generic value that is blurring the image? I used fh and fv of 1.3 and things looked much better without much aliasing if at all. I know spline was much sharper but as I said there were dark lines around edges. I understand there will be undesirable things happening in certain patterns like brick or criss crossed clothing designs and have seen this in some modern dvds.

If converting yv12 to yuy2 with the bug fix script in 2.5.8, does the bilinear lowpass here or any other time when upsizing? I would assume it's just the downsize that the lowpass is applied but considering some bugs that have come to light like mpeg1 chroma placement, I am paranoid.

This lowpass is why everyone rags on bilinear but I find the spline useful only if I want to sharpen my clips and I usually don't like the outcome of sharpening, even LSF, when I view the final dvd. I probably oversharpened as a little goes a long way as far as that's concerned. For awhile there I was using spline48 then 144 and I do not like the effect those produced. Two tap spline is what I would prefer. Anything more warps some details because they are sharpened too much.

Something to watch out for: I noticed some ringing introduced by msuper when sharp is set above bilinear interpolation.

[tormento]

cretindesalpes, is it possible to have the previous version of dither? I am getting strange results with the newer and I'd like to compare them.

[cretindesalpes]

What operations are more correctly done in linear scale light:
downsizing I assume, denoising (mdegrain)?, smoothlevels 16bit (altering y levels)?
Although it's technically correct, denoising in linear light isn't necessary, because usually noise pixel values are close to each other. Operating in linear light is better when averaged pixel values are far. Moreover, the current modified MDegrain accepts only 8-bit input, and the conversion to 8-bit linear is not recommended because of the important resolution loss in the dark range.

And when using bilinear to downsize, because every other filter rings or has a dark line around edges, is the lowpass an accurate calculation obtaining the highest sustainable frequency without aliasing or is it a conservative generic value that is blurring the image?
The value is more or less arbitrary, but should be a decent compromise. It all depends on the interpolator frequency response shape. For example, the linear interpolator has a quite slow rolloff (huge aliasing excepted for highly oversampled data) and starts attenuating significantly the frequency content of the picture far before the Nyquist frequency. So if you increase fh and fv, you'll get a sharper picture but more aliasing. The visual result highly depends on the content nature. Anyway, for almost all filter (excepted sinc-based ones with a high number of taps), the default fh and fv values will bring you both aliasing and smoothing. Another thing to consider is that the ringing on the final video resulting from a large kernel is not necessarily bad. If the picture is scaled again during display, these ringing may contain valuable information that can help the display resizer to reconstitute fine details, instead of creating ringing and bluring where they didn't exist on the video. So it may look better at the native resolution, but worse when scaled.

If converting yv12 to yuy2 with the bug fix script in 2.5.8, does the bilinear lowpass here or any other time when upsizing?
I'm not sure to understand the question. From a signal processing point of view, the resizer works like this:
virtually upsample the picture to an infinite sampling frequency (like a continuous signal) by multiplying it with a Dirac impulse train (or grid, because we are in 2D). Muliplying is space domain is equivalent to convolving in frequency domain. Here, we get the picture spectrum mirrored and replicated. In one dimension: 0|¯\_|Fn becomes: ..._/¯¯¯\___/¯|¯\___/¯¯¯\_...
Convolve the signal with the normalised kernel impulse. This is where filtering happens, because the kernel is intended to be an approximation of a brickwall lowpass filter. fv and fh are implicitly divided by an additional factor in case of downscaling. Convolving in space domain is equivalent to multiplying spectrums in frequency domain, and we try here to clear everything above the desired cutoff frequency.
Then, the filtered signal is sampled again according to the new size, offsets, etc. This is where aliasing occurs, because the impulse spectrum may have a frequency content higher than both new and old nyquist frequencies, letting the rests of the mirrored parts fold and overlap where they shouldn't.
I don't know if I'm clear, but the important thing to understand it that there is always some kind of lowpass filtering during a scaling. Even upscaling.

cretindesalpes, is it possible to have the previous version of dither? I am getting strange results with the newer and I'd like to compare them.
Yes, they should be all available, just change the version numbers in the zip file url. What kind of strange result are you getting?

[tormento]

Increase in file size. I'll test and report.

[atra dies]

Although it's technically correct, denoising in linear light isn't necessary, because usually noise pixel values are close to each other. Operating in linear light is better when averaged pixel values are far. Moreover, the current modified MDegrain accepts only 8-bit input, and the conversion to 8-bit linear is not recommended because of the important resolution loss in the dark range.

Nevermind. I now realize mvtools only had lsb out and should have known that's what was meant from the lsb part of the doc.
I wanted to raise y levels before mdegrain but I don't care now as I think I want to denoise before any other filtering anyway.
Dogway says his lsb in is a hack, he's on vacation and file set to private anyway.

I'm kinda spooked after I looked at that degrain comparison thread. I don't want to lose fine details but this is an old technoscope movie from the early 70s that has a cbr of 7mbps and I want to shrink file size while degraining. Mdegrain removes that blocking that results when encoding the film grain to 480p.

As for the resize, I want to test results now by viewing the downsize through an upsizer that most closely emulates an HDTV upsizer. What kernel would you recommend?
Before I just let AVSpmod upsize it and I think it uses point or nearest neighbor for its display.

[tormento]

What? So if I want to use mdegrain, I have to put it outside of the 16 bit section?
Try newer SMDegrain. It works in 16bit domain.

[dadix]

It would be so good if this script would have a deblock function

[Serenato]

Hello!
I have a problem with the green screen after adding GradFun3() to the script. It goes like this:
LoadPlugin ("C:\Program Files (x86)\AviSynth 2.5\plugins\dither.dll")
Import ("C:\Program Files (x86)\AviSynth 2.5\plugins\dither.avsi")
Import ("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_xxpand_multi.avsi")
v1=DirectShowSource("D:\video.mkv",audio=false)
v2=DirectShowSource("D:\tips01.mp4",audio=false)
v1 ++ v2
TextSub("D:\logo.ass")
TextSub("D:\Op-Ed 01.ass")
ConvertToYV12()
GradFun3 ()

I have a copy of mt_masktools-26-x64.dll in the SySWOW64 folder and mvtools2.ll, mt_xxpand_multi.avsi, MT.dll, dither.dll in my avisynth's plugin folder. I'm rather confused, so perhaps I pasted something to the wrong place. The script is running, but the preview video is green.
Thanks for your help in advance!

[atra dies]

Hello!
I have a problem with the green screen after adding GradFun3() to the script. It goes like this:
LoadPlugin ("C:\Program Files (x86)\AviSynth 2.5\plugins\dither.dll")
Import ("C:\Program Files (x86)\AviSynth 2.5\plugins\dither.avsi")
Import ("C:\Program Files (x86)\AviSynth 2.5\plugins\mt_xxpand_multi.avsi")
v1=DirectShowSource("D:\video.mkv",audio=false)
v2=DirectShowSource("D:\tips01.mp4",audio=false)
v1 ++ v2
TextSub("D:\logo.ass")
TextSub("D:\Op-Ed 01.ass")
ConvertToYV12()
GradFun3 ()

I have a copy of mt_masktools-26-x64.dll !

You are running 2.5 so you need the masktools 25 dll.
masktools26 dll is for avisynth 2.6.
Also any plugin in the plugin folder is auto loaded so you don't need to load it in the script.

[cretindesalpes]

Try to remove your masktools dll file. If it still outputs a green screen without complaining about a missing function, that's because you have another masktool file loaded by Avisynth. Find and remove it, and retry until Avisynth can't find the masktools functions. Then you can copy again the right masktools dll in your plugin folder.

[Serenato]

Okay, I deleted every masktools and tried to run the script after I pasted the mt_masktools-26-x64.dll to the SySWOW64 folder, but it gave the green screen again. I removed it, then I got the error saying "there is no function named "mt_expand". Is it normal, right?

[cretindesalpes]

Make sure that your dll is from the Masktools version 2 alpha 48. Previous versions won't work and give a green screen. The dll should be located in the Avisynth plugin folder, not in SySWOW64. Also, I don't know if it works with the x64 version; try the 32-bit dll too.

[Serenato]

Oh, wow, it's really working! Thank you a lot!

[TheProfileth]

BTW I have been wondering this for a while but, why is there no dither_lut16xy(z)? I mean if you can already handle dither_lut16 what is the difference between adding another variable? Or am I wrong to think that it is that simple?

[cretindesalpes]

TheProfileth:

See my reply to SSH4 (http://forum.doom9.org/showthread.php?p=1522318#post1522318).

[ajp_anton]

Amount of RAM needed (hope I did it correctly):
8-bit LUT (x, xy, xyz): 256B, 64kiB, 16MiB
16-bit LUT (x, xy, xyz): 64kiB, 4GiB, 256TiB

edit: Yes I did it wrong, the 16-bit LUTs should be double that (forgot the output is also 16-bit). So 128kiB, 8GiB and 512TiB.

[TheProfileth]

TheProfileth:

See my reply to SSH4 (http://forum.doom9.org/showthread.php?p=1522318#post1522318).

Ah sorry did not see that, makes sense though I was thinking about it after I made that post, just considering the massive increase in numerical values available I am amazed you even got dither_lut16 to work. Well either way I guess my question is, is it ok to use mt_lutxy on a 16bit clip? If not is there something that could be done to help atleast achieve similar results?

[ajp_anton]

Well the lut16 isn't that difficult really. I was wrong in my previous post, but it still only needs 128kiB of RAM.
You can easily do it by combining two mt_lutxy's. One for MSB, one for LSB, and they both take MSB and LSB as separate inputs.

For that same method, for a lutxy16 you would need to use two of "mt_lutxyzw" and use up pretty much all your memory.

For a LUT for an arbitrary function, you can't really get around the amount of RAM needed. You would need something that calculates the pixels individually on the fly, or maybe a special case that can be simplified somehow.

edit: A semi-lutxy16 can be made though where one of the input clips is 8-bit, using two mt_lutxyz =).
edit2: You can do any lutxy where the input clips have a combined bitdepth of 24-bits, so 16/8 or 12/12 for example. Which combination it is doesn't really matter, you just have to re-order the bits into 3x 8-bit, so they can be imported with mt_lutxyz.

[gyth]

You would need something that calculates the pixels individually on the fly
Does anything do this already or is it so slow that anyone skilled enough to make it would know they shouldn't?

[titlis]

While seeing post about gamma correction,
some idea came up to me.

dither_convert_8_to_16()
dither_y_gamma_to_linear(curve="709")
dither_y_linear_to_gamma(curve="srgb")
ditherpost()

Would this compensate transfer function difference between
Rec.709 and sRGB?
(As most computer monitor use sRGB transfer function)

[cretindesalpes]

titlis:

This would be a good approximation, indeed. But just an approximation because the gamma curve should be applied on RGB components, not the signal luma. The latter method would involve a double colorspace conversion, which is lossy (chroma blurring or ringing). Pick your poison. Anyway, correctly compensating the Rec 709 transfer function should be the player or display's job, via a color profile. Or you'd have to have a mix of tags in the encoded stream: 709 matrix and color primaries with sRGB transfer characteristics, but I don't know if it's even possible.

gyth & ajp_anton:

There are many possible alternatives to the straigth 2D- or 3D-LUTs in 16 bits. But they require some kind of compromises: interpolation, computation on the fly, etc. This makes the coding more complex and the use less interesting than the usual 8-bit LUTs. So it's possible, but requires efforts…

[titlis]

Didn't Gamma apply only to the Luma?
I thought chroma is independent to gamma.

[tormento]

Is this AddGrainC 1.61 (http://forum.doom9.org/showthread.php?p=1534816#post1534816) compatible with your script?

[cretindesalpes]

Try it. If you find a problem, please report it.

[tormento]

I've done some encodings with almost 8-10GB memory usage with 12 threads on 8 cores, stability seems to be ok ;)

I'll watch results in the following days and I'll report quality.

[TheProfileth]

So first off I should say I am trying to use the width of the frame to help autofind a good thr for Smoothgrad but while doing this I ran into this issue.
The error that is occuring for me, is whenever I use either Smoothgrad or Dither_limit_dif16, if I set thr to be something like thr=(704/720) it says "thr must be a strictly positive number" same thing happens if I set it to be thr=(960/1280) the weird part is if I actually put in the decimal amount such as 0.75 for 960/1280 or 0.97777777777777777777777777777778 for 704/720 it works.
Also if I try to calculate externally the amount outside of the function such as doing
for SD
input=last
w=input.width()
thr=w/720
input.Dither_convert_8_to_16()
smoothgrad(thr=thr,u=2,v=2)
for HD
input=last
w=input.width()
thr=w/1280
input.Dither_convert_8_to_16()
smoothgrad(thr=thr,u=2,v=2)
it still gives me issue
the thing that might be part of the issue is how the numbers are being stored because it seems to mess up also if I do
input=last
w=input.width()
thr=w/720
thr=thr*thr
thr=thr/thr
input.Dither_convert_8_to_16()
smoothgrad(thr=thr,u=2,v=2)
it says that I am dividing by zero :scared:
when was the last time that
x>0
(x*x)/x=0
Very confused I am in this weird venture either way I hope someone can help me with this, hopefully it is just me being silly and not some big issue or something.

[cretindesalpes]

In Avisynth, dividing an integer number by another integer number gives an integer number. Therefore 704 / 720 = 0. If you want the correct result, specify the operands as floating point data: 704.0 / 720.0 = 0.9777… Or Float(blah1) / Float(blah2) if you use variables. Also for thr != 0, thr=thr*thr then thr=thr/thr will give 1, not thr.

Just out of curiosity, why setting the thr value according to the picture size? I would understand for radius, but thr? :confused:

[TheProfileth]

In Avisynth, dividing an integer number by another integer number gives an integer number. Therefore 704 / 720 = 0. If you want the correct result, specify the operands as floating point data: 704.0 / 720.0 = 0.9777… Or Float(blah1) / Float(blah2) if you use variables. Also for thr != 0, thr=thr*thr then thr=thr/thr will give 1, not thr.

Just out of curiosity, why setting the thr value according to the picture size? I would understand for radius, but thr? :confused:
Thanks I hope that helps, and btw I was doing it for both thr and radius just mainly had the issue with thr :D

[Yellow_]

Hi I'm getting an error with "Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", chromak="bicubic", lsb_in=true, output="rgb48y")" now which never used to happen, using Dither 1.13.0 onwards all give same error. Have the correct masktools 2a 48.

error is: mt_lutxy:unsupported colorspace. Masktools only support planar YUV colorspaces (YV12, YV16, YV24) dither.avsi, lines 773, 752, 263

Source is YV12 and the only line after ffmpegsource2(file) is Dither_Convert_8_to_16()

Just converting 8 to 16 works, gives me stacked video over green frame, adding:

Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="709") gives me a 'noisy' green frame so everything working as expected except adding Dither_convert_yuv_to_rgb...... before gives planar error. :-(

[cretindesalpes]

Are you using mt_masktools-25.dll with Avisynth 2.6 ? In other words, can you feed the masktools functions with Y8 clips ? That would explain. I'll try to find a decent workaround for this case.

Edit : in the meantime, in dither.avsi, try to replace the "vers < 2.60" with "true" line 206, 214, 217, 317 and 384.

[Yellow_]

Actually no, not using 2.6. I'm using 2.5 with mt_masktools-25.

[Kraise]

Hey, first of all thanks for making these great filters.
I only used avisynth for small things over the years so I'm still not too familiar with it. I was directed to this topic on how to encode to 10bit and make the gradients smooth in anime.

I read through half the topic not understanding a thing almost, and just got done reading some of the documentation but I still feel confused, so I have a couple of questions...

1: I can't notice banding in my DVD footage, should I still use this if I'm encoding to 10bit?

1.1: Is DVD/Blu Ray already 10 bit from the source .m2ts / .vobs?

1.2: Can we bring already 8bit sources and achieve the debanding effect still? I.E. if I wanted to bring an anime opening and add karaoke effects to it and render it back out lossless.



Sorry if those were dumb questions, I don't even know a proper script but here's one I just made up, please tell me if it's right.




If my source had no noise, and I just wanted to perform debanding before encoding I would do this? Then send it to megui?


mpeg2source("E:\Backup\VTS_05_2.d2v")
Crop(0, 0, -0, -0)
Lanczos4Resize(640, 460)

GradFun3(lsb=true)
Dither_convey_yuv4xxp16_on_yvxx ()
avs2yuv -raw "E:\script.avs" -o - | x264-10bit --demuxer raw --input-depth 16 --input-res 640x460 --fps 23.976 --output "E:\test.mp4" -



I tried previewing just this in VirtualDub and I get a strechted aspect ratio and the screen has fadded green lines and artifacts over it. Is that normal?
[CODE]mpeg2source("E:\Backup\VTS_05_2.d2v")
Crop(0, 0, -0, -0)
Lanczos4Resize(640, 460)

GradFun3(lsb=true)





and if my source had noise and I wanted to denoise and sharpen I would do this? (I added my favorite sharpener, I don't know if that's how it works or not, but you can't sharpen the footage after using dfttest() with normal sharpeners right?

LSFmod(strength=120)

mpeg2source("E:\Backup\VTS_05_2.d2v")
Crop(0, 0, -0, -0)
Lanczos4Resize(640, 460)
dfttest (sigma=16, lsb=true)
f = DitherPost (mode=-1)
s = LSFmod(strength=120)
d = mt_makediff (s, f)
DitherPost ()
mt_adddiff (last, d, u=2, v=2)
SmoothGrad ()
Dither_convey_yuv4xxp16_on_yvxx ()
avs2yuv -raw "E:\script.avs" -o - | x264-10bit --demuxer raw --input-depth 16 --input-res 640x460 --fps 23.976 --output "E:\test.mp4" -

[cretindesalpes]

Yellow_:

Sorry I couldn't reproduce the problem here. I definitely don't understand what's wrong here.

Kraise:

1. If your source has no banding and you're encoding to 10 bits with enough bitrate, you probably don't need to do anything.

1.1 DVD and bluray are 8 bit only. But this doesn't mean that the master wasn't in a higher bitdepth.

1.2 Yes. That's what GradFun3() is for.

If you try previewing a 16-bit clip (in stacked format or ready to encode), this is normal that you get garbage. Just replace Dither_convey_yuv4xxp16_on_yvxx () with DitherPost() for a convenient preview (or just remove the line and set lsb=false in GradFun3).

Also, you should be careful when mixing 8 and 16 bit processing. In your latest example, dfttest (lsb=true) generates a 16-bit clip and pass it to LSFMod, but LSFMod excepts only 8-bit clips. And the subsequent mt_makediff() will receive input clips with different height, this is not good. Same thing just a few lines below, SmoothGrad excepts a 16-bit clip. Here is a more correct script, using a simple mask :


dfttest (sigma=16, lsb=true) # last is now 16 bits
f = DitherPost (mode=-1) # 16 -> 8 bits
s = f.LSFmod(strength=120) # 8 -> 8
mask = mt_lutxy (s, f, "x y != 255 0 ?", u=0, v=0) # 8, 8 -> 8
s16 = s.Dither_convert_8_to_16 () # 8 -> 16
Dither_merge16_8 (last, s16, mask) # 16, 16, 8 -> 16
SmoothGrad () # 16 -> 16
Dither_convey_yuv4xxp16_on_yvxx () # 16 -> enco


You can also see post #250 (http://forum.doom9.org/showthread.php?p=1518953#post1518953) of this thread for more details about mixing 8- and 16-bit processing; the proposed method is slightly different.

[Yellow_]

Yellow_:

Sorry I couldn't reproduce the problem here. I definitely don't understand what's wrong here.

You were right I was running 2.6 alpha without knowing it, due to the freaking mess which is the sourceforge 'latest version' link giving me 2.6 Alpha even from within the 2.5 branch of Avisynth at Sourceforge. :-) So of coarse downloading 2.5 from 2008 + latest Directshow zip solves problems.

[titlis]

Didn't Gamma apply only to the Luma?
I thought chroma is independent to gamma.

oops... sorry
I was a bit stupid


Now I got it
Gamma is really a RGB thing right?

[canuckerfan]

Is there a way to use DitherPost() without denoising first? For example, I have a clip that is already denoised. And I don't want to apply dfttest(lsb=true) because it will denoise my clip twice. Instead, how can the clip be changed to 16-bit so that DitherPost() can be applied to it?

[mandarinka]

There is normal fucntion for 8 to 16 bit conversion: Dither_convert_8_to_16()

However, by itself it does nothing but add zeros as the bottom (least significant) bits. Of course, when you do not want to smooth your video, that's good. It doesn't make sense to do that just so that you can run ditherpost (that will only convert it back to 8bit).

If you actually want to do some dithering (debanding?) on your video, you could use gradfun3 which takes 8bit as input but can output 16bit as well as 8bit (if you want 8bit output, it actually has to use ditherpost itself).

Anyway, to answer your question properly, you need to specify what do you intend to do by using ditherpost...

[canuckerfan]

Anyway, to answer your question properly, you need to specify what do you intend to do by using ditherpost...
Basically to reduce banding and to avoid giving the clip a "glossy/shimmery" look. Maybe GradFun3 by itself can do this and I won't need to Dither?

[mandarinka]

Yeah, in that case, you'd use a debanding filter (gradfun3 or that other one, flashkyu3deband) and you can just start straight from the 8bit source. If you want to encode 8bit h.264, that would be it (gradfun3 works in 16bit internally which it converts to 8bit using ditherpost unless you set lsb=true). If you wanted to encode in 10-bit, then you should output 16-bit from gradfun3 and then use the special function to get 10-bit video for purposes of x264 encoding through avs2yuv:

gradfun3(lsb=true)
fslg_dither_to_n_bits(10, outn = true) # add mode=x if you want to specify which dithering to use in ditherpost
Dither_convey_yuv4xxp16_on_yvxx()

[cretindesalpes]

Dither 1.14.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) package released:
Multi-threading: most dither.dll functions are now multi-threaded with the help of AVSTP (http://forum.doom9.org/showthread.php?t=164407), excepted Dither_box_filter16 and SmoothGrad (later, maybe). The speed improvement is not that impressive but still welcome.
Added staticnoise parameter to GradFun3.
MVTools v2.6.0.0:
MDegrainN: internally uses MDegrain1/2/3 when tr ≤ 3, for optimal speed.
MCompensate: added multi-compensation mode for easy use with temporal filters.
MAnalyse, MSuper, MCompensate, MDegrainN: added multithreading (but partially, not for all code paths).
MAnalyse: fixed a corruption of the global motion vector.
Dfttest v1.9.1: fixed a regression with internal dithering.
Documentation revisited.

[Dogway]

whoa unexpected! Thanks a lot!

I'm away of avisynth now but I will test when I clear a bit the work space. I wanted to ask if it is possible to default the "quiet" parameter of dfttest to true, because I'm making it to perform as a high pass prefilter for the smdegrain function, so it is backwards compatible with vanilla dfttest.

edit: btw I'm having a dilemma on whether to default truemotion true as mvtools default, or false like mctemporaldenoise, so I had in mind to test the related parameters. Because in the documentation I saw you were also hesitant on these parameters as well I'll let you know when I get to any conclusion.

edit2: and I think there is a bug for mdegrainN when using avstp.dll

[tormento]

Thanks, I'll test it in my worst encodings ;)

[atra dies]

MAnalyse: fixed a corruption of the global motion vector

Would this make a difference here?:

super = MSuper(pel=1, sharp=0)
backward_vec2 = MAnalyse(super, isb = true, blksize=4,delta = 2, overlap=0,temporal=true)
backward_vec1 = MAnalyse(super, isb = true, blksize=4,delta = 1, overlap=0,temporal=true)
forward_vec1 = MAnalyse(super, isb = false, blksize=4,delta = 1, overlap=0,temporal=true)
forward_vec2 = MAnalyse(super, isb = false, blksize=4,delta = 2, overlap=0,temporal=true)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)

I am in the middle of an encode and would gladly start it over using new dither plugin if there is any bug at all with the previous version.

OT:
(Mdegrain recommendation/my opinion after a lot of trying out different parameters:
no supersampling, pel > 1 blurs,
blksize is much more accurate at 4 for SD, anything greater blurs with no supersampling,
mdegrain3 and above introduces artifacts at scene change.)

[cretindesalpes]

Dogway:

You're right! I'll release a fix soon.

atra dies:

When analysing a block, the global motion vector computed from the previous level and used as "second try" (after the "no motion" vector) is truncated so it does not point to an area located out of the picture. Previously, it was never reset to its initial value when passed to the next block, so when it was truncated once (very frequent because analysis begins at the top left of the frame), it became a bad guess for all other blocks and global motion estimation was actually ignored. Anyway the difference was minor in my tests, but I think it really depends on the scene. I think you can safely terminate your encoding with the previous MVTools…

[SSH4]

cretindesalpes
Looks like bug in new build of MVtools2 mod from your dither 1.14 pack.
mdegrainN crashed on any tr. For example:
AVCSource("g:\Endhiran.dga")
Crop(0, 132, -0, -132) #1920х816
tr = 6 # Temporal radius
super = MSuper ()
multi_vec = MAnalyse (super, multi=true, delta=tr)
MDegrainN (super, multi_vec, tr, thSAD=400, thSAD2=150)


MVTools2 from 1.13 work well.
Windows 7 64bit
Avisynth 2.6 MT
crash on both Setmode(2) and Setmode(5)

"exception c0000005 at address 0001043e"

on SD source with more complex script QTGMC, 2xMdegrainN, dfttest work well 0__0

[cretindesalpes]

Sorry for these bugs, I found that I just left half-finished code in MDegrainN :eek: . But still, I changed lots of code everywhere, so take these new Dither/MVTools with care.

Bugs fixed in Dither 1.14.1 / MVTools 2.6.0.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
MAnalyse: fixed potential crash in multithreading mode.
MDegrainN: fixed systematic crash in multithreading mode.
All MDegrain functions: fixed potential crash when thSAD is set to 0.

I would be interested to know the performance gain you have with the multithreaded versions. On my 4-core rig, I generally obtain a ×2.5 speed increase for a ×3.2 CPU usage (MT mode not activated).

[Dogway]

setmtmode(5,2)
SD source
setmtmode(2,2)
Crop(0, 160, -0, -160)
smdegrain(tr=3,thSAD=300,lsb=true)

597mb
80-90% CPU load
1.950fps

577mb
50-60% CPU load
1.300fps

+50% on 2.53gh intel core 2 duo
I think you gain +25% per thread.

Is it possible to default quiet to true in dfttest?

why mflow only uses 2 vectors?, can't it benefit from more vectors as mdegrain does?

[tormento]

Test setup:

2600k@4500MHz with 16GB
x264 v2183 64bit
avs4x264mod v0.6.2 compiled with LARGEADDRESSAWARE flag

SetMemoryMax(2048)
LoadPlugin("D:\eseguibili\media\dgdecnv\DGDecodeNV.dll")
DGSource(Some1080p)
ChangeFPS(last,last,true)
SMDegrain (tr=6,PreFilter=2,contrasharp=false,lsb=true,mode=6)

Dither 1.13.0: 1.95 fps 62.686.533 bytes
Dither 1.14.1: 3.76 fps 62.036.889 bytes

SetMTMode(3,8)
SetMemoryMax(2048)
LoadPlugin("D:\eseguibili\media\dgdecnv\DGDecodeNV.dll")
DGSource(Some1080p)
ChangeFPS(last,last,true)
SetMTMode(2,8)
SMDegrain (tr=6,PreFilter=2,contrasharp=false,lsb=true,mode=6)

Dither 1.13.0 MT: 6.54 fps 62.704.727 bytes
Dither 1.14.1 MT: 6.43 fps 62.039.911 bytes

[06_taro]

- Add "Y"/"U"/"V" in "Dither_merge16_8", fully support -65535~5, default value: Y=3, U=2, V=2 as mt_merge

- Fix Dither_lut16 with Y/U/V<=0 (previous Dither_lut16(Y=-x) returns Y(msb)=x and Y(lsb)=x, while the correct result should be Y(msb)=x/256 and Y(lsb)=x%256)

Function Dither_merge16_8(clip src1, clip src2, clip mask, bool "luma", int "y", int "u", int "v"){
vers = VersionNumber()

luma = Default(luma, false)

y = Default(y, 3)
u = Default(u, 2)
v = Default(v, 2)

w = mask.Width()
h = mask.Height()
Assert(src1.Width() == w)
Assert(src2.Width() == w)
Assert(src1.Height() == h * 2)
Assert(src2.Height() == h * 2)

msk = ( luma && (vers < 2.6 || ! src1.Dither_isY8()) )
\ ? mask.MergeChroma( mask.YToUV(mask).BilinearResize(w, h) )
\ : mask

msb1 = src1.Dither_get_msb()
lsb1 = src1.Dither_get_lsb()
msb2 = src2.Dither_get_msb()
lsb2 = src2.Dither_get_lsb()

p1a = Dither_lutxy8( msb1, msk, expr="256 y - x * ", Y=(y==3?3:1), U=(u==3?3:1), V=(v==3?3:1) )
p1b = Dither_lutxy8( lsb1, msk, expr="256 y - x * 256 /", Y=(y==3?3:1), U=(u==3?3:1), V=(v==3?3:1) )

p2a = Dither_lutxy8( msb2, msk, expr=" y x * ", Y=(y==3?3:1), U=(u==3?3:1), V=(v==3?3:1) )
p2b = Dither_lutxy8( lsb2, msk, expr=" y x * 256 /", Y=(y==3?3:1), U=(u==3?3:1), V=(v==3?3:1) )

p1 = Dither_add16(p1a, p1b, wrap=True)
p2 = Dither_add16(p2a, p2b, wrap=True)

Dither_add16(p1, p2, wrap=True)

( y==2 || u==2 || v==2 ) ? mt_lutxy( src1, Y=(y==2?4:2), U=(u==2?4:2), V=(v==2?4:2) ) : nop
( y==4 || u==4 || v==4 ) ? mt_lutxy( src2, Y=(y==4?4:2), U=(u==4?4:2), V=(v==4?4:2) ) : nop
( y==5 || u==5 || v==5 ) ? mt_lutxy( mask.Dither_convert_8_to_16, Y=(y==5?4:2), U=(u==5?4:2), V=(v==5?4:2) ) : nop

return ( y<=0 || u<=0 || v<=0 ) ? Dither_lut16( Y=(y<=0?y:2), U=(u<=0?u:2), V=(v<=0?v:2) )
\ : last

}

Function Dither_lut16 (clip src,
\ string "expr", string "yexpr", string "uexpr", string "vexpr",
\ int "y", int "u", int "v")
{
s_msb = Dither_get_msb (src)
s_lsb = Dither_get_lsb (src)

d_msb = Dither_lut16_msb (s_msb, s_lsb,
\ expr=expr, yexpr=yexpr, uexpr=uexpr, vexpr=vexpr, y=y, u=u, v=v)
d_lsb = Dither_lut16_lsb (s_msb, s_lsb,
\ expr=expr, yexpr=yexpr, uexpr=uexpr, vexpr=vexpr, y=y, u=u, v=v)

StackVertical (d_msb, d_lsb)
}

Function Dither_lut16_msb (clip msb, clip lsb,
\ string "expr", string "yexpr", string "uexpr", string "vexpr",
\ int "y", int "u", int "v")
{
expr = Default (expr, "x")
yexpr = Default (yexpr, expr)
uexpr = Default (uexpr, expr)
vexpr = Default (vexpr, expr)
expr = Dither_lut16_expr (expr, True)
yexpr = Dither_lut16_expr (yexpr, True)
uexpr = Dither_lut16_expr (uexpr, True)
vexpr = Dither_lut16_expr (vexpr, True)

mt_lutxy ( msb, lsb,
\ expr=expr, yexpr=yexpr, uexpr=uexpr, vexpr=vexpr, y=(y<=0?y/256:y), u=(u<=0?u/256:u), v=(v<=0?v/256:v) )
}

Function Dither_lut16_lsb (clip msb, clip lsb,
\ string "expr", string "yexpr", string "uexpr", string "vexpr",
\ int "y", int "u", int "v")
{
expr = Default (expr, "x")
yexpr = Default (yexpr, expr)
uexpr = Default (uexpr, expr)
vexpr = Default (vexpr, expr)
expr = Dither_lut16_expr (expr, False)
yexpr = Dither_lut16_expr (yexpr, False)
uexpr = Dither_lut16_expr (uexpr, False)
vexpr = Dither_lut16_expr (vexpr, False)

mt_lutxy ( lsb, msb,
\ expr=expr, yexpr=yexpr, uexpr=uexpr, vexpr=vexpr, y=(y<=0?y%256:y), u=(u<=0?u%256:u), v=(v<=0?v%256:v) )
}

[zero9999]

when run on AviSynth 2.6, Dither_convert_yuv_to_rgb(output="rgb48yv12") throws the following error: Dither_convey_rgb48_on_yv12: r,g,b must be YV12. Probably because the the convert function uses Y8 because it's available and the convey function not supporting it.

[cretindesalpes]

06_taro:

Thank you, I'll add these ones in the next version.

zero9999:

Same thing. In the meantime, you can change the IsYV12() in the assert line with IsYUV() and add just below:
r = r.ConvertToYV12 ()
g = g.ConvertToYV12 ()
b = b.ConvertToYV12 ()

[06_taro]

Bug report (Dither_resize16):
ColorBars(width=1068, height=600, pixel_type="YV12")
Dither_convert_8_to_16.Dither_resize16(1280, 720, kernel="bicubic")

Crashes since v1.12.1, works fine with v1.12.0 and older versions.
If the source resolution changes, e.g., width=1920, height=1080, or the kernel is not "bicubic", this script might not crash.

[cretindesalpes]

Dither 1.15.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added the luma parameter to Dither_merge16 and defaulted u and v to 2 (copy) excepted when luma = true (process). This behaviour is now identical to mt_merge.
Added plane filtering in Dither_merge16_8. The function now internaly uses Dither_merge16 and is much faster (×20 on a 4-core CPU).
Added support for negative plane filter values in Dither_lut16. Thanks to 06_taro for the suggestion.
Set quiet = true as default in dfttest 1.9.2.
Fixed a bug in Dither_resize16 making it crash for some combinations of source and destination sizes.

[Dogway]

10 minutes after I release v1.9d? hmmmm.... ok hahahah. No really, thanks a lot, I will have a look at to what I can implement or need to change for v1.95d. : )

[pandy]

is there any way to reduce bit depth with dither to less than 8 bits?

Yes. Reduce the signal before dithering and amplify it afterwards.
bits = 4 # Valid range: 0 - 8
mul = String (Pow (2, 8 - bits))

Dither_convert_8_to_16()
Dither_lut16 (expr="x "+mul+" /")
DitherPost ()
mt_lut ("x "+mul+" *", y=3, u=3, v=3)

Thank You cretindesalpes, sorry for very late reply, it work almost OK.

[Lenchik]

How to apply Spline144Resize (http://forum.doom9.org/showthread.php?p=1356563#post1356563) analog with Dither_resize16 "impulse " notation?

[cretindesalpes]

You'd need to enter the 6-point spline polynomials into a math software and to make it calculate the coefficients with a reasonable oversampling. It would be probably painful and take longer than the few minutes I just spent to add the generic spline kernel in Dither_resize16. Thanks for the pointers, it will be delivered in the next release.

[Boulder]

I've been getting a lot of VirtualDub or x264pipe crashes with the new MVTools version. The crash occurs during the initialization of the script and it always points to fftw3.dll. The version supplied with Dither v1.13.3 seems to work, at least I couldn't make it crash by constant reloading of the script in VDub. I'm on Win7 64-bit with SEt's latest MT build of 32-bit Avisynth.

While fftw3.dll is the subject, do you happen which version of it should be used? There are many of them floating out there and apparently you have to rename a specific build to fftw3.dll since there's now 3 different dlls included in the latest FFTW release.

[cretindesalpes]

Please post your script.

[Boulder]

I'll send it to you in a PM, the function I use is very, very ugly spaghetti code... :)

[cretindesalpes]

I couldn't reproduce the crash (latest Avisynth 2.6 MT). I tried with various sources, everything is playing fine for me here. Have you tried to disable the MT modes? Anyway when the script is highly memory-intensive, sometimes VDub cannot reload it (F2) correctly. I think it tries to keep both versions until the second one is ready. Close the script first then open it again.

For FFTW, I have both the old version (fftw3.dll, 1627136 bytes, 2004-01-30) and the "new" ones (libfftw3-3.dll, libfftw3f-3.dll and libfftw3l-3.dll, 2009-07-19, probably not the most recent files). These versions are different, I think you should get a genuine file, not rename one of the newest files.

[Boulder]

OK, I'll try lowering the SetMemoryMax settings first and see if it works better then.

Regarding FFTW3.dll, I found this: http://forum.doom9.org/showthread.php?t=156028 , near the bottom of the first post. It seems that you can rename libfftw3f-3.dll to fftw3.dll if you want to upgrade. I don't know how MAnalyse uses fftw3 so it's not certain that there actually are any changes, but I'll see if anything weird happens anyway.

[Lenchik]

http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=179 and http://www.mediafire.com/file/8b6dhdat888i9do/SeparateResize_v1.4.2.7z
As ideas for Dither_resize16 from links above:
- setting separate algorithms for luma and chroma planes;
- some algorithms from madVR;
- some non-ringing algorithms maybe (i don't know if we can use Repair in 16bit without bugs of any kind).

[cybersharky]

DGSource("D:\magnum\title00.dgi",fieldop=0)
crop(0, 4, 0, -4)
LanczosResize(640,480)
tr = 6 # Temporal radius
super = MSuper (pel=2)
multi_vec = MAnalyse (super, multi=true, delta=tr)
MDegrainN (super, multi_vec, tr, thSAD=400, thSAD2=150)

Does temporal radius have anything to do with overlap?
Because I see in the examples overlap is used in MAnalyse, when using MDegrain2/3

[06_taro]

http://www.nmm-hd.org/newbbs/viewtopic.php?f=7&t=179 and http://www.mediafire.com/file/8b6dhdat888i9do/SeparateResize_v1.4.2.7z
As ideas for Dither_resize16 from links above:
- setting separate algorithms for luma and chroma planes;
- some algorithms from madVR;
- some non-ringing algorithms maybe (i don't know if we can use Repair in 16bit without bugs of any kind).

An example for Spline64 luma kernel with SoftCubic75 chroma kernel(but no non-ringing algorithm, RemoveGrain16 and Repair16 is necessary for many scripts to be ported from 8-bit to 16-bit, and adding YV16/YV24 support is also interesting):
Dither_resize16(dest_width, dest_height, kernel="Spline64")
\ .MergeChroma(Dither_resize16(dest_width, dest_height, kernel="Bicubic", a1=0.75, a2=0.25))

[cretindesalpes]

cybersharky:

No, the temporal radius has nothing to do with the overlap. However, omitting the overlap (default is 0) may cause blocking-like artifacts.[/QUOTE]

Lenchik / 06_taro:

Thank you for the suggestions. Actually I prefer keeping the resizer function "simple", and use scripting for the non-linear parts and plane recombination. Anyway I will add a built-in function for colorspace conversions (including simple chroma subsampling conversions) probably in combination with resizing, because scripting these things is quite complicated and not very efficient.

In your example you could use the plane filters to avoid unnecessary computations that will be thrown out:
Dither_resize16(dest_width, dest_height, kernel="Spline64", u=1, v=1)
\ .MergeChroma(Dither_resize16(dest_width, dest_height, kernel="Bicubic", y=1, a1=0.75, a2=0.25))

Even without 16-bit RemoveGrain, it's possible to work in 8 bits for ringing cancellation and merge the modified pixels with the 16-bit results. This would be a good approximation for visible ringing. Pixel-sized artifacts (aliasing, ringing, halos…) generally don't need 16-bit processing to be fixed as they often result in large value changes.

[06_taro]

Agree with keeping the resizer "simple". And applying such a filter on 8-bit clip and adding the diff is usually enough in terms of visual effect. :) But if the aimed output clip is 16-bit, would it result in some artefacts in LSB, such as mosquito around edges for deringing? And although such artefacts in LSB shouldn't be noticeable after dithered to 8-bit, I'm worried if they may reduce the efficiency of compression.

[Lenchik]

Same questions as 06_taro's but for aiming output in 10 bit (http://forum.doom9.org/showpost.php?p=1519194&postcount=252) to 10 bit x264 build.

[cretindesalpes]

Agree with keeping the resizer "simple". And applying such a filter on 8-bit clip and adding the diff is usually enough in terms of visual effect. :) But if the aimed output clip is 16-bit, would it result in some artefacts in LSB, such as mosquito around edges for deringing? And although such artefacts in LSB shouldn't be noticeable after dithered to 8-bit, I'm worried if they may reduce the efficiency of compression.

These artifacts are small relative to the edges that generate ringing and will be most likely crushed by the compression. And artifacts caused by the low-bitdepth processing are much smaller than the visual artifacts caused by the method itself which is far from perfect, or than the noise level that comes with most lossy or real-life video source.

Below is a quick attempt (http://screenshotcomparison.com/comparison/118826). I adopted a slightly different approach than just adding the 8-bit difference.

# Gradient
amt_l = 0.25
amt_c = 0.125
BlankClip (length=1, width=640, height=480, pixel_type="YV12", color_yuv=$000000)
KillAudio ()
base = last
x = "x 0.5 - 32768 * "+String(amt_c)+" * 32768 + 0 65535 clip "
y = "y 0.5 - 32768 * "+String(amt_c)+" * 32768 + 0 65535 clip "
z = "x y + 1 - 32768 * "+String(amt_l)+" * 32768 + 0 65535 clip "
msb = base.mt_lutspa (mode="relative", yexpr=z+"8 >>u 255 &u", uexpr=x+"8 >>u 255 &u", vexpr=y+"8 >>u 255 &u", y=3, u=3, v=3)
lsb = base.mt_lutspa (mode="relative", yexpr=z+" 255 &u", uexpr=x+" 255 &u", vexpr=y+" 255 &u", y=3, u=3, v=3)

# Text
txt = msb
txt = txt.Subtitle ("This is a test.\n\n--\\||//@=#§", align=5, lsp=1, text_color=$E0E020, halo_color=$202020)
txt = txt.Subtitle ("\nTest with light ringing.", align=5, lsp=1, text_color=$848484, halo_color=$808080)
msk = mt_lutxy (msb, txt, "x y != 255 0 ?", y=3, u=3, v=3)
lsb = mt_merge (lsb, base, msk, y=3, u=3, v=3)
msb = txt

StackVertical (msb, lsb)

# Rings. Source: http://www.imagemagick.org/Usage/img_photos/rings_lg_orig.png
rings = ImageSource ("rings_lg_orig.png").Trim (0, -1)
rings = rings.ConvertToYV12 ().Crop (0, 0, 0, base.Height ()).Dither_convert_8_to_16 ()
StackHorizontal (last, rings)


# Resize
dw = Width() * 2
dh = Height()/2 * 2

main = Dither_resize16 (dw, dh, kernel="Spline64" )
nrng = Dither_resize16 (dw, dh, kernel="Gauss", a1=100)

m8 = main.DitherPost (mode=-1)
n8 = nrng.DitherPost (mode=-1)
f8 = m8.Repair (n8, 1)
k8 = mt_lutxy (m8, f8, "x y != 255 0 ?", y=3, u=3, v=3)
f16 = f8.Dither_convert_8_to_16 ()
f16 = nrng.Dither_limit_dif16 (f16, thr=0.5)
Dither_merge16_8 (main, f16, k8, y=3, u=3, v=3)


# Display
main.Subtitle ("Spline64") + last.Subtitle ("nrSpline64")
DitherPost (mode=6)
last
\ + mt_lut (expr="x 16 &u 17 x 15 &u - 1 x 15 &u + ? 14 *", u=-128, v=-128)
\ + k8.GreyScale ().Subtitle ("Ringmask")

[SilaSurfer]

Hey cretindesalpes. Since I'm using a lot of your functions from Dither package, I decided to incorporate Dither_resize16 in my general workflow. Now I want to produce the same output as ResampleHQ, meaning gamma aware resizing, and I wanted to check if I got it correctly:

Dither_convert_8_to_16 ()
Dither_y_gamma_to_linear()
Dither_resize16(w, h, kernel="blackman", taps=5) - I'm using Blackman as my resizer for Bluray sources where I resize down to 720p.
Dither_y_linear_to_gamma()
Ditherpost(mode=-1,ampo=0,ampn=0)

As you can see my sources are YV12. Is this ok?

[cretindesalpes]

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="709", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="709", mode=-1)
But unless you really have a reason, prefer mode=6 (or 0) to mode=-1.

[SilaSurfer]

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="709", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="709", mode=-1)
But unless you really have a reason, prefer mode=6 (or 0) to mode=-1.

Thanks cretindesalpes. About mode parameter I was just posting an example. I will incorporate this into my denoising and dithering workflow from your Dither tools. What about if I would want to downsize to SD, I would need to change Matrix parameter to 601?

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="601", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="601", mode=-1)

Like so?

[cretindesalpes]

Yes for Dither_convert_rgb_to_yuv(), but keep "709" in Dither_y_linear_to_gamma(). This is the same transfer characteristic anyway. I just should add an alias for convenient use.

[SilaSurfer]

Thanks, cretindesalpes

Do you have any convolutions examples for your resizer?

[cretindesalpes]

Dither 1.16.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added generic spline kernel to Dither_resize16.
Added the SMPTE 240M gamma curve and aliases for BT.601 and SMPTE 170M.
Fixed a bug related to chroma subsampling and multi-threading in Dither_merge16.
Bug fixed in MVTools.

——— Edit ———

SilaSurfer:

# Equivalent to mt_convolution (horizontal="1 5 -1 7 -1 5 1", vertical="1", u=3, v=3) for stack16 clips
w = Width ()
h = Height () / 2
Dither_resize16 (w, h, kernelh="impulse 1 5 -1 7 -1 5 1", fh=-1, center=false)

[Reel.Deel]

Thanks cretindesalpes!!

[SilaSurfer]

Dither 1.16.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added generic spline kernel to Dither_resize16.
Added the SMPTE 240M gamma curve and aliases for BT.601 and SMPTE 170M.
Fixed a bug related to chroma subsampling and multi-threading in Dither_merge16.
Bug fixed in MVTools.

——— Edit ———

SilaSurfer:

# Equivalent to mt_convolution (horizontal="1 5 -1 7 -1 5 1", vertical="1", u=3, v=3) for stack16 clips
w = Width ()
h = Height ()
Dither_resize16 (w, h, kernelh="impulse 1 5 -1 7 -1 5 1", fh=-1, center=false)

Thanks cretindesalpes for updated version and the example.

[SilaSurfer]

Dither_convert_8_to_16 ()
Dither_resize16 (1280, 720, kernel="impulse -1 6 -1",
\ fh=-1, fv=-1, cnorm=true, center=false)
DitherPost (mode=-1)

I also tried your example from the docs and the result is just beautiful. But I had to change it from this:

Dither_convert_8_to_16 ()
Dither_resize16 (Width (), Height () / 2, kernel="impulse -1 6 -1",
\ fh=-1, fv=-1, cnorm=true, center=false, y=3, u=2, v=2)
DitherPost ()

If I specify my own resolution, that is without /2, I get some color distortion, I had to remove y=3,u=2,v=2 parameters to normalize it. Is it save to use the way I posted it in the first example? Thanks for your patience.

[Boulder]

Out of interest: have you had any chance to look at the improvements made to MVTools v2 by the SVP team?

[cretindesalpes]

Not yet. But I'll have a look when possible.

[ajp_anton]

Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y")
Dither_y_gamma_to_linear (curve="709", u=1, v=1)
Dither_resize16 (w, h, kernel="blackman", taps=5, u=1, v=1)
Dither_y_linear_to_gamma (curve="709", u=1, v=1)
Dither_convert_rgb_to_yuv (
\ SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ matrix="709", mode=-1)
But unless you really have a reason, prefer mode=6 (or 0) to mode=-1.Why go to RGB for resizing?

[cretindesalpes]

It’s not for resizing, it’s for gamma correction (to do things correctly).

[ajp_anton]

Isn't gamma_to_linear doing the gamma "correction"? Or is linear scale not well defined for the UV planes or something?

[Lenchik]

ajp_anton
You may find these links useful for understanding idea of linear light resize and gamma correction.
http://www.glennchan.info/articles/technical/chroma/chroma1.htm
http://archive2.avsforum.com/avs-vb/showthread.php?p=4811927&&#post4811927
http://www.4p8.com/eric.brasseur/gamma.html
http://www.imagemagick.org/Usage/resize/#resize_colorspace
http://forum.doom9.org/showthread.php?p=1279308#post1279308 (and posts nearby)

[ajp_anton]

Tested with both "earth_lights" and that monk image, and indeed I only get correct results in RGB. Maybe they are extreme examples, but then again, why are we resizing in linear light unless we want it perfect =).
Don't know exactly how subsampling should be handled though, didn't test it.

[06_taro]

cretindesalpes, the BT.709 coefficients matrix used in dither package is:
kr = 0.2125
kg = 0.7154
kb = 0.0721

That is the outdated version in early draft of ITU.709, and the latest one in ITU.709-5 is:
kr = 0.2126
kg = 0.7152
kb = 0.0722

Hope to correct it in the next release.

[cretindesalpes]

Thanks. I've taken the coefficients from the Avisynth documentation (http://avisynth.org/mediawiki/Color_conversions), without checking if they were right. It's fixed now, it will be available in the next release. I also corrected the Avisynth page.

[06_taro]

Another useful thing is introducing YCgCo matrix in Dither_Convert_YUV_TO_RGB/Dither_Convert_RGB_TO_YUV, as 8-bit RGB can be losslessly converted to 8-bit Y and 9-bit Cg/Co (full-range and YV24). Padding such YCgCo to 9/10-bit and encoding them with x264 has been possible now, and madVR can render YCgCo matrix correctly. But old YCgCo tools (jmac698's script (http://forum.doom9.org/showthread.php?p=1510465#post1510465) or xv's plugin (http://forum.doom9.org/showthread.php?p=1510268#post1510268)) can only converted 8-bit RGB to 8-bit YCgCo, which is still lossy. It would be great if Dither implement it.

[tormento]

cretindesalpes, have you noticed LLVM freeware compiler has added CUDA support?

I'd like to see some compiles to understand if it could help heavy scripts.

Thanks for any effort ;)

[SilaSurfer]

hello cretindesalpes. I was wondering something. I'm working with a source (bluray) which inherits a lot of "bad" grain. My goal is to remove it and redither the source, but I want to use just noise without bayer matrix in Ditherpost(ampo=0,ampn=?). The noise just adds that feeling to the movie. I was thinking "Mode=2" in DitherPost, correct? Or can I use Error Diffusion modes just with noise? I don't wanna use Grandfun3 :p. Thanks in advance.

[cretindesalpes]

Dither 1.17.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added the YCgCo matrix in RGB conversions.
More dithering settings exposed in RGB conversions.
Fixed a very small error in BT.709 coefficients.
Bug fixed in MVTools: artifacts related to overlap in MDegrainN with tr > 3 in multithreading mode.

SilaSurfer:

Error diffusion modes (6-8) would be right for this. Anyway if you don't want the ordered dithering with modes 0-5, only the noise, you can just set ampo=0 as you suggested. You can also generate the noise with Dither_add_grain16 before dithering with mode 6.

[jmartinr]

Dither 1.17.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added the YCgCo matrix in RGB conversions.


:thanks:

[SilaSurfer]

SilaSurfer:

Error diffusion modes (6-8) would be right for this. Anyway if you don't want the ordered dithering with modes 0-5, only the noise, you can just set ampo=0 as you suggested. You can also generate the noise with Dither_add_grain16 before dithering with mode 6.

Sorry cretindesalpes for my previous question. That moment I still felt effects of a birthday party which ended couple of hours before my writing, :rolleyes: so I posted a confusing question. I'm going to make a simple post. I always use error diffusion modes with Ditherpost, I just wondered if I can dither with those modes just using noise.

My Regular use:

...
DitherPost(mode=6,ampo=1.0-1.5, anpn=0.5-0.8) - my setup for many sources.


This case:

...
Ditherpost(mode=6, ampo=0, ampn=2) - will this be Ok, it shouldn't break anything right?

Again sorry for confusing post and taking already precious time from you. BTW Thanks for your work (especially Mvtools2), I've completely switched to your functions in my video encodings. Thank you for sharing your work with us here. Best regards

[cretindesalpes]

Ditherpost(mode=6, ampo=0, ampn=2)
It is valid but won't have any specific effect. In error diffusion modes, setting ampo to a value below 1 is currently ignored. Use mode=-1 if you don't want the noise specifically added by the error diffusion (it turns the correlated quantization error into a noise whose power is mainly located in high frequencies)

[SilaSurfer]

It is valid but won't have any specific effect. In error diffusion modes, setting ampo to a value below 1 is currently ignored. Use mode=-1 if you don't want the noise specifically added by the error diffusion (it turns the correlated quantization error into a noise whose power is mainly located in high frequencies)

Thanks for your answer. So Modes 0-5 can have ordered dithering deactivated setting ampo=0, while modes 6-8 just ignore it. Got it. I decided to go with my usual setting and I'm going to add some grain with Dither_add_grain16. Just one more question, your MVTools2 mod MdegraiNN is internally multithreaded right, like MVDegrainMulti? I get speed ups compared to standard MVTools2 from Fizick.

[Yellow_]

On the same subject as SilaSurfer, I've been using 06_taro's 16bit mod of GrainFactory3 instead of Dither's 16bit addgrain, so when it comes to DitherPost if I use mode=-1 to avoid additional noise, ampo doesn't matter for modes 6 - 8? Most natural dither at expense of bitrate would be choice of 6 - 8?

[cretindesalpes]

ampo is ignored in mode 6–8 for values < 1 only. When set to a value > 1, the diffused error is augmented from ampo-1, which actually amplifies the dithering pattern in a somewhat different way from just lowering the target bitdepth ; it's more progressive. But I don't know if it's really useful.

And yes, MDegrainN is multithreaded (as well as MSuper and MAnalyse, but partially for these ones), with the help of avstp.dll.

[GMJCZP]

Hi cretindesalpes:

When I use version 2.0.6.2 MVTools (I have not tested the latest version) I have problems with SetMTMode, I hang up VirtualDubMod and the error message tells me about a ntdll.dll file (bug report). I use Windows XP SP3 and the version 31/03/2012 2.6.0 SEt Avisynth MT. If I use MVTools 2.5.11.3 SetMTMode works well, but with version 2.0.6.2 I have to disable SetMTMode.

[the_weirdo]

@GMJCZP:

You may want to check the latest version (2012.05.06) of SEt's Avisynth 2.6 MT.
http://forum.doom9.org/showthread.php?p=1574708#post1574708
Not sure if that fix is related to your problem though.

[GMJCZP]

I did a test with the new versions of MVTools and Avisynth and seems to work best, however it would be good to check if there is a relationship with ntdll.dll and operation of MVTools.

[ajp_anton]

dither_add16, merge16, and maybe others complain about the clips' format being different. The only difference is their length, and is fixed with a trim.
Afaik, the regular mt functions just loop the last frame of the shortest clip. Can you fix that?

[cretindesalpes]

I don't think it has to be "fixed". Trying to combine clips of different length is often a sign that something is borked in the script or the source files, and sometimes it's something subtle. I'd rather have the problem detected as soon as possible, not after hours of encoding. And if mixing different lengths is desired, I would explicitly Trim or LengthenClip at the appropriate place. Frankly, I had too many headaches because of a lack of checks on the plug-in side. But what do others think about it?

[ajp_anton]

I've used this to merge a single frame to a longer clip, as well as merge multiple recordings from a single VHS tape. The latter is what I'm doing right now. It's easier to not care about their lengths, just sync, merge, and then worry about trimming it.
It's not a big deal though, it was just unexpected and spent half an hour trying to figure out why they had different "formats"...

[cretindesalpes]

Oh I can just make the message more explicit.

[cybersharky]

QTGMC( Preset="Slow",FPSdivisor=2)
o=last
super = MSuper(pel=2,sharp=2)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
bc1 = o.MCompensate(super,backward_vec1)
fc1 = o.MCompensate(super,forward_vec1 )

Interleave(fc1,o,bc1)
MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)
SelectEvery(3,1)

How do I use MDegrain2 as the denoiser with MCompensate?

[StainlessS]

You would (I think) need a new Super & set of vectors on the interleaved clip as the ones already calculated
relate to the original (o) clip and not the interleaved one.
Could well be wrong but think that there is little to be gained by doing motion compensation twice.
Suggest that you just use instead, the function linked below after QTGMC.
http://forum.doom9.org/showthread.php?p=1508289#post1508289 # By Didee, MCDegrain

EDIT: Changed above link to point to specific post rather than thread.

EDIT: Ref the code in prev post, if you throw away the vec2 vectors, then you could use as is
using some denoiser that takes a single frame either side of the frames to denoise eg FFT3DFilter,
instead of the call to MDegrain2 (which requires two frames either side of frame to denoise).

EDIT: If this is for B5 Ivana's head spots, then perhaps QTGMC has already applied temporal denoising
and partially migrated some of the spots into adjacent frames making it difficult to remove the spots later.
Despotting really should be done before any temporal processing, perhaps on even/odd separated fields,
separately.

[GMJCZP]

Continued instability in MVTools (see my previous post). :(

PD: Or is it a incompatibildad with the latest version of avisynth?

[cretindesalpes]

QMJCZP:

Please be more specific. I would need your script, as simple as possible to make it crash, and an excerpt of your source clip that exhibits the problem.

[GMJCZP]

Here the sample and script:

Here (http://www.mediafire.com/?vaiwldoae21pft7)

Thanks.

[ajp_anton]

I feel like I missed something trivial, but is there an easy way to convert RGB24 to RGB48?

[cretindesalpes]

QMJCZP:

Thank you. I'll check your sample.

ajp_anton:

To convert from RGB24 to the format I call RGB48Y:
Interleave (
\ ShowRed (pixel_type="YV12"),
\ ShowGreen (pixel_type="YV12"),
\ ShowBlue (pixel_type="YV12"))
Dither_convert_8_to_16 ()

[ajp_anton]

That's what I ended up using. I just thought there'd be an easier way. Why not make dither_convert_8_to_16 support RGB also?
BTW, any reason not to have RGB48 as stacked RGB instead of 3x stacked Y-planes?

[cretindesalpes]

Because these tools are designed to work with planar data, which RGB is not. I could make them support any possible colorspace but this is a lot of work I don't want to do. My main goal for the version 1 of this package is to bring new possibilities in Avisynth processing, not to achieve production-class tools. That's why everything looks like clumsy hacks ; I just don't care about elegant design because I didn't plan any specification or anything, I just go where it looks interesting at the moment.

[cybersharky]

It appears I'm also encountering MVTools issues.

SetMemoryMax(512)
SetMTMode(2,0)
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mvtools2.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\avstp.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\dither.dll")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\QTGMC-3.33.avsi")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\dither.avsi")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mt_xxpand_multi.avsi")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mt_masktools-25.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\RemoveGrainSSE2.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\NNEDI3.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\TTempSmooth.dll")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\FFT3DFilter.dll")
Import("D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\McDegrainSharp.avsi")
LoadPlugin("D:\MeGUI_OneClick_Preview\tools\dgindex\DGDecode.dll")
mpeg2source("C:\Users\me\Documents\DVDrips\BABYLON5_S4_DISC4_E15\VIDEO_TS\VTS_01_1.d2v")
QTGMC( Preset="Slow",FPSdivisor=2)
LanczosResize(640,368)
McDegrainSharp()
GradFun3 ()


Faulting application name: x264.exe, version: 0.124.2197.0, time stamp: 0x4f97d104
Faulting module name: mvtools2.dll, version: 2.6.0.3, time stamp: 0x4faf77d0
Exception code: 0xc0000005
Fault offset: 0x000a2a61
Faulting process id: 0x1a0c
Faulting application start time: 0x01cd39855f35be4d
Faulting application path: D:\MeGUI_OneClick_Preview\tools\x264\x264.exe
Faulting module path: D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mvtools2.dll
Report Id: a725c16f-a578-11e1-8831-1c6f65d777b9
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2012-05-24T08:15:42.000000000Z" />
<EventRecordID>27978</EventRecordID>
<Channel>Application</Channel>
<Computer>my-PC</Computer>
<Security />
</System>
<EventData>
<Data>x264.exe</Data>
<Data>0.124.2197.0</Data>
<Data>4f97d104</Data>
<Data>mvtools2.dll</Data>
<Data>2.6.0.3</Data>
<Data>4faf77d0</Data>
<Data>c0000005</Data>
<Data>000a2a61</Data>
<Data>1a0c</Data>
<Data>01cd39855f35be4d</Data>
<Data>D:\MeGUI_OneClick_Preview\tools\x264\x264.exe</Data>
<Data>D:\MeGUI_OneClick_Preview\tools\avisynth_plugin\mvtools2.dll</Data>
<Data>a725c16f-a578-11e1-8831-1c6f65d777b9</Data>
</EventData>
</Event>

[Dogway]

Just wanted to add my 2cents. I got a thread deadlock while encoding to FFV1, I don't know if it's relevant since I don't encode as often but I used:
setmtmode(2,2)
smdegrain(...)
I wonder if that's right, or avstp.dll has crashes with MT, I chose to take out avstp.dll for that specific encode. Hope it helps...

[Boulder]

I had issues with MVTools internal multithreading and SetMTMode, disabling internal multithreading fixed them.

[Boulotaur2024]

Yeah, I've always had troubles using anything else than SetMtMode(5) or no SetMtMode at all iirc, the recommended SetMtMode(3) MySource() SetMtMode(2) SMDegrain always crashed eventually even though it first looked like it was about to work.

By the way a huge thanks to CretinDesAlpes (chapeau l'artiste si tu me permets), it really felt like night and day between not using your tools at all and learning how to finally properly use them (it's not so easy though). The added value is like 200% on the final encode, I've always wanted to thank you but couldn't for some reason. You have all my respect... and more

[cretindesalpes]

Boulotaur2024:

Thanks for your support!

Dogway:

I couldn't reproduce the deadlock. I used smdegrain() with default settings, and encoded ~10000 PAL frames with FFV1 in CLI FFmpeg.

EDIT: I tried with tr=4 and it freezed after 4500 frames. Now I have something to investigate!

cybersharky:

I couldn't reproduce the crash. I tried encoding about 20000 frames. However:
– I don't know the number of threads you used, as you just specified 0 in the SetMTMode parameter. I tried with 4 threads (my native configuration).
– It's strongly advised to run the source filter (mpeg2source here) in MT mode 5 and to switch to mode 2 afterwards.

QMJCZP:

I couldn't make your script crash. But I have a few more questions:
– Could you confirm that the script crashes with the exact source sample you provided in the zip file? It is very short, and the script loaded a different file.
– The SetMTMode() functions were commented in your script, and I have no idea what was the number of threads it actually used. This matters because of the related memory usage, the cache memory being fixed to 512 KB.
– I couldn't find any function called PreF_MDeGrain2.avsi (apparently from John Meyer), so I had to bypass it. I would need it to check the script.
– While looking for a link to this function, I stumbled across a Didée's post (http://forum.doom9.org/showpost.php?p=1345390&postcount=1285) regarding a Clense instability in MT mode. And your script uses Clense via RemoveDirt2. So you could try to replace it with the Didée's suggestion.

[GMJCZP]

Thanks for your support!

QMJCZP:

I couldn't make your script crash. But I have a few more questions:
– Could you confirm that the script crashes with the exact source sample you provided in the zip file? It is very short, and the script loaded a different file.
– The SetMTMode() functions were commented in your script, and I have no idea what was the number of threads it actually used. This matters because of the related memory usage, the cache memory being fixed to 512 KB.
– I couldn't find any function called PreF_MDeGrain2.avsi (apparently from John Meyer), so I had to bypass it. I would need it to check the script.
– While looking for a link to this function, I stumbled across a Didée's post (http://forum.doom9.org/showpost.php?p=1345390&postcount=1285) regarding a Clense instability in MT mode. And your script uses Clense via RemoveDirt2. So you could try to replace it with the Didée's suggestion.

Thanks cretindesalpes:

I will answer in order:

- The source is the same, what happens is that as I have no internet itself I could not upload a file larger. The script is generic, because it is for a series of DVDs.
- Indeed, I use SetMTMode (), what happens is grayed out to prevent the crash, for your testing simply remove the # to the left of the command. My PC is a Core 2 Duo E4400 with 2GB RAM.
- Sorry for not having included PreF_MDeGrain2 function, here the script:

function PreF_MDeGrain2(clip src)
{

preNR = src.unfilter(-6,-6)

preNR_super = preNR.MSuper(pel=2, sharp=1)
src_super = src.MSuper(pel=2, sharp=1, levels=1)

backward_vec4 = MAnalyse(preNR_super, isb = true, delta = 4, overlap=4)
backward_vec2 = MAnalyse(preNR_super, isb = true, delta = 2, overlap=4)
forward_vec2 = MAnalyse(preNR_super, isb = false, delta = 2, overlap=4)
forward_vec4 = MAnalyse(preNR_super, isb = false, delta = 4, overlap=4)

src.MDegrain2(src_super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,thSAD=400)
}

- I'll try the Didée's suggestion. Thanks for the tip.

Whatever else you need let me know.
Good luck, and thanks again.

[cretindesalpes]

QMJCZP:

Thank you. But I still couldn't make the script crash.

Dogway:

OK, I think I found the problem. Please update avstp (http://forum.doom9.org/showthread.php?p=1564516#post1564516).

[Lenchik]

Someone may be interested in implementation of such conversion through Dither plugin: http://forum.doom9.org/showthread.php?p=639432#post639432

[nhope]

I'd be interested in suggestions for dealing with the banding that you see starting at about 03:00 in my test video on YouTube (http://www.youtube.com/watch?v=u5qmfIKGMKA), as the blue opening of the cave fades up from black.

The clip was shot in HDV (1440x1080-60i). There is no significant banding in the original clip. There is some noise. For YouTube I bobbed to 30p as follows, then wrote an RGB UT Video Codec lossless intermediate for further editing in Vegas, then encoded it to x264 in MeGUI at crf 18.

#Clip frameserved in RGB from Sony Vegas Pro
AviSource("d:\fs.avi")
ConvertToYV12(interlaced=true)
AssumeTFF
QTGMC( Preset="Slower", EdiThreads=2 )
Spline36Resize(1920,1080)
AssumeFPS(29.97)

I can provide source if it helps. If so, the original .m2t HDV file, or the lossless file from later in the process?

[Reel.Deel]

Hi nhope, I'm assuming your original HDV source is YUV 4:2:0. Correct? Why not deinterlace in it's original color space and save to a lossless file and then import to Vegas? It saves you from having extra unnecessary color space conversions.

BTW the link to your video on youtube does not work.

[-Vit-]

I'd be interested in suggestions for dealing with the banding that you see starting at about 03:00 in my test video on YouTube (http://www.youtube.com/watch?v=u5qmfIKGMKA), as the blue opening of the cave fades up from black.
Before you start debanding it you could try to remove the cause of the banding. Maybe x264 or QTGMC. For the occasional manual fade you can use x264 zones: for example add: --zones 100,200,b=1.5 to the command line to give 1.5x bitrate to frames 100 to 200. Add multiple zones so: --zones 100,200,b=1.5/400,500,b=1.5. You can also adjust the quantizer with q=x instead of b=x, but I find that value hard to guess at. (Edit: CRF 18 sounds plenty for 1080, but sometimes x264 seems to underestimate the bitrate needed on fades)

In QTGMC, adding DCT=5 sometimes helps with fades.

[nhope]

BTW the link to your video on youtube does not work.
Thanks. Now fixed (http://www.youtube.com/watch?v=u5qmfIKGMKA).

I'm assuming your original HDV source is YUV 4:2:0. Correct?
Correct.

Why not deinterlace in it's original color space and save to a lossless file and then import to Vegas? It saves you from having extra unnecessary color space conversions.

What source method would you suggest? I wanted to use DGIndex/MPEG2Source directly on the file, having read that it's the preferred option (http://forum.videohelp.com/threads/343362-Avisynth-MPEG2Source) for MPEG source, but it "expanded" my colorspace to PC levels, even if I selected "TV Scale" in the video options. Same with FFVideoSource. If I "squeezed" that back with SmoothLevels(preset="pc2tv"), the result is much further away from the original than if I go via RGB then YV12 as in the script I posted.

Before you start debanding it you could try to remove the cause of the banding...
Thanks. I'll try out your suggestions.

Notwithstanding -Vit-'s suggestions, which of the following scripts makes more sense for avoiding banding in this scenario? Or something else? And should I be looking at using a "coarser" dither?

...
QTGMC()
Spline36Resize(1920,1080)
dfttest (lsb=true)
DitherPost ()

...
QTGMC()
Spline36Resize(1920,1080)
GradFun3 ()

[nhope]

Well, I'm getting a pretty good result with this...

QTGMC( Preset="Slower" )
Spline36Resize(1920,1080)
dfttest
GradFun3 (radius=16)

... but the overall luminance of the "bounces" up and down compared to the more noisy original. Adding ampo=2 made that worse (although it seems to give a little less banding when studying an individual frame).

Now I'm wondering if all I should be doing to avoid this banding in the fade on YouTube is just to add noise with AddGrainC? Or if it's simply unavoidable no matter what I do due to YouTube's encoder and the Flash Player?

I'm also interested in what the code might be for implementing a mask that would make the filter only operate on the blue water and not the darker areas of the diver and near-black cave.

Here's a link to a frame grab (http://dl.dropbox.com/u/21489814/cave-after-qtgmc-no-denoise-no-debanding.png) after QTGMC and resizing, but without any denoising or debanding/dithering. And here's a link direct to the part of the video (http://www.youtube.com/watch?v=u5qmfIKGMKA#t=00m30s) with the "radiating" banding. Obviously it behaves differently at different resolutions.

[cretindesalpes]

It will be very difficult to get rid of the band crawling in a Youtube video. Maybe you can try GradFun3 (mode=6, ampn=4, staticnoise=true) on this fade, but I doubt it will be of any use.

You could also give a try to the LumaDB (http://www.mediafire.com/?gfvlpplt03rmm#wty3h3izyk5md) script.

[Reel.Deel]

What source method would you suggest?

Hi nhope, for the most part I usually use DGIndexNV and sometimes FFMS2. I use them on my Canon DSLR files with the Cinestyle profile and the colors do not get expanded nor squeezed.

Maybe it would be best if you upload a sample from the original file.

[poisondeathray]

Probably not much you can do. Even if you "fix" it on your end before uploading, by the time YT re-encodes it - the "banding" will come back . You need lots of bitrate to retain dither and noise, and 10bit helps but that's not applicable here







What source method would you suggest? I wanted to use DGIndex/MPEG2Source directly on the file, having read that it's the preferred option (http://forum.videohelp.com/threads/343362-Avisynth-MPEG2Source) for MPEG source, but it "expanded" my colorspace to PC levels, even if I selected "TV Scale" in the video options. Same with FFVideoSource. If I "squeezed" that back with SmoothLevels(preset="pc2tv"), the result is much further away from the original than if I go via RGB then YV12 as in the script I posted.



They don't "expand" anything . Both FFMS2 and MPEG2Source serve the original YV12. It might be vegas and studio RGB workflow , but you have to be more clear on the exact steps you are using

[nhope]

Thanks for the replies. LumaDB at default settings is very bandy. GradFun3 (mode=6, ampn=4, staticnoise=true) is looking very promising! It previews more nicely than everything else I've tried. When I went to encode in MeGUI so I could upload some further tests to YouTube I was mysteriously struck by this problem (http://forum.doom9.org/showthread.php?p=1576774#post1576774) so I'll report back about the colorspace stuff after I've sorted that out.

[raido]

I've been using...

zzz_denoise ()
DitherPost (mode=2)

It's awesome. I workes really well at reducing posterization. Typically my sources have noise that I haven't fully eliminated. Sometimes when this noise is encoded, the posterization "flickers" on the borders between each band. This filter is the closest thing I've found that comes close to completely eliminating them while retaining detail. I also got a 10% reduction in file size (using crf 21). The thing is that it's really slow (getting about 3 fps on a 720x576 source). I tried setmtmode 2,3,4 and they don't seem to work. Can this be multithreaded?

[Revgen]

Here is >>>> dither-1.17.0.zip <<<< (http://ldesoras.free.fr/src/dither-1.17.0.zip).

The link isn't working for me. AVSTP link isn't working either.

EDIT

Link works now. Thank You.

[Lenchik]

Dither_convert_yuv_to_rgb (lsb_in=true)
Dither_convert_rgb_to_yuv (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2),
\ lsb=true, output="YV24")
Is this the only way to convert from YV12 to YV24 or YV16? Is there or will be any way to convert it without going to RGB?

[cretindesalpes]

Dither 1.18.0: (http://forum.doom9.org/showthread.php?p=1386559#post1386559)

Added Dither_quantize to dither to bitdepths higher than 8 (almost the same function as previously posted on this thread).
Added Dither_srgb_display for Y'CbCr preview and accurate screen captures.
More meaningful error messages about wrong clip formats.
SmoothGrad and Dither_box_filter16 now work correctly on picture of width > 2048
Fixed a bug in multithreaded Dither_resize16.
Bug fixed in MVTools (multithreaded MCompensate).

Lenchik:

Here is another solution to go from YV12 to YV24, assuming progressive frames and MPEG-2 chroma placement:
# Stack16 clip on input

w = Width ()
h = Height () / 2
u = UToY8 ().Dither_resize16 (w, h, kernel="spline36", src_left=0.25, u=1, v=1)
v = VToY8 ().Dither_resize16 (w, h, kernel="spline36", src_left=0.25, u=1, v=1)
YToUV (u, v, last)

# Stack16 clip on output

I hope I got the src_left coefficient right, chroma placement always give me headaches. Use w = Width () / 2 and src_left=0 to obtain 4:2:2 instead of 4:4:4.

[wOxxOm]

Added Dither_quantize to dither to bitdepths higher than 8 (almost the same function as previously posted on this thread).That old function with 10bit dithering hack wasn't applying dither to msb and copied it as is, actually dithering only the least significant 2 bits - and what about Dither_quantize?

[cretindesalpes]

I was referring to this function (http://forum.doom9.org/showthread.php?p=1519194#post1519194), which works as expected.

[wOxxOm]

Oh, thanks, I overlooked that one. However why isn't there a 'mode' parameter?

[cretindesalpes]

You're right, I forgot it… :eek:

You can redownload the archive, I've just fixed it without changing the version number.

[Reel.Deel]

Thanks for the update cretindesalpes! I noticed that avstp is still v.1.0.0 instead of v.1.0.1. I don't know if this is by mistake or not.

[Tempter57]

cretindesalpes

This is taken from the dither.avsi documentation:
dfttest (lsb=true)
SmoothGrad ()
Dither_quantize (10, mode=6,reducerange=true)
Dither_convey_yuv4xxp16_on_yvxx ()

[cretindesalpes]

Reel.Deel:

Yes, a mistake again, shame on me. The package is updated to v1.18.1 with the right dll version. I hope I got everything right this time.

Tempter57:

Use reducerange=true only if you want to encode with --input-depth 10. In this example, the bitdepth was still 16. But it wasn't very clear actually, so I elaborated a bit more.

[shark000X]

cretindesalpes, hi!

Thank you very much for the great tools that for last years stay to be revolutionary from a technical point of view.

It seems, if we use
Dither_convert_8_to_16()
Dither_quantize(8, mode=6, reducerange=false)
no matter which mode, the result is the same as if Dither_quantize is not used at all. Is it a feature, or my mistake?
In the above example the reducerange=true variant just flips the MSB and LSB, no other changes also.
May be the Crop() is enough here to go back to the natural 8 bits?

Thanx

[mandarinka]

The first command sets LSB part to zeros. Therefore, the second command dithers those zeros into the unchanged MSB. It's sorta logical for the MSB part (8-bit) to be unchanged after that. And since the LSB was zeros too, nothing changed there too.

[Reel.Deel]

Hello, I keep getting the green screen problem when I use Dither_convert_8_to_16(). I'm using the latest MaskTools 2. I searched hi and low for older versions of masktools on my system but came up with nothing. Dither was working just fine about a month ago and everything seemed to be ok. I went back to and older version of dither and it did not fix it. I'm using Windows XP 32bit and running SEt's latest Avisynth. Any suggestions will be greatly appreciated.

[cretindesalpes]

Please post your script.

[mandarinka]

Are you sure you have the right dll too? mt_masktools-25.dll for avisynth 2.5.8, or mt_masktools-26.dll for avisynth 2.6?

[SilaSurfer]

cretindesalpes hello.

Could you post an example of multicompensating temporal-spatio denoiser like FFt3dfilter or Dfttest using Multi vectors. I can't get it right.

[Reel.Deel]

Hello, this is the script I have been using. I have tried the official version of MastTool2 v2.0a48 and also Vit's MaskTool2 v2.0a48 modded for the updated 2.6 interface. Both seem to produce the same green screen.

setmemorymax(1024)

#Load Plugins

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Dither\avstp.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Dither\dither.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Dither\mt_masktools-26.dll")
#LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Dither\mt_masktools-26 vit.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Dither\mvtools2.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Dither\dfttest.dll")

#Load Scripts

Import("C:\Program Files\AviSynth 2.5\plugins\Dither\dither.avsi")
Import("C:\Program Files\AviSynth 2.5\plugins\Dither\mt_xxpand_multi.avsi")

#=============================================================================

SETMTMode(3,0)

#Source

FFVideoSource("MVI_1397.MOV")

SetMTmode(2)

Dither_convert_8_to_16 ()

Dither_resize16 (1280, 720)

I only get the green screen problem lower half after using Dither_convert_8_to_16 (). If I add Dither_resize16 (1280, 720) the green screen problem goes away and the LSB looks correct ( at least I think :) ) Is this the expected behavior of dither?

I was snooping around the dither.avsi, out of curiosity is this all that Dither_convert_8_to_16 is? I know I must be missing something.

Function Dither_convert_8_to_16 (clip src)
{
StackVertical (src, src.Dither_gen_null_lsb ())
}

Function Dither_gen_null_lsb (clip src)
{
vers = VersionNumber ()
p_t = (vers < 2.60) ? "YV12" : Dither_undef ()
BlankClip (src, pixel_type=p_t, color_yuv=0)
}

Function Dither_undef () {}

[Lenchik]

Here is another solution to go from YV12 to YV24, assuming progressive frames and MPEG-2 chroma placement
Great script! Thank you! Never thought that simply putting clips of proper dimensions to YToUV will create YV24 or YV16.
Cheat sheet of chroma placement for such script (and maybe for interlaced material in advance) would be great too.

[cretindesalpes]

SilaSurfer:

tr = 3 # Temporal radius
sup = MSuper ()
vec = sup.MAnalyse (multi=true, delta=tr, blksize=8, overlap=4)
MCompensate (sup, vec, tr=tr, thSAD=200)
dfttest (sigma=8, tbsize=tr*2+1)
SelectEvery (tr*2+1, tr)

Reel.Deel:

It's perfectly normal to have a green half screen on the bottom part. It means all LSB data are 0, which is normal right after a Dither_convert_8_to_16. If you put a DitherPost() just after, you'll see that everything is fine. The symptom of a wrong Masktools version is 100 % green screen whatever the original content.

[Reel.Deel]

Thanks for fast responce cretindesalpes!

I guess I just got a little paranoid after seeing the naked results of Dither_convert_8_to_16. :)

[SilaSurfer]

cretindesalpes thanks for helping out.

[atra dies]

"709" ITU-R BT.709 transfer curve for digital video
"601" ITU-R BT.601 transfer curve, same as "709"
"170" SMPTE 170M, same as "709"
"240" SMPTE 240M (1987)
"srgb" sRGB curve

These are the current possible curve values in the dither.html for linear to gamma to linear. I used default for SD content a month or two ago and remember seeing it defaulted to srgb and then looked that up to see if it was the same as 601. (I am not clear on this stuff, I just read something about srgb and 709 are the same till you get near zero, there's a hook to prevent something and shadows are lifted or something like that) Were the curves right then? I can easily do them over.

For SD, use 601, which is the same as 709, which is for HD too? srgb is for computer images?

Also, I'm not complaining just letting you know in case you want to know in the dither html I see things copied to their inverse functions like The matrix used to convert the Y’CbCr pixels to R’G’B’. is found under Dither_convert_rgb_to_yuv when it should be rgb to ycbcr pixels, right?

[cretindesalpes]

The default transfer curve in the gamma-linear functions has always been sRGB, and still is. It is mostly used for still pictures and computer display. For SD video, use the "601" curve, which is the same as for HD. I think these formulas are right unless I did something wrong and nobody noticed. Anyway the video curve is clearly different from the sRGB one, not just for the darkest shades. It looks brighter in general, when going from gamma to linear.

Also, thank you for reporting the copypasta mistake in the documentation.

[atra dies]

Ugh this is why we lowly end users need example scripts explaining what to use for SD or HD. Now I have to do some encodes over, but only a couple HAHAHAHA!, cause I waited and did things that didn't require a resize until I was sure of what was going on and what settings to use (and saved my vafs). I tell you what though, I compared a default to a 601 linear resize and didn't see a difference. But when I put 601 on to linear and srgb back to gamma then there were big problems.

OK it's my fault cause I saw it was srgb and misread a page when checking to see if that meant 601 too (or the page had it wrong) but I think 601/709 should be default cause avisynth is mainly for yv12 digital video IMO. At the time 601 was not listed and I thought 709 was just for 709.

I also wish for a 16bit colormatrix mod cause the convert to rgb solution for 709 to 601 is really slow (for rgb48) and doesn't look all that different from tritical's Colormatrix plugin as it is.

Edit: OK I just saw this wi-pedi page (http://en.wikipedia.org/wiki/File:SRGB_gamma.svg) and the difference between the srgb and 601 curves is the red and black respectively, right? They don't look much different to me. :confused:

[cretindesalpes]

The black curve on the wikipedia page is the ideal gamma 2.2 transfer curve. Here are the transfer curves that the Dither gamma functions use:

http://sadpanda.us/images/1033118-45XNOXO.png

[Akai-Shuichi]

I don't know if this is the right place to post this, but I was wondering is there any way at all that it is possible to fix the banding in this?
http://i49.tinypic.com/30sjzt3.jpg

I have tried GradFun3, but it doesn't seem to do much to it... I'm kind of noobish at avisynth for the record.

[Are_]

Hi,

I found gradfun3() speed is inconsistent between modes on my cpu (smode 2 is way faster than it should be).

I ran a few short tests using this simple script:
ColorBars(pixel_type="YV12")
trim(0,1000)
gradfun3(smode=3)
Test report on Windows 7 (http://paste.kde.org/506108/)
Test report on Linux (wine) (http://paste.kde.org/506120/)
Test report on Windows 7 (laptop) (http://paste.kde.org/506156/)

On Linux it only gets worst :(

At first, I was thinking it was cause my amd cpu on my main box, because a friend of mine with an I5 reported to me he was getting the expected fps from the readme, but the quick test on my i5 laptop reported about the same results as my main box. Is the speed of smode 2 ok?

All test were done with last build of Avisynth 2.6 MT and the latest version of all plugins (I also tested a little Avisynth 2.5.8 but it was the same).

[atra dies]

This happened to me before where I thought "this is way faster than I remember" then I realized I had smode=3 which is an "undocumented mode". 2 is the bilateral slow one. In the code you just posted, it says smode=3, which was fast for me months ago when it happened.

Akai-Shuichi, looks like jpeg or mpeg artifacts which I use mdegrain almost always and that smooths it a little but something stronger would be deblock_qed or deblock or some type of smoothing. You could use the smoothing in smoothadjust16 as well as adjust levels.

[Keiyakusha]

I don't know if this is the right place to post this, but I was wondering is there any way at all that it is possible to fix the banding in this?
http://i49.tinypic.com/30sjzt3.jpg

I have tried GradFun3, but it doesn't seem to do much to it... I'm kind of noobish at avisynth for the record.
IMHO it needs not debanding but levels fixing and strong but masked smoothing.

[cretindesalpes]

smode 3 is an experiment I started but I was unhappy with it. I had to make a new release for other reasons and I decided to leave this unfinished code in the script so I could continue working on it later. I thought it was funny to mention a dissuasive "undocumented mode" in the doc, but it is even funnier to see that people actually use it (http://www.google.fr/search?hl=all&q="gradfun3"+"smode%3D3")!

As for speed, smode 2 is not that slow nowadays, mainly because partial sampling is activated for the bilateral filter. Set subspl=1 for maximum slowness. It also relies on threading from avstp.dll, so the discrepancies may be coming from that too.

[Are_]

Oh I see, thx for the clarification. Even at the maximum radius (48), smode 2 is still faster than smode 1, that's a pretty impressive speed boost from initial versions.

Thx for your great tools, now I can use them with more relief :)

[atra dies]

Maybe I'm being dumb and paranoid here, but is convert to 16 supposed to have this overlap at the border?

And also I want to double the chroma instead of interpolating it to convert to yuy2. Is this right for 720x480 clips? I'm pretty sure it is but I don't want to make a stupid mistake since I mainly do normal, simple things with avisynth.

u=UToY().Dither_convert_8_to_16().dither_resize16(360,480,kernel="point").Dither_get_msb ().converttoyuy2
V=VToY().Dither_convert_8_to_16().dither_resize16(360,480,kernel="point").Dither_get_msb ().converttoyuy2
YToUV(U, V, last.converttoyuy2)

[cretindesalpes]

Overlap: check the content of the planes by converting them to luma only. I don't know how your software converts YV12 to RGB for display, this may be an interpolation issue.

Conversion to YUY2: Don't use Dither_get_msb() to convert from 16 to 8 because it would leave the lsb information aside, use only DitherPost(). Anyway your example doesn't even need 16-bit processing, because you just pointresize (no interpolation) and keep the result in 8 bits. And it's even simpler on Avisynth 2.6: ConvertToYUY2 (chromaresample="point").

[Gavino]

And it's even simpler on Avisynth 2.6: ConvertToYUY2 (chromaresample="point").
Because of differing chroma placement between YV12 and YUY2,
ConvertToYUY2 (chromaresample="point")
has to be followed by
MergeChroma(PointResize(width, height, 0, 1))
to get the desired effect of doubling every chroma line.

[atra dies]

I thought it looked like a resize artifact but I see it at 100% too if that matters which I suspect it doesn't since you say it's an YV12 to RGB issue. I am using AvsPmod 2.2.1 and at sat=0 I don't see it. Avisynth stackvertical isn't doing it, I don't think, and I see it with mdegrain lsb out but since it adds a border I am guessing it doesn't matter for that. I see no signs of it with dither post either (but maybe I don't know what I'm talking about). Used dither display as srgb and still saw it with point kernel.

After finding out about resize and converttoyuy2 bugs for 2.5.8 I don't trust internal filters and want to use dither for almost everything. Point resize doesn't alter lsb so I cut it off and it also doesn't lowpass, right? I don't want to unnecessarily lowpass or supersample the video when processing.

[atra dies]

Nevermind, it went away when I converted to yuy2 using what I posted above although I changed the resize width/height to accept any clip like Gavino's yv12toyuy2.

[tormento]

cretindesalpes could you please put some older version online? I am getting strange glitches in latest encodings I made and I am trying to understand why.

[Are_]

http://ldesoras.free.fr/src/avs/

[Yellow_]

hi, I notice in dither tools docs examples using Gradfun3 as final op to go from 16bit to 8bit instead of a dither function, it is under the simple debanding heading I think, is using Gradfun3 a suitable alternative to a dither method specifically if further processing outside a avisynth is envisaged.

Also when going to RGB48 is a dither method involved, if so which is default, same query for going to 10bit lossless h264.

Cheers

[cretindesalpes]

GradFun3() calls DitherPost() internally, but does other things before to deband the input clip. Dithering is only the process of reducing the bitdepth with less perceptual loss than the naive way. If you don't need debanding, just call DitherPost() on your 16-bit clip.

When you're going to RGB48Y, there is no dithering involved, because dithering is only used when reducing the bitdepth. However Dither_quantize() does dithering and has the same default settings as DitherPost().

[Yellow_]

Great, thanks.

GradFun3() calls DitherPost() internally

Of coarse, slaps head :o

[tormento]

cretindesalpes: there's a bit of discussion about the glitch I found. If you mind it's there (http://forum.doom9.org/showthread.php?t=165306).

[cretindesalpes]

Dither v1.19.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added kernel inversion to Dither_resize16. Similar to debilinear(), but works differently.
Added cropping parameters to Dither_srgb_display.
Fixed a bug preventing to use the 4:1:1 colorspace in some scripted functions.
MVTools v2.6.0.5: MCompensate, MDegrainN: fixed a bug causing occasionally horizontal magenta stripes in multithreading mode.

[tormento]

Thx. I'll try ASAP.

[XMEN3]

Sorry if it's been asked before...is possible x64 version of this filter?

thanks

[cretindesalpes]

Not a the moment.

[raido]

I stumbled on this thread after searching for "Death Note" and "debanding". I have some debanding and I wanted to get rid of it. I am using the following script...


# functions that load clip here...

animeivtc(mode=1,aa=0,killcomb=2)

crop(0,2,0,0)
blackmanresize(720,400,taps=4)

fft3dfilter(sigma=3,bw=32,bh=32,ow=16,oh=16,plane=4)
limitedsharpenfaster(strength=200)
toon(0.5)
ediaa()
awarpsharp2()
gradfun3(thr=0.4,smode=2)
tweak(sat=1.1)



I still get banding, although it is much less noticable. I remembered reading a while ago that for sources with banding that dithering or adjusting the video levels can fix the problem. Since I'm relativily new to avisynth I was hoping someone here could explain to me what that means or give me a general idea about how to combat banding in general. From what I can tell it is a problem that occurs primarily in areas with low tonal/color range. I read that it can be fixed by dithering or increasing the bits per pixel. I understand that the dither package uses ordered dithering. I'm assuming that the other technique would be to increase the bit per pixel of the image, right? Is increasing the bits per pixel similar to adjusting video levels? Is this an x264 setting? Sorry if these questions have obvious answers, like I said, I'm new to this and I'm just trying to understand the potential solutions to the problems with my video source.

[mastrboy]

The Death Note dvd's should not any noticeable banding as i remember, so you are probably creating the banding with: "fft3dfilter(sigma=3,bw=32,bh=32,ow=16,oh=16,plane=4)"

[TheSkiller]

You can be almost sure that Tweak(sat=1.1) does, more or less, introduce (chroma) banding because it does not work with high precision and dithering.

Therefore, I'd suggest to put Gradfun3() at the end and use SmoothTweak (http://forum.doom9.org/showthread.php?t=154971)(saturation=1.1) instead.


I don't think adjusting levels would reduce banding, that seems illogical. Well, maybe exept if you adjust the levels so that the banding gets hidden in total blackness. :p

[raido]

The Death Note dvd's should not any noticeable banding as i remember, so you are probably creating the banding with: "fft3dfilter(sigma=3,bw=32,bh=32,ow=16,oh=16,plane=4)"

Is there an alternative denoiser that I should try on this anime? Fft3dfilter has always been my goto filter. I'll remove it when I get home and test it. Is there a more subtle alternative? This source has quite a bit of grain in it for my tastes so I went for an aggressive denoiser.


You can be almost sure that Tweak(sat=1.1) does, more or less, introduce (chroma) banding because it does not work with high precision and dithering.
Therefore, I'd suggest to put Gradfun3() at the end and use SmoothTweak (http://forum.doom9.org/showthread.php?t=154971)(saturation=1.1) instead.
I don't think adjusting levels would reduce banding, that seems illogical. Well, maybe exept if you adjust the levels so that the banding gets hidden in total blackness. :p

I was not aware of this plugin. Thanks! I'll give it a shot. Also, I didn't know what was meant by "adjusting video levels". Does this mean adjusting brightness/contrast/gamma etc. etc.? Hate to ask a novice question, but I'm not a pro like you guys.

[Wilbert]

You can be almost sure that Tweak(sat=1.1) does, more or less, introduce (chroma) banding because it does not work with high precision and dithering.

Therefore, I'd suggest to put Gradfun3() at the end and use SmoothTweak(saturation=1.1) instead.

It didn't in the old days (at least the dithering). Try

Tweak(sat=1.1, dither=true)

[TheSkiller]

Is there an alternative denoiser that I should try on this anime? Fft3dfilter has always been my goto filter.There are a dozen of denoising filters and scripts that you could try but my advice to you being new to this is: don't overdo the filtering. ;) A light bit of noise acts as a dither somewhat, filtering a video so that there is absolutely zero noise left makes it easy for banding to become visible. Try it.

If you want to try other denoisers you could give MC_Spuds (http://avisynth.org/mediawiki/MC_Spuds) a shot for example. I have used that on a pretty grainy VHS tape Walt Disney animated movie and got great results.
It's not a very newb-friendly script though, you need tons of plugins. :p



@ Wilbert Thanks, I didn't know Tweak now comes with a dithering option.

[raido]

I tried MC_spuds and I keep getting a error that says manalyse does not have an argument named "pel" I have mvtools2 and I can see in the documentation that Manalyse does indeed not have an argument named pel. Why is this passed as an argument to the function?

[StainlessS]

mvtools v1 uses pel arg in MVAnalyse().


MVAnalyse (clip, int "blksize", int "blksizeV", int "pel", int "level", int "search", int "searchparam", int "pelsearch", bool "isb", int "lambda", bool
"chroma", int "delta", bool "truemotion", int "lsad", int "pnew", int "plevel", bool "global", int "pzero", int "overlap", int "overlapV", string
"outfile", int "sharp", int "rfilter", clip "pelclip", int "dct", int "divide", bool "mc", int "idx", int "sadx264")


v2

MAnalyse (clip super, int "blksize", int "blksizeV", int "levels", int "search", int "searchparam", int "pelsearch", bool "isb", int "lambda", bool
"chroma", int "delta", bool "truemotion", int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", int "overlapV",
string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", int "badrange", bool "isse", bool "meander", bool "temporal", bool "trymany")

[raido]

Ah, I see....

/slapshead

I added the mvtools listed here (http://avisynth.org/mediawiki/MC_Spuds) which links to mvtools2. When the script said it couldn't find MVAnalyse, I thought it meant MAnalyse.

[raido]

Okay I did a few encodes with denoisers such as fft3dfilter, mc_spuds, and dfttest (and zzz_denoise). I was hoping to find a combination that removed the noise without introducing color banding but I wasn't able to do this very well. It seems that what everyone in this thread has been saying all along is true; that you have to leave in some grain in order to combat banding. The only way I was able to do both was using the following technique...

My source had minor banding in it but running fft3dfilter(sigma=3,bw=32,bh=32,ow=16,oh=16,plane=4) resulted in even more color banding. I compensated for this by following it up with gradfun3(thr=0.96) which removed the banding. When I encoded it using the x264 animation preset I still got banding. Then I found a couple of threads that recommended raising AQ, raising Psy-RD, and raising Psy-Trellis. I increased these values to 1.2, 1.00, and 0.8 respectivily. I encoded using high profile with a crf value of 18. My encoded video was much larger but the dithering was preserved. There was no noise and color banding was virtually eliminated. The cost of this was a large file size and fairly aggressive x264 settings that preserved the dithering. Personally, the picture still looks a little soft but I won't sharpen it because that seems to bring out the banding again.

[jmartinr]

The cost of this was a large file size and fairly aggressive x264 settings that preserved the dithering. Personally, the picture still looks a little soft but I won't sharpen it because that seems to bring out the banding again.

Was the result any better than just no filtering at all and encoding with crf 18?

[raido]

I won't make the noob mistake of saying my encodes are better since the word better invariable boils down to "in what way".

I was in a position where I was forced to choose between leaving in the grain/noise (i.e. no filtering or very minor smoothing/grain removal) or taking the grain out and dealing with color banding. I hate both grain and color banding. If you are the type that can tolerate either color banding or noise, then no, my result will not look any better. However, if you want a denoised and debanded result, then yes, my result will look better. My result is 856x480 and it even looks pretty good on my HD TV. I guess the trade off is file size, my files went from 65-70 MB to 185 MB when I increased AQ, Psy-RD, and Psy-Trellis.

[TheSkiller]

Another option would be to deband during playback, using ffdshow (either with it's "Deband" option or by implementing Gradfun3 or Gradfun2dbmod via the AviSynth option). Of course, that's only possible when using a PC for playback. File sizes would be just as small as they used to be and chances are the played back video looks even better debanded.


Edit: Btw, GradFun3(thr=0.96) is quite aggressive, it will soften the picture in certain areas.

[raido]

Another option would be to deband during playback, using ffdshow (either with it's "Deband" option or by implementing Gradfun3 or Gradfun2dbmod via the AviSynth option). Of course, that's only possible when using a PC for playback. File sizes would be just as small as they used to be and chances are the played back video looks even better debanded.

Unfortunately I use XBMC for my playback. It uses Mplayer. I read that Mplayer does have a deband on playback option but that it is not implemented in XBMC.

Edit: Btw, GradFun3(thr=0.96) is quite aggressive, it will soften the picture in certain areas.

Quite right. I think it is on the soft side. I tried doing some sharpening after gradfun3, but that just reintroduced the banding when encoding. I tried several sharpeners but none seemed to work. In my opinion, the picture is softer, but it was tolerable (well, more tolerable than the banding). I don't know enough about avisynth but I was hoping that there was a way to create a mask that selected areas with low tonal range and applied the debanding in those areas. I may play around with lower thr values in the future.

[cretindesalpes]

I was hoping that there was a way to create a mask that selected areas with low tonal range and applied the debanding in those areas.
That's what GradFun3 does; it automatically sets thr_det according to thr. If you need to keep thr high because of important banding steps, try to lower thr_det a bit (2 or 3, but not less) to preserve more details and textures.

[jmartinr]

@raido

fft3d introduces banding. I'd recommend motion adaptive denoising:

# degrain
super = MSuper(pel=2, sharp=1)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
MDegrain2(super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD=400, lsb=true)
DitherPost()

[raido]

@cretindesalpes

Ah, good to know. I found this to work quite set to 3. Thank you for the wonderful plugin!

@jmartinr

I pulled out fft3dfilter and used your script. It was really REALLY effective a "calming" the grain. I use the term calming because it still seemed to keep a minor amount, enough to prevent color banding but not enough to be very noticeable. The result was comparable to what I was getting using my aggressive gradfun3 script except that yours preserved detail much better. The thing is, I have no idea what that script does. Can you give me a brief explanation? It looks like it creates a "super" clip to do some type of vector analysis. How exactly does this work?

[ajp_anton]

The thing is, I have no idea what that script does. Can you give me a brief explanation? It looks like it creates a "super" clip to do some type of vector analysis. How exactly does this work?MSuper just prepares the clip for motion analysis. It creates multiple stacked versions of the same clip at different sizes for motion vectors of different length and precision.
MAnalyse finds the motion vectors using the super clip.
MDegrain2 uses these vectors for what it does. The 2 stands for its temporal resolution (2 frames forward, 2 backward), MDegrain1/3 also exist. Afaik it overlaps the frame block by block with matches it made from neighboring frames, strength depending on settings (thSAD and maybe others).

[jmartinr]

@ raido

I can't give you a brief explanation, sorry. See http://avisynth.org.ru/mvtools/mvtools2.html

[raido]

@ajp_anton
Thank you for the explanation, so MDegrain3 would stomp out the grain even further. I would just have to create another backward and forward vector and add them as arguments.

@jmartinr
No worries, you've helped me a lot already. I think I have a basic understanding of how MDegrain works. I see from that link that mvtools has many more useful applications. Thanks for bringing it to my attention.

Gonna go generate some CPU cycles now...

[ajp_anton]

Is there a way to use a non-16bit-aware linear filter in ~16 bits?
I had some vague ideas that may lose one bit (so 15 bits), but haven't had access to a computer with Avisynth to test anything.

[Abyssal]

For someone like me who does not make magic happen when it comes to avisynth (new to this), is there anything I should look at particularly when setting Gradfun3 up? Meaning what parameters are the things you generally want to look at when working with simple debanding? I mean, it's a lot of parameters and many that I don't think is necessary to learn about if you know nothing to begin with. For now I'm only messing with thr, thr detection, range, mode and mask - Is there any other important stuff I need to know when setting it up for the particular source I want to use it on?

[cretindesalpes]

ajp_anton:

This was more or less what Dither1Pre and Dither2Pre (8-bit input, 16-bit output) tried to achieve but this method isn't very effective and is subject to artifacts. The main problem is the limited output precision of the main filter.

Abyssal:

Mostly:

- thr: the intensity of the debanding. Tradeoff between debanding power and preservation of the textures and low-contrast details.
- radius: a large radius is able to smooth very low gradients when color steps are distant, but may fail on steeper ones.
- mode: tradeoff between visual smoothness and compressability
- smode: quality of the restored gradients, misc artifacts.

You can also play with the mask, depending on the amount of noise and details on the source.

[cretindesalpes]

Dither 1.20.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added Dither_removegrain16 (modes 1–4, 11, 12, 19 and 20 only) and Dither_repair16 (modes 1–4 and 11–14 only).
Added threshold, radius and elasticity settings for chroma to GradFun3, as well as the usual plane filters. Better compatibility with the 4:1:1 colorspace.
Added Dither_sub16 and optimised Dither_add16 for SSE2.
Fixed a regression for 4:2:0 DV-PAL chroma placement in Dither_resize16.
Colorspace conversions using point-sampling for chroma are now more consistent regarding chroma placement.

[mandarinka]

Ah, the chroma parameters are a great idea, because last time I was working with gradfun3, the filtering kept changing hues of flat areas a lot on stronger settings (and weaker ones were... weak). no idea why, iirc it was with smode=1.

[Reel.Deel]

Cretindesalpes, thank you for the update and Dither_removegrain16 / Dither_repair16.

[mastrboy]

Added Dither_removegrain16 (modes 1–4, 11, 12, 19 and 20 only) and Dither_repair16 (modes 1–4 and 11–14 only).


Nice, speed wise, is it SSE(2/3) optimized?
I want to run some benchmarks compared to the original removegrain and need to know which dll file to use....

[cretindesalpes]

Nice, speed wise, is it SSE(2/3) optimized?
I want to run some benchmarks compared to the original removegrain and need to know which dll file to use....

Yes, it is optimised for SSE2, but don’t expect something anywhere close to the original RemoveGrain. First because I suck at optimizing and I don’t want to spend weeks on assembly stuffs that I couldn’t read and fix two months after. Secondly because it has to handle 16-bit data instead of 8 which divides the optimal pixel throughput by 2. Third because the stack16 format has a large overhead for small operations like these.

However the good thing is that it is internally multithreaded, so it’s not that bad.

[mastrboy]

I see, so no real point in comparing them speed wise.
Understand completely about the assembly code though, once tried to change a hardcoded timeout value in a assembly bootcode app, took me hours to just find the timeout code within that mess (Not commenting code should be illegal).

The multithreading is based on avstp i guess? So we get the advantages of having a shared thread pool...

[Reel.Deel]

So I noticed that Dither_removegrain16 modes 0-4 dont output the LSB while modes 11-20 do. Is that the expected behaviour?

PS it's very nice to see removegrain16 work on all planar colorspace. :)

[cretindesalpes]

Reel.Deel:

If your input is 8 bits and comes straight from Dither_convert_8_to_16(), it’s normal. RG modes 1–4 don’t create new values, they just shuffle existing pixels.

mastrboy:

Yes it uses avstp.

[TheProfileth]

cretindesalples it is really cool to see you porting removegrain modes to dither tools. Do you plan to also port the removegrainHD modes/filters as well? I think it would be really cool to see some of those filters function with the help of 16bit accuracy.
On a side note, have you ever considered just doing a 16bit median filter with a variable radius?
Currently the only viable median filter with a variable radius that I know of is medianblur.
Would be really cool to see a proper 16bit median filter, and while I don't do much coding myself a median filter shouldn't be that hard to make given that you already have removegrain(mode=4) in dither tools
Much love for your excellent work at advancing avisynth as a whole.
Take care

[Keiyakusha]

I was using dither 1.19 and avisynth 2.6 32bit mt from 16may 2012 (don't remember who built it, maybe SEt) on win7 x64
then i copied following files from 1.20 to avisynth plugin folder: avstp.dll; dither.avsi; dither.dll; mt_xxpand_multi.avsi (with overwrite)
then copied dither's mvtools2.dll and avstp.dll on top of that. (with overwrite)
After that when I open avs script in some software and reload it few times it crashes. I did something wrong?
After rolling back to dither 1.19 all is fine again.
I didn't used any part of the dither package in the actual script when it crashed.

[cretindesalpes]

I can’t see what you could have done wrong but I would need a little more information to know if there is a problem with v1.20. What kind of crash? (some programs can export a detailed crash report) Is it reproducible? Is it script-dependant? Is it dependant on the host application? Are you auto-loading the plug-ins?

[Keiyakusha]

What kind of crash? (some programs can export a detailed crash report)Is it dependant on the host application?Sorry forgot that. Avsp says "error loading avisynth" and says probably some plugins are unstable. Then crashes, or survives sometimes(but preview window resets). VirtualDub just hangs so no reports for you... Probably not dependant, not sure that i have other apps where i need/can to reload scripts often.

Is it script-dependant?
I don't have much time trying lots of scripts. I used, few colorspace conversions, removegrain, resize, f3kdb, mpeg2source
Videoclip was containing 1 frame only, odd-width yv24 was involved, if that matters...
EDIT: i realise that removegrain doesn't supports yv24 i used it before conversion
EDIT2: and this is classic removegrain, not new one from dither of course.

Is it reproducible?
From 10 reloads it for sure happens once. Wasn't able to reproduce with 1.19 unless I'm really unlucky. No other plugins or scripts was changed for some time And I was already doing similar processing before and never noticed crash.

Are you auto-loading the plug-ins?
It is probably obvious now but yes. I wouldn't be posting here otherwise. It looks like autoload-related issue.

[nhope]

From the 3rd post:

dfttest 1.9.2 (mod16)
MVTools 2.6.0.5 (mod16)

Does the "(mod16)" mean that these versions only work with mod16 resolution? Do they differ from the original versions in that respect?

[cretindesalpes]

Keiyakusha:

Thanks for your report, tell me if you have news about this crash.

nhope:

No, it just means it’s a modification to support 16-bit clips.

[Keiyakusha]

cretindesalpes
I realize that its probably impossible to fix something or even identify where is the problem, I'm fine with that. But just in case want to ask what news you expect? Since the changes introduced in 1.20 is not interesting to me I planned to just use 1.19, which I used since you 1st time posted it and never experienced crash (or how should i call this, autoload fail?)... I haven't updated other plugins. No one really develops them these days...
Howewer I just remembered, one more plugin I updated. AddgrainC... but i used it since you posted it and it was ok. And to dither 1.20 updated only today.

[nhope]

@cretindesalpes Thanks. I was confusing it with the abbreviation for "Modulus 16".

[zero9999]

Using Avisynth 2.6 Dither_convert_yuv_to_rgb(), Dither_srgb_display(), etc. fail because Dither_convert_yuv_to_rgb() passes Y8 data to Dither_lut16_msb/Dither_lut16_lsb which passes it directly to mt_lutxy. But mt_lutxy doesn't support Y8 so it fails.

[cretindesalpes]

Masktools2 do support Y8. Be sure you have only "mt_masktools-26.dll" in your plug-in folder, not the "mt_masktools-25.dll" version which is for Avisynth 2.5.x.

[zero9999]

Masktools2 do support Y8. Be sure you have only "mt_masktools-26.dll" in your plug-in folder, not the "mt_masktools-25.dll" version which is for Avisynth 2.5.x.
now i feel like a complete idiot. somehow the 2.5 masktools seems to have made it back into my plugins folder along with the 2.6 version.

[Yellow_]

hi, cretindesalpes, do the dither tools with Avisynth 2.5 cover the problem I'm having with some odd raw UYVY:

http://forum.doom9.org/showthread.php?p=1590264&posted=1#post1590264

The source is JFIF encoded full range chroma + luma, chroma +- 128

Described here by Poynton:

http://books.google.co.uk/books?id=dSCEGFt47NkC&pg=PA175&lpg=PA175&dq=chroma%2BJFIF%2Bvideo&source=bl&ots=OL4zeJ7Vyr&sig=aoNstetOv-HN7PpZ3i-dOj2JxVs&hl=en#v=onepage&q=chroma%2BJFIF%2Bvideo&f=false

One solution suggested makes use of kassandro's RemoveGrain:

http://home.pages.at/kassandro/RemoveGrain/

[cretindesalpes]

Yellow_:

If it’s a signed/unsigned integer problem, I would replace 127 with 128 in the lut expression: "x 128 + 256 %". Unfortunately I don’t know much about JFIF.

—

Dither 1.21.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added the seed parameter to Dither_add_grain16. Reworked Dither_srgb_display for mor accurate aspect ratios and transfer characteristics, added gamma parameter. Gamma transfer functions have now an optional gamma correction parameter. Added Dither_median16 for median and quantile clipping of any radius, but slow and without temporal component at the moment.

[TheProfileth]

—

Dither 1.21.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added Dither_median16 for median and quantile clipping of any radius, but slow and without temporal component at the moment.

Thank you so much :o this is what I have been waiting for, for a long time.

[TheProfileth]

BTW cretindesalpes, regarding the temporal element of the current 16 bit median that you recently (and thankfully) added I was wondering if you were planning on implementing the basic limitational system from removegrain that worked quite well in degrainmedian that being this


We consider the same 3x3x3 cube.
New pixel candidate value (newp) is clipped by values of neighbor pixels from pair (bound1 and bound2).
But mode1-mode5 use more safe criterion (weight) of optimal pixel pair,
We take into account also the difference of new value from central pixel old value (oldp) .

Mode=0 of DeGrainMedian (similar to mode=9 of RemoveGrain) uses weight=|bound1 -bound2|, it is strongest.
Mode=1 of DeGrainMedian (more strong than mode=8 of RemoveGrain) uses weight=|oldp - newp|+4* |bound1 - bound2|
Mode=2 of DeGrainMedian (similar to mode=8 of RemoveGrain) uses weight=|oldp - newp|+2* |bound1 - bound2|
Mode=3 of DeGrainMedian (similar to mode=7 of RemoveGrain) uses weight=|oldp - newp|+ |bound1 - bound2|
Mode=4 of DeGrainMedian (similar to mode=6 of RemoveGrain) uses weight=2*|oldp - newp|+ |bound1 - bound2|
Mode=5 of DeGrainMedian (similar to mode=5 of RemoveGrain) uses weight=|oldp - newp|, it is weakest.

I do understand that this would probably require some more inventive thinking or complex coding in order to extend this past the originally intended 3 frames being processed at once. While we are on the topic of degrainmedian, if you were to implement a limit parameter that was to limit the amount of change per pixel, would it she represented in standard 8bit form to us then converted to its 16bit equivalent or would you allow for direct 16bit numbers? By that I mean will the limit parameter be between 0 and 255 or between 0 and 65535? Maybe I am being a bit presumptuous by asking this in advance though.

Also I would like to ask some questions regarding the currently implemented bilateral filter. You stated in the documentation that the this implementation is an approximation of a bilateral filter, like many other image processing tools do. The spatial weighting is not a gaussian curve but a box, and the value weighting is a trapezoid.
I assume this is for the sake of speed and from what I read regarding using, what is probably a similar implementation here
http://people.csail.mit.edu/sparis/siggraph07_course/slides08/06_implementation.pdf
it probably works fine.
While I don't necessarily know if that is even a viable option, I was interested to read/see this explanation of how to fix the "staircase effect" caused by the weighting in bilateral filters here http://people.csail.mit.edu/sparis/siggraph07_course/slides08/09_limitations.pdf
Not sure if this is necessarily useful however I thought it was interesting.
However could you instead use something akin to the method utilized in variableblur's binomialblur which
works by repeating a 5x5 or 3x3 kernel based on pascals triangle multiple times to blur the image. With a variance above 1 the result is very close to a true gaussian blur and much faster
Actually regarding this, I was also wondering if you could possibly port binomialblur to a 16bit version,with the exception of the box filter, as I don't particularly like the effects of box filtering, there are currently no good ways to blur things in 16bit. I would really appreciate this.
And as always keep up the good work cretindesalpes :)

[cretindesalpes]

I was wondering if you were planning on implementing the basic limitational system from removegrain that worked quite well in degrainmedian
Not at the moment. I'll stay with pure median/quantile for now, and we'll see if people use it to build more elaborate tools.

if you were to implement a limit parameter that was to limit the amount of change per pixel, would it she represented in standard 8bit form to us then converted to its 16bit equivalent or would you allow for direct 16bit numbers?
16 bit of course. It wouldn't make sense to use 8-bit values as you couldn't limit the changes below one 8-bit step. Anyway it's already possible to limit changes for any filter by using diff, lut (limiting) and add. Or you can use Dither_limit_dif16, but it limits differently.

For the staircase effect of the bilateral filter, have you encountered some cases where it shows unexpectedly? I designed it mainly for debanding purpose and I think the current implementation does correctly its job, thanks to the trapezoid approximation of the gaussian value weighting.

If you want gaussian blur, you can approximate it efficiently with a cascade of identical box filters (2–4 would do the job). This is the Pascal's triangle trick you mentioned previously, and it works with large box kernels too. Another possibility is to use Dither_resize16 with a gaussian kernel, without changing the picture size but using (negative) values for fh and fv, between -1 and 0. Increase the number of taps too. It's slow but accurate.

[TheProfileth]

cretindesalpes thanks for your responce, sorry I haven't responded sooner.
However I do have a question. Do you think you could implement a 16bit version of mt_average? I think something like this would be very nice to have, currently I am using redaverage (http://forum.doom9.org/showthread.php?t=163018) to do this, speaking of which it would be nice if you could port the parts of RAverageW and RAverageM to dithertools.

[nibus]

(edited to remove incorrect information)

[wOxxOm]

nibus, try replacing RemoveGrainSSE2.dll with SSE3 one (or plain RemoveGrain.dll), also ensure you have a new beta version from the developer's forum.

[cretindesalpes]

TheProfileth:

Why would you need an average function in Dither tools since you already have it in RedAverage?

nibus:

I don't own a 2600K but I'll try to reproduce this issue when I have time on my poor Phenom II 965. In the meantime, could you test if it crashes also with source plugins different of DGSource, as I don't have it and could not test in the same conditions as you. Also, what is the input resolution? I guess 1920x1080.

[TheProfileth]

TheProfileth:

Why would you need an average function in Dither tools since you already have it in RedAverage?

Because I have had issues with RedAverage plus the fact that I assume it would benefit from avstp and such multithreading and therefor be faster.
Plus the way that RedAverage handles it's settings is a bit off-putting.

[Keiyakusha]

nibus
If you not using new things that was added in current mvtools2, you can try different mvtools2 versions, as dither one is slowest out of three variants (including original) if you use it with mt avisynth.

BTW, back then when I reported about crashes I was using the same CPU as nibus. And they also was non-deterministic. Didn't tested if anything changed with new dither versions though as I'm no longer using it.

[nibus]

I don't own a 2600K but I'll try to reproduce this issue when I have time on my poor Phenom II 965. In the meantime, could you test if it crashes also with source plugins different of DGSource, as I don't have it and could not test in the same conditions as you. Also, what is the input resolution? I guess 1920x1080.

I apologize cretindesalpes, but it was not a problem with your modified mvtools2.dll. I just needed to adjust my usage of SetMTMode. I have it working again without crashes. Sorry for the false alarm :o

[Yellow_]

hi cretindesalpes, could you suggest what causes this grid appearance in this image extracted from AVSPmod window on this occasion but I see it in the final written 16bit images when they are sharpened as a post operation.

http://dl.dropbox.com/u/74780302/convert_yuv_rgb_RGB48Y.png

Its 16bit lsb=true output. I'm using a 16bit modded version of MCTD (Motion Compensated Temporal Denoise) to denoise and convert to 16bit RGB, then convey as rgb48_on_YV12 to Imagemagick for 16bit tif's.

I don't think this is a problem with MCTD script as I think the weave appearance happens even with a simple denoise lsb=true output and then piped to Imagemagick.

[cretindesalpes]

It looks like packed (not stacked) 16-bit data (YUV or even RGB) displayed as 8-bit YUV. How did you achieve this (script details)?

[Yellow_]

Here's the script minus the Loadplugins("").

SetMemoryMax(512)

ffmpegsource2("ffv1.mkv", threads=1)

MCTD(dfttest(sigma=0.25, lsb=true), chroma=true, settings="very low")

Dither_convert_yuv_to_rgb(matrix="601", tv_range=false, cplace="MPEG2", lsb_in=true, output="rgb48y")
#Dither_y_gamma_to_linear (tv_range_in=false, tv_range_out=false, curve="709")
Dither_convey_rgb48_on_yv12 (SelectEvery (3, 0),SelectEvery (3, 1),SelectEvery (3, 2) )

Opened in AVSPmod and using vdcrimms excellent PipeRGB to Imagemagick.

http://forum.doom9.org/showthread.php?p=1599178#post1599178

Wicked script for batch processing to 16bit image sequences avoiding high memory usage / crashing.

But this wierdness has been going on for a while even just using the CLI for piping, also just simply using 16bit mods of SMDegrain or dfttest, without MCTD, not sure whether it's just my set up, or bad usage of Dither tools or a bug.

Here's a link to a straightforward denoised frame to stacked:

http://dl.dropbox.com/u/74780302/grid.png

Using this:

baselayer = ffmpegsource2("Mov.MOV", threads=1)

dfttest(baselayer, sigma=0.25, lsb=true, Y=true, U=true, V=true)

To Illustrate grid I added ConvertToRGB(matrix="PC.601") to the bottom of the above script and saved the image frame from AVSPmod.

But I see same grid in 8bit images converted from 16bit tifs written from Imagemagick if the 8bit images are sharpened for example. Perhaps this is due to the 16bit to 8bit image conversion via numerous methods outside of Avisynth like The Gimp, ffmpeg and IM not dithering in that conversion?

[cretindesalpes]

Is the grid you are referring to the occasional patterns shown in the LSB part of the picture? If so, there is nothing wrong. They must result from blocking or other compression artefacts. Please post a few untouched frames of your source if you want me to check this issue.

[TheSkiller]

cretindesalpes, I have a question about chaining dither_convert_X_to_X(). :)

I'm in the position where I need to convert a native RGB24 clip to YV12 to process it with certain filters. After that I need RGB24 again for further processing in a NLE. It's an unfortunate situation but there's no way around it.

My question therefore is: Do you recommend chaining dithered color space conversions, or should I rather use AviSynth's internal conversions except for the very last conversion?

My concern is the dither gets "stacked" which might cause artifacts or blurring?

[cretindesalpes]

Converting with dithering (error diffusion mode without additional noise) will always be better than no dithering at all. If the filters you use only affect specific parts of the picture, you can use a mask and copy the untouched areas from the original RGB24 clip instead of the double conversion. This will at least save the effects of the chroma subsampling.

[TheSkiller]

Thanks a lot for your input. :)
Chroma subsampling is another issue right... :rolleyes:

[Yellow_]

hi, thanks here's a link to the file:

http://dl.dropbox.com/u/74780302/MVI_0561.mp4

I wonder if the 'grid' appearance is made to look worse by my set up, I recently updated my NVidia graphics card driver and I've noticed today seeing a definite pattern viewing images and even images uploaded from an ipad, so looks like display scaling issues outside of avisynth / AVSPmod making things look worse.

[cretindesalpes]

Indeed, the "grid effect" comes from blocking beeing smoothed by the denoising filter. So be reassured, there’s nothing wrong here.
o = FFVideoSource ("MVI_0561.mp4", threads=1)
f = o.dfttest(sigma=0.25, lsb=true, Y=true, U=true, V=true)
l = f.Dither_get_lsb ()
Interleave (l, o.Dither_convert_8_to_16 ().histogram_luma_16 (), f.histogram_luma_16 ())

Function histogram_luma_16 (clip src, int "amp", bool "tv_range")
{
amp = Default (amp, 16)
tv_range = Default (tv_range, true)

mi = (tv_range ? 16 * 256 : 0)
ma = (tv_range ? 235 * 256 : 65535)
rpa = (ma + 1 - mi) / amp - 1
mis = String (mi)
rpas = String (rpa)
amps = String (amp)
e = rpas+" x "+mis+" - "+rpas+" 2 * % "+rpas+" - abs - "+amps+" * "+mis+" +"

src
Dither_lut16 (e, u=1, v=1)
DitherPost (mode=6, u=-128, v=-128)
}

[cretindesalpes]

Indeed, this is a bug related to Dither_quantize in general. It's not specifically caused by the ampo value, althought it increases the probability to get this kind of artefact, as well as reducerange=true. I'm not sure I can do something unless rewriting the quantization function in a native form instead of scripting.

[mandarinka]

So the problem is from some sort of error in Dither_quantize?
I usually use that to dither to 10-bit for encoding. Does this affect other modes (6-8) besides the default?

[cretindesalpes]

Actually as soon as the dither amplitude is greater than unity (ampo + ampn > 1), there is a risk of getting this kind of inaccuracy in Dither_quantize, whatever the dither method. When reducing to 10 bits, it affects a small range of pixel values below any multiple of 16 (on a 8-bit scale). Keeping ampo + ampn ≤ 1 shouldn't create any noticeable artefact.

[Jenyok]

How to use Dither functions in the script below (super resolution upscale) ?
Thanks.
.

LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\MVTOOLS-V2_5_11_3\mvtools2.dll")
LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\NNEDI3\nnedi3.dll")
LoadPlugin("C:\PROGRAM FILES\AVISYNTH 2.5\PLUGINS\NNEDI_20070920\nnedi.dll")



#
# http://forum.doom9.org/showthread.php?t=142704
# Author: Efenstor
#
function MSR(clip clp)
{
# Constants
# Change those constants to tune for the best result of interpolation

# MSuper
#
hpad = 8 # 4
vpad = 8
pel = 4 # 2
rfilter = 4 # 4
sharp = 2 # 2
isse = true

# MAnalyse
#
blks = hpad
blksV = vpad
overlap = blks / 2
overlapV = blksV / 2
search = 5
searchparam = 3
dct = 5

# MCompensate
#
tSAD = 10000
tCD1 = 400
tCD2 = 130

# Overlay
#
opacity = 0.5



brc = BilinearResize(clp, Width(clp) * 2, Height(clp) * 2)
# function MSuper(clip, int "hpad", int "vpad", int "pel", int "levels", bool "chroma", \
# int "sharp", int "rfilter", clip "pelclip", bool "isse", bool "planar")
#
super = MSuper(brc, \
hpad = hpad, \
vpad = vpad, \
pel = pel, \
levels = 0, \
chroma = true, \
sharp = sharp, \
rfilter = rfilter, \
isse = isse, \
planar = false)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec1 = MAnalyse(super, \
isb = true, \
delta = 2, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)


# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec2 = MAnalyse(super, \
isb = true, \
delta = 1, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec3 = MAnalyse(super, \
isb = false, \
delta = 1, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)

# function MAnalyse(clip super, int "blksize", int "blksizeV", int "level", int "search", int "searchparam", \
# int "pelsearch", bool "isb", int "lambda", bool "chroma", int "delta", bool "truemotion", \
# int "lsad", int "plevel", bool "global", int "pnew", int "pzero", int "pglobal", int "overlap", \
# int "overlapV", string "outfile", int "dct", int "divide", int "sadx264", int "badSAD", \
# int "badrange", bool "isse", int "full", bool "meander", bool "temporal")
#
vec4 = MAnalyse(super, \
isb = false, \
delta = 2, \
blksize = blks, \
blksizeV = blksV, \
levels = 0, \
search = search, \
searchparam = searchparam, \
pelsearch = pel, \
lambda = 0, \
chroma = true, \
truemotion = true, \
lsad = 1200, \
plevel = 0, \
global = true, \
pnew = 50, \
pzero = 50, \
pglobal = 0, \
overlap = overlap, \
overlapV = overlapV, \
dct = dct, \
divide = 0, \
sadx264 = 0, \
badSAD = 10000, \
badrange = 24, \
isse = isse, \
meander = true, \
temporal = false, \
trymany = false)

# clp = nnedi(clp, field=0, dh=true)
# clp = TurnLeft(clp)
# clp = nnedi(clp, field=0, dh=true)
# clp = TurnRight(clp)
# clp = NNEDI3_rpow2(clp, rfactor=2, cshift="spline64resize", qual=2, nsize=2, nns=2)
clp = NNEDI3_rpow2(clp, rfactor=2, cshift="lanczos4resize", qual=2, nsize=2, nns=2)

clp = Sharpen(clp, 1)

# function MSuper(clip, int "hpad", int "vpad", int "pel", int "levels", bool "chroma", \
# int "sharp", int "rfilter", clip "pelclip", bool "isse", bool "planar")
#
super = MSuper(clp, \
hpad = hpad, \
vpad = vpad, \
pel = pel, \
levels = 0, \
chroma = true, \
sharp = sharp, \
rfilter = rfilter, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c1 = MCompensate(clp, \
super, \
vec1, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD , \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c2 = MCompensate(clp, \
super, \
vec2, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD, \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c3 = MCompensate(clp, \
super, \
vec3, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD, \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

# function MCompensate(clip source, clip super, clip vectors, bool "scbehavior", float "recursion", \
# int "thSAD", bool "fields", int "thSCD1", int "thSCD2", bool "isse", bool "planar")
#
c4 = MCompensate(clp, \
super, \
vec4, \
scbehavior = true, \
recursion = 0, \
thSAD = tSAD, \
fields = false, \
thSCD1 = tCD1, \
thSCD2 = tCD2, \
isse = isse, \
planar = false)

t1 = Overlay(clp, c1, opacity=opacity)
t2 = Overlay(clp, c2, opacity=opacity)
t3 = Overlay(clp, c3, opacity=opacity)
t4 = Overlay(clp, c4, opacity=opacity)
f1 = Overlay(t1, t2, opacity=opacity)
f2 = Overlay(t3, t4, opacity=opacity)
last = Overlay(f1, f2, opacity=opacity)

return (last)
}

[cretindesalpes]

What "Dither functions" do you want to use? The question is strange: "How to use a screwdriver in this pie recipe?" What exactly do you expect to achieve or improve with the Dither tools?

[Jenyok]

cretindesalpes
.
I need very good quality in super resolution upscale (up to 2 times, up to 4 times) with enhanced small and very small details in video.

[TheProfileth]

cretindesalpes
.
I need very good quality in super resolution upscale (up to 2 times, up to 4 times) with enhanced small and very small details in video.
And I need him to finish that filter called AddDetail() ;)
By just saying what you "need" or want it doesn't really give much direction on what to change, that is, if there is anything at all to change. At most your function might be more accurate or have less banding but dithertools isn't magic and in general super resolution is more of a misnomer than an actual thing. It assumes that magic things can occur when you interpolate frame data and that all frame data is static because if it isn't then your clip ends up either full of artifacts or no more detailed than it started.

[Elvellon]

Thank you for the dither package! I use it constantly.
I have a problem.
If the 16 bit clip is YV12 or YV24 (then Dither_convey_YUV4xxP16_on_YVxx()) then everything encodes OK. Now I try to encode 4:2:2 (YV16 or YUY2) and everything is garbled.
Can it be fixed? Is there a workaround with some lossless adding/removing of chroma with YV24 in AVS and YV16 in x264?

[sneaker_ger]

Try SwapUV() between YV16 source and dither. Otherwise post the complete script and piping/encoding parameters.

[Elvellon]

FFVideoSource("random mkv, 8 bit 4:2:0")
BilinearResize(640,360) http://i53.fastpic.ru/big/2012/1226/83/da347a2923101ebe735a7d6c3efdfa83.jpeg
ConvertToYV16() # or YUY2
Dither_convert_8_to_16() # looks ok/expected up to this point
# SwapUV() http://i52.fastpic.ru/big/2012/1226/e0/32b2502aa3f7a23d8301bc0c712d95e0.jpeg
Dither_convey_YUV4xxP16_on_YVxx() # almost the same with and without swap http://i53.fastpic.ru/big/2012/1226/ac/418b64ccec0fcf63896dcbaaa83814ac.jpeg

Then
avs4x264mod -L x264_10bit_x86 "in.avs" --demuxer raw --input-depth 16 --input-res 640x360 --input-csp i422 --output-csp i422 --seek 100 --frames 1 --qp 0 --output "out.mkv"
Load via hacked 10 bit ffms: http://i51.fastpic.ru/big/2012/1226/e8/bcaae42ea362acdf922e65221ef940e8.jpeg

avs2yuv_x86 -raw -csp i422 "in.avs" -o - | x264_10bit_x86 --demuxer raw --input-depth 16 --input-res 640x360 --input-csp i422 --output-csp i422 --output "out.mkv" --seek 100 --frames 1 --qp 0 -
same result

If I change YV16 to YV24 in the script and i422 to i444 in the encoding parameters, everything is fine: http://i53.fastpic.ru/big/2012/1227/50/d9693137b5c31ca5d4c981c3dfea1450.jpeg

The "random mkv" was for reproducibility, the file I've been struggling with is ProRes. The result is the same, green blotches etc. Of course if I input it directly to x264 it encodes 4:2:2 fine. So somehow the colorspaces in the AVS > pipe > x264 process don't connect somehow.

[sneaker_ger]

I'm also having trouble making it work. Let's wait for cretindesalpes.

Or would fmtconv or flash3kyuu be an alternative for you? I don't know exactly why you need to use the dither tools.

[Elvellon]

Thanks, let's wait.
Actually I mostly do some basic denoising/resizing (+ QTGMC outside of dither). So I don't want to get into VapourSynth just yet.
Besides others have noticed some banding problems in the video that I can't see/fix so I'll just report the fix/workaround to them.

[cretindesalpes]

Sorry, my bad. Dither_convey_yuv4xxp16_on_yvxx() implementation was a bit naive and doesn’t work when the chroma subsampling is not the same on the X and Y axis. Here is a fixed version, before I release an update:

Function Dither_convey_yuv4xxp16_on_yvxx (clip src, bool "bigendian")
{
bigendian = Default (bigendian, false)

src
msb = Dither_get_msb ()
lsb = Dither_get_lsb ()

(bigendian) ? Interleave (msb, lsb) : Interleave (lsb, msb)

d = (Dither_get_chroma_subspl_h () != Dither_get_chroma_subspl_v ())
m = (d) ? ConvertToY8 () : last
u = (d) ? UToY8 () : last
v = (d) ? VToY8 () : last

m = m.TurnRight ().AssumeFieldBased ().AssumeTFF ().Weave ().TurnLeft ()
u = (d) ? u.TurnRight ().AssumeFieldBased ().AssumeTFF ().Weave ().TurnLeft () : u
v = (d) ? v.TurnRight ().AssumeFieldBased ().AssumeTFF ().Weave ().TurnLeft () : v

(d) ? YToUV (u, v, m) : m
}

[Elvellon]

Great, thank you!

[cretindesalpes]

Dither 1.22.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) is the last release of the year:
Added temporal processing to Dither_median16.
Added Dither_resize16nr, a simple ringing-cancellation wrapper around Dither_resize16.
Added a noring parameter to conversion functions between YUV and RGB.
Added lsb, center and bkgcolor to Dither_srgb_display.
Replaced Dither_convey_yuv4xxp16_on_yvxx with Dither_out, which also works correctly with YV16 input. The old function is still there for compatibility, mapped on the new one.
Dither_addborders16 now works with RGB clips.
Badly designed and buggy resizing functionalities in Dither_srgb_display were rewritten.
I’ll try to backport the new or improved features from fmtconv to Dither as soon as I’m done with the fmtconv optimisations. In the meantime, I wish you all a happy new year.

[ganymede]

Thank you very much Mr. DesAlpes, and happy new year to you !
Dither_out, which also works correctly with YV16 inputDo you mean that it is possible to pipe 4:2:2 data outside avisynth, using a fake YV12 stream, or is 4:2:0 the only option ?

[Lenchik]

WhyDither_resize16nr (Width (), Height () / 2, kernel="impulse -1 20 -1",
\ fh=-1, fv=-1, cnorm=true, center=false, y=3, u=2, v=2)give greenish output while Dither_resize16 with same parameters don't?

[Tanbheer]

your filter functions are so hard to understand. I have always used GradFun2DBmod so far for removing banding.
Can you give me an equivalent parameter of GradFun3 for this parameter GradFun2DBmod(thr=1.4,thrC=1.4,mode=3,str=0.8,strC=0.0).
I want to try & compare

[SAPikachu]

My friend maven mawen found a bug in Dither_quantize with bitdepth=14. After debugging I think the culprit is in Dither_quantize_out_n.

Test case:


ColorBars
ConvertToYV12.Grayscale

dither_convert_8_to_16

StackHorizontal( \
f3kdb(y=0, cb=0, cr=0, grainy=0, grainc=0, input_mode=1, input_depth=16, output_mode=1, output_depth=14, dither_algo=1), \
Dither_quantize(14, reducerange=true, mode=-1) \
)



As of Dither 1.22.0, LSB part of Dither_quantize's output is totally different from f3kdb's. If I get it right, ofs2 in Dither_quantize_out_n is incorrect when output bit depth is 14 (exponent becomes -1), so the result is incorrect.

(Actually I don't fully understand this function, so please correct me if I am wrong. :P )

[mawen1250]

Function Dither_quantize (clip src, int bitdepth, bool "reducerange",
\ int "mode", float "ampo", float "ampn", int "pat", bool "dyn", bool "prot",
\ clip "mask", float "thr", bool "interlaced", int "y", int "u", int "v",
\ bool "staticnoise")
In addtion, due to the definition of "int bitdepth", I can only use Dither_quantize(10) but Dither_quantize(bitdepth=10) returns error.

[cretindesalpes]

Dither 1.22.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559) (update from v1.22.0 (http://ldesoras.free.fr/src/avs/dither-1.22.0-to-1.22.1.zip)) should address the problems raised by Lenchik, SAPikachu and mawen1250:
Fixed green frame output in Dither_resize16nr for some combination of the y, u and v parameters.
Fixed Dither_quantize for 14-bit output. It is not perfect but much better than before.
Made the bitdepth parameter in Dither_quantize a named argument.
Fixed var and uvar scaling in Dither_add_grain16 to match more accurately AddGrainC.

ganymede:

Yes, you should be able to pipe 16-bit 4:2:2 with a fake YV16 stream. Make sure that x264 reads planar 4:2:2.

Tanbheer:

Try this:
GradFun3 (thr=1.4*0.3, smode=2, lsb=true)
Dither_add_grain16 (var=0.8, uvar=0, soft=2)
DitherPost (mode=6)
Should be close, but not exactly the same because GradFun2DbMod applies the grain through a luma mask.

Dogway:

I’m going to investigate this issue in the next days.

[Tanbheer]

your filter functions are so hard to understand. I have always used GradFun2DBmod so far for removing banding.
Can you give me an equivalent parameter of GradFun3 for this parameter GradFun2DBmod(thr=1.4,thrC=1.4,mode=3,str=0.8,strC=0.0).
I want to try & compare

Anyone have any answer plz. :o

[Are_]

Anyone have any answer plz. :o

Just two post above yours.

http://forum.doom9.org/showthread.php?p=1609899#post1609899

[S_Prince]

Hello All..

I Am Getting A Problem By Using MDegrain3..

i Am using this script :

h = Height ()
w = Width ()

super = MSuper(pel=2, sharp=1)
bv3 = MAnalyse(super, isb = true, delta = 3, overlap=8, blksize=16)
bv2 = MAnalyse(super, isb = true, delta = 2, overlap=8, blksize=16)
bv1 = MAnalyse(super, isb = true, delta = 1, overlap=8, blksize=16)
fv1 = MAnalyse(super, isb = false, delta = 1, overlap=8, blksize=16)
fv2 = MAnalyse(super, isb = false, delta = 2, overlap=8, blksize=16)
fv3 = MAnalyse(super, isb = false, delta = 3, overlap=8, blksize=16)

MDegrain3(super, bv1, fv1, bv2, fv2, bv3, fv3, thSAD=400, lsb=true)

msb = last.Crop (0, 0, w, h)
lsb = last.Crop (0, h, w, h)
DitherPost (msb, lsb)


But It is giving the following error:

http://img163.imageshack.us/img163/1709/239ql.png


Please Anyone Can tell Me How To Get Rid of This Error..The it Will Be Really Helpful.. :o

[cretindesalpes]

S_Prince:

You need the modified MVTools from the dither package instead of the regular one.

You can also replace the 3 last lines with just DitherPost().

[S_Prince]

S_Prince:

You need the modified MVTools from the dither package instead of the regular one.

You can also replace the 3 last lines with just DitherPost().
Thanks For The Reply.

Okay.Is this 1 called Modifed MVtools 'mvtools-2.6.0.5'..I Found it in dither package ;)

& Yes I shall try to replace the last 3 lines with ditherpost.I'll Surely try I will redirect the result to You.



Thanks

[Tanbheer]

Tanbheer:

Try this:
GradFun3 (thr=1.4*0.3, smode=2, lsb=true)
Dither_add_grain16 (var=0.8, uvar=0, soft=2)
DitherPost (mode=6)
Should be close, but not exactly the same because GradFun2DbMod applies the grain through a luma mask.
Thank you very much for your reply
I tried it. Giving me this error. I have used your modified mvtools2 also. but same error
http://imgit.me/i/9C6b3X4.jpg

[cretindesalpes]

You need a more recent version of AddGrainC (I think v1.4 or later ). Check this one (http://forum.doom9.org/showthread.php?p=1587215#post1587215), for example.

[Tanbheer]

You need a more recent version of AddGrainC (I think v1.4 or later ). Check this one (http://forum.doom9.org/showthread.php?p=1587215#post1587215), for example.

Thanks a lot man . It works now. Now i need to find a good de-noising filter which works best with GradFun3.
VagueDenoiser always worked great with GradFun2DBmod. I am sure your denoising scripts on dither package are great. Will try & check it out. Again very hard function i must say. There are so many parameters & options there :o Its like reading a rocket science.

[TheProfileth]

For the longest time I have been getting this error every time I try to use rgb or yv24 with dither tools it is odd.
For example I am using something close to one of the examples in the documentation AVCSource("stuff.dga")
Dither_convert_8_to_16 ()
Dither_convert_yuv_to_rgb (lsb_in=true,matrix="709", output="rgb48y",noring=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709",output="yv24", mode=6,noring=true)http://eypic.net/images/errorfif.png
Is something up with my mt_masktools or am I just doing something wrong?

[martin53]

With mv2tools.dll from the package 2.6.0.5 that is contained in dither 1.22.1, I frequently get the following error message:
MVTools: vector clip is too small (corrupted?)
([GScript], line 42)
([GScript], line 55)
([GScript], line 63)
(_FSubstitute.avsi, line 744)
...
With my 2nd most recent mvtools2.dll from the package mvtools-2.5.11.9-svp, I never see this error.

Unfortunately, I can not yet provide a simple example script that produces the error. It is produced when using my _FSubstitute script from the 'bye by blurred frames (http://forum.doom9.org/showthread.php?t=166446)' thread on various source clips. That script uses MVAnalyse in a not everyday manner during script compile time of the RTE to get global motion information with MDepan.
I posted version 01/13/2013 of that script with the actual code (from StainlessS) in the mentioned lines. The first line of the error msg is 'cGM.RT_AverageLuma(currentframe, w=1, h=1)', for your reference.

RT_AverageLuma() in this line forces the global motion clip cGM to be evaluated during script compile, so MDepan writes the global motion figures to the logfile. The logfile must be available for reading a few lines below when the motion figures are read to the string variable sMD.

[StainlessS]

Martin53, This will not solve the mvtools problem but,

you can use RT_YankChain(currentframe) instead of RT_AverageLuma(currentframe,w=1,h=1).
Also a little faster than I might have expected.
It just forced fetches the frame without any sampling.

[martin53]

( StainlessS,
thanks, will download your new package)

Here is more detail regarding the MVtools issue.

I stripped down my function a bit, and added debugging operations. Currentframe is just a copy of current_frame and is 6 at the end, iDelta is 1. Frames 2,5,6 are processed until the error occurs. With frame 2 and also iDelta=1, the whole script works good. With frame 5, the error currently appears, but in another function of the script that uses MVtools on the same frame after this function. After each source code change, there may be a then reproduceable issue with frame 5 in this function, or not before frame 6. I could not figure out a relation between the code changes (like is the ImageWriter function dot-appended to MAnalyse or in a separate line) and the first failing frame number. The failing frames also depend on how I navigate through the clip. The previous description is true when starting with frame 1 and then go forward frame by frame.

c_ = (iDelta==-1) ? c : c.FreezeFrame(currentframe-1, currentframe-1, currentframe+iDelta)
cS = c_.MSuper(pel=4)
cV = cS.MAnalyse(truemotion=true, global=true, isb=false, delta=1, search=3, pelsearch=4, searchparam=1, pzero=255, plevel=1, blksize=16, blksizeV=16, overlap=4, divide=2)
RT_debug("[<-]#"+string(currentframe), "Vector clip w=", string(cV.width)," h=",string(cV.height))
cVM = cV.ImageWriter(ScriptDir+"\cV", 0, 0, type="ebmp").RT_AverageLuma(currentframe, w=1, h=1)
cVM = 0
c
cGM = c_.MDepan(cV, log=LOGFILE, zoom=zoom>0, rot=rot>0, range=0, thSCD1=thSCD1, thSCD2=thSCD2)
cGM.RT_AverageLuma(currentframe, w=1, h=1) ### ERROR LINE ### # Force frame read of Global Motion clip, and MDepan testing.
cGM = 0 # Destructor of temp Global Motion clip called here and log file available after here (created in Destructor)
c # Hopefully destructor of temp last clip called here and log file available after here

sMD = RT_ReadTxtFromFile(LOGFILE)

- DebugView (from 1st blue code line) looks the same for all 3 processed frames: '[4168] RT_Debug: [<-]#6 Vector clip w= 33000 h= 1'
- an 'image' of all 3 processed vector clip frames is written (2nd blue code line), see attached archive if you are able to analyse that
- still, an error message over the clip says that the line with the red note fails because of a too small vector clip
- The new parameter mt=false with the MVtools functions did not help

I was wondering if the vector clip is not available at all at the right time - but the existence of the outputs of the blue lines indicates that it is there when MDepan is called...

Please tell me how I can contribute more.

[cretindesalpes]

I haven't checked this issue in detail yet, but I suspect something related to ScriptClip or other runtime functions. Actually, MVTools vector clips need that the filter that generated them (MAnalyse, MRecalculate…) is still instantiated when they are used. The reason is simple: the vector clips use a hack to attach private data to the clip descriptor (before the first frame is generated). The number of audio channels actually conveys a pointer to this data structure, which belongs to the filter, not to the clip. If the filter is deleted before the clip, or if multiple vector clips from different filters are merged, weird things can happen. The original MVTools have a minimal protection to check the validity of a vector clip but this was not enough. I added more error checking in my mod16 version, hence it's less tolerant to errors.

Of course there could be another reason but this is the first thing that comes in mind.

[martin53]

Are you suggesting that there is no solution to the issue, and that e.g. my script might have suffered from a hidden problem all the time?

I maybe could check this if I knew how to detect a certain suspicious misbehaviour of the previous versions.
I am sceptical though because then I think I would have seen sporadic unexpected results, but the whole thing *seems* to run satisfyingly with the mentioned older dll.

Of course, it would be a very sad end to the story if MVtools were just unable to do what is needed here.

The task was to get global motion data (pan, zoom rotate) between two arbitrary frames of a clip, in order to precompensate the global motion before e.g. MFlowInter resynthesizes a frame. My research had shown that the synthesized results are *much* better this way.

A hack to mix up a clip data structure with a filter instance data structure indeed sounds weird. Although I don't understand the reasons it was made this way, it sounds possible to add that structure to every frame of the vector clip instead, or solve the problem any other way - would that be something you could try?

EDIT: Or even if MDepan had a sister function that just returned the string MDepan writes to the log file. Then I could omit a bunch of tricks.

[Lenchik]

As a suggestion to future releases of dither plugin:
Add support for Rec.2020 (http://en.wikipedia.org/wiki/Rec._2020) colorimetry conversions. Don't know whether it is difficult keeping in mind that Rec. 2020 defines a color depth of either 10-bits or 12-bits (http://www.itu.int/rec/R-REC-BT.2020-0-201208-I/en).

[Chyrka]

Hi!
What the hell is:
http://thumbnails102.imagebam.com/23572/7164b4235710556.jpg (http://www.imagebam.com/image/7164b4235710556)
when I use

Dither_convert_8_to_16 ().Dither_resize16(784,574,0, 2, -0, -0,"spline64")
DitherPost (mode=-1)
instead of
http://thumbnails101.imagebam.com/23572/ddf343235710574.jpg (http://www.imagebam.com/image/ddf343235710574)
when I use

Spline64Resize(784,574,0,2,0,0)
:confused:

It doesn't happen with zero cropping
Dither_resize16(784,574,0, 0, -0, -0,"spline64")
Also usacale/downscale/in-out picture size doesn't matter.
And heignt of messed up stripe at the bottom is proportional to top-cropped pixel amount.

[cretindesalpes]

Chyrka:

It looks like an discrepancy between the avs and dither resizing functions regarding the value of 0 for src_width and src_height. In Dither_resize16, 0 means "input width or height" whereas in XxxResize it works exactly like in Crop(). I'll probably change this behaviour in the next release to match the Avisynth resizers.

Lenchik:

It's possible for the R’G’B’<->Y’Cb’Cr’ conversion but keep in mind that the RGB colorspace has not the same primaries (at all) as the Rec.601/Rec.709 colorspaces. It would require a conversion between these colorspaces with a 3D LUT.

Dogway:

How could I solve the problems you report if you delete your posts?

[Overdrive80]

Hi, actually I am using ditherpack for filtering. However I had have troubles with mvtools for this pack. I am doing vfr file with this script:

In first pass, I deinterlace video file encoding with lagarith:

DGDecode_mpeg2source("C:\Users\Isra\Desktop\Bleach\VID 2\VideoFile1.d2v", info=3)

ColorMatrix(hints=true, threads=0,interlaced=true)

part1=trim(0,38811).tfm().tdecimate()
part2_in=trim(38812,41089)


part2=part2_in.QTGMC( Preset="very Slow", EdiMode="EEDI2",fpsdivisor=2).aaf().assumefps(part1.framerate)

part1++part2

In secong pass, filtering video:

SetMemorymax(1024)
setmtmode(5,4)

avisource("F:\001\1. Desentralazar.avi")

#Desentrelazado
#Crop & resize

crop(4,0,-4,0).Spline64resize(720,480)

# Filtrado

Setmtmode(3)

original=last

aa=spline64Resize(width(original)*2,height(original)*2).SangNom(aa=29).spline64Resize(width(original),height(original)).mergechroma(original).finesharp(sstr=1.8)


prefiltrado= dfttest(sigma=4,tbsize=1,tmode=0,lsb=true).ditherpost(mode=2)

super=Msuper(prefiltrado)

backward_vec3 = MAnalyse(super, isb = true, delta = 3, overlap=4,search=4)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4,search=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4,search=4)

forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4,search=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4,search=4)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, overlap=4,search=4)

prefiltrado.MDegrain3(super,backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,
\plane=3,lsb=true).ditherpost(mode=2)

contrasharpening(last,aa)

toon(0.15)

gradfun3()

If I use plane=4 or 1, I got this https://dl.dropbox.com/u/19135067/example.avi

And if I use other option crash operation. Log:

VirtualDub crash report -- build 32842 (release)
--------------------------------------

Disassembly:
02d0b3c0: 0feada pminsw mm3, mm2
02d0b3c3: 660ff9cb psubw xmm1, xmm3
02d0b3c7: 660f6f5c2460 movdqa xmm3, [esp+60h]
02d0b3cd: 660f65d9 pcmpgtw xmm3, xmm1
02d0b3d1: 660f6fe1 movdqa xmm4, xmm1
02d0b3d5: 660fd54c2450 pmullw xmm1, [esp+50h]
02d0b3db: 660ff94c2470 psubw xmm1, [esp+70h]
02d0b3e1: 660f65a4248000 pcmpgtw xmm4, [esp+80]
0000
02d0b3ea: 660f6ffd movdqa xmm7, xmm5
02d0b3ee: 660f71d701 psrlw xmm7, 01h
02d0b3f3: 660ff9c7 psubw xmm0, xmm7
02d0b3f7: 660fe5c1 pmulhw xmm0, xmm1
02d0b3fb: 660f6f8c249000 movdqa xmm1, [esp+90]
0000
02d0b404: 660f71f002 psllw xmm0, 02h
02d0b409: 660f6fd3 movdqa xmm2, xmm3
02d0b40d: 660fdb5c2440 pand xmm3, [esp+40h]
02d0b413: 660febd4 por xmm2, xmm4
02d0b417: 660f6ff5 movdqa xmm6, xmm5
02d0b41b: 660ff9f0 psubw xmm6, xmm0
02d0b41f: 660fdfd6 pandn xmm2, xmm6
02d0b423: 660f6fc1 movdqa xmm0, xmm1
02d0b427: 660febd3 por xmm2, xmm3
02d0b42b: 660fdbe5 pand xmm4, xmm5
02d0b42f: 660febd4 por xmm2, xmm4
02d0b433: 660fdbc2 pand xmm0, xmm2
02d0b437: 660fdfca pandn xmm1, xmm2
02d0b43b: 660f73d901 psrldq xmm1, 01h
02d0b440: 660f67c1 packuswb xmm0, xmm1
02d0b444: 660fd60402 movq [edx+eax], xmm0 <-- FAULT
02d0b449: 660f6dc0 punpckhqdq xmm0, xmm0
02d0b44d: 660fd60401 movq [ecx+eax], xmm0
02d0b452: 83c008 add eax, 08h
02d0b455: 836c242001 sub dword ptr [esp+20h], 01h
02d0b45a: 89442428 mov [esp+28h], eax
02d0b45e: 0f85c8fcffff jnz 02d0b12c
02d0b464: 5f pop edi
02d0b465: 5e pop esi
02d0b466: 5b pop ebx
02d0b467: 8be5 mov esp, ebp
02d0b469: 5d pop ebp
02d0b46a: c21c00 ret 001c
02d0b46d: cc int 3
02d0b46e: cc int 3
02d0b46f: cc int 3
02d0b470: 83e940 sub ecx, 40h
02d0b473: e968f1ffff jmp 02d0a5e0
02d0b478: cc int 3
02d0b479: cc int 3
02d0b47a: cc int 3
02d0b47b: cc int 3
02d0b47c: cc int 3
02d0b47d: cc int 3
02d0b47e: cc int 3
02d0b47f: cc int 3
02d0b480: b818040100 mov eax, 00010418
02d0b485: e826a80100 call 02d25cb0
02d0b48a: a1f8b0d302 mov eax, [02d3b0f8]
02d0b48f: 33c4 xor eax, esp
02d0b491: 89842414040100 mov [esp+10414], eax
02d0b498: 53 push ebx
02d0b499: 55 push ebp
02d0b49a: 56 push esi
02d0b49b: 57 push edi
02d0b49c: 8d84241c020000 lea eax, [esp+21c]
02d0b4a3: 33f6 xor esi, esi
02d0b4a5: 50 push eax
02d0b4a6: 8d7c2418 lea edi, [esp+18h]
02d0b4aa: 33ed xor ebp, ebp
02d0b4ac: 89742414 mov [esp+14h], esi
02d0b4b0: e89b010000 call 02d0b650
02d0b4b5: 83c404 add esp, 04h
02d0b4b8: 8bcf mov ecx, edi
02d0b4ba: 51 push ecx
02d0b4bb: 6a01 push 01h
02d0b4bd: 68 db 68h
02d0b4be: 0000 add [eax], al

Built on Aegis on Fri Dec 24 13:18:44 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7601) [Service Pack 1]

EAX = 47920020
EBX = 000001e0
ECX = b86dffe0
EDX = b87345e0
EBP = 38e0b574
ESI = 38e0b5bc
EDI = 00443214
ESP = 38e0b4d0
EIP = 02d0b444
EFLAGS = 00010a03
FPUCW = 027f
FPUTW = ffff

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'dither'...

...writing address 00054600.

Pointer dumps:

EAX 47920020: 53535351 53535353 53535353 52525253 53535353 53535353 53535353 53535353
ESI 38e0b5b8: 000001e0 02d2f1c8 00000000 00000000 00000000 00000000 00000000 00000000
EDI 00443210: 0000b887 8b8b9900 000000b8 a3f7ca23 000000b8 0005d12b 83800000 958900d2
ESP 38e0b4d0: 142cce38 4fc30020 47920020 00000000 00000000 00000416 000003f6 01392cb0
38e0b4f0: 0000005a 000003f7 47920020 08310000 00054600 b87345e0 08364600 b86dffe0
38e0b510: 536e536c 536f5370 536e536e 536e536d 06660666 06660666 06660666 06660666
38e0b530: 00150015 00150015 00150015 00150015 065e065e 065e065e 065e065e 065e065e
EBP 38e0b570: 47922a50 47974620 02d0ae55 38e0b5bc 00000000 00054600 47920020 47974620
38e0b590: 4fc30020 4fc84620 ea888816 00000000 38e0f6ac 000002d0 4fc84620 00054600
38e0b5b0: 00000000 4fc30020 000001e0 02d2f1c8 00000000 00000000 00000000 00000000
38e0b5d0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000

Thread call stack:
02d0b444: dither!_AvisynthPluginInit2@4 [02cf0000+a460+10fe4]
02d0ae55: dither!_AvisynthPluginInit2@4 [02cf0000+a460+109f5]
02d0ad48: dither!_AvisynthPluginInit2@4 [02cf0000+a460+108e8]
02d0ad07: dither!_AvisynthPluginInit2@4 [02cf0000+a460+108a7]
02d0aba8: dither!_AvisynthPluginInit2@4 [02cf0000+a460+10748]
02d16151: dither!_AvisynthPluginInit2@4 [02cf0000+a460+1bcf1]
02d0a75b: dither!_AvisynthPluginInit2@4 [02cf0000+a460+102fb]
6e56cde5: AviSynth!avs_delete_script_environment [6e500000+68d20+40c5]
6e56b9f2: AviSynth!avs_delete_script_environment [6e500000+68d20+2cd2]
6e56add3: AviSynth!avs_delete_script_environment [6e500000+68d20+20b3]
02cfd960: dither!_AvisynthPluginInit2@4 [02cf0000+a460+3500]
02b649b6: avstp!avstp_wait_completion [02b60000+4830+186]
02b64827: avstp!avstp_enqueue_task [02b60000+47f0+37]
02cfb4e2: dither!_AvisynthPluginInit2@4 [02cf0000+a460+1082]
02b64761: avstp!avstp_destroy_dispatcher [02b60000+4720+41]
02cf5e11: dither!00005e11
77ae33f7: ntdll!RtlInterlockedFlushSList [77ab0000+32775+c82]
77acfab2: ntdll!NtAllocateVirtualMemory [77ab0000+1faa0+12]
77aee605: ntdll!NtdllDefWindowProc_A [77ab0000+3e5e2+23]
77ae3426: ntdll!RtlInterlockedFlushSList [77ab0000+32775+cb1]
02ea5a74: lagarith!00005a74
77ade35c: ntdll!RtlInitUnicodeString [77ab0000+2e1f8+164]
77ae3426: ntdll!RtlInterlockedFlushSList [77ab0000+32775+cb1]
77ae3054: ntdll!RtlInterlockedFlushSList [77ab0000+32775+8df]
77b1031c: ntdll!RtlFindMessage [77ab0000+5fcbd+65f]
77b10343: ntdll!RtlFindMessage [77ab0000+5fcbd+686]
77acf9b9: ntdll!NtSetEvent [77ab0000+1f9a4+15]
77aeb591: ntdll!RtlpUnWaitCriticalSection [77ab0000+3b56c+25]
77aeb562: ntdll!RtlIntegerToUnicodeString [77ab0000+3b19d+3c5]
77acf8b1: ntdll!NtWaitForSingleObject [77ab0000+1f89c+15]
77aeb4c4: ntdll!RtlIntegerToUnicodeString [77ab0000+3b19d+327]
76099d45: msvcrt!malloc [76090000+9cee+57]
7609b0d7: msvcrt!??2@YAPAXI@Z [76090000+b0c9+e]
77acf9b9: ntdll!NtSetEvent [77ab0000+1f9a4+15]
6e56601a: AviSynth!0006601a
6e564e6a: AviSynth!00064e6a
02d16151: dither!_AvisynthPluginInit2@4 [02cf0000+a460+1bcf1]
02d0a75b: dither!_AvisynthPluginInit2@4 [02cf0000+a460+102fb]
6e56cde5: AviSynth!avs_delete_script_environment [6e500000+68d20+40c5]
6e56b9f2: AviSynth!avs_delete_script_environment [6e500000+68d20+2cd2]
6e56add3: AviSynth!avs_delete_script_environment [6e500000+68d20+20b3]
02cf4edc: dither!00004edc
6e564e6a: AviSynth!00064e6a
02d16151: dither!_AvisynthPluginInit2@4 [02cf0000+a460+1bcf1]
02d0a75b: dither!_AvisynthPluginInit2@4 [02cf0000+a460+102fb]
6e56cde5: AviSynth!avs_delete_script_environment [6e500000+68d20+40c5]
6e56b9f2: AviSynth!avs_delete_script_environment [6e500000+68d20+2cd2]
6e5711a7: AviSynth!DllCanUnloadNow [6e500000+6ea80+2727]
7705338a: kernel32!BaseThreadInitThunk [77040000+13378+12]
77aebf42: ntdll!RtlInitializeExceptionChain [77ab0000+3bedf+63]
77aebf15: ntdll!RtlInitializeExceptionChain [77ab0000+3bedf+36]

-- End of report

[cretindesalpes]

I couldn't reproduce the crash but got the artefacts. They seem related to only MSuper/MAnalyse/MDegrain in MT mode 3 (it works well in MT mode 2, in single-threaded mode or stepped frame by frame manually in MT 3), and not dependent on the lsb parameter. I'm going to investigate this further as I don't know much about what MT mode 3 does.

[Overdrive80]

I tried use different modes on MT, but result is the same. However, I encoding without MDegrain and not occurs.

[TheProfileth]

Hey cretindesalpes, so I have been messing with stuff lately and I rediscovered a filter that I quite liked that utilizes a standard median but makes it a bit more applicable to general use the only issue was that it was excruciatingly slow and didn't support 16bit. I am speaking of BucketMedian where the latest version cane be found here (https://github.com/chikuzen/BucketMedian/). https://github.com/chikuzen/BucketMedian/
Either way my request of you was if you could possibly implement the limiting system into the dithertools median. Would be very useful especially given its ability to help in the process of debanding by homogenizing regions seen here (http://2.bp.blogspot.com/-DHnJAnGEaYY/T38SwtO2n1I/AAAAAAAAAJ8/CL4M7ormi0o/s1600/BucketTest000328.png) the introduction of the limiters would make the median filter an even more helpful tool for both denoising and debanding.
Also keep up the good work :)

[Elvellon]

Hi, there's a discussion going on a local forum. Is 16 bit resizing only useful when the unscaled image has been denoised and/or debanded in 16 bits, i.e., the noise/dither has been messed with and we need to keep 16 bits through resizing? Is Dither_convert_8_to_16().Dither_resize16(W,H).DitherPost() rather useless?

[mandarinka]

Hmm, I wanted to test something and needed a 16bit version of plain avisynth merge(), to do a 50:50 weighting of two clips. What would be a good way to script that?

I was thinking about Dither_merge16 with luma=true and a mask with luma set to 32768, but that sounds hacky (also, is that a proper luma center for this purpose?)

[06_taro]

Why not simply use Merge(a16, b16, weight) for those 16-bit clips?

For 16-bit stacked msb+lsb clips:
new_value16
= a16 * ( 1 - weight ) + b16 * weight
= ( msb_a * 256 + lsb_a ) * ( 1. - weight ) + ( msb_b * 256 + lsb_b ) * weight
= ( msb_a * ( 1. - weight ) + msb_b * weight ) * 256 + ( lsb_a * ( 1. - weight ) + lsb_b * weight )
which equals to StackVertical(Merge(msb_a, msb_b), Merge(lsb_a, lsb_b)), and equals exactly to Merge(a16, b16) as merge is only a per-pixel linear combination without overflow/underflow.
So Merge works for 16-bit inputs as well.

[mandarinka]

Isn't there going to be a +-0,5 rounding error in the merged MSB? That sorta makes it sorta pointless to process with LSB after it...

Unless I am viewing it wrong.

[06_taro]

Ah my fault, yes merge is done in floating point and msb may need to be rounded.

[mandarinka]

Hmm, I'll go with:

Dither_merge16(16bitclipA,16bitclipB, Dither_convert_8_to_16().dither_lut16("32768"),luma=true))

[bennynihon]

cretindesalpes, I just stumbled on this filter after getting some unsatisfactory banding when encoding Blu-ray disks using x264 at crf 22. Amazing work.

But I'm curious what settings people would suggest when wanting to encode film Blu-ray sources. Currently, I'm using the following as it seems to be a great compromise in terms of quality and speed.

GradFun3(thr=1.2 * 0.3, smode=0, lsb=true)
Dither_add_grain16(var=0.8, uvar=0, soft=2)
DitherPost(mode=0)

But given that these are clean Blu-ray sources without any discernible banding in the source, the GradFun3() call may be unnecessary. Ultimately what I'm after is the ability to selectively add dither to flat areas of a frame, to reduce the likelihood that banding will occur in these areas after encoding it with x264. As it is now, I feel as if I'm first debanding the source, then adding grain to the entire frame, then dithering. I'd like to selectively add grain/dither to flat areas as I mentioned. Any pointers as to what might be good options to try? Thank you!

[bennynihon]

Is there a way to combine the MSB portion of one source, and the LSB portion of another to create a 16-bit stacked clip? Basically, I'd like to use Dither_add_grain16, but only add the grain to the LSB of the source.

[mandarinka]

clipA.Dither_get_lsb() will give you teh least significant bits separately from clipA

clipB.Dither_get_msb() will give you the most significant bits from another clipB

You can combine it simply with stackvertical, in this case: stackvertical(clipB.Dither_get_msb(),clipA.Dither_get_lsb())

PS.
No disrespect, but due to the way gradfun3 works, it oversmooths the source quite a bit. Maybe try to use flash3kyuu_deband (http://forum.doom9.org/showthread.php?t=161411) first - it is much milder to the source (detail) as far as my experience goes and debands pretty well at the same time.

[bennynihon]

clipA.Dither_get_lsb() will give you teh least significant bits separately from clipA

clipB.Dither_get_msb() will give you the most significant bits from another clipB

You can combine it simply with stackvertical, in this case: stackvertical(clipB.Dither_get_msb(),clipA.Dither_get_lsb())

PS.
No disrespect, but due to the way gradfun3 works, it oversmooths the source quite a bit. Maybe try to use flash3kyuu_deband (http://forum.doom9.org/showthread.php?t=161411) first - it is much milder to the source (detail) as far as my experience goes and debands pretty well at the same time.

Thanks. stackvertical was what I was looking for. Adding grain just to the LSB and then dithering isn't as powerful against banding when encoding as simply letting it apply grain to all 16-bits. I'll continue doing that.

[x265]

should i use a dehalo filter before or after converting to 16bits using the Dither_convert_8_to_16( )? Is the code correct?

DGSource("C:\Users\Abhijith Nair\Desktop\location\Episode.dgi")
Interp=NNEDI3(nns=4, qual=2, pscrn=4)
deint = tdeint(edeint=Interp, emask=tmm())
tfm(pp=6, micmatching=2, slow=2, clip2=deint)
smartfade(dgm=true)
tdecimate(mode=1)
Episode=trim(1918, 30688)
Preview=trim(32368, 32727)
Opening=avisource("opening.avs")
Ending=avisource("ending.avs")
AlignedSplice(Opening, Episode, Ending, Preview)
Dehalo_Alpha_mt(ss=3.0)
Dither_Convert_8_to_16()
Dither_crop16(2,4,-2,0)
GradFun3(thr=0.48, smode=2, lsb_in=true, lsb=true)
Dither_Quantize(bitdepth=10, reducerange=true, mode=0)
Dither_Out()

[feisty2]

should i use a dehalo filter before or after converting to 16bits using the Dither_convert_8_to_16( )?

depands on your dehalo filter supports 16bit or not
if your dehalo script supports 16bit process first upsample to 16bit and turn on "lsb_in" parameter

[x265]

Should i use smode=3 or 2 for the gradfun3 filter?

[ajp_anton]

Nevermind =).

[Yellow_]

Hi, have a quick query regarding function: Dither_convert_rgb_to_yuv

Converts an RGB clip to YUV using high bitdepth calculations.

src

An 8 or 16-bit RGB clip to be converted to RGB. if only src is defined, the clip is considered as 8-bit RGB. If g and b are defined too, the three are assumed as 16-bit stacked clips containing the red, green and blue channels.... I guess this is for importing "rgb48y", ie: R,G,B planes as YCC? Is it possible to import 16bit per channel RGB imagery through Dither Tools, or should I be looking at another plugin, sure there was one sometime ago that would handle 16bit?

**EDIT**

No bother I've resolved this query. :-) But got a new one.

[Yellow_]

I have a problem using dfttest packaged with Dither Tools with 16bit planar YCC 4:4:4 YV24

This is an extract of the script:

MSB = RawReadPlanar("yuv444p16le.yuv", "YUV444", 1088, 448, packing="16:0:8").ConvertToYV24()
LSB = RawReadPlanar("yuv444p16le.yuv", "YUV444", 1088, 448, packing="16:8:8").ConvertToYV24()
StackVertical(LSB, MSB)

When I add:

Dfttest(sigma=0.25, lsb=true, lsb_in=true)

The output is a complete frame as the original source except it's obviously stacked16 but exactly 1 quarter of the image (bottom left hand quarter has red, blue and green chroma noise overlaid the 'proper' image. I guess this is because it's 4:4:4 but whether it's a bug or just unsupported I'm unsure. It only happens with a denoiser.

[Reel.Deel]

Dfttest only supports YV12 and YUY2. That's strange how it doesn't give an error with YV24.
I've tried using it with Y8 and I got the colorspace error. That's the only reason I know. :)

[Yellow_]

Thanks, ok I'll go on a hunt for a denoiser for YV24 or find a work around.

[zero9999]

This change in AviSynth 2.6a4 broke GradFun3 with radiusc:


* Force int call arguments to user script function float params to be explicit floats.


fix:

Function GradFun3 (clip src, float "thr", int "radius", float "elast",
\ int "mask", int "mode", float "ampo", float "ampn", int "pat",
\ bool "dyn", float "dthr", int "smode", float "wmin", float "thr_det",
\ float "thr_edg", int "debug", int "subspl", bool "lsb", bool "lsb_in",
\ bool "staticnoise", float "thrc", int "radiusc", float "elastc",
\ int "y", int "u", int "v")

[spawnbsd]

cretindesalpes, now that x264 supports BT2020 colorimetry, any chance we can get an update to dither tools with proper BT2020 support ? =)

[cretindesalpes]

Yes people already asked me about BT.2020. Sorry for my slow replies, I’m not visiting the forum regularly these days and didn't code much for the Dither tools either. I want to finish optimizing fmtconv for Vapoursynth first, then backport the changes to Dither and finally add new features. I may release a quickfix in the meantime for various problems reported with the latest Avisynth 2.6a4.

[Sparktank]

Just popped in to tell you how much I love finding this filter!
It's worked magic for downscaling my BD's to DVD.
Still learning a lot of what it has to offer but the skies and walls have never looked so much better.
Thank you for your hard work and looking forward to any updates in the future. :)

[cretindesalpes]

Dither 1.23.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):

dfttest 1.9.4: Compatibility with the new Avisynth 2.6 colorspaces, excepted Y8.
Partial implementation of the BT.2020 color conversions. Only the non-constant luminance matrix is implemented.
Optimized Dither_merge16 for SSE2.
Fixed the radiusc type in GradFun3 (int), so it now should work correctly in Avisynth 2.6 alpha 4.
Fixed a bug in Dither_merge16 affecting the chroma of YV12 clips with luma=true. Thanks to tp7 for reporting these issues.

[Yellow_]

I'll check later when at my machine, but does the dfttest 1.9.4 compatibility fix with 2.6 color spaces perhaps solve the problem I was having that I PM'd you about a while ago and mentioned here http://forum.doom9.org/showpost.php?p=1632471&postcount=696, which would be great news. :-)

[martin53]

Dither 1.23.0
Cretindesalpes,
thanks for that great package!

I am unsure if lines 381, 382 of dither.avsi should read
scale_y = (tv_range) ? (219.0 / 255.0) : 1.0
scale_uv = (tv_range) ? (112.0 / 255.0) : 0.5
to produce a proper float result?:confused:
Also, in dither.html, in the dither_convert_rgb_to_yuv description: I think tv_range defines the output clip pixel range.

[StainlessS]

(219 / 255.0)

The less precise variable will be coerced to the same format as the more precise variable, eg int 219 will be converted to Float 219.0.
So eg Float(219) / 255, or 219.0 / 255, or 219 / Float(255), or 219 / 255.0, all same result.
EDIT: Probably exactly the same in all programming languages (but there may be exceptions).
Also in eg C, char + int, char would be promoted to type int before addition, and results in int.

[cretindesalpes]

martin53:

As StainlessS said, these int should be promoted to float.
You’re right about the tv_range in the doc, I just fixed it. Thank you for reporting it.

[Overdrive80]

@cretindesalpes Actually, I´m using mvtools2 mod for you from ditherpack. I have tested MdegrainN with bad results, as it introduces blocking artifacts. Screenshots:

Source (DVD):
http://imageshack.us/a/img811/4026/c9md.png

MDegrainN:


original=last
aa=antialiasing
prefiltrado= denoiser_spatial

tr = 5 # Temporal radius

super = MSuper (prefiltrado)
multi_vec = MAnalyse (super, multi=true, delta=tr,dct=10)
prefiltrado.MDegrainN (super, multi_vec, tr, thSAD=400,thSADC=200, thSAD2=250,lsb=true,
\thSADC2=200).ditherpost(mode=2)

gradfun3()

http://imageshack.us/a/img594/7236/w06p.png

MDegrain3:


original=last
aa=antialiasing
prefiltrado= denoiser_spatial

super=Msuper(prefiltrado)

backward_vec3 = MAnalyse(super, isb = true, delta = 3, overlap=4,search=4)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4,search=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4,search=4)

forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4,search=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4,search=4)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, overlap=4,search=4)

prefiltrado.MDegrain3(super,backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,
\thSAD=370,plane=4,lsb=true).ditherpost(mode=2)

gradfun3()

http://imageshack.us/a/img27/4458/yf1q.png

I hope this is useful and you can fix. Greetings.

[cretindesalpes]

Overdrive80:
Indeed, the blocking shown in your example is typical from an overlap parameter left to its default value (0).

*
* *

Dither 1.24.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):

Added all modes up to 24 to Dither_removegrain16, thanks to tp7. Most of them are SSE2-optimised.
Added modes 15–18 to Dither_repair16 (not optimised for SSE2).

[Overdrive80]

For comparison, users should control the variables as few as possible. In your instance, you did not set either dct, overlap, search in MAnalyse or thSAD, thSADC, thSAD2, thSADC2 in MDegrain to the same value in both scripts, and that confuses people.

I dont said that was a comparative. I did say that MdegrainN did introduce blocking artefacts, is that all.

@Cretindesalpes Ok, I gonna review it. Thanks.

[sqrt(9801)]

Hi.

I've been using Dither quite a bit lately as a replacement for ResampleHQ, and I've stumbled upon what looks like a bug with Dither_resize16.

For example, this script
BlankClip(width=1024, height=1024, pixel_type="Y8")

Dither_convert_8_to_16
Dither_resize16(32, 32, kernel="spline36")
DitherPost(mode=6)
throws an unknown exception (tested with Avisynth 2.6a5 and Avisynth+ r1561).
It works when using Dither_resize16 with another kernel like Bicubic, Spline16, or Lanczos (with taps=2 or less), but the issue reappears when downsampling further.

Does anyone else have this problem, or is my PC ready for the scrap heap ? (I am using Win7 x86, btw)

[Dogway]

I do, I've been experiencing this issue for several years but cretindesalpes never cared for it. This is why I never got to downsample HD to SD using Dither_resize16().

[feisty2]

I dont said that was a comparative. I did say that MdegrainN did introduce blocking artefacts, is that all.

@Cretindesalpes Ok, I gonna review it. Thanks.

don't use "dct" other than "0" or "5" for mdegrain, it will produce blocking artifacts

[cretindesalpes]

sqrt(9801), dogway:
Extreme downsampling ratios in both directions (32x is big) are likely to fail, yes. I’m not sure how to fix it. Please bear with me, I’m not perfect. But you can do it in several steps (1024 → 256 → 32). However, downsampling from HD to SD should work without any problem, I use it daily. And it’s wrong I don’t care, I don’t know where you read this. It’s just I don’t know what to do and haven’t an infinite time to think about every issue.

[Dogway]

I had crashes on my new system as well, for speed purposes I ended up using bicubic with Dither_resize16(), but let me confirm again with spline36. Normally these HD->SD scripts are so simple (denoise+resize) that I can exclude the RAM/buffer being the culprit, either way I will use MP_Pipeline next time which uses cores on a better fashion, and then be dead sure.

[Boulder]

Out of interest, are there any differences between "Dither_convert_8_to_16() - Dither_resize16(x,y) - Ditherpost(mode=-1)" and ResampleHQ(x,y) ? Using Dither seems to be somewhat faster.

[cretindesalpes]

IIRC ResampleHQ always resizes in linear RGB colorspace. So there are at least two more conversions involved.

[Boulder]

But in real-life applications like encoding to a lossless format after the resize operation, "a meaningless edge"?

[foxyshadis]

Well, linear downsampling is VERY different from gamma-corrected resampling. Highlights are kept more accurately with linear, with gamma they're darkened. The smaller you go the darker it gets. Too bad you have to choose between that or dithering here.

[mawen1250]

When I use some expression like this, it gives me unexpected result.
Dither_lut16(" x 32768 <= x x 32768 - ? ")
While this one works correctly.
Dither_lut16(" x 32768 <= x x 32768 - ? ")

After checking Functions in dither.avsi, I reallized that Dither_lut16 replaces " x " in <str> with <repstr>, so it doesn't deal with " x x " correctly.
Dither_replace_string is only used by Dither_lut16 so I think it can be fixed like this:
Function Dither_replace_string (string str, string findstr, string repstr)
{
strl = LCase (str)
findstrl = LCase (findstr)
lenf = StrLen (findstr)

pos = FindStr (strl, findstrl)
rep = (pos <= 0) ? "" : LeftStr (str, pos-1) + repstr + Dither_replace_string (MidStr (str, pos + lenf - 1), findstr, repstr)

return ((pos <= 0) ? str : rep)
}

[spawnbsd]

cretindesalpes, any chance we can get a 64bit build of dither tools ? Now that Avisynth+ is taking off and has a 64bit build available, 64bit is starting to look like a viable option again.

[StainlessS]

@mawen1250,
dither-1.24.0.zip contains this mod which seems to work OK

Function Dither_replace_string (string str, string findstr, string repstr)
{
strl = LCase (str)
findstrl = LCase (findstr)
lenf = StrLen (findstr)

pos = FindStr (strl, findstrl)
pos2 = (pos <= 0) ? 1 : pos # EDIT: seems superflous
rep = (pos <= 0) ? "" : LeftStr (str, pos-1) + repstr + Dither_replace_string (MidStr (str, pos + lenf), findstr, repstr)

return ((pos <= 0) ? str : rep)
}


EDIT: On comparing with your version, the "pos2 = (pos <= 0) ? 1 : pos" line in v1.24.0 seems superflous.
Your version with the added '-1' causes Avisynth halt without producng clip or error message and is not working as you expected.
EDIT: Your version keeps trying to replace the same string over and over again, until it runs out of stack.

EDIT: This

colorbars

Function Dither_replace_string (string str, string findstr, string repstr)
{
strl = LCase (str)
findstrl = LCase (findstr)
lenf = StrLen (findstr)

pos = FindStr (strl, findstrl)
# pos2 = (pos <= 0) ? 1 : pos
rep = (pos <= 0) ? "" : LeftStr (str, pos-1) + repstr + Dither_replace_string (MidStr (str, pos + lenf), findstr, repstr)

return ((pos <= 0) ? str : rep)
}

S = " x 32768 <= x x 32768 - ? "
F=" x "
R=" 123 "
O=Dither_replace_string(S,F,R)
RT_DebugF("%s\n%s",S,O)
S2 = " x 32768 <= x x 32768 - ? "
O2=Dither_replace_string(S2,F,R)
RT_DebugF("%s\n%s",S2,O2)

return Last


Produces this:

00000013 72.02262115 [1948] RT_DebugF: x 32768 <= x x 32768 - ?
00000014 72.02263641 [1948] RT_DebugF: 123 32768 <= 123 x 32768 - ? # Problem
00000015 72.03427887 [1948] RT_DebugF: x 32768 <= x x 32768 - ?
00000016 72.03430176 [1948] RT_DebugF: 123 32768 <= 123 123 32768 - ? # OK


AMENDED as per Gavino post below.

[Gavino]

S2 = " x 32768 <= x x 32768 - ? "
O2=Dither_replace_string(S,F,R)
RT_DebugF("%s\n%s",S2,O2)
Shouldn't this be S2?

Try F=" x " R=" 123 " and you will see the issue mawen1250 is talking about.

[Gavino]

EDIT: Your version keeps trying to replace the same string over and over again, until it runs out of stack.
Only if the search string is a single character.
I think his point is that the way the function is used in Dither_lut16, both the search and replacement strings always have added spaces, so his fix works.

[StainlessS]

Thank you big G, you got me there.
As omission of space could cause problem, how bout this:



colorbars

Function Dither_replace_string_MOD (string str, string findstr, string repstr) {
strl = LCase (str) # case independent for FindStr
findstrl = LCase (findstr) #
lenf = StrLen (findstr)
pos = FindStr (strl, findstrl)
lenr = Strlen(repstr)
# Below string comparison 'RightStr(repstr,1) == LeftStr(findstr,1)' is case insignificant
# If last char in replace matches 1st char in find, then
# we replace with all but last char of replace, and re-search remainder but with prepended first char of replace.
rep = (pos <= 0)
\ ? ""
\ : (lenf>1 && lenr>1 && RightStr(repstr,1) == LeftStr(findstr,1))
\ ? LeftStr (str, pos-1) + LeftStr(repstr,lenr-1) + Dither_replace_string_MOD (RightStr(repstr,1)
\ + MidStr (str, pos + lenf), findstr, repstr)
\ : LeftStr (str, pos-1) + repstr + Dither_replace_string_MOD (MidStr (str, pos + lenf), findstr, repstr)
return ((pos <= 0) ? str : rep)
}

S1 = " x 32768 <= x x 32768 - ? "
F=" x "
R=" ABC "
O1=Dither_replace_string_MOD(S1,F,R)
RT_DebugF("S1='%s' : F='%s' R='%s'\nO1='%s'\n",S1,F,R,O1)
R="ABC"
O2=Dither_replace_string_MOD(S1,F,R)
RT_DebugF("S1='%s' : F='%s' R='%s'\nO2='%s\n",S1,F,R,O2)

S2 = " x 32768 <= x x 32768 - ? "
F=" x "
R=" ABC "
O3=Dither_replace_string_MOD(S2,F,R)
RT_DebugF("S2='%s' : F='%s' R='%s'\nO3='%s\n",S2,F,R,O3)
R="ABC"
O4=Dither_replace_string_MOD(S2,F,R)
RT_DebugF("S2='%s' : F='%s' R='%s'\nO4='%s\n",S2,F,R,O4)
return Last



00000005 2.31286550 [2940] RT_DebugF: S1=' x 32768 <= x x 32768 - ? ' : F=' x ' R=' ABC '
00000006 2.31392574 [2940] RT_DebugF: O1=' ABC 32768 <= ABC ABC 32768 - ? '
00000007 2.31412077 [2940] RT_DebugF:
00000008 2.33118415 [2940] RT_DebugF: S1=' x 32768 <= x x 32768 - ? ' : F=' x ' R='ABC'
00000009 2.37454796 [2940] RT_DebugF: O2='ABC32768 <=ABCABC32768 - ?
00000010 2.37459588 [2940] RT_DebugF:
00000011 2.39530826 [2940] RT_DebugF: S2=' x 32768 <= x x 32768 - ? ' : F=' x ' R=' ABC '
00000012 2.39533472 [2940] RT_DebugF: O3=' ABC 32768 <= ABC ABC 32768 - ?
00000013 2.39536095 [2940] RT_DebugF:
00000014 2.40840435 [2940] RT_DebugF: S2=' x 32768 <= x x 32768 - ? ' : F=' x ' R='ABC'
00000015 2.41166806 [2940] RT_DebugF: O4='ABC32768 <=ABCx 32768 - ?
00000016 2.41172385 [2940] RT_DebugF:

so long as repstr surounded by space, no problems.
Not much testing but seems to work. Spot any problems Gavino ?

PS, Whatever happend to the 'quick reply' spell checker when I was on 'vacation' ???

EDIT:
This causes silent crash in Mawen1250 mod, no error message, could be hard to trace bug

S1 = " x 32768 <= x x 32768 - ? "
F="x"
R="A"
O1=Dither_replace_string(S1,F,R)


but not above mod which produces this


00000005 2.15423131 [2144] RT_DebugF: S1=' x 32768 <= x x 32768 - ? ' : F='x' R='A'
00000006 2.15425491 [2144] RT_DebugF: O1=' A 32768 <= A A 32768 - ? '

[mawen1250]

The original Dither_replace_string works correctly for pure linear string replacement, but it is only used in Dither_lut16, while the string replacement in Dither_lut16 is somewhat meant to be non-linear...
So the modified Dither_replace_string should be renamed to something like Dither_expr_replace_string IMO.

[cretindesalpes]

Because there isn’t currently any operator starting with the letter "x", I think replacing " x" instead of " x " in Dither_lut16_expr() is a simpler solution.

[cretindesalpes]

Dither 1.25.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added a 64-bit version. Source code projects ported to MSVC 2012.
Added modes 5–10 to Dither_repair16.
Added ref clip to GradFun3.
The order of the luma, y, u and v parameters in Dither_merge16 is now the same as in Dither_merge16_8. The documentation has been fixed accordingly.
Slight speed improvement in Dither_srgb_display when there is no resizing nor gamma correction.
Fixed a problem in Dither_lut16 with expressions containing two consecutive “x” arguments.
Fixed artefacts (overflow occuring when input pixels are close to 0) and inaccuracy in mode 20 of Dither_removegrain16.
Edit: important bug found. Please use the version 1.25.1 instead.

[tormento]

Added a 64-bit version. Source code projects ported to MSVC 2012.
Do you plan to release x64 version for other dll in the package too?

[cretindesalpes]

Do you mean avstp.dll? Most likely not. With this 64-bit version, I’m supporting AviSynth+, not the old 64-bit AviSynth which is abandoned. Avs+ already has frame-based multi-threading capabilities, and I’ll probably make avstp use its internal thread pool interface in the future.

[tormento]

Do you mean avstp.dll?
And MVTools too.

[mawen1250]

It seems plane copy of Dither_merge16 is broken in 1.25.0.

[cretindesalpes]

Damn! Your’re right. Fixed in Dither 1.25.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559)

[DeathAngelBR]

Just wondering... is this the expected encode speed for a Q8400 and 4GB RAM?

(avstp.dll is in the autoload plugin folder)

http://i.imgur.com/BaGjJp6.png

setmemorymax(512)

LoadPlugin("D:\Edição de video\MeGUI\tools\dgavcindex\DGAVCDecode.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\removegrain\RemoveGrainSSE2.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\removegrain\RepairSSE2.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\removegrain\RSharpenSSE2.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\mt_masktools-26.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\AddGrainC.dll")
LoadPlugin("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\dither.dll")
Import("D:\Arquivos de programas\AviSynth 2.5\plugins\outros\dither.avs")

AVCSource("I:\BDMV Symphogear G Vol. 2\BDMV\STREAM\00003.dga")
spline64resize(1280,720)
dither_convert_8_to_16()
gradfun3(lsb_in=true,lsb=true,smode=2)
dither_out()

[cretindesalpes]

No idea. If you want to check the speed of the script, use AVSMeter. If you want to check the speed of x264, keep only AVCSource + resize and use 10bit-x264 with 8-bit input.

[spawnbsd]

And MVTools too.

@ cretindesalpes, any chance we'll get a 64bit mvtools build as well ?

[cretindesalpes]

Not right now. This is a serious beast, and I have things to finish before (porting dfttest and eedi3 to 64-bits/VS/Avs+/high bit depth). But I plan to work on it. I think Myrsloik and tp7 are already poking it too.

[feisty2]

are you planning on porting nnedi3 to high bit depth too?

[Bernardd]

Hello

When I try Martin53's AWB function, i get error " Unable to call Dither_convert_yuv_to_rgb(). Required dither package 1.23.0 or higher installed ? http://forum.doom9.org/showthread.php?p=1386559"

I have writted this short script for checking

LoadPlugin("C:\RT_Stats_25&26_dll_v1.30_20131219\Avisynth26\RT_Stats26.dll")
LoadPlugin("C:\dither-1.25.1\win32\dither.dll")
assert(RT_FunctionExist("""Dither_convert_yuv_to_rgb"""), "Unable to call Dither_convert_yuv_to_rgb().")
return last

I get the same error. I use avisynth 2.60A5 (CVS 20130920, ICL 10).

What is missing ?

Thanks

[the_weirdo]

@Bernardd:

Function Dither_convert_yuv_to_rgb is implemented as a script function in dither.avsi, so you need to import dither.avsi or have it in autoload folder.

[Bernardd]

@ the_weirdo :

Thanks for your quick answer. It is ok now.

[cretindesalpes]

are you planning on porting nnedi3 to high bit depth too?
No, other people took charge of it, I’ll leave it to them.

[Overdrive80]

One dude, how I can use below script with dither, for use high-bitdepth in his processing. Thanks.


ColorYUV( gain_y=-10, off_y=0, gamma_y=-8, cont_y=0,\
gain_u=4, off_u=0.0, gamma_U=-5, cont_u=0,\
gain_v=-4, off_v=0.0, gamma_v=-5, cont_v=0)

[cretindesalpes]

Overdrive80:

The code below should emulate ColorYUV in 16 bits:
Dither_convert_8_to_16 ()
Dither_lut16 (
\ yexpr=coloryuv_to_expr16 (-10, 0, -8, 0),
\ uexpr=coloryuv_to_expr16 ( 4, 0, 0, 0),
\ vexpr=coloryuv_to_expr16 ( -4, 0, 0, 0),
\ y=3, u=3, v=3)
DitherPost ()

Function coloryuv_to_expr16 (float gain, float off, float gamma, float cont)
{
pscale = 256.0
vscale = 65536
a = String ((gain + pscale) / pscale)
c = String ((cont ) / pscale)
o = String (off)
g = String (pscale / (gamma + pscale))
v = String (vscale)
h = String (vscale / 2)

e = "x "+a+" * x "+h+" - "+c+" * + "+o+" + "+v+" / "+g+" ^ "+v+" *"

return (e)
}

Set gamma_u and gamma_v to 0, they are ignored in the original ColorYUV.

[Overdrive80]

Thanks for your answer. ^^

[jackoneill]

No, other people took charge of it, I’ll leave it to them.

Who did?

Uh, seems the line cretindesalpes was quoting didn't come through. It was about someone porting nnedi3 to 16 bit.

[Popwax]

I think patched dfttest-1.9.4 have a bug handling "plane" parameters from "nfile" or "nstring".
In "vi_src.width >> vi.GetPlaneWidthSubsampling (b);", maybe we should use "1<<b" instead of "b" here.

[Lenchik]

Is it possible to resize using a Sigmoidal Colorspace (http://www.imagemagick.org/Usage/resize/#resize_sigmoidal) with Dither package?

[cretindesalpes]

Popwax:

Yes you’re right, there’s something like this to fix it. I’ll work on it when back home.

Lenchik:

Yes it’s possible, it’s just a LUT and its inverse to add. I will probably add sigmoid curves in a next release.

[Motenai Yoda]

Sorry, but this is right?
Function Dither_lut16_expr (string expr, bool gmsb)
{
# We use spaces to enclose " x" because we don't want operators
# like "max" or "exp" to have their "x" replaced…
repstr = (gmsb) ? " x 256 * y + " : " y 256 * x + "
expr = Dither_replace_string (" " + expr, " x", repstr)
part = Dither_expr_part (gmsb)
expr = expr + part

return (expr)
}

what's the meaning? doesn't use lsb as msb and viceversa?

[cretindesalpes]

Yes, it’s to cancel the swap between the LSB and MSB input clips in the mt_lutxy parameters when processing the LSB output. I don’t remember exactly why it was needed but I think it’s to simplify the handling of the y, u and v plane parameters.

[Motenai Yoda]

I wrote a Lutxy12 and a Lutxy16_8, can someone test them?

###############################
########## Yoda_Lutxy12 ##########
###############################

function Yoda_Lutxy12(clip src, clip src2, string "expr", string "yexpr",
\ string "uexpr", string "vexpr", int "y", int "u", int "v")
{
#needed coz if x is (n*256) -8 < x < n*256 it can't be rounded to n*256 with lsb only.
src_quant = src.Yoda_Lutxy12_aux(y=y, u=u, v=v, up= true)
src2_quant = src2.Yoda_Lutxy12_aux(y=y, u=u, v=v, up= false)

s1_msb = src_quant.Dither_get_msb()
s1_lsb = src_quant.Dither_get_lsb()
s2_msb = src2_quant.Dither_get_msb()
s2_lsb = src2_quant.Dither_get_lsb()

ss_lsb = mt_adddiff(s1_lsb, s2_lsb, y=y, u=u, v=v)

clip_msb = Yoda_Lutxy12_get(s1_msb, s2_msb, ss_lsb, expr,
\ yexpr, uexpr, vexpr, y, u, v, True)

clip_lsb = Yoda_Lutxy12_get(s1_msb, s2_msb, ss_lsb, expr,
\ yexpr, uexpr, vexpr, y, u, v, False)

stackvertical(clip_msb, clip_lsb)

#handling chroma planes if y,u or v are 2 or 4
(Defined (y)) ? (y == 2) ? MergeLuma(src) :
\(y == 4) ? MergeLuma(src2) : last : last

(Defined (u)) ? (Defined (v)) ? (u == 2 && v == 2) ? MergeChroma(src) :
\(v == 2) ? ytouv(last.utoy(),src.vtoy()) :
\(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
\(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
\(Defined (v) && v == 2) ? ytouv(last.utoy(),src.vtoy()) : last

(Defined (u)) ? (Defined (v)) ? (u == 4 && v == 4) ? MergeChroma(src2) :
\(v == 4) ? ytouv(last.utoy(),src2.vtoy()) :
\(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
\(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
\(Defined (v) && v == 4) ? ytouv(last.utoy(),src2.vtoy()) : last
}


function Yoda_Lutxy12_aux(clip src, int "y", int "u", int "v", bool "up")
{
expr = (up) ? "x 247 > 0 x 16 / round 16 * ?" : "x 247 > 0 x 16 / round 128 + ?"
src_lsb=src.Dither_get_lsb()
lsb_l=src_lsb.Mt_lut(expr, y=y , u=u, v=v)
lsb_r=src_lsb.Mt_lut("x 247 > 129 128 ?", y=y , u=u, v=v)
msb_l=src.Dither_get_msb().mt_adddiff(lsb_r, y=y , u=u, v=v)
stackvertical(msb_l,lsb_l)
}

Function Yoda_Lutxy12_get (clip "s1_msb", clip "s2_msb", clip "ss_lsb",
\ string "expr", string "yexpr", string "uexpr", string "vexpr",
\ int "y", int "u", int "v", bool "gmsb")
{
expr = Default (expr, "x")
yexpr = Default (yexpr, expr)
uexpr = Default (uexpr, expr)
vexpr = Default (vexpr, expr)
gmsb = Default(gmsb, False)

expr = Yoda_lut12_expr (expr, gmsb)
yexpr = Yoda_lut12_expr (yexpr, gmsb)
uexpr = Yoda_lut12_expr (uexpr, gmsb)
vexpr = Yoda_lut12_expr (vexpr, gmsb)

y = (Defined (y) && y < 0) ? (gmsb) ? y / 256 : y % 256 : y
u = (Defined (u) && u < 0) ? (gmsb) ? u / 256 : u % 256 : u
v = (Defined (v) && v < 0) ? (gmsb) ? v / 256 : v % 256 : v

mt_lutxyz (s1_msb, s2_msb, ss_lsb, expr=expr, yexpr=yexpr,
\ uexpr=uexpr, vexpr=vexpr, y=y, u=u, v=v)
}

Function Yoda_lut12_expr (string expr, bool gmsb)
{
repstr1 = " x 256 * z z 16 % - + "
repstr2 = " y 256 * z 16 % 16 * + "

expr = Dither_replace_string (" " + expr, " x", repstr1)
expr = Dither_replace_string (" " + expr, " y", repstr2)
part = Dither_expr_part (gmsb)
expr = expr + part

return (expr)
}

#################################
########## Yoda_Lutxy16_8 ##########
#################################

function Yoda_Lutxy16_8(clip src, clip src2, string "expr", string "yexpr",
\ string "uexpr", string "vexpr", int "y", int "u", int "v")
{
s_lsb = src.Dither_get_lsb()
s_msb = src.Dither_get_msb()
s2_c = src2.Dither_convert_8_to_16()

clip_msb = Yoda_Lutxy16_8_get(s_msb, src2, s_lsb, expr,
\ yexpr, uexpr, vexpr, y, u, v, True)
clip_lsb = Yoda_Lutxy16_8_get(s_msb, src2, s_lsb, expr,
\ yexpr, uexpr, vexpr, y, u, v, False)

stackvertical(clip_msb, clip_lsb)

#handling chroma planes if y,u or v are 2 or 4
(Defined (y)) ? (y == 2) ? MergeLuma(src) :
\(y == 4) ? MergeLuma(src2) : last : last

(Defined (u)) ? (Defined (v)) ? (u == 2 && v == 2) ? MergeChroma(src) :
\(v == 2) ? ytouv(last.utoy(),src.vtoy()) :
\(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
\(u == 2) ? ytouv(src.utoy(),last.vtoy()) : last :
\(Defined (v) && v == 2) ? ytouv(last.utoy(),src.vtoy()) : last

(Defined (u)) ? (Defined (v)) ? (u == 4 && v == 4) ? MergeChroma(src2) :
\(v == 4) ? ytouv(last.utoy(),src2.vtoy()) :
\(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
\(u == 4) ? ytouv(src2.utoy(),last.vtoy()) : last :
\(Defined (v) && v == 4) ? ytouv(last.utoy(),src2.vtoy()) : last
}

Function Yoda_Lutxy16_8_get (clip "s1_msb", clip "src2", clip "s1_lsb",
\ string "expr", string "yexpr", string "uexpr", string "vexpr",
\ int "y", int "u", int "v", bool "gmsb")
{
expr = Default (expr, "x")
yexpr = Default (yexpr, expr)
uexpr = Default (uexpr, expr)
vexpr = Default (vexpr, expr)
gmsb = Default(gmsb, False)

expr = Yoda_lut16_8_expr (expr, gmsb)
yexpr = Yoda_lut16_8_expr (yexpr, gmsb)
uexpr = Yoda_lut16_8_expr (uexpr, gmsb)
vexpr = Yoda_lut16_8_expr (vexpr, gmsb)

y = (Defined (y) && y < 0) ? (gmsb) ? y / 256 : y % 256 : y
u = (Defined (u) && u < 0) ? (gmsb) ? u / 256 : u % 256 : u
v = (Defined (v) && v < 0) ? (gmsb) ? v / 256 : v % 256 : v

mt_lutxyz (s1_msb, src2, s1_lsb, expr=expr, yexpr=yexpr,
\ uexpr=uexpr, vexpr=vexpr, y=y, u=u, v=v)

}

Function Yoda_lut16_8_expr (string expr, bool gmsb)
{
repstr1 = " x 256 * z + "
repstr2 = " y 256 * "
expr = Dither_replace_string (" " + expr, " x", repstr1)
expr = Dither_replace_string (" " + expr, " y", repstr2)
part = Dither_expr_part (gmsb)
expr = expr + part

return (expr)
}


removed all ">>" "<<" stuff to make them faster.

[cretindesalpes]

Dither 1.26.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Modded versions of dfttest and MVTools2 aren’t bundled with Dither anymore. Get them separately (at the same location).
Added sigmoid curves to Dither_y_gamma_to_linear and Dither_y_linear_to_gamma.
Added Dither_sigmoid_direct and Dither_sigmoid_inverse functions.
Added the sigmoid parameter to Dither_srgb_display.
Dither_y_gamma_to_linear and Dither_y_linear_to_gamma now process data contained in the chroma planes when u or v are set to 3, instead of just copying them.
— * —

Motenai Yoda:

Thanks. I’ll check it later.

[Sparktank]

Dither 1.26.0

dither-1.26.1

http://i.imgur.com/T5dSeWQ.gif <strike>Is the update a minor thing? I hadn't even noticed it changed versions until I had to update my laptop (low priority udpates).</strike>

EDIT: Ah, the small change doesn't affect my normal work environment.

Here's what the changelog says in the doc for those who are interested:
v1.26.1, 2014.06.05

Fixed a wrong saturation in Dither_sub16 on the last pixels of a clip whose width is not mod-8.

[Motenai Yoda]

@cretindesalpes can u optimize luts by set uexpr and vexpr?

if expr is set then even with u=1, v=1 or chroma="copy", uexpr and vexpr will be calculated (at least with 06_taro's mod).

ie
yexpr = (Defined (y) && (y != 3)) ? "x" : yexpr
uexpr = (Defined (u) && (u == 3)) ? uexpr : "x"
vexpr = (Defined (v) && (v == 3)) ? vexpr : "x"

[colours]

Calculating uexpr and vexpr is effectively free since it only happens during initialisation unless you're using Dither_lut16/etc. in ScriptClip (or other runtime filtering scenarios).

In that case, what you should be doing is to figure out how to avoid runtime filtering instead of performing what would be a microoptimisation for normal people doing normal filtering.

Edit: Someone reminded me that tp7's Masktoolsv2 fork (https://github.com/tp7/masktools) doesn't generate unnecessary LUTs, which may be more relevant than what I wrote above.

[Motenai Yoda]

Calculating uexpr and vexpr is effectively free since it only happens during initialisation unless you're using Dither_lut16/etc. in ScriptClip (or other runtime filtering scenarios).
yep but for some of my scripts it takes up to 30 mins to initialize (without any ScriptClip).


Edit: Someone reminded me that tp7's Masktoolsv2 fork (https://github.com/tp7/masktools) doesn't generate unnecessary LUTs, which may be more relevant than what I wrote above.
:eek: I definitively will try it!
That guy is making a lot of interesting stuff.

[tormento]

:eek: I definitively will try it!
That guy is making a lot of interesting stuff.
Any compiled build around to test?

[Reel.Deel]

Any compiled build around to test?

Check under the GitHub release section (https://github.com/tp7/masktools/releases); also for future reference this is the discussion thread: http://forum.doom9.org/showthread.php?t=169832

[StainlessS]

Just thought I'de point out that FFTW library used by eg FFT3DFilter as fftw3.dll and dftTest as libfftw3f-3.dll (same dll, just different name)
has been updated twice since the one included in dither tools. Last update March 16 2014, v3.3.4. http://www.fftw.org/

[TheProfileth]

I would like to report a bug regarding Dither_median16 when I use dither_median16 if I set rt=0 and setting rx=1 and ry =1 then the output seems to be broken like it is all black except for about one line, however if I set rx and ry to something higher than 1 and leave rt set to 0 it outputs as expected, the same happens if you simply set rt=1 while rx=1 and ry=1. The weird thing about this behavior is that you can only exhibit it by using the default settings of the function which specifically say
Dither_median16 (
clip src,
int rx (1),
int ry (1),
int rt (0),
int ql (undefined),
int qh (undefined),
int y,
int u,
int v
)
I used this to repeatedly recreate the problem

Dither_convert_8_to_16
Dither_median16()
ditherpost(mode=-1)

[asarian]

(Original reply replaced by this Dither presentation.)


Download

>>>> dither-1.26.1.zip <<<< (http://ldesoras.free.fr/src/avs/dither-1.26.1.zip)
Main scripts and plug-ins from the Dither package, including avstp.dll.


Is it just me, or has the Dither download site disappeared?!

[Sparktank]

Is it just me, or has the Dither download site disappeared?!

Yes, it seems down (for now).
http://www.downforeveryoneorjustme.com/http://ldesoras.free.fr

Site Status reports a specific error: 500 (timeout)
http://www.sitestatus.net/

I'm sure it's temporary for now.

EDIT: teh actual domain "free.fr" is up.
http://www.isitdownrightnow.com/free.fr.html

[asarian]

Yes, it seems down (for now).
http://www.downforeveryoneorjustme.com/http://ldesoras.free.fr

Site Status reports a specific error: 500 (timeout)
http://www.sitestatus.net/

I'm sure it's temporary for now.

EDIT: teh actual domain "free.fr" is up.
http://www.isitdownrightnow.com/free.fr.html

Thx. :)

[asarian]

(Original reply replaced by this Dither presentation.)


Download

>>>> dither-1.26.1.zip <<<< (http://ldesoras.free.fr/src/avs/dither-1.26.1.zip)
Main scripts and plug-ins from the Dither package, including avstp.dll.
>>>> dfttest 1.9.4 <<<< (http://ldesoras.free.fr/src/avs/dfttest-1.9.4.zip)
Mod16 version of dfttest.
>>>> MVTools 2.6.0.5 <<<< (http://ldesoras.free.fr/src/avs/mvtools-2.6.0.5.zip)
Mod16 version of MVTools 2
Masktools 2 a48 (http://manao4.free.fr/) or above is required. Previous Masktools versions will silently fail and cause the greenish screen of death. Please update your Masktools before installing the Dither script.




Hmm, if I'm to believe my hover-text, my MaskTools are still at 1.15.10 (I could have sworn they were at 2 a48). They're on the same French site, though, which is still down. There used to be an AviSynth filter repository, iirc.

[feisty2]

@asarian
I copied all those stuff from my plugins folder, hope it helps for now.
https://www.sendspace.com/file/i4atjp

[asarian]

@asarian
I copied all those stuff from my plugins folder, hope it helps for now.
https://www.sendspace.com/file/i4atjp

Thank you!! Got it! :)

[asarian]

Dither requires Masktools 2 alpha 48 or later, and asks me to remove mt_masktools.dll, mt_masktools-25.dll and mt_masktools-26.dll. When I do, however, QTGMC throws an error:

"There is no function called mt_makediff."

Is there way to use Masktools 2 a48, Dither, AND mt_masktools-25.dll?!

Thanks.

[feisty2]

delete mt_masktools-25.dll and update to avisynth 2.6 or avs+

[asarian]

delete mt_masktools-25.dll and update to avisynth 2.6 or avs+

Upgraded everything to 2.6. Still getting:

"There is no function named mt_makediff."

On using QTGMC. I supposed I could find an older version of mt_masktools-26 again, but Dither specifically says to remove it.

EDIT: French site is up again. :) Found the missing dll and installed it again! :)

[feisty2]

Upgraded everything to 2.6. Still getting:

"There is no function named mt_makediff."

On using QTGMC. I supposed I could find an older version of mt_masktools-26 again, but Dither specifically says to remove it.

EDIT: French site is up again. :) Found the missing dll and installed it again! :)

how many masktools dll files are placed in your plugins folder, you can only put one in the folder and delete all the rest

[asarian]

how many masktools dll files are placed in your plugins folder, you can only put one in the folder and delete all the rest

I was able to download the full masktools just now, so I installed just mt_masktools-26 now, and it's working again. :)

I'm getting this error on your script, now, though:

Eedi3 does not have a named clip 'mclip'

I still have an eedi3 0.9.1 (for 2.5, it says); but link to the same version at least is dead too.

[feisty2]

I was able to download the full masktools just now, so I installed just mt_masktools-26 now, and it's working again. :)

I'm getting this error on your script, now, though:

Eedi3 does not have a named clip 'mclip'

I still have an eedi3 0.9.1 (for 2.5, it says); but link to the same version at least is dead too.

you need eedi3mod like I said in your upscale thread, its way much faster

[asarian]

you need eedi3mod like I said in your upscale thread, its way much faster

Sorry, my bad, missed that. :o

[asarian]

Odd. Seems Dither_Convert_8_to_16 () doubles my vertical resolution (not horizontal). Is it supposed to do that?

[foxyshadis]

16-bit in avisynth is done that way, the first 8 bits of each pixel in one half and the other 8 bits in the other, called stack16.

[asarian]

16-bit in avisynth is done that way, the first 8 bits of each pixel in one half and the other 8 bits in the other, called stack16.

Huh. That's strange. :) So, if you end your script with Dither_Convert_8_to_16 (), and start to encode it (with, say, x264, like I did), you'll wind up with a doubled vertical resolution?! That can't be right.

Or is Dither_Convert_8_to_16 () just meant as some sort of intermediate (internal) state?

[feisty2]

Huh. That's strange. :) So, if you end your script with Dither_Convert_8_to_16 (), and start to encode it (with, say, x264, like I did), you'll wind up with a doubled vertical resolution?! That can't be right.

Or is Dither_Convert_8_to_16 () just meant as some sort of intermediate (internal) state?

double height is a hack in avisynth used to convey 16bpc clips on 8bpc colorspaces, the "normal" image on the upper half is called "MSB" (first 8bits of each channel), the crappy look stuff on the lower half is called "LSB" (9-16 bits of each channel) with MSB+LSB stacked together, the clip is actually at 16bpc precision, not common 8bpc precision, "ditherpost" I wrote in your upscale thread will accept 16bpc format image and dither it to 8bpc so you can encode the video directly

[foxyshadis]

x264-tmod (https://astrataro.wordpress.com/2014/04/24/x264-rev2431-tmod/) and avs4x264mod (https://astrataro.wordpress.com/2013/08/31/avs4x264mod-0-9-1/) can both read 16-bit avisynth scripts without dithering to 8-bit first. (They'll internally dither to either 8 or 10 bit, whichever one you encode with.) Ditherpost should only be used if you have to have 8bit output on the other end.

[feisty2]

I know, I just don't wanna confuse asarian who seems hasn't used ditherpackage before :)

[asarian]

x264-tmod (https://astrataro.wordpress.com/2014/04/24/x264-rev2431-tmod/) and avs4x264mod (https://astrataro.wordpress.com/2013/08/31/avs4x264mod-0-9-1/) can both read 16-bit avisynth scripts without dithering to 8-bit first. (They'll internally dither to either 8 or 10 bit, whichever one you encode with.) Ditherpost should only be used if you have to have 8bit output on the other end.

Thank you both for your detailed explanations! :)

[Sparktank]

avs4x264mod (https://astrataro.wordpress.com/2013/08/31/avs4x264mod-0-9-1/)

Just today, Taro updated it and renamed it.

avs4x26x (https://astrataro.wordpress.com/2014/08/28/avs4x26x-0-10-0/)

It now supports x265, x264 and x262, why the renaming of the tool.

[cretindesalpes]

I would like to report a bug regarding Dither_median16

Oh! Sorry. Thanks for the report. Fixed in Dither 1.26.2 (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

[asarian]

May I ask, in

src.DitherPost (mode=6, y=3, u=1, v=1)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
MergeRGB (r, g, b)
}

What is the function of the SelectEvery() here? I thought that was just for interlaced and pulldown stuff.

Thanks.

[feisty2]

May I ask, in

src.DitherPost (mode=6, y=3, u=1, v=1)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
MergeRGB (r, g, b)
}

What is the function of the SelectEvery() here? I thought that was just for interlaced and pulldown stuff.

Thanks.

it's for rgb48y format, a hacked format to store 16bpc rgb on y8 colorspace, r channel g channel and b channel are interleaved on the y plane of 3 neighbor frames, so need to merge 3 neighbor frames together to restore the correct rgb image

[feisty2]

rgb48y is a dope hack, with this format, you can convert your clips to rgb colorspace and still use common yuv only filters on them, filtering in rgb can return significantly better results usually

[Motenai Yoda]

Any alternative to Dither_median16() for mt_inpand or mt_expand at 16bit?

[feisty2]

are you sure there are mt_inpand and mt_expand tools at 16bpc?

[Motenai Yoda]

are you sure there are mt_inpand and mt_expand tools at 16bpc?

I mean for something like mt_inpand and mt_expand, actually Dither_Median16 is quite slow.

@cretindesalpes I found some shift/distortion caused by Dither_Resize16 used with Dither_Box_Filter16

ie
imagesource("4.2.03.tiff")
converttoyv12()
StackHorizontal(last,last,last,last)
http://www.imagebam.com/image/3180fd349209628

imagesource("4.2.03.tiff")
converttoyv12()
StackHorizontal(last,last,last,last)
Dither_convert_8_to_16()
orig=last
Dither_resize16(2500,height/2)
Dither_box_filter16(radius=2, y=3, u=3, v=3)
Dither_resize16(orig.width,orig.height/2)
ditherpost()
http://www.imagebam.com/image/577eab349209579
(look at about (2048/2500)*2048)

edit: ok isn't related to Dither_Resize16 anyway, that's something Dither_Box_Filter16 shows about at 2048th column.

I fixed this way for now...
Function Dither_Box_Filter16_Fixed(clip c, int "radius", int "y", int "u", int "v")
{
part1=c.crop(0,0,c.width/2 + 96,0).Dither_box_filter16(radius=radius, y=y, u=u, v=v)
part2=c.crop(c.width/2 - 96,0,0,0).Dither_box_filter16(radius=radius, y=y, u=u, v=v)
return stackhorizontal(part1.crop(0,0,c.width/2,0),part2.crop(96,0,0,0))
}

[feisty2]

http://forum.doom9.org/showthread.php?t=169832
MedianBlur2 is an 8bpc version of dither_median16, and works very fast

[Motenai Yoda]

http://forum.doom9.org/showthread.php?t=169832
MedianBlur2 is an 8bpc version of dither_median16, and works very fast

well but I need something that replace the target pixel with the min or max pixel from the 3×3-pixel reference square with 16bpc in/out.
Dither_median16 can do this as
Dither_median16(ql=0,qh=0) or Dither_median16(ql=8,qh=8)
but isn't optimized at all.

[tormento]

Just thought I'de point out that FFTW library used by eg FFT3DFilter as fftw3.dll and dftTest as libfftw3f-3.dll (same dll, just different name)
has been updated twice since the one included in dither tools. Last update March 16 2014, v3.3.4. http://www.fftw.org/
A bit off topic but can't understand difference between libfftw3-3.dll, libfftw3f-3.dll and libfftw3l-3.dll. Which should I use?

[Sparktank]

libfftw3-3.dll,
libfftw3f-3.dll,
and libfftw3l-3.dll

The "Readme-Windows" doc (of FFTW) in the source says:

There are three libraries:
single precision (float),
double precision,
and extended precision (long double).

And the header file designates them as:

fftw = double precision
fftwf = single precision (float)
fftwl = extended precision (long double)

The dfttest "Readme" doc (of dfttest) says to use the "f" version.
Requires libfftw3f-3.dll to be in the search path.
fftwf = single precision (float)

[Sparktank]

There's also this post...

Regarding FFTW3.dll, I found this: http://forum.doom9.org/showthread.php?t=156028 , near the bottom of the first post.

It seems that you can rename libfftw3f-3.dll to fftw3.dll if you want to upgrade.

I don't know how MAnalyse uses fftw3 so it's not certain that there actually are any changes, but I'll see if anything weird happens anyway.

Boulder hasn't reported back any problems, so I assume that works, too.

[Boulder]

Yep, no problems here. I don't recall which version I have but I'd assume any recent version will work.

[cretindesalpes]

I found some shift/distortion caused by […] Dither_Box_Filter16

Thanks for the report. Fixed in Dither 1.26.3 (http://forum.doom9.org/showthread.php?p=1386559#post1386559).

[colours]

http://forum.doom9.org/showthread.php?t=169832
MedianBlur2 is an 8bpc version of dither_median16, and works very fast

The reason it's "very fast" is because it's assembly optimised. Dither_median16 isn't, and also uses a full sort instead of a fast selection algorithm like quickselect.

Motenai Yoda: If you're looking to getting fast 16-bit mt_xxpand equivalents, but don't know how to write code to do this, it wouldn't be difficult to hack something up using CLExpr (assuming you can use OpenCL). Probably something like cl_exprxyz(last,pointresize(width,height,-1),pointresize(width,height,1),"x y z max max",lsb=true).

[asarian]

Speaking of speed, whilst I love the Dither package, I noticed Dither_resize16() and Dither_resize16nr() are rather slow.

Is there a way to speed them up maybe? (special MT mode, GPU assistence, etc).

Thanks.

[feisty2]

rather slow? dither_resize16 runs faster than realtime processing on my computer when upscaling 480p to 1080p

[asarian]

rather slow? dither_resize16 runs faster than realtime processing on my computer when upscaling 480p to 1080p

Probably related to MT mode again, I reckon. in Mode 1 it runs much faster. :)

[asarian]

On a more serious note, seems SmoothGrad() wreaks havoc a bit on more darker areas:

SmoothGrad() de-GRAD-ation (https://www.dropbox.com/s/2cfcfrtpwy3gmp1/smoothgrad.jpg?dl=0)

Obviously it lessens the amount of banding (already present in the source); but all gets so smudged, that the darker areas almost disappear.

I didn't use it with a vector or anything, just a single SmoothGrad() call over everything.

[cretindesalpes]

You should not use SmoothGrad alone to reduce banding, unless using extremely light settings. It’s a simple building block. It requires edge and detail masking to avoid the mentioned problems. Prefer using GradFun3 or f3kdb. If you’re processing dark scenes, you’ll probably need a luma mask too.

[Boulder]

Out of interest, is it generally useful to apply GradFun3 if the output is then encoded using x264 (with general settings not targeted for grain retention, and usually crf 18-19)?

[asarian]

You should not use SmoothGrad alone to reduce banding, unless using extremely light settings. It’s a simple building block. It requires edge and detail masking to avoid the mentioned problems. Prefer using GradFun3 or f3kdb. If you’re processing dark scenes, you’ll probably need a luma mask too.

Thx. I figured I'd need masking. :)

As for GradFun3, I thought that was only to *prevent* banding (whereas SmoothGrad is used to *remove* existing banding).

[feisty2]

all smoothing filters (denoise filters, blurring filters, etc) can deband your clip more or less, cuz banding it very self is a kind of noise too. as for PREVENTING banding occurring on banding free clips, it's a job for dither filters, not smoothing filters

[asarian]

all smoothing filters (denoise filters, blurring filters, etc) can deband your clip more or less, cuz banding it very self is a kind of noise too. as for PREVENTING banding occurring on banding free clips, it's a job for dither filters, not smoothing filters

Exactly. :) So, I thought GradFun3 was to *prevent* banding (like GradFun2DBmod does) when compressing starts (x264).

[feisty2]

gradfun3 is a smoothing filter, dither filter is something like ditherpost

[feisty2]

a deband (smoothing) filter takes low precision banding clips and outputs high precision banding free clips
a dither filter takes high precision banding free clips and outputs low precision clips via dither algorithms, so banding won't jump outta noplace on your downsampled low precision clips

[asarian]

a deband (smoothing) filter takes low precision banding clips and outputs high precision banding free clips
a dither filter takes high precision banding free clips and outputs low precision clips via dither algorithms, so banding won't jump outta noplace on your downsampled low precision clips

Thx. :)

Well, the Dither manual has some excellent examples for using masks with these. I guess I should go try em out, instead of just applying SmoothGrad() on the entire frame.

[feisty2]

gradfun3 is simply a shortcut of smoothgrad+mask filters+limit filters, everything you need has been built in inside the func, that's why smoothgrad is just a building block of gradfun3, use gradfun3, you don't have to worry about mask stuff anymore

[Sparktank]

I just noticed the document is updated and very organized!

Excellent work! :) I love the new TOC.

Any chance for cretindesalpes to hijack thread?
Like copy/move his main post (with updates) to be post #0 (or -1) and push the first post (thread starter) to post #1?
Or maybe just move to new thread where he gains control over thread starter.
I'm pretty used to how it is now, but for simplistic reasons, I think it would fair better with Dither_package being first post and title of thread for newcomers.

Keep up the good work, cretindesalpes.

[StainlessS]

Any chance for cretindesalpes to hijack thread?

I guess that would require osgZach consent, he is still around last on-line 26 aug 2014.

It is not without precedent, I took over a thread originally by Forensic.

[Motenai Yoda]

Probably something like cl_exprxyz(last,pointresize(width,height,-1),pointresize(width,height,1),"x y z max max",lsb=true).
I don't think this can work, maybe a more complex one like (also note pointresize is in 8bit domain)

h=cl_exprxyz(last,dither_resize16(width,height /2,-1,0,-1,0,kernel="point"),\
dither_resize16(width,height /2,1,0,width +1,0,kernel="point"),"x y z max max",lsb=true)
v=cl_exprxyz(last,dither_resize16(width,height /2,0,-1,0,-1,kernel="point"),\
dither_resize16(width,height /2,0,1,0,(height /2) +1,kernel="point"),"x y z max max",lsb=true)
cl_exprxy(h,v,"x y max",lsb=true)

btw my gpu doesn't support opencl over 1.0.

[cretindesalpes]

Dither 1.26.4 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed bug in Dither_limit_dif16 and SmoothGrad when thr and elast are both too low or too high. Thanks to mirkosp for reporting it.
The radius value is more thoroughly checked in GradFun3 in order to make related error messages more explicit.

[asarian]

Dither 1.26.4 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed bug in Dither_limit_dif16 and SmoothGrad when thr and elast are both too low or too high. Thanks to mirkosp for reporting it.
The radius value is more thoroughly checked in GradFun3 in order to make related error messages more explicit.

Hmm, when I upgraded, I saw both avsi files were identical to the one I already had, LOL. :confused:

Also, your mvtools also comes with its own avstp.dll file. The dither package also has one (which appears to be created an hour later). So, which one should I be using now?

Thx.

[cretindesalpes]

dither.avsi has changed. Check the version on the header (1.26.4)
The avstp.dll should be the same too, the date difference probably comes from conversions to local time (with and w/o DST).

[asarian]

dither.avsi has changed. Check the version on the header (1.26.4)
The avstp.dll should be the same too, the date difference probably comes from conversions to local time (with and w/o DST).

Thx. :)

[cretindesalpes]

Dither 1.26.5 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed Dither_resize16 to make pure vertical resizing multi-threaded.

[bxyhxyh]

Hi, I can't understand Dither_lut8, Dither_lutxy8 and Dither_lutxyz8.
Documentation says These functions are the equivalent of the Masktools mt_lut, mt_lutxy and mt_lutxyz functions. Here, they generate 16-bit data from 8-bit clips.
But it generates clip like this

source - https://dl.dropboxusercontent.com/u/58215671/source.png
dither_lut8() - https://dl.dropboxusercontent.com/u/58215671/result.png

What am I doing wrong?

[colours]

bxyhxyh, it doesn't look like there's anything wrong there.

Remember that you need to scale your output by 256 when going from 8-bit to 16-bit to get something meaningful. In other words, expr="x 256 *" would be the "no-op" for Dither_lut8, not expr="x". Likewise for the other 8-to-16 LUT functions.

(Of course there are caveats and occasionally it makes sense to not scale the input, but I'm guessing this is not one of those times.)

[bxyhxyh]

From documentation, I thought dither_lut8() and dither_lutxy8() work on normal 8bit clip. And result would be 16bit stacked clip.

But it is not even 16bit stacked clip.

Do you have an example that uses dither_lut8() properly?

[feisty2]

very simple, you can keep your old 8bpc lut expr and add "256 *" at the end of it, and it will work on dither_lut8

[colours]

But it is not even 16bit stacked clip.

It does have twice the height of the input clip, so I'm not sure why you don't think it's a stack16 clip.

Like I said, you can check it for yourself with Dither_lut8("x 256 *",u=3,v=3), which would be a slower version of Dither_convert_8_to_16.

[bxyhxyh]

Ok. As you and feisty said it works. But is it 256 or 257?
0=0 and 255(8bit white)*257=65535(16bit white) (not 256=65536 here)

[colours]

The convention is to multiply by 256; both DitherPost and f3kdb_dither assume this convention when reducing the bit depth. BT.709 and BT.2020 also specify this depth conversion convention.

Unless you intentionally want to screw up the levels or the source is full-range 8-bit and you want the output to be full-range 16-bit, don't multiply by 257.

[StainlessS]

Ok. As you and feisty said it works. But is it 256 or 257?
0=0 and 255(8bit white)*257=65535(16bit white) (not 256=65536 here)

I dare say that you spotted your little faux pas, but just in case you did not,

255 * 256 = 65280 ($FF * $100 = $FF00)
256 * 256 = 65536 ($100 * $100 = $10000)

[bxyhxyh]

The convention is to multiply by 256; both DitherPost and f3kdb_dither assume this convention when reducing the bit depth. BT.709 and BT.2020 also specify this depth conversion convention.

Unless you intentionally want to screw up the levels or the source is full-range 8-bit and you want the output to be full-range 16-bit, don't multiply by 257.

Ok. Thanks.

[bxyhxyh]

Hi, sorry for double post

This shifts my video. It shouldn't do shift, right?
Dither_convert_8_to_16().Dither_resize16(1280,720,0,5,0,0,kernel="spline16").Ditherpost()

So I'm using.
Dither_convert_8_to_16().Dither_resize16(1280,720,0,5,0,-0.0001,kernel="spline16").Ditherpost()

Is it a bug or not?

[TheSkiller]

RTFM. ;)


dither_resize16 (
clip src,
int width,
int height,
float src_left (0),
float src_top (0),
float src_width (0),
float src_height (0),
string kernel ("spline36"),
float fh (1.0),
float fv (1.0),
int taps (4),
float a1 (undefined),
float a2 (undefined),
float a3 (undefined),
int kovrspl (1),
bool cnorm (true),
bool center (true),
string cplace ("mpeg2"),
int y (3),
int u (3),
int v (3),
string kernelh (""),
string kernelv (""),
float totalh (0),
float totalv (0),
bool invks (false),
bool invksh (invks),
bool invksv (invks),
int invkstaps (5)
)


You are telling dither_resize16 to crop before resizing.

[bxyhxyh]

RTFM. ;)
You are telling dither_resize16 to crop before resizing.

Yes, but it is not cropping like standard resizers, it is shifting.

[atra dies]

My avsi doesn't have function dither_resize16, ditherpost, latest 1.26.5 and says 1.26.3 at the top? Am I missing something?

[Reel.Deel]

That's odd. Try redownloading? Anyways dither_resize16, ditherpost are part of dither plugin not the script.

[atra dies]

Oops, updated from a sandbox which doesn't work.
Alright now.

[Boulder]

cretindesalpes,

have you happened to have taken a look at the improvements mentioned by chainik_svp here: http://forum.doom9.org/showthread.php?p=1695899#post1695899 ? Just wondering if those could be included in your MVTools2 build :)

[Overdrive80]

Hi, using gradfun3(mode=2,3,4 or 5) in flat areas I see vertical lines, is it normal???

Source: https://dl.dropboxusercontent.com/u/19135067/Sample.demuxed.m2v

Script used:

assumetff()

tfm(order=1,pp=6,mode=4).tdecimate(mode=1)

#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-
#FILTRADO
#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-

original=last

#/*Prefiltrado*/#

pre=original.dfttest(sigma=8.5,tbsize=1,lsb=true).ditherpost()

mascara=original.mt_edge("hprewitt",thY1=28, thY2=55,thc1=0,thc2=255).mt_inflate(thY=16)

#return mascara

prefiltrado=mt_merge(pre, original, mascara,u=2,v=2)

#/*Filtrado temporal*/#

tr = 6

super= MSuper(prefiltrado,mt=true)

multi_vec = MAnalyse (super, multi=true, delta=tr, overlap=4,search=4)

prefiltrado.MDegrainN (super, multi_vec, tr, thSAD=470, thSAD2=170,plane=4,lsb=true)

#/*Otros*/#

gradfun3(mode=2,lsb_in=true)



For seeing better this effect I had resized to * 2, and this is the result:

Mode 2: http://i57.tinypic.com/12192yv.png

[Izuchi]

If one were to apply 16bit filtering and then encode to 10bit using x264, which method would be better quality wise?


16bit filtering, then using Dither_Out() and --input-depth 16; or
16bit filtering, then using Dither_quantize (10,reducerange=true) + Dither_out () and --input-depth 10


Would inputting a 16bit clip versus a 10bit clip be better since it has higher bits?

[wOxxOm]

Izuchi, 10-bit dithering by DitherTools is better because it's more sophisticated and customizable whereas x264 internal 16->10bit transformation is rather primitive. The only case where 16-bit undithered input would have been justified is a not-yet-supported 16-bit Hi16P profile. Anyway whichever method you choose I doubt there'll be a noticeable visual difference unless you encode textureless 3D-rendered stuff with smooth artificial color transitions. However there might be a size difference if you go with Dither_quantize's mode=1 which is better compressible.

[feisty2]

Direct 16 bit input to x264 = rounding to 10 bit

[the_weirdo]

The only case where 16-bit undithered input would have been justified is a not-yet-supported 16-bit Hi16P profile.

H.264 doesn't have such profile and I don't think they'll ever add that profile to H.264 specs.

[mawen1250]

x264 always uses Sierra-2-4A error diffusion for bit depth down-conversion, which is an encode-friendly dithering method.
fmtconv supports it but DitherTools doesn't, so I'd rather directly output 16bit for TV range content. (for PC range content x264 and DitherTools always applies bit shifting for bit depth conversion which is not correct)

[fabioseixal]

Hello, I just want to thank cretindesalpes for his oustanding job :)
Thank you very much, you made me have a lot more quality in my personal encodings.

[Dogway]

x264 always uses Sierra-2-4A error diffusion for bit depth down-conversion, which is an encode-friendly dithering method.

Friendlier than ordered dither?

Dither documentation encourages the pre-dither step with Dither_quantize(), which is defaulted to mode=0.

I thought that high bitdepth input would benefit from other x264 float point calculations like motion analysis, etc. As this is put it looks like feeding x264 a 8-bit Sierra-2-4A dithered source would yield the same results as if the input was the same source in it undithered 16-bit form, is this right?

[Izuchi]

Direct 16 bit input to x264 = rounding to 10 bit

Yes, that's the part I wasn't quite sure of: whether to output 16-bit and let x264 handle the bit depth conversion to 10-bit; or output directly to 10-bit in which case x264 doesn't need to perform any further bit depth conversions since both the input script and output video will be the same bit depth.

[Sparktank]

Yes, that's the part I wasn't quite sure of: whether to output 16-bit and let x264 handle the bit depth conversion to 10-bit; or output directly to 10-bit in which case x264 doesn't need to perform any further bit depth conversions since both the input script and output video will be the same bit depth.

Well, letting the Dither filter dither down to 10bit, you have options of which dither algorithm you want.

The documentation has some info on how certain dither algorithms affect the output.

Actually, I think it was flash3kyuu_deband that specified how different dither algorithms affected the outcome (especially in an x264 encode).

From the flash3kyuu help.txt:
dither_algo:
3: Floyd-Steinberg dithering
* Visual quality of mode 3 is the best, but the debanded pixels may easily be destroyed by x264, you need to carefully tweak the settingsto get better result.

I would think having the option to choose different algorithms (suited to your encoding needs) would be a better choice than being limited a singular option (now that we know how x264 dithers).
Entirely source-dependent.
Blu-ray movies would yield different results than an HD camcorder.

There's more to read in the flash3kyuu help.txt. Or whatever it's called.

But, I can't imagine the dither algorithm is proprietary and different than most other dither algorithms.

[cretindesalpes]

Direct 16 bit input to x264 = rounding to 10 bit
No. As mawen1250 stated, x264 uses Sierra-2-4A to reduce bitdepth. This algorithm is similar to Floyd-Steinberg quality-wise, but a bit more computationally-efficient.

It was better to convert to 10 bits in Avisynth in the early days of 10-bit encoding because of a bug in x264 (or a wrong design), but this is not required anymore. As Dogway said, you can still use 10-bit ordered dithering to keep the best possible gradients, but I’m not sure it’s worth the effort, as 10-bit banding is more or less beyond the visual perception or hardware capabilities. You’d better spend these precious bits on other features.

[feisty2]

No. As mawen1250 stated, x264 uses Sierra-2-4A to reduce bitdepth. This algorithm is similar to Floyd-Steinberg quality-wise, but a bit more computationally-efficient.


I blame it for the harm of rumors :)

[mawen1250]

Friendlier than ordered dither?

Dither documentation encourages the pre-dither step with Dither_quantize(), which is defaulted to mode=0.

I thought that high bitdepth input would benefit from other x264 float point calculations like motion analysis, etc. As this is put it looks like feeding x264 a 8-bit Sierra-2-4A dithered source would yield the same results as if the input was the same source in it undithered 16-bit form, is this right?

Obviously ordered dither is more encode-friendly, but it will generate undesired patterns in the image.
When encoding in low bitrate 8bit, I prefer ordered dither to avoid banding. For 10bit or high bitrate 8bit, random dither is a better choice.

[Dogway]

Maybe I had it wrong from the beginning but to put in other words, what's the benefit of feeding high bitdepth video to x264 instead of a pre-dithered source? Is it only for the Sierra algo that Dither lacks?


Also wanted to ask if it's possible to change chroma placement within Dither tools, as for now I'm using the next for a YV12 DV PAL clip. I guess I am right to assume the clip uses DV chroma placement but, is there a way to know for sure? (I captured using WinDV as Type 1, and then ignoring I couldn't edit with that converted to Type 2 with Enosoft DV Processor)
ConvertToYV12(matrix="Rec601", interlaced=true,ChromaInPlacement="DV",ChromaOutPlacement="MPEG2")


This is as far as I went using Dither but I'm not sure if converting to rgb and back is worth, result differs a lot from the above.
Dither_convert_yuv_to_rgb(lsb_in=false,cplace="DV",chromak="spline36",output="rgb48y",interlaced=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv(r,g,b,lsb=false,cplace="mpeg2",chromak="spline36",mode=6,interlaced=true)

[TheSkiller]

I guess I am right to assume the clip uses DV chroma placement but, is there a way to know for sure?Depends on the decompression Codec. If you use Cedocida (http://forum.doom9.org/showthread.php?t=94458) (highly recommended) it's configuration allows you to output the raw YV12 (DV chroma placement) or a MPEG2 corrected one – your choice.

[Izuchi]

I just want to confirm something to make sure I read the Dither documentation right (you can never be too safe).

If I were to only use 16-bit filters and then reduce the bitdepth down to, say 8-bit, would the following filter-chain as an example be correct?


src
Dither_convert_8_to_16()
HQDeringmod(lsb=true,lsb_in=true,lsb_out=true)
SMDegrain(lsb=true,lsb_in=true,lsb_out=true)
GradFun3(lsb=true,lsb_in=true)
DitherPost()


So basically, what I'm asking is that I should be able to use as many 16-bit filter calls as I want without the results going awry, right?
As long as I remember to include the lsb=true,lsb_in=true,lsb_out=true parameters for each successive filter to make sure its stacked 16-bit data, of course.

[cretindesalpes]

Dogway: I’m not sure about the interlaced 4:2:0 DV chroma placement. If someone knows a serious technical source, I’m all ears. What is currently implemented in Dither is described here (http://www.mir.com/DMG/chroma.html). The code in Avisynth (avisynth/src/convert/convert_planar.cpp, ConvertToPlanarGeneric constructor) looks different.

Izuchi: Yes, you’re right.

[Dogway]

cretindesalpes: Yes, I followed the same page diagrams and ended up with this (http://forum.videohelp.com/threads/369658-Converting-chroma-placement-from-DV-to-MPEG2-manually?p=2369301&viewfull=1#post2369301).
So it can be done manually with Dither, it matches Cedocida's conversion, and as you say Avisynth conversion is different, wrong IMO.
In theory the above Dither conversion should also work, but it's different too, although not an ideal method (due two a double chroma resize) maybe it could be fixed, I'm not sure.

Mainly I wanted to ask about high bitdepth input in x264, trying to clear up this post (http://forum.doom9.org/showthread.php?p=1704794#post1704794). I thought it did something else other than a "ditherpost" to Sierra algo, like some high bitdepth motion analysis, or else. In that regard an imaginary 8 bit Sierra dithered input would yield the same output as a 16-bit input? If that was the case the only benefit of high bitdepth input x264 builds would be for "10bit in->10bit out".

[Dogway]

I still think the above should be double checked. Converting a DV chroma encoded YUV format to RGB should produce a chroma cositing output (all planes aligned). Then from RGB back to YUV I use cplace="mpeg2", so my output should have MPEG2 chroma placement. Unless I'm reading the "cplace" setting wrongly.
It's possible that the interlaced flag could be causing the error, although I haven't checked without it.

Also a reminder of the above, is there any benefit using high bitdepth input for x264 when rendering to 8bit?

What I really came for is that I think there's a bug in Dither_resize16nr(), I was doing some tests and when using for output a resolution around 3K the noring cancellation wasn't working. I'm not sure if it's due to high demanding memory, I compared against Resize8 where it was working.

[feisty2]

Read the script and you will find out noring does not work on large factor upscale
It's not a bug, it's designed like this

[Dogway]

yes, found the note in the script, it's undocumented though.

[foxyshadis]

Does anyone know a way to deal with LWLibAVVideoSource's broken stack16 format? It opens 10-bit files as (msb/lsb) xxxxxx12/34567890 instead of 12345678/90xxxxxx.

Edit: Never mind, Dither_lut16("x 64 *",u=3,v=3) works perfectly.

[feisty2]

Does anyone know a way to deal with LWLibAVVideoSource's broken stack16 format? It opens 10-bit files as (msb/lsb) xxxxxx12/34567890 instead of 12345678/90xxxxxx.

Edit: Never mind, Dither_lut16("x 64 *",u=3,v=3) works perfectly.

LWLibavvideosource ("xxx",xxx,format="yuv4xxp16")

[foxyshadis]

Thanks! I didn't realize that was an option.

[Motenai Yoda]

Edit: Never mind, Dither_lut16("x 64 *",u=3,v=3) works perfectly.
or use "x 6 <<u"

[Boulder]

cretindesalpes,

sorry to bother you again, but would you be willing to add the speedup changes of svp's MVTools and the optimized SATD functions that the VapourSynth version has to your latest MVTools build?

I've been investigating switching to VapourSynth but changing some of my custom functions might not be too easy if not feasible at all :(

[Sparktank]

Would it be possible to add Jinc to the resizers (for upscale only) ?
http://forum.doom9.org/showthread.php?p=1655388#post1655388

[cretindesalpes]

Boulder:
Maybe if I can find the time to do it, but it is a lot of work and I’m not sure it is worth it (IIRC most of the SVP speed-up was related to private GPU code).

Sparktank:
Not at the moment. The JINC resizer is not of the separable kind and would require a different code path which is more work than just adding a kernel.

I’m currently busy merging the resizer codes from fmtconv and Dither, adding AVX2 support and a few other improvements while working on 32/64-bit/VS-dfttest/eedi3 at the same time, so things go slower than ever.

[Sparktank]

Sparktank:
Not at the moment. The JINC resizer is not of the separable kind and would require a different code path which is more work than just adding a kernel.

I thought it might be different due to being an upscaler only.
Just wanted to check.

I’m currently busy merging the resizer codes from fmtconv and Dither, adding AVX2 support and a few other improvements while working on 32/64-bit/VS-dfttest/eedi3 at the same time, so things go slower than ever.

Sounds like a workload!
Can't wait to see the future updates. :thanks:

[Boulder]

Is it possible to keep this whole package in high bit depth and just use Ditherpost over "output" at the end? I am very unsure how the Masktools2 calls react.


cleanedclip = MDegrain1(c,supermdg,bv1,fv1,thsad=200,lsb=true)
limited = mt_lutxy(cleanedclip,c,"x y < x 1 + x y > x 1 - x ? ?",U=3,V=3)
output = CSMDG(original,limited,supermdg,bv1,fv1,true)
return output

Function CSMDG(clip original, clip denoised, clip cssuper, clip cb1, clip cf1,\
bool HD, bool "chroma", bool "preblur", bool "preblurc", int "preR", bool "limit",\
string "blur", int "RGmode", int "RGmodeU", int "Repmode", int "RepmodeU", int "overshoot")
{
# contra-sharpening: sharpen the denoised clip, but don't add more to any pixel than what was removed previously.
# script function from Didee from the VERY GRAINY thread

chroma = Default(chroma, true )
preblur = Default(preblur, true )
preblurc = Default(preblurc,chroma )
preR = Default(preR, 1 )
limit = Default(limit, true )
RGmode = Default(RGmode, 11 )
RGmodeU = Default(RGmodeU, RGmode )
Repmode = Default(Repmode, 1 )
RepmodeU = Default(RepmodeU,Repmode)
overshoot = Default(overshoot, 0 )

HD ? eval("""
cb1=original.MCompensate(csSuper, cb1)
cf1=original.MCompensate(csSuper, cf1)
pmax = original.mt_logic(cb1, "max").mt_logic(cf1, "max")
pmin = original.mt_logic(cb1, "min").mt_logic(cf1, "min")""") : NOP()

s = denoised.MinBlur(HD?2:1,1) # Damp down remaining spots of the denoised clip.
allD = mt_makediff(original,denoised) # The difference achieved by the denoising.
ssD = mt_makediff(s,HD?s.removegrain(20,-1).\
removegrain(20,-1):\
s.removegrain(11,-1)) # The difference of a simple kernel blur.
ssDD = ssD.repair(HD?ssD.repair(allD,1):allD,HD?12:1) # Limit the difference to the max of what the denoising removed locally.
ssDD = SSDD.mt_lutxy(ssD,"x 128 - abs y 128 - abs < x y ?") # abs(diff) after limiting may not be bigger than before.

denoised.mt_adddiff(ssDD,U=2,V=2) # Apply the limited difference. (Sharpening is just inverse blurring)
HD ? mt_clamp(last,pmax,pmin,overshoot,overshoot,chroma="copy first") : last

return last
}

[cretindesalpes]

The contra-sharpening part alone can be converted to 16 bits (it has already been done). It think the mt_logic min/max can be converted too. However MCompensate is not compatible with 16-bit input or output.

[colours]

It think the mt_logic min/max can be converted too.

Someone wrote that too. (http://pastebin.com/D25HuLQ5) It is unfortunately a bit of a hack and also kinda slow. It could probably be optimised a little, but it'd definitely be slower than native code anyhow.

Using CLExpr would be faster if available (e.g. cl_exprxy(a,b,"x y max",lsb=true)), but apparently not everyone can use CLExpr.

[Boulder]

Fortunately MCompensate is run on a regular 8-bit super clip so that can be ignored.

Could someone please give me a hand in converting that blurb? I've been trying to get my head around converting the Masktools-related calls but I don't seem to understand the logic behind it. I did find Contra-Sharpen mod 16 but it seems to do many other things than the old simple contrasharpening. Using OpenCL shouldn't be a problem, at least NLMeansCL2 runs on my GPU.

[cretindesalpes]

rean: There is no definitive answer. Use ordered dithering (default setting for DitherPost and GradFun3) on the Dither side.
On the x264 side, increase --aq-strength a bit, and maybe the psy-trellis too. The exact values will obviously depend on the actual content and other compression settings.

[kuchikirukia]

Ok, how do I deband for 10 bit encoding? I shouldn't need dither since that's just a band-aid for 8 bit, right?
I've got a source that needs debanding, I'm resizing from 1080p to 720p (since I notice Dither can resize I'm throwing that in), and I'm encoding x264 10 bit through MeGUI. Suggestions?

[cretindesalpes]

kuchikirukia: On your video converted to 16 bits, use gradfun3() with lsb = true and lsb_in = true. Start with a low thr, like 0.2, and increase it by 0.1 steps until the banding is gone. Then send the result to your encoding application via Dither_out().

[kuchikirukia]

So I need avs2yuv?

Also, I'm antialiasing with santiag which can downscale as a part of its output. Dither can also resize. Which should I use? And how do I combine the 8 bit antialiasing with 16 bit debanding?

And, finally, does debanding improve compressibility? Since I'm 3/4ths done with this encode, I probably won't bother starting over if it doesn't save space. If it does, that's certainly an incentive add a high bit depth workflow to my repertoire.

[cretindesalpes]

kuchikirukia:

1. Yes, avs2yuv, avs2pipemod, or any modified x264 build that can take 16-bit data natively. You can also convert to 8 bits with DitherPost(mode=6) and let x264 convert it back to 10 bits before encoding.

2.a If resizing in 16 bits does a significant improvement for you, use Santiag to AA then call Dither_resize16 to resize. Otherwise, just go with Santiag. A bit better solution would be writing a small wrapping function around Dither_resize16 taking the same arguments (resize + crop) as the regular Avisynth resizers, and pass it to Santiag as scaler_post argument. However it will have to take 8-bit data as input and output.
But there is a third way: use an edge mask to select between Santiag and Dither_resize16 outputs.

2.b For mixed processing, please refer to the Dither doc: “Combining 8- and 16-bit processing.”

3. Sometimes, but by a tiny amount. It depends on your source.

[kuchikirukia]

If I just put the 8 bit first I shouldn't need anything special to combine them, right?

8 bit antialiasing/resize --> <convert to 16 bit> --> deband --> output.

Also, it was said earlier that x264's 16->10bit conversion was primitive. How do I use Dither to convert and what do I set for input? The wiki seems to show two different ways and I can't figure out where it's 10 bit output and where it's actually 16 bit.

a = Dither_get_lsb ()
b = Dither_get_msb ()
c1 = a.mt_lut ("x 6 >>", y=3, u=3, v=3)
c2 = a.mt_lut ("x 2 << 255 &u", y=3, u=3, v=3)
DitherPost (c1, c2) # Add the mode you want and other parameters here
mt_lut ("x 6 <<", y=3, u=3, v=3)
StackVertical (b, last)
Dither_out ()


Dither_quantize (
clip src,
int bitdepth,
bool reducerange (false),
int mode,
float ampo,
float ampn,
int pat,
bool dyn,
bool prot,
clip mask,
float thr,
bool interlaced,
int y,
int u,
int v,
bool staticnoise
)
Dither a clip to a given bitdepth. It’s possible to keep the resulting clip at the initial bitdepth of 16 or to reduce it to the specified depth.
To encode a clip in 10 bits using x264, set bitdepth=10 and reducerange=true, followed by Dither_out and set --input-depth 10 in the x264 parameters.

For a set of tools supposedly for high bit depth use, that wiki sure is light on how to actually use it.

[foxyshadis]

Also, it was said earlier that x264's 16->10bit conversion was primitive. How do I use Dither to convert and what do I set for input? The wiki seems to show two different ways and I can't figure out where it's 10 bit output and where it's actually 16 bit.

Don't know where you read that, but it isn't true. It uses Sierra-2-4A to dither both up and down. Also, in x264 10bit everything is internally 16-bit (except for decoded refs, which have to equal the bit-depth), so dithering down to 10 just so it can scale back up to 16 is crazy. x265 also uses Sierra-2-4A, but it internally keeps all frames at exactly the bit-depth of the encoding.

[cretindesalpes]

If I just put the 8 bit first I shouldn't need anything special to combine them, right?
Right. But you were previously mentioning using Dither’s 16-bit resizer.

Also, it was said earlier that x264's 16->10bit conversion was primitive.
Not really. There was a bug in the early 10-bit x264 versions but it has been fixed for a long time.

How do I use Dither to convert and what do I set for input? The wiki seems to show two different ways and I can't figure out where it's 10 bit output and where it's actually 16 bit.

a = Dither_get_lsb ()
b = Dither_get_msb ()
c1 = a.mt_lut ("x 6 >>", y=3, u=3, v=3)
c2 = a.mt_lut ("x 2 << 255 &u", y=3, u=3, v=3)
DitherPost (c1, c2) # Add the mode you want and other parameters here
mt_lut ("x 6 <<", y=3, u=3, v=3)
StackVertical (b, last)
Dither_out ()
This was the old method. I’ll remove it from the doc, as Dither_quantize now exists and is preferred. Thanks for having reported this inconsistency.

In short, use Dither_quantize(bitdepth=10, reducerange=true).Dither_out() if you really want to send 10-bit data to x264. Just use Dither_out() to send 16-bit data (preferred).

For a set of tools supposedly for high bit depth use, that wiki sure is light on how to actually use it.
The wiki looks outdated. Not surprising, as trying to maintain an exact copy of the official doc in a different format must be extremely boring (I’m not responsible for this).

[kuchikirukia]

Ooh, that's nice.

Thank you!

[Reel.Deel]

The wiki looks outdated. Not surprising, as trying to maintain an exact copy of the official doc in a different format must be extremely boring (I’m not responsible for this).

Thanks for pointing that out. I'm not responsible for that either but I'll try to update it to coincide with the current documentation. The dither wiki page is fairly new (created in March 2015) so it should be current but it seems that v1.23.0 was used as reference :(.


Regarding the official dither documentation there's some dead/outdated links in the troubleshooting section.

You surely have an old Masktools 2 version installed somewhere and autoloaded by Avisynth. Search on your disks the filenames mt_masktools.dll, mt_masktools-25.dll and mt_masktools-26.dll. Remove the culprits and install the latest Masktools 2 version from there.

The link to manao's homepage is dead. It's probably best to link to the updated MaskTools b2 (https://github.com/tp7/masktools/releases).

Also this:
Your RemoveGrain version is too old. Remove RemoveGrain*.dll, Repair*.dll and SSE?Tools.dll and install the version 1.0b "beta" from there, but do not copy the SSE3 files. This is not the latest version, but the most compatible with all the existing scripts around there.

The correct RemoveGrain link: http://avisynth.nl/index.php/RemoveGrain
It also might be best to recommend users to use RgTools (https://github.com/tp7/RgTools/releases). No more doubts on which binary to use :).

[kuchikirukia]

While you're at it, there's a syntax error:
mask_details= details.mt_binarize (lvl_details)>

And if you want to edit the santiag page, there's a mistake there as well:
http://avisynth.nl/index.php/Santiag

scaler_pos

pos doesn't work. It's "post" as in the example:
scaler_post="Spline36Resize")

[Reel.Deel]

@kuchikirukia

I am responsible for the santiag page :p. Fixed; thanks for pointing that out.

[kuchikirukia]

All I can say is thank God for great error reporting. I would've given up on avisynth in a day if MeGUI didn't tell me exactly where to look.

[cretindesalpes]

Reel.Deel: thanks. I updated the documentation for the upcoming 1.27 release.

[Desbreko]

Also, I'm antialiasing with santiag which can downscale as a part of its output. Dither can also resize. Which should I use? And how do I combine the 8 bit antialiasing with 16 bit debanding?

If you don't need santiag specifically, you could use xaa (http://www.mediafire.com/download/sygi04y47eknvc2/xaa_v1.1.1.avsi) for this. When outputting 16-bit, it will retain the lsb from working in 16-bit whenever possible, including combining the 8-bit antialiased clip with the 16-bit resized clip with an edge mask.

So, you could do something like this:

# 8-bit 1080p source
xaa(ow=1280, oh=720, mode="nnedi3", lsb=true)
GradFun3(thr=0.2, lsb=true, lsb_in=true)
Dither_out()

[kuchikirukia]

Tried it and it was slower than santiag for no apparent gain. I only noticed one difference in one scene and it was in santiag's favor.

[Desbreko]

How fast xaa runs is highly dependent on what settings you use, so it might be worth tinkering to see if you can get a better result than my example. Or santiag may just be better suited to your source since it works somewhat differently from xaa.

[Dogway]

for confirmation, does this apply as "Random Dither"?:
DitherPost(mode=-1, ampn=1)

[sneaker_ger]

I don't understand. Why does ampn do anything when we are only rounding?

/edit:
It's just adding noise? Then I don't think it's equal to dithering. Noise is just noise, it does not preserve gradients.

[cretindesalpes]

Dogway: yes.

sneaker_ger: We are just rounding, but after the noise addition. Actually rounding is implemented as dithering using a pattern full of zeroes.

Dithering is related to noise. It breaks correlated quantization noise and turns it into uncorrelated noise, perceptually much less annoying. Adding noise before quantization is a valid way to do it but has a poor global SNR (depends on the noise amount).

[Dogway]

Do you think it can work as an intermediate for further filtering despite the poor SNR? IMO rounding creates some ugly banding and F-S which I was using before is meant to be an "end look" due to its pattern based condition from what I read here (http://forum.doom9.org/showthread.php?p=1727107#post1727107).

I'm kinda switching the old DitherBuildMask method for hacky lsb_in with sub16/add16, which hopefully makes more sense, even more with the above random dither intermediate.

[cretindesalpes]

You’d better use error-diffusion dithering like Floyd-Steinberg rather than noise + quantization. The result will always be much better, no matter what the further processing steps will be. The only drawback is that it will be a bit slower.

[Dogway]

edit:nevermind, just read your post here (http://forum.doom9.org/showthread.php?p=1728059#post1728059).

[feisty2]

or better, do not dither at all, add native high bit depth support to filters lack of that

[cretindesalpes]

Dither 1.27.0 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Added cplaces, cplaced and csp to Dither_resize16.
Backported some code from fmtconv to Dither_resize16.
Added the slice parameter to disable multi-threaded slicing for error-diffusion dithering (DitherPost and related functions).
Added Dither_min_dif16.
AVX2 optimisations for Dither_resize16, Dither_add16, Dither_sub16, Dither_median16, Dither_max_dif16, Dither_min_dif16 and Dither_limit_dif16.
Added the BT.1886 transfer cruve to the gamma conversion functions.
Added the dispm parameter to change the conversion mode in Dither_srgb_display.
Improved the quality of the subsampling in Dither_bilateral16.
Radius is no longer limited in Dither_bilateral16 as well as in GradFun3 in smode 2.
Fixed a bug in the chroma filling (u or v < 0) with non-YV12 colorspaces.

[Sparktank]

Thanks a lot for the continual work!
Nice update.

Looking forward to playing with it.

[Bloax]

Clearly the future is motion-based dither that is easily predicted by lossy encoders.

[feisty2]

Clearly the future is motion-based dither that is easily predicted by lossy encoders.

Clearly the future is the death of low bit depth

[real.finder]

after update to 1.27.0

I can't use nnedi3_resize16 in upscale

in avspmod will show this

Traceback (most recent call last):
File "avsp.pyo", line 9061, in OnMenuVideoToggle
File "avsp.pyo", line 13855, in ShowVideoFrame
File "avisynth.pyo", line 462, in GetFrame
WindowsError: [Error -1073741795] Windows Error 0xC000001D

[StainlessS]

Looks like 0xC000001D might be Illegal Instruction (there seem to be other things attributed if you google it eg bad device driver).
Provide cretindesalpes with details of OS and CPU, perhaps some new assembler instructions added.
EDIT: Would not hurt to mention Avisynth version too.

[real.finder]

Looks like 0xC000001D might be Illegal Instruction (there seem to be other things attributed if you google it eg bad device driver).
Provide cretindesalpes with details of OS and CPU, perhaps some new assembler instructions added.
EDIT: Would not hurt to mention Avisynth version too.

cpu intel(r) core(tm) i7 cpu x 980 @ 3.33ghz

os windows server 2008 r2

avs http://forum.doom9.org/showthread.php?t=148782

2015.02.20 or 2013.03.09

[Groucho2004]

after update to 1.27.0

I can't use nnedi3_resize16 in upscale

in avspmod will show this

Traceback (most recent call last):
File "avsp.pyo", line 9061, in OnMenuVideoToggle
File "avsp.pyo", line 13855, in ShowVideoFrame
File "avisynth.pyo", line 462, in GetFrame
WindowsError: [Error -1073741795] Windows Error 0xC000001D
I can confirm the same crash. It doesn't seem to matter which Avisynth version is used.
I also tried to insert a another simple 8 Bit filter (sharpen) using the recommended methods, crashes with the same error.

[cretindesalpes]

Fixed in dither 1.27.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed a wrong code path in Dither_limit_dif16 causing a crash on CPU without AVX2 instruction set. Thanks to real.finder and Groucho2004 for their report.
Documentation typos fixed thanks to Colours.

[LouieChuckyMerry]

Hello and thanks in advance for any help :) . Please, I'm wondering if the following script has the proper order and syntax:

Blu-raySourceVideo
SMDegrain(Lsb=True,Lsb_Out=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
# DitherPost()
Dither_Out()

for encoding with 10-bit x264 (adding "--demuxer raw --input-depth 16 --sar 1:1" to the x264 command line). That is, is it correct to go: denoise, sharpen, deband? And are the Dither lines for FastLineSharpenMod then onto GradFun3 the best method? Thanks in advance for any help.

[cretindesalpes]

Indeed, this looks correct. The filter order doesn’t matter much regarding the 8/16-bit pipeline. It’s more important to focus on the actual processing. Moreover, strengths and thresholds depends heavily on the filter order. Of course it’s easier to insert the 8-bit filters before the 16-bit ones. Debanding generally comes at the end because it may have to repair damages done by previous filters.

[LouieChuckyMerry]

Thanks for your answer, cretindesalpes, I appreciate it. I've been slowly learning a bit here and a bit there and wanted to make sure I was putting the pieces together correctly. Another question, if I may. I updated Dither a few days ago, from v1.25.1 to v1.27.1 (I'm a bit slow) and in the process unwittingly added the avstp.dll to my plugins folder. I typically use the basic above script with SEt's MT AviSynth 2.6 like so:

SetMemoryMax(2048)
SetMTMode(3,7)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("D:\Temp\[0000]ReEncTemp\TheUnbearableLightnessOfBeing[S]{1988}[720p]\OriginalVideo,OriginalAudio[ViGi].mkv")
SetMTMode(2)
SMDegrain(tr=3,thSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True,PreFilter=2)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Lsb_In=True,Lsb=True)
# DitherPost
Dither_Out()

This has always given me good results (except for the occasional corrupt source), but with the avstp.dll in my plugins folder the output video had nasty artifacts every few seconds. Once I figured out that the avstp.dll was the cause of the glitches I found my way to your thread (http://forum.doom9.org/showpost.php?p=1564516&postcount=1) and, after reading the "Short user manuel" I don't understand why the avstp.dll would cause so much trouble. To my obviously wrong understanding it should help. Or is the problem that SMDegrain and FastLineDarkenMod aren't AVSTP enabled?

[Groucho2004]

To my obviously wrong understanding it should help. Or is the problem that SMDegrain and FastLineDarkenMod aren't AVSTP enabled?
avstp certainly helps when you don't use AVS MT. Before trying to figure out what the problem may be, what speeds do you get when you don't use MT? Make sure that you use mvtools2.dll from this post (http://forum.doom9.org/showthread.php?p=1386559#post1386559). Most of the plugins in your script (mvtools, dither) can actually run multi-threaded through avstp.dll, so that should be worth a try. Multi-threading Mvtools and Dither through avstp is very efficient, CPU usage/speed scale almost linear.

Edit: I did a couple of tests with your script, SMDegrain is quite the CPU hog. You may be better off using AVS MT and disabling avstp, for example using Boulder's method below.


My guess regarding the artefacts you're seeing - AVS MT sometimes doesn't play well with internally multi-threaded plugins. In case of dither.dll (and mvtools 2.6.0.5), the presence of avstp.dll enables the internal multi-threading.

[Boulder]

You can use the line "avstp_set_threads(x)" (without the quotes) to control the amount of internal threads being used with filters that utilize avstp.dll. I have "avstp_set_threads(1)" in my plugins folder as avstp.avsi so that is always the default, and then I can change it to something else in my script if needed. Whenever I switch off MT, I set the avstp threads higher to compensate. With QTGMC that has given a nice boost without the stability issues.

[cretindesalpes]

Check the result with avstp multithreading but without avisynth MT. MT is not reliable when scripts are complex and heavy. Try to increase the memory available for avisynth too.
EDIT: Do you know which plug-in or script part causes the glitches?

[Groucho2004]

Fixed in dither 1.27.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed a wrong code path in Dither_limit_dif16 causing a crash on CPU without AVX2 instruction set. Thanks to real.finder and Groucho2004 for their report.
Documentation typos fixed thanks to Colours.

Still crashing sometimes with mpc-be(hc) when closing the player (AltF4). Error 0xC0000005 (access violation).

[cretindesalpes]

Groucho2004: avs script + input video format?

[Groucho2004]

Groucho2004: avs script + input video format?
Input is 1080p AVC clip cut from BD. This happens with every source format/filter, by the way.

The only way I can get it to crash is MPC (HC or BE). With VDub, AVSMeter, AVSPMod it does not crash whatever I try.


For the 8 bit filter I used "sharpen()" for simplicity. I usually would use LSF().

LoadPlugin("E:\Apps\VideoTools\DGDecNV\DGDecodeNV.dll")
DGSource("F:\Aliens1\aliens1.dgi")
w = 1280
h = round((last.height() / 1.5) / 2.0) * 2
Dither_convert_8_to_16()
S16 = last
DitherPost(mode = -1)
sharpen(0.5)
Dither_convert_8_to_16()
s16.Dither_limit_dif16(last, thr = 1.0, elast = 2.0)
Dither_resize16(w, h, kernel = "Spline64")
DitherPost()

[LouieChuckyMerry]

avstp certainly helps when you don't use AVS MT. Before trying to figure out what the problem may be, what speeds do you get when you don't use MT? Make sure that you use mvtools2.dll from this post (http://forum.doom9.org/showthread.php?p=1386559#post1386559). Most of the plugins in your script (mvtools, dither) can actually run multi-threaded through avstp.dll, so that should be worth a try. Multi-threading Mvtools and Dither through avstp is very efficient, CPU usage/speed scale almost linear.

Edit: I did a couple of tests with your script, SMDegrain is quite the CPU hog. You may be better off using AVS MT and disabling avstp, for example using Boulder's method below.

My guess regarding the artefacts you're seeing - AVS MT sometimes doesn't play well with internally multi-threaded plugins. In case of dither.dll (and mvtools 2.6.0.5), the presence of avstp.dll enables the internal multi-threading.

I'll try to run some with-without MT tests this weekend, and I'm using the version of mvtools that you linked, thanks. And given that the artifacts occur only when the avstp.dll is in my plugins folder I'd reckon you're right about the not playing well.


You can use the line "avstp_set_threads(x)" (without the quotes) to control the amount of internal threads being used with filters that utilize avstp.dll. I have "avstp_set_threads(1)" in my plugins folder as avstp.avsi so that is always the default, and then I can change it to something else in my script if needed. Whenever I switch off MT, I set the avstp threads higher to compensate. With QTGMC that has given a nice boost without the stability issues.

Thanks for that, I'll check it out this weekend. Is it literally:

avstp_set_threads(1)

as an .avsi file? Also, if I used "avstp_set_threads(0)" would that "deactivate" it?


Check the result with avstp multithreading but without avisynth MT. MT is not reliable when scripts are complex and heavy. Try to increase the memory available for avisynth too.
EDIT: Do you know which plug-in or script part causes the glitches?

That's the thing I don't understand. Without the avstp.dll in my plugins folder the above script (Post #905) reliably outputs glitch-free video (dozens of movies, and I've watched most of them) using SEt's MT AviSynth 2.6, provided the source isn't somehow corrupt. It's only when I decided to ignorantly place the avstp.dll in my plugins folder a couple days ago, when updating SMDegrain and Dither, that I discovered this issue. It took me a bit of testing to figure it out, but it's definitely the avstp.dll that's causing the glitches, and that's obvious, major artifacts every 2-3 seconds.

This weekend I'll run some tests with the avstp.dll and without MT AviSynth and see what happens. Thanks for your help.

[cretindesalpes]

LouieChuckyMerry: this is my experience using MT. I had stability issues on complex scripts (before writing avstp).
And when there is a problem with avstp, it generally does not comes from avstp itself, but from the plug-in using avstp for multi-threading its algorithm. So knowing which plug-in creates the artefacts in conjunction with avstp is important for debugging. Dogway, who wrote the improved SMDegrain version, reported me strange artefacts using avstp (most likely in the MVTools) but unfortunately I never could reproduce them.

Groucho2004: Thanks. I tried your script but I couldn’t reproduce the crash (using MPC-HC from CCCP 2015-02-22 Beta with MadVR 0.88.14). Is it avs MT or a standard version?

[Groucho2004]

Groucho2004: Thanks. I tried your script but I couldn’t reproduce the crash (using MPC-HC from CCCP 2015-02-22 Beta with MadVR 0.88.14). Is it avs MT or a standard version?
I'm using the release MPC-HC v1.7.9. With AVS 2.6, and 2.6 MT, opening and closing (Alt-F4) the script several times will produce the crash.
With AVS+ r1576, things are slightly different. No crash, no error message, but mpc-hc stays in memory after closing.

I just tried with a simpler script:
w = 1280
h = round((last.height() / 1.5) / 2.0) * 2
Dither_convert_8_to_16()
Dither_resize16(w, h, kernel = "Spline64")
DitherPost()

Still crashes but possibly less often.

The best way to test this seems to be with AVS+ r1576 since it causes mpchc to crash (i.e stay in memory) more reliably.
As I mentioned, all this does not happen with the previous Dither tools.

In case you need this info, the CPU is a Sandy Bridge i5 2500K, OS is XPSP3.

[Boulder]

Thanks for that, I'll check it out this weekend. Is it literally:

avstp_set_threads(1)

as an .avsi file? Also, if I used "avstp_set_threads(0)" would that "deactivate" it?Yes, it's just that inside the .avsi file. Setting threads to 1 kind of disables internal multithreading, at least I've not had any glitches with that setting and I've used it for quite some time now.

Unfortunately MVTools is quite unstable, I'd say 95% of my crashes have pointed to mvtools2.dll. And Avisynth MT is a yet another story..

[Dogway]

I thought the avstp issues were resolved (see last post (https://web.archive.org/web/20130521103336/http://doom10.org/index.php?topic=2178.msg10388#msg10388))

I have been using it for a while now, latest SEt's MT on Win7 with avstp and no issues... I recommend you to try to nail it down. Take all the plugins out and test with a very simple script using avstp and mvtools2, something along those lines. Try also to use avisource, instead of LSMASHSource just in case, and encode to lossless instead of x264 (trying different things while still using avstp).

[LouieChuckyMerry]

LouieChuckyMerry: this is my experience using MT. I had stability issues on complex scripts (before writing avstp).
And when there is a problem with avstp, it generally does not comes from avstp itself, but from the plug-in using avstp for multi-threading its algorithm. So knowing which plug-in creates the artefacts in conjunction with avstp is important for debugging. Dogway, who wrote the improved SMDegrain version, reported me strange artefacts using avstp (most likely in the MVTools) but unfortunately I never could reproduce them.

I guess it's good that you couldn't reproduce them, ha ha. If I use the MinimumPlugins+AVSTP&SEtMTAviSynt (http://www.mediafire.com/download/ynuj3srin9xo9ft/MinimumPlugins+AVSTP&SEtMTAviSynth.7z) to encode this TestClip (http://www.mediafire.com/download/mz6r8ema5ck12zd/TestClipShawshank2.7z) (which is just one example of many) using the following script with either FFMS2 or L-SMASH Works:

SetMemoryMax(2048)
SetMTMode(3,7)
"SourcePath"
SetMTMode(2)
SMDegrain(TR=3,ThSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
### Add "SMode=1" to GradFun3, Someday ###
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
### Preview Source OR Send 16-bit Output To x264 10-bit ###
# Trim()
# DitherPost()
Dither_Out()

with my i7 3820QM, 12 GB RAM rig, the output video has artifacts (more motion means more artifacts) every time. However, if I remove the AVSTP.dll then the output video has no artifacts.

I know this is probably a silly question (and most likely beyond my understanding), but what how do you use AVSTP to output video without artifacts? As Groucho2004 and Boulder suggested above?


Boulder: thanks for the explanation. Now to bed...

[cretindesalpes]

Dogway: Huhuhu my memory must be failing, I remember that we had this issue unresolved for a very long time but I forgot the end of the story…

LouieChuckyMerry: Thanks, I’ll try to reproduce it with your material.

[Dogway]

@cretindesalpes: I haven't seen this for a while but I got to reproduce the issue here with LouieChuckyMerry's source, still trying to figure out the culprit. Looks like the more complex the script is most likely are the artifacts to arise.

[Groucho2004]

with my i7 3820QM, 12 GB RAM rig, the output video has artifacts (more motion means more artifacts) every time. However, if I remove the AVSTP.dll then the output video has no artifacts.

I know this is probably a silly question (and most likely beyond my understanding), but what how do you use AVSTP to output video without artifacts? As Groucho2004 and Boulder suggested above?
With your set of DLLs and script, I can also reproduce the artefacts.
Solutions (tested):

Don't use avstp if you're using "SetMTMode(()" and vice versa. Besides, there is no advantage in using both anyway.
Don't use SMDegrain.


I guess (2) is not an option so just stick with (1).

[Dogway]

Groucho2004: Saying SMDegrain is a hog, is mostly saying mvtools2 is a hog... I don't think it's wise to tell people to not use one of the best temporal denoisers out there. In SMDegrain most of the code is for YUY2 and LSB support, as well as chroma switches to speed up the script, because of this chroma optimization code I have discovered some mvtools2 bugs here (http://forum.doom9.org/showthread.php?p=1551305#post1551305), as is possibly the case here too.

As you will see below there's not much going on inside the script.
SetMemoryMax(1024)
SetMTMode(5,6)
ffvideosource("TestClipShawshank2.mkv")
SetMTMode(2,6)

#SMDegrain(tr=3,thSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True,PreFilter=2)
#the same as above
super_search = MinBlur(2,1).Dither_Luma_Rebuild(S0=1.0,c=0.0625,lsb=true,uv=1).MSuper(rfilter=4,pel=1,hpad=16,vpad=16,chroma=false)

bv3 = super_search.MAnalyse(isb = true, delta = 3, overlap=8, blksize=16, truemotion=false,chroma=false)
bv3 = super_search.MRecalculate( bv3, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
bv2 = super_search.MAnalyse(isb = true, delta = 2, overlap=8, blksize=16, truemotion=false,chroma=false)
bv2 = super_search.MRecalculate( bv2, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
bv1 = super_search.MAnalyse(isb = true, delta = 1, overlap=8, blksize=16, truemotion=false,chroma=false)
bv1 = super_search.MRecalculate( bv1, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
fv1 = super_search.MAnalyse(isb = false, delta = 1, overlap=8, blksize=16, truemotion=false,chroma=false)
fv1 = super_search.MRecalculate( fv1, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
fv2 = super_search.MAnalyse(isb = false, delta = 2, overlap=8, blksize=16, truemotion=false,chroma=false)
fv2 = super_search.MRecalculate( fv2, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
fv3 = super_search.MAnalyse(isb = false, delta = 3, overlap=8, blksize=16, truemotion=false,chroma=false)
fv3 = super_search.MRecalculate( fv3, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)

MDegrain3(MSuper(levels=1,pel=1,hpad=16,vpad=16,chroma=false), bv1, fv1, bv2, fv2, bv3, fv3, plane=0, thSAD=400, lsb=true)

F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)

GradFun3(Radius=16,Lsb_In=True,Lsb=True)

DitherPost(mode=6)


I normally process chroma so maybe there it goes why I never encountered this issue.

In this case (not processing chroma), if I turn Dither_Luma_Rebuild() "uv" parameter to 3 (process chroma) I don't get artifacts. If after MDegrain3() you just add the DitherPost() line you have some chances of not getting artifacts but in a real scenario you normally do some stuff after lsb_out... looks like some chroma-memory related issue in mvtools2.

[Groucho2004]

Groucho2004: Saying SMDegrain is a hog, is mostly saying mvtools2 is a hog... I don't think it's wise to tell people to not use one of the best temporal denoisers out there.
1. Your logic is flawed.
SMDegrain is a hog because it uses mvtools excessively. I can put 1000 PointResize() one after another and it will be very slow even though a single Pointresize() is very fast. If you use mvtools for a simple mdegrain with a delta of 1, it will also be fast.

2. At no point did I tell people to not use SMDegrain. I simply stated that it is slow. If you have trouble understanding my last post, let me put it in other words:
Either use SetMTMode() or avstp. If you use both, SMDegrain will cause artefacts.

[LouieChuckyMerry]

cretindesalpes: sorry for the trouble, I wish that I was capable of actually helping.

With your set of DLLs and script, I can also reproduce the artefacts.
Solutions (tested):

Don't use avstp if you're using "SetMTMode(()" and vice versa. Besides, there is no advantage in using both anyway.
Don't use SMDegrain.


I guess (2) is not an option so just stick with (1).

Thanks for the help, and you're right: two isn't an option (don't use SMDegrain? Never!), so I'll remove the AVSTP.dll from my plugins folder for now, then when able I'll try your and Boulder's earlier suggestions and try to learn more.

[feisty2]

switch to vaporsynth
solution once for all
:)

[foxyshadis]

If your GPU is reasonably powerful, you can swap SMDegrain with KNLMeansCL. SMDegrain (especially with dfttest) is great but it is slooooowwww, and KNLMeansCL is nearly as good.

It supports 16bit (and float in VS) natively (except not 16bit RGB64 in Avisynth, only YUV). Give it a shot, maybe it'll relieve the bottleneck.

[Groucho2004]

If your GPU is reasonably powerful, you can swap SMDegrain with KNLMeansCL. SMDegrain (especially with dfttest) is great but it is slooooowwww, and KNLMeansCL is nearly as good.
Indeed, this has been suggested before in a similar scenario (http://forum.doom9.org/showthread.php?p=1728514#post1728514).

[cretindesalpes]

LouieChuckyMerry: I think I nailed the problem. The culprit was DitherPost. It was a frame pointer released a bit too early, before the corresponding frame buffer was fully read. OK in a single-threaded Avisynth but definitely wrong in MT. Please test this dither.dll (http://ldesoras.free.fr/src/avs/dither-test.zip) and tell me if it works better on your side.

Groucho2004, maybe it addresses your issue too.

[Dogway]

it fixed it for me, thank you.

[Groucho2004]

Groucho2004, maybe it addresses your issue too.
No, still the same.

[LouieChuckyMerry]

LouieChuckyMerry: I think I nailed the problem. The culprit was DitherPost. It was a frame pointer released a bit too early, before the corresponding frame buffer was fully read. OK in a single-threaded Avisynth but definitely wrong in MT. Please test this dither.dll (http://ldesoras.free.fr/src/avs/dither-test.zip) and tell me if it works better on your side.

Thank you for that :) . I've presently a several-day encode running, but I'll do some testing as soon as it's finished (should be Saturday noonish) then post the results.

[LouieChuckyMerry]

Happy Saturday! I ran multiple tests with the new Dither.dll and the AVSTP.dll and there were no artifacts, only shiny shiny video. Thank you very much, cretindesalpes.

[real.finder]

Happy Saturday! I ran multiple tests with the new Dither.dll and the AVSTP.dll and there were no artifacts, only shiny shiny video. Thank you very much, cretindesalpes.

so, in this case there is no need for "the slice parameter to disable multi-threaded slicing for error-diffusion dithering (DitherPost and related functions)."

"Set it to false to disable plane slicing. Slicing is used in multithreading and can create artefacts at slice boundaries with error-diffusion algorithms. Disabling slicing reduces the threading preformances."

?

[Groucho2004]

Happy Saturday! I ran multiple tests with the new Dither.dll and the AVSTP.dll and there were no artifacts, only shiny shiny video. Thank you very much, cretindesalpes.
So, you're not using AVS MT any more?

:Edit
Just in case you're still using both, SetMTMode() and avstp:

I used your script (more or less) from post #917 and ran 2 tests with and without avstp:

Script:
SetMemoryMax(1024)
SetMTMode(5)
LoadPlugin("E:\Apps\VideoTools\DGDec\DGDecode.dll")
MPEG2Source("F:\Test\test.d2v", idct = 4)
SetMTMode(2)
SMDegrain(TR=3,ThSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
DitherPost()

Result with avstp.dll present:
[Runtime info]
Frames processed: 2000 (0 - 1999)
FPS (min | max | average): 4.767 | 238636 | 22.04
Memory usage (phys | virt): 1076 | 1172 MB
Thread count: 12
CPU usage (average): 98%
Time (elapsed): 00:01:30.737


Result with avstp.dll removed:
[Runtime info]
Frames processed: 2000 (0 - 1999)
FPS (min | max | average): 4.947 | 238636 | 25.27
Memory usage (phys | virt): 1063 | 1171 MB
Thread count: 9
CPU usage (average): 95%
Time (elapsed): 00:01:19.130


These results make of course sense since using just SetMTMode() removes the complexity of two multi-threading "layers".

[cretindesalpes]

Thanks everyone for your reports. I’ll release a new official version later.

real.finder: these are not the same kind of artefacts. They’re almost invisible, unless you dramatically amplify the contrast to see the dithering patterns on large, flat areas (here (http://screenshotcomparison.com/comparison/136301), amplified 128x). They are caused by the error not diffusing between simultaneously processed areas.

[real.finder]

Thanks everyone for your reports. I’ll release a new official version later.

real.finder: these are not the same kind of artefacts. They’re almost invisible, unless you dramatically amplify the contrast to see the dithering patterns on large, flat areas (here (http://screenshotcomparison.com/comparison/136301), amplified 128x). They are caused by the error not diffusing between simultaneously processed areas.

I see, how about make slice=false by default in DitherPost mode 6 and up?

[cretindesalpes]

It disables avstp multithreading within the planes so it’s much less efficient in general. The visual quality gain is insignificant and mostly restricted to corner cases. Another solution is to add a very subtle noise, something like ampn=0.02, which keeps you away from most of the artefacts related to error diffusion algorithms.

[LouieChuckyMerry]

so, in this case there is no need for "the slice parameter to disable multi-threaded slicing for error-diffusion dithering (DitherPost and related functions)."

"Set it to false to disable plane slicing. Slicing is used in multithreading and can create artefacts at slice boundaries with error-diffusion algorithms. Disabling slicing reduces the threading preformances."

?

I'm really sorry, but I don't understand your question :o . I was referring to my issue from Post #917, that with that script, the AVSTP.dll, and the new Dither.dll there are no longer artifacts. Sorry for the confusion.


So, you're not using AVS MT any more?

:Edit
Just in case you're still using both, SetMTMode() and avstp:

I used your script (more or less) from post #917 and ran 2 tests with and without avstp:

See Above

These results make of course sense since using just SetMTMode() removes the complexity of two multi-threading "layers".

Thanks for the tests, the results are good to know. I'm still using SEt's MT AviSynth 2.6, without the AVSTP.dll in the Plugins folder, since my typical usage is the script from Post #917 and I never had the AVSTP.dll in my Plugins folder until I put it there, well, just because when updating Dither some days ago. I want to try the things you and Boulder suggested a few posts ago but there's not enough time in a day; every time I learn something new about AviSynth that leads to two more things, which leads to four more things... AviSynth³ ;) .

[Motenai Yoda]

@cretindesalpes

IIRC converting from YUV to RGB with Dither_convert_yuv_to_rgb() will clip into sRGB/bt.709 colorspace, there is a way to get an arbitrary colorspace (ie using bt.601 to get a bt.601's RGB, the same for bt.2020), or better in CIE 1976 (L*, u*, v*)?


ps how much Dither_min_dif16() is faster than ?


quant= int(pow(((2*radius)+1),2) - 1)

dark = Dither_median16(rx=radius, ry=radius, rt=0,ql=0,qh=0, y=3, u=1, v=1)
bright = Dither_median16(rx=radius, ry=radius, rt=0,ql=quant,qh=quant, y=3, u=1, v=1)

darkdiff = Dither_sub16(dark, sharp, wrap=False, y=3, u=1, v=1)
brightdiff = Dither_sub16(sharp, bright, wrap=False, y=3, u=1, v=1)

darkclamp = Dither_add16(sharp, darkdiff, wrap=False, y=3, u=1, v=1)
brightclamp = Dither_sub16(darkclamp, brightdiff, wrap=False, y=3, u=1, v=1)
clamp = (radius==1) ? Dither_Repair16(sharp, last, 1, -1) : brightclamp

[cretindesalpes]

Motenai Yoda: For BT.2020 (non-constant luminance version), it’s already possible, but assuming your RGB data is already in the BT.2020 gamut. The conversion to Luv is not linear so it’s not possible without a huge modification of the conversion function.

Dither_min_dif16 is fast and should be faster than the piece of code you quoted (I haven’t check what it does so I can’t confirm both are equivalent). But the best way to know is to check yourself.

[LouieChuckyMerry]

Hello :) . I'm encoding SD animation (early seasons Simpsons) and was wondering if there might be any noticeable benefit to upping the default Radius=12 to Radius=16? I've run tests and can't see a difference on my 14" 1600x800 laptop screen, but I'm hoping to someday own the world's biggest, er, a large screen TV, so I thought I'd ask before doing 200+ episodes in case I'd see the benefit on a much bigger panel. Thanks for your experienced opinion(s).

[foxyshadis]

If you can't see it at full size (bigger than your screen), you can rely on your future TV to handle it well. The only way it could be worse is if it's the cheapest 6-bit TN screen ever, which are getting hard to find, and even then the dithering from debanding will help.

Just make sure you double-check after the first episode that the compression didn't wipe out the debanding and make it ugly again. (If you're encoding to 8-bit.) That's why debanding on playback is useful.

[LouieChuckyMerry]

If you can't see it at full size (bigger than your screen), you can rely on your future TV to handle it well. The only way it could be worse is if it's the cheapest 6-bit TN screen ever, which are getting hard to find, and even then the dithering from debanding will help.

I'm really sorry, foxyshadis, but I'm not certain I understand the above. Do you mean that, although I'm currently unable to check the output on a larger screen, any reasonable large screen TV I might possibly own in the future will most likely benefit from raising the GradFun3 radius from 12 to 16 even though the source is SD? I just want to be sure ;) .


Just make sure you double-check after the first episode that the compression didn't wipe out the debanding and make it ugly again. (If you're encoding to 8-bit.) That's why debanding on playback is useful.

I'm encoding the DVD's with 10-bit x264 after denoising, sharpening, and debanding in a 16-bit pipeline (all hail Dither!). Debanding on playback sounds like a good future project...

[Groucho2004]

GradFun3
Alternatively, you might give "flash3kyuu_deband (http://forum.doom9.org/showthread.php?t=161411)" a try. It supports 16 bit stacked format, is very fast and in my opinion works very well.

[LouieChuckyMerry]

Alternatively, you might give "flash3kyuu_deband (http://forum.doom9.org/showthread.php?t=161411)" a try. It supports 16 bit stacked format, is very fast and in my opinion works very well.

But that would entail more testing! "Can't read you. Can't read you. Can't read you..."

[Groucho2004]

But that would entail more testing!
That's part of the fun, isn't it?

[LouieChuckyMerry]

Dang, you're too fast. Seriously though, what settings would you recommend that would be comparable to GradFun3(ThR=0.55,SMode=2,StaticNoise=True,Lsb_In=True,Lsb=True)? Also, so I can start simultaneous encodes before bed, do you think I'd see an improvement on a future 50" TV upping the GradFun3 radius from 12 to 16 with an SD source?

[Groucho2004]

Dang, you're too fast. Seriously though, what settings would you recommend that would be comparable to GradFun3(ThR=0.55,SMode=2,StaticNoise=True,Lsb_In=True,Lsb=True)? Also, so I can start simultaneous encodes before bed, do you think I'd see an improvement on a future 50" TV upping the GradFun3 radius from 12 to 16 with an SD source?
Never used GradFun3, so I can't recommend settings.

[cretindesalpes]

Increasing the radius helps with very progressive gradients, when the distance between the bands is large. It helps making the gradient really smooth without flat-gradient-flat-gradient staircases (bumps in the derivative function) which can be visible sometimes.

If you don’t get this kind of artefacts, there is no need to increase the radius.

[LouieChuckyMerry]

Never used GradFun3, so I can't recommend settings.

Ahhh, then given your experience with Flash3KYUUDeband could you recommend settings for SD animation (early seasons Simpsons) that you'd consider a good starting point for testing?


Increasing the radius helps with very progressive gradients, when the distance between the bands is large. It helps making the gradient really smooth without flat-gradient-flat-gradient staircases (bumps in the derivative function) which can be visible sometimes.

If you don’t get this kind of artefacts, there is no need to increase the radius.

So the increased radius is good for things like early morning-late evening skies and such in film, but wouldn't really do much for big blocks of solid colors found in animation? Thanks for the answer :) .

[Groucho2004]

Ahhh, then given your experience with Flash3KYUUDeband could you recommend settings for SD animation (early seasons Simpsons) that you'd consider a good starting point for testing?
For 16 bit processing, I usually use something like this:
f3kdb(sample_mode = 2, keep_tv_range = true, input_mode = 1, output_mode = 1)

Pretty much the defaults, does exactly what I want. I found the clips I linked to here (http://forum.doom9.org/showthread.php?t=172321) very useful for testing de-banding filters.

[Keiyakusha]

f3kdb at defaults adds quite a bit of grain on top of debanding. Personally I can't imagine using it without setting grainY=0, grainC=0. I think defaults were tweaked for 8bit video, but with 10bit video there are no problems retaining f3kdb's deband without any grain. At least when it comes to real-world content.

[foxyshadis]

I'm really sorry, foxyshadis, but I'm not certain I understand the above. Do you mean that, although I'm currently unable to check the output on a larger screen, any reasonable large screen TV I might possibly own in the future will most likely benefit from raising the GradFun3 radius from 12 to 16 even though the source is SD? I just want to be sure ;) .

I'm encoding the DVD's with 10-bit x264 after denoising, sharpening, and debanding in a 16-bit pipeline (all hail Dither!). Debanding on playback sounds like a good future project...

I mean, view it at 100% so there's absolutely no scaling or processing being applied. Sure, you can't fit it all on the screen at once, but you can scroll -- but if you're already encoding to 10-bit after debanding then I doubt you could possibly do better. Expanding the radius further just slows encoding down for a difference you might never notice even side-by-side.

Since your TV will likely never play AVC 10-bit, you'll have to play it from PC anyway, where you can apply further debanding or light noise. If you're encoding to HEVC 10-bit, a future TV will probably be able to play it, so just double-check the encode anyway and what you see is what you'll get, unless the TV has some kind of grain-generation.

[LouieChuckyMerry]

For 16 bit processing, I usually use something like this:
f3kdb(sample_mode = 2, keep_tv_range = true, input_mode = 1, output_mode = 1)

Pretty much the defaults, does exactly what I want. I found the clips I linked to here (http://forum.doom9.org/showthread.php?t=172321) very useful for testing de-banding filters.

Finally ran some tests. I prefer the results with GradFun3 but Flash3KYUUDeband is definitely faster and doesn't look at all bad. It's, to simplify, as if GradFun3 has relatively large areas of "movement" (blocking?) and Flash3KYUUDeband has relatively small areas of "movement".


f3kdb at defaults adds quite a bit of grain on top of debanding. Personally I can't imagine using it without setting grainY=0, grainC=0. I think defaults were tweaked for 8bit video, but with 10bit video there are no problems retaining f3kdb's deband without any grain. At least when it comes to real-world content.

Thanks! Now I have to run more tests ;) . This might explain my technical observation of large vs. small areas of "movement", too.

Edit: I ran some tests and much prefer the results with "GrainY=0,GrainC=0" added to the F3KDB script; the results are quite similar to those of GradFun3 (and still a bit faster, too). Thanks for the suggestion, Keiyakusha.


I mean, view it at 100% so there's absolutely no scaling or processing being applied. Sure, you can't fit it all on the screen at once, but you can scroll -- but if you're already encoding to 10-bit after debanding then I doubt you could possibly do better. Expanding the radius further just slows encoding down for a difference you might never notice even side-by-side.

Since your TV will likely never play AVC 10-bit, you'll have to play it from PC anyway, where you can apply further debanding or light noise. If you're encoding to HEVC 10-bit, a future TV will probably be able to play it, so just double-check the encode anyway and what you see is what you'll get, unless the TV has some kind of grain-generation.

For fear of being labeled more ignorant, how would I "view it at 100%"? I understand what you typed, but I'm not sure how to "upscale" the video.

[foxyshadis]

For fear of being labeled more ignorant, how would I "view it at 100%"? I understand what you typed, but I'm not sure how to "upscale" the video.

Sorry about that; AvsPmod can do that, plus it makes life so much easier anyway. When you right-click on the preview image, there's a Bit Depth option and you need to select "Stacked yuv420p16" (or whatever format you're working in if it's something else), then you will see your true 16-bit output with no downscaling, and you can scroll around as you like.

[LouieChuckyMerry]

foxyshadis: thanks for that. I checked out AvsPMod some time ago but had trouble with it not responding on my older (4GB RAM) laptop when working with HD sources. Hopefully it'll be more friendly with an SD source.

Edit: happily it's much more responsive with an SD source, but when I right-click the preview window there's no "Bit Depth" option. It's version 2.5.1, which as far as I can tell is the latest. Granted I'm a bit slow sometimes, but searching the right-click options and menu bar options I don't see anything regarding "Bit Depth" or playback possibilities other than "YUV -> RGB".

EditEdit: hmmm, seems there's a newer version (I found it here (http://www.videohelp.com/software/AvsP/old-versions#download)), 2.5.1-r426-x86-04874ed, which has a "Bit depth" option when right-clicking on the preview window. Cool...

[Dogway]

cretindesalpes, I wanted to ask if you have any hints on to what causes the mdegrain DC artifacts. I used to get them when BTB or near it values, but recently I'm including some averaged PC levels autogain to smdegrain's prefilter and I get them on almost every flash (when swift dark-bright frames occur), this puzzles me because I don't remember getting this with similar flashes on TV levels. I know you used to tackle that, I'm fixing them with repair() but just wondering if this is a bug or by design.

[cretindesalpes]

Dogway: I’m not sure to understand your terminology… What is BTB?

If you process your search clip with dynamic levels, you’re going to encounter DC artefacts in the filtered clip. This is more or less “by design”, because MDegrain averages the content of the input frames. Averaging the content of frames of different brightness will give a frame of variable brightness, depending on the weight of each input frame. Because these weights (inverse SAD) vary from frame to frame and from block to block in a single frame, as result of MAnalyse, you’re likely getting blocking artefacts during flashes or fades. Even with subtle values, the effect is noticeable and annoying. In any case, this is not related to PC or TV levels.

If you want to counter that, you have to replace the DC (and the lowest frequencies) of the filtered clip with the one of the original clip. Generally you don’t want to change them while denoising.

[Sparktank]

What is BTB?

Blacker Than Black, IIRC.

WTW = Whiter Than White

[Boulder]

Which is the better way to go what comes to final quality:

1) Process normally in 8-bit domain (denoise etc. which I cannot translate to high bit depth stuff) --> convert to 16-bit video --> deband (GradFun3) --> output 16-bit data to 10-bit x264

2) Process normally in 8-bit domain (denoise etc. which I cannot translate to high bit depth stuff) --> convert to 16-bit video --> deband (GradFun3) --> convert back to 8-bit data (DitherPost) and output to 10-bit x264

3) Process normally in 8-bit domain (denoise etc. which I cannot translate to high bit depth stuff) --> deband (GradFun3) --> output 8-bit data to 10-bit x264

My guess is that number 1 is the way to go, but I'm not sure if x264 is capable of proper dithering back to 10 bits?

[Desbreko]

1 is what you want to do for 10-bit encoding. x264 uses high-quality dithering for its bit depth conversions.

Or if you want control over the dithering, you could get a patched build of x264 that will skip bit depth conversions when the input and output depths are the same and use Dither_quantize to dither to 10-bit in your script. I'm not sure if there are any up-to-date builds with that patch, though.

2 and 3 are the same since GradFun3 works in 16-bit and just calls DitherPost to convert its output back to 8-bit without lsb=true.

[Groucho2004]

denoise etc. which I cannot translate to high bit depth stuff
Just curious - why not?

[Boulder]

There's some stuff that my skills don't cover, such as the prefilter for the analysis clip which is Flux5framesT (probably cannot be translated to 16-bit stuff)

function Flux5framesT(clip c, int "th", int "thC", bool "chromamotion")
{
chromamotion = default(chromamotion, true)
th = default(th, 2)
thC = default(thC, chromamotion ? th : 0)
med = chromamotion ? ytouv(c.utoy8().median5t(), c.vtoy8().median5t(), c.median5t()) : c.median5t().mergechroma(c)
avg = c.temporalsoften(2, th, thC, 24, 2)
output = interleave(c, med, avg).clense(grey=!chromamotion).selectevery(3,1)
return output
}

Also MAnalyse requires 8-bit input anyway and there's the contrasharpening part after the denoising by MDegrain which makes my head spin. I've seen the 16-bit mod of contrasharpening but to me it looks like it's trying to be much too fancy compared to Didée's original, brilliant idea.

I've also tried comparing the gamma-aware resizing of Dither and ResampleHQ (I almost always downsize to 720p) and found out that the method by Dither doesn't scale well with SetMTMode. The performance sometimes fluctuates quite a lot and I haven't found a good reason for that.

[Groucho2004]

There's some stuff that my skills don't cover, such as the prefilter for the analysis clip which is Flux5framesT (probably cannot be translated to 16-bit stuff)
I see. In that case, as already suggested, denoise in 8 bit, convert to 16, deband, reduce to 10 bit, feed to x264 10 bit.

[Sapo84]

Also MAnalyse requires 8-bit input anyway and there's the contrasharpening part after the denoising by MDegrain which makes my head spin. I've seen the 16-bit mod of contrasharpening but to me it looks like it's trying to be much too fancy compared to Didée's original, brilliant idea.

MAnalyse requires 8-bit but MDegrain can output in 16-bit with lsb=true, so it shouldn't be a problem at all, unless you're really against the 16-bit mod of contrasharpening.

[feisty2]

In that case you should probably switch to vaporsynth, vaporsynth mvtools takes 16bits or even 32bits (floating point) precision and, no concerns about MT stuff no more

[Boulder]

Yeah, that's one option. I've tried to do the switch at least once but haven't had the time to properly familiarize myself with the syntax. Loading stuff is easy but there's always more than that :)

[feisty2]

I don't find vaporsynth script (Python) and avisynth script much different...
Both are simple scripting languages, not monster like C++...

[Boulder]

Once you get used to Python (and coding in general, after 20 years..) :) In fact, I've had to do some very small coding on Python at work to create some helper jobs so maybe I'll get around to it in the near future.

I have a feeling that the slowdown issue with Dither's gamma-aware resizing is cache related. I just need to find out which portion of the script causes the severe slowdown because directly compared, there is no problem with it.

[Groucho2004]

I've also tried comparing the gamma-aware resizing of Dither and ResampleHQ (I almost always downsize to 720p) and found out that the method by Dither doesn't scale well with SetMTMode. The performance sometimes fluctuates quite a lot and I haven't found a good reason for that.
Do you have avstp.dll in your plugin directory? The multi-threading of avstp and SET's AVS MT don't go well together. If you have the file, remove it from the auto-load directory.

[Boulder]

I do have it but it's set to single-thread mode with avstp_set_threads(1) in my script, that should cover it.

[Groucho2004]

I do have it but it's set to single-thread mode with avstp_set_threads(1) in my script, that should cover it.
I'd still try without it, I tested it with "avstp_set_threads(1)" and it's presence still influenced the results.

[LouieChuckyMerry]

I see. In that case, as already suggested, denoise in 8 bit, convert to 16, deband, reduce to 10 bit, feed to x264 10 bit.

Happy Friday! Do you reckon there's a noticeable difference between the above and simply feeding the 16-bit output to 10-bit x264?

[feisty2]

Happy Friday! Do you reckon there's a noticeable difference between the above and simply feeding the 16-bit output to 10-bit x264?

"reckon", aussie?
no, I guess, x264 got "Sierra-2-4A error diffusion", something similar to typical FS (ditherpost mode 6)

[Groucho2004]

"reckon", aussie?
Info on the origin of the word can be found here (http://www.oxforddictionaries.com/definition/english/reckon).
I was rather sure that this word is commonly used in AE as well. Maybe it just fell through the cracks when spelling dictionaries for phones were created. :D

[fvisagie]

commonly used in AE

I reckon quite a few Safricans also use it. So not a reliable indicator of origin.

[feisty2]

I was rather sure that this word is commonly used in AE as well.

not now... unless someone like 50 yrs old "country bumpkin" (no offense)

[LouieChuckyMerry]

"reckon", aussie?
no, I guess, x264 got "Sierra-2-4A error diffusion", something similar to typical FS (ditherpost mode 6)

I'm using "Dither_Out()" to feed 10-bit x264; would there still be no apparent difference? Apologies for my obtuseness.

Info on the origin of the word can be found here (http://www.oxforddictionaries.com/definition/english/reckon).
I was rather sure that this word is commonly used in AE as well. Maybe it just fell through the cracks when spelling dictionaries for phones were created. :D

Thanks, good to learn more ways to spread The Tao Of Reckon ;) .

I reckon quite a few Safricans also use it. So not a reliable indicator of origin.

I picked it up watching weekend westerns as an impressionable youth. Nice to know I'd feel at home in South Africa :D .

[feisty2]

I'm using "Dither_Out()" to feed 10-bit x264; would there still be no apparent difference? Apologies for my obtuseness.
you can either dither down to 10bits and x264 will take it and skip the bit depth conversion, or pipe that 16bits stuff directly to x264 and x264 will "Sierra-2-4A" it to 10bits
so, no apparent difference

[LouieChuckyMerry]

Thanks for the clarification, feisty2 :) .

[Desbreko]

Does vanilla 10-bit x264 skip the bit depth conversion now if you feed it 10-bit? Last I heard, it would convert to 16-bit first and then dither down to 10-bit.

[feisty2]

the internal bit depth will always be, 16bits, like, that's an elementary data type to computers
you got char, which is an 8bits integer, and can be used to do, stuff, to 8bits vids
and short, which is a 16bits integer, and in between, it's just, nothing, 10bits integer? never heard of it
8<bit depth<=16 just gotta be padded to 16bits first and truncated (or dithered) to the desired bit depth when all the internal processes are done

[sneaker_ger]

Does vanilla 10-bit x264 skip the bit depth conversion now if you feed it 10-bit? Last I heard, it would convert to 16-bit first and then dither down to 10-bit.
I think that has remained unchanged but does not matter anyways. At least that's how I always understood it.
x264 uses a variant on Sierra-2-4A error diffusion, a faster alternative to Floyd-Steinberg error diffusion. x264's implementation is explicitly designed such that if the input is upscaled within x264 to a higher bit depth, then downscaled using error diffusion, the result will be losslessly identical to the input.

[LouieChuckyMerry]

the internal bit depth will always be, 16bits, like, that's an elementary data type to computers
you got char, which is an 8bits integer, and can be used to do, stuff, to 8bits vids
and short, which is a 16bits integer, and in between, it's just, nothing, 10bits integer? never heard of it
8<bit depth<=16 just gotta be padded to 16bits first and truncated (or dithered) to the desired bit depth when all the internal processes are done

Interesting information to this ignorant and curious individual. Any idea why 10-bit x264 isn't instead 16-bit x264?

[feisty2]

H.264 got no profile with support to 16bits precision (High444pp supports bit depth up to 14bits)

[colours]

Does vanilla 10-bit x264 skip the bit depth conversion now if you feed it 10-bit? Last I heard, it would convert to 16-bit first and then dither down to 10-bit.

Must've been quite a few years ago when you last heard, then. I think bypassing the depth filter has been around since 2012/2013.

the internal bit depth will always be, 16bits, like, that's an elementary data type to computers
H.264 got no profile with support to 16bits precision (High444pp supports bit depth up to 14bits)

Wrong. Bit depth is not the size of the integer format used to contain data.

[feisty2]

Right, 8-15bits ain't got padded to 16bits, uint16_t is used for, it's just the smallest data type that covers the whole range of 9-16bits
My mind was floating somewhere else when I said that :D

[tormento]

>>>> dfttest 1.9.4 <<<< (http://ldesoras.free.fr/src/avs/dfttest-1.9.4.zip)
Mod16 version of dfttest.
>>>> MVTools 2.6.0.5 <<<< (http://ldesoras.free.fr/src/avs/mvtools-2.6.0.5.zip)
Mod16 version of MVTools 2
Any chance to get a x64 compile of them?

At least of modified MVTools.

Please. =P

[feisty2]

Go vaporsynthing

[tormento]

Go vaporsynthing

VS MVTools goes up to 3 frames and there is no official SMDegrain script. I need from 4 to 6 frames and dogway support ;)

[feisty2]

VS MVTools goes up to 3 frames and there is no official SMDegrain script. I need from 4 to 6 frames and dogway support ;)

http://forum.doom9.org/showthread.php?p=1736069#post1736069
see EDIT10

[tormento]

http://forum.doom9.org/showthread.php?p=1736069#post1736069
see EDIT10
LOL 24 frames. Thank you.

Seriously, I want to solve this issue on standard AviSynth or AviSynth+ before starting to learn new tricks.

[feisty2]

the source code is there, compile to x64 yourself

[tormento]

the source code is there, compile to x64 yourself

Hey Siri, compile MVtools in x64 flavour. :D

Compiling is beyond my capabilities.

[feisty2]

I don't know what you mean by "compile MVtools in x64 flavour."
How about a web search for it?

seriously... avs mvtools is real horrible (like, code monster), and no one wants to touch it....

[Reel.Deel]

Any chance to get a x64 compile of them?

At least of modified MVTools.

Please. =P

Both x64 plugins can be found here: http://avisynth.nl/index.php/AviSynth%2B#AviSynth.2B_x64_plugins

[tormento]

Both x64 plugins can be found here: http://avisynth.nl/index.php/AviSynth%2B#AviSynth.2B_x64_plugins

:thanks:

It's a pity AviSynth+ has a problem of image corruption.

[tormento]

cretindesalpes, I wanted to ask if you have any hints on to what causes the mdegrain DC artifacts. I used to get them when BTB or near it values, but recently I'm including some averaged PC levels autogain to smdegrain's prefilter and I get them on almost every flash (when swift dark-bright frames occur), this puzzles me because I don't remember getting this with similar flashes on TV levels. I know you used to tackle that, I'm fixing them with repair() but just wondering if this is a bug or by design.

I suppose it's similar to what I get with AVS+.

[Dogway]

I suppose it's similar to what I get with AVS+.
No, it's not. It's an "issue" related to mvtools2 on an unreleased version.
If I understood correctly your problem is with KNLmeansCL right?

[tormento]

No, it's not. It's an "issue" related to mvtools2 on an unreleased version.
If I understood correctly your problem is with KNLmeansCL right?
No, image corruption is related to AVS+ (32/64) and SMDegrain, prefilter 2 or 4.

[Dogway]

Those are quantile and KNLMeansCL, narrow down the culprit.

[tormento]

Those are quantile and KNLMeansCL, narrow down the culprit.

Other modes not tested, as 2 (software) and 4 (hardware) gives me similar if not identical corruption.

[LouieChuckyMerry]

tormento: I don't know if this is related, but I had corruption issues with SMDegrain that were due to having the AVSTP.dll in my plugins folder. As soon as I removed it the corruption stopped.

[tormento]

tormento: I don't know if this is related, but I had corruption issues with SMDegrain that were due to having the AVSTP.dll in my plugins folder. As soon as I removed it the corruption stopped.
AVSTP has been removed as I use MT.

Even tests with single thread were done without that library.

However: :thanks: