About DitherPost Mode

[MysteryX]

I just saw that DitherPost has a mode option. I've read you should use mode=0 when it is the final output and mode=6 when doing further processing.

Could someone tell me more about how this works?

When AviSynthShader returns 8-bit data after processing 16-bit data, it doesn't do any of that. What could be the side-effects or loss opportunity?

The upcoming version of AviSynthShader supports returning planar output which gives me the opportunity of doing quick extra processing on the output. I run this HLSL shader on the output for each channel and dithering like DitherPost is doing (either mode 0 or 6) could be added here.


sampler s0 : register(s0);
float4 main(float2 tex : TEXCOORD0) : COLOR {
return float4(tex2D(s0, tex).yyy, 1);
}


Any comments or recommendations?

[MysteryX]

I found this option in DirectX 9. Not sure what time of dithering it applies.

m_pDevice->SetRenderState(D3DRS_DITHERENABLE, TRUE)


From what I'm reading Floyd Steinberg and Ordered Dithering are the best to use (mode 6 and mode 0 of DitherTools). I also read that Floyd Steinberg couldn't be implemented in HLSL because it requires sequential analysis. I would have to find HLSL code for Ordered Dithering; unless that's what DirectX is applying by default.

[MysteryX]

This is from Madshi

One thing you should look into is whether video compression works better with error diffusion or with ordered dithering. I don't know that. It's possible that lossy encoders like x264 or x265 prefer one or the other. Maybe someone in the AviSynth doom9 section knows that? You should probably use the algo that works best for video compression.

I don't know what D3DRS_DITHERENABLE does. It will probably depend on the GPU driver, so I wouldn't recommend using it.

I think MPC-HC does ordered dithering via HLSL.


I searched through .hlsl files within MPC-HC source code (https://github.com/mpc-hc/mpc-hc/find/develop) and couldn't find anything.

[vivan]

I don't believe mpc-hc uses dithering, EVR is limited to 8 bit processing (which leads to a shitton of banding).

OpenGL has an dithering option too. But there's slight problem - they believe that rounding is a valid dithering method.
And even if DirectX 11 really forces GPU to use ordered dithering - 4x4 with static pattern is crap.

Anyway, the point is - screw drivers, implement it yourself. Ordered dithering is trivial, and madshi also made some improvements (like "opposite color"). The only hard part is figuring out what matrix to use - I used algorithm described in some paper madshi linked somewhere.

[MysteryX]

One thing you should look into is whether video compression works better with error diffusion or with ordered dithering. I don't know that. It's possible that lossy encoders like x264 or x265 prefer one or the other. Maybe someone in the AviSynth doom9 section knows that? You should probably use the algo that works best for video compression.


I think this answers the question, from DitherTools documentation

The default mode=0 will help you optimize the dithering for optimum encodings when no further non-edge processing is done. Use mode=6 (error diffusion) if further processing will be done.

[bxyhxyh]

For comparison purpose.
Try this on some dark frames.
Dither_convert_8_to_16()
dither_resize16(320,180)
ditherpost(mode=0) #and mode=6
spline36resize(1280,720)
You'll see visible "pattern" for mode=0

It's exactly as in dithertools documentation said.

If i upscale clips, I wouldn't choose mode=0 for mid processing.
For some cases you don't need to upscale to see this "pattern".
So mode=0 is no no for my general use.

[feisty2]

6, definitely.
mode 6 = FS error diffusion, classic
mode 0=ordered

mode 6 looks good if uncompressed, but it might not survive from lossy compression.
mode 0 looks, well, not good as mode 6, but it's designed specifically for lossy compression, so...

[thecoreyburton]

What about modes 7 and 8? Are you able to implement them into your script and if so are there any advantages for those over mode 6?

[MysteryX]

Mode 6 is impossible to implement via HLSL because each pixel depends on the value of other pixels.

I implemented Dither mode=0 in AviSynthShader v1.5 although it hasn't been fully tested. The only way to have Dithering mode=6 is to return the data as Stack16 with lsb_out and then use DitherPost(mode=6). Reading the memory back from the GPU is a serious bottleneck so you don't want to return LSB data unless the gain is worth it or you have further 16-bit processing to do.

[MysteryX]

For comparison purpose.
Try this on some dark frames.
Dither_convert_8_to_16()
dither_resize16(320,180)
ditherpost(mode=0) #and mode=6
spline36resize(1280,720)
You'll see visible "pattern" for mode=0

It's exactly as in dithertools documentation said.

If i upscale clips, I wouldn't choose mode=0 for mid processing.
For some cases you don't need to upscale to see this "pattern".
So mode=0 is no no for my general use.
I honestly have a very hard time finding a case where banding occurs.

I did a test between
Resize+DitherPost(mode=-1)
Resize+DitherPost(mode=6)
Spline16
ResizeShader with integrated Ordered Dither

I haven't been able to reproduce the banding but the difference I notice is that the shapes are better defined.

I did notice something else during that test.

DitherPost slightly distorts the colors!! Test 1 and 2 gave slightly different colors than test 3 and 4

and DitherPost is used... well... everywhere you're using 16-bit

[hello_hello]

I honestly have a very hard time finding a case where banding occurs.

I use noise filtering a bit and that tends to bring it on, but as a rule I just put gradfun3() at the end of a script and don't have to think about it much (I rarely process in 16 bit).

There's a link to a couple of images in this post (http://forum.doom9.org/showthread.php?p=1745200#post1745200).
If the 8 bit image is indicative of real world banding it's reasonably severe and not easy to repair. Banding is possibly easier to prevent than fix.

I found the 10 bit image more interesting. No matter how I open it, it shows much the same banding as the 8 bit image (I'm connected to a Plasma via VGA at the moment), except if I open it with MPC-HC (it's the only media player I have decoding with LAV). When I open it with MPC-HC (VMR9 renderer) there's no banding at all. I can only assume LAV is applying dithering when decoding it as 8 bit and whatever dithering it uses, it works well.
You could probably try a similar experiment using L-Smash to open it in 16 bit mode and dither it down from there.

[bxyhxyh]

I honestly have a very hard time finding a case where banding occurs.
16-bit
Please test this carefully. By pattern I meant something like chessboard.
Maybe try some anime or something?