Doom9's Forum - View Single Post - Scaling, colorspace conversion, and dithering library

mawen1250 · 16th April 2015, 15:39

test script

Code:

src = core.std.BlankClip(width=120, height=120, format=vs.YUV444P8, color=[0,0,0])

dst = core.z.Depth(src, dither='ordered', depth=16, fullrange_in=True, fullrange_out=True)

dst = core.std.ShufflePlanes([dst, dst, dst], [0,1,2], vs.RGB)
dst = core.std.Expr(dst, 'x', vs.RGB48)
dst_stack = core.fmtc.nativetostack16(dst)

dst_stack.set_output()

I use full range YUV444 as input, to test both luma and chroma.
The result YUV444Px is copied to RGB48 planes to avoid YUV->RGB conversion.
Finally convert native-16bit to stacked-16bit format to better visualize the MSB/LSB in RGB24 image.

ordered

none or random

error_diffusion

1. There're underflow issues. For example, 0 in 8bit chroma converted to 16bit is -128, it should be cliped to 0, but the result is 65408.
2. There're also overflow issues with dither enabled. For example, 65535 in 16bit chroma converted to 15bit is 32767.25, then the resulting dithering pattern is mixed with 32767 and 32768, exceeding valid range [0,32767].
3. The ordered dither is always applied even if no range conversion is needed, and the dithering amplitude is always 1 in 8-bit scale no matter what the output depth is. I wonder if it is designed to be this or not.
4. In the above examples, the right most part is not affected, but in some other tests this doesn't happen. I'm not very sure in which conditions will this happen.

16th April 2015, 15:39	#22 \| Link
mawen1250 Registered User Join Date: Aug 2011 Posts: 103	Here're some issues I found test script Code: src = core.std.BlankClip(width=120, height=120, format=vs.YUV444P8, color=[0,0,0]) dst = core.z.Depth(src, dither='ordered', depth=16, fullrange_in=True, fullrange_out=True) dst = core.std.ShufflePlanes([dst, dst, dst], [0,1,2], vs.RGB) dst = core.std.Expr(dst, 'x', vs.RGB48) dst_stack = core.fmtc.nativetostack16(dst) dst_stack.set_output() I use full range YUV444 as input, to test both luma and chroma. The result YUV444Px is copied to RGB48 planes to avoid YUV->RGB conversion. Finally convert native-16bit to stacked-16bit format to better visualize the MSB/LSB in RGB24 image. ordered none or random error_diffusion 1. There're underflow issues. For example, 0 in 8bit chroma converted to 16bit is -128, it should be cliped to 0, but the result is 65408. 2. There're also overflow issues with dither enabled. For example, 65535 in 16bit chroma converted to 15bit is 32767.25, then the resulting dithering pattern is mixed with 32767 and 32768, exceeding valid range [0,32767]. 3. The ordered dither is always applied even if no range conversion is needed, and the dithering amplitude is always 1 in 8-bit scale no matter what the output depth is. I wonder if it is designed to be this or not. 4. In the above examples, the right most part is not affected, but in some other tests this doesn't happen. I'm not very sure in which conditions will this happen. Last edited by mawen1250; 16th April 2015 at 16:05.