Comparison of ColorMatrix Methods

[MysteryX]

Just a few things to add to AviSynth.nl

Under Processing High Bit-depth Video with AviSynth and Multipurpose (http://avisynth.nl/index.php/High_bit-depth_Support_with_Avisynth), AviSynthShader should be there.

And in this list: Deep Color Tools (http://avisynth.nl/index.php/Category:Deep_color_tools)

Also in the Colorimetry page (http://avisynth.nl/index.php/Colorimetry), it should mention that ColorMatrix can cause banding due to rounding errors, and suggest 16-bit-depth alternatives such as Dither Tools and AviSynthShader for colorspace conversion without banding.

By the way, does anyone know how the 'hints' parameter of ColorMatrix works internally? I'm wondering whether that could be implemented in AviSynthShader easily.

[Chikuzen]

By the way, does anyone know how the 'hints' parameter of ColorMatrix works internally? I'm wondering whether that could be implemented in AviSynthShader easily.

"hints" is a hack for some filters to communicate informations besides struct VideoInfo to the next filter.
The informations(color matrix, RFF flags, field matching, is combed frame or not, etc...) are embedded with rewriting the lowest 1bit in each 64bytes of upper left of frame data.
The first 32bits over the overwritten 64bits are the magic number(0xDEADBEEF) to indicate existence of hints.
The following 32bits are the flags which show informations.
If you'd like to know more about "hints", read source code of DGMPEGDec.

[Groucho2004]

Also in the Colorimetry page (http://avisynth.nl/index.php/Colorimetry), it should mention that ColorMatrix can cause banding due to rounding errors
What rounding errors?

[MysteryX]

Chikuzen, wow that's an ugly hack!!

What rounding errors?
Because the processing is done with 8-bit, a very slight rounding occurs during the color matrix conversion from Rec601, and again when convert to Rec709. And because the effect of the matrix shift is also subtle, it can cause banding to appear on soft gradients. That's the reason why many are instead using DitherTools to adjust the color matrix.

Someone sent me a screenshot of an anime where the banding was obvious and I did the test for myself to witness this.

[Groucho2004]

Chikuzen, wow that's an ugly hack!!


Because the processing is done with 8-bit, a very slight rounding occurs during the color matrix conversion from Rec601, and again when convert to Rec709. And because the effect of the matrix shift is also subtle, it can cause banding to appear on soft gradients. That's the reason why many are instead using DitherTools to adjust the color matrix.

Someone sent me a screenshot of an anime where the banding was obvious and I did the test for myself to witness this.
I see. I wasn't sure if you were referring to 8 bit processing or the rounding errors that the colormatrix SSE2/MMX routines introduce.

[MysteryX]

I don't know whether the SSE2/MMX introduces any more rounding errors than without, all I know is that the output of ColorMatrix can introduce some banding that is greatly reduced by using 16-bit processing. With DitherTools, there is still a little bit of banding but it is much smaller. ColorMatrixShader may have slightly more banding than DitherTools because of the YV12-YV24 conversion (unless you use DitherTools for that conversion) but it's close.

There are others who know more about this.

[Groucho2004]

With DitherTools, there is still a little bit of banding but it is much smaller.
Probably. Unfortunately, DitherTools is 10 times slower.

[MysteryX]

Probably. Unfortunately, DitherTools is 10 times slower.
Yeah I saw that when I tested it; AviSynthShader with ColorMatrixShader isn't super fast but is much faster.

[MysteryX]

I did some comparison between the various color matrix shift methods. Look at the banding on the right head.

Original (YV12)
http://s20.postimg.org/6o99rlgbd/Band_Original.png (http://postimg.org/image/6o99rlgbd/)

ColorMatrix(mode="Rec.601->Rec.709")
http://s20.postimg.org/ic3bm55g9/Band_Color_Matrix.png (http://postimg.org/image/ic3bm55g9/)


Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y")
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=0)

http://s20.postimg.org/z2epbh3vd/Band_Dither_Tools.png (http://postimg.org/image/z2epbh3vd/)

ColorMatrixShader(MatrixIn="601")
http://s20.postimg.org/cf41j2dp5/Band_Shader.png (http://postimg.org/image/cf41j2dp5/)

This is with a version of ColorMatrixShader that isn't released yet. The existing version lost all benefits of 16-bit processing by calling ConvertToYV24() and ConvertToYV12() which added the banding anyway. The next version will require DitherTools and will automatically convert to YV24 the right way.

I'm seeing, however, that ColorMatrixShader is just very slightly darker. I really don't know what could be causing this.


Performance-wise (AVS+ with 8 threads):

ColorMatrix

FPS (min | max | average): 195.2 | 95838 | 625.9
Memory usage (phys | virt): 123 | 116 MiB
Thread count: 21
CPU usage (average): 61%


DitherTools

FPS (min | max | average): 4.883 | 106293 | 50.77
Memory usage (phys | virt): 619 | 617 MiB
Thread count: 27
CPU usage (average): 79%


AviSynthShader

FPS (min | max | average): 3.200 | 1000000 | 39.49
Memory usage (phys | virt): 989 | 1135 MiB
Thread count: 63
CPU usage (average): 78%


mmm... AviSynthShader is the slowest on my system, but you might get better performance on yours. If the GPU returns 8-bit data instead of 16-bit data, performance goes up to 59fps, but then there is banding to convert back to YV12. With YV24 source, AviSynthShader gives 75fps.

I think that for stand-alone conversion, DitherTools is better, but if using any other AviSynthShader function, the best is to do the colorspace conversion at the same time.

Woah... just saw the memory usage of AviSynthShader. No idea why memory usage went so high!! I run much more complex shader scripts and the memory usage is generally lower than that!! Not sure what's going on here. mmmm.... on the plus side, I'm not getting the weird memory usage discrepancy between 32-bit and 64-bit versions in this case! Here it is high for both for some unknown reason.

Edit: I think I'll kill the ColorMatrixShader stand-alone function. If it requires DitherTools anyway, it has no benefit over Dither Tools. Performance is lower, it requires YV12-YV24 conversion and it depends on Dither Tools anyway. Colorspace conversion can still be made through any other function.

[Chikuzen]

@MisteryX
I think that the biggest bottleneck of your plugin is Convert(To/From)Shader.

I did a tiny comparison as follows(YV24->RGB32->YV24).

LoadPlugin("Shader.dll")
LoadPlugin("PlanarTools.dll")
src=ColorBars(1920, 1080, "YV24").Trim(0, 9999)
src.ConvertToShader(1).ConvertFromShader(1, format="YV24")
#src.PlanarToRGB32(BlankClip(src, pixel_type="Y8", color_yuv=$000000)).PackedToPlanar()


AvisynthShader: 121.934sec(82.012fps)
PlanarTools (https://github.com/chikuzen/PlanarTools): 16.442sec(608.199fps)

You should optimaze those two filters first if you want to make fast your plugin.

[Groucho2004]

I did some comparison between the various color matrix shift methods. Look at the banding on the right head.
I prepared a little banding comparison between ColorMatrix and DitherTools as well, you can download it here (https://www.dropbox.com/s/699cxnljtlv0bzz/TestBanding.7z?dl=0).
If there is a difference, it's really small.

Changing assumefps(25, 1) to (5, 1) might make it more visible.

[Motenai Yoda]

Looks to me as Colormatrix sports less banding...
maybe a better source, will give more sense to a banding comparison than a full banded one.

[Groucho2004]

I prepared a little banding comparison between ColorMatrix and DitherTools as well, you can download it here (https://www.dropbox.com/s/699cxnljtlv0bzz/TestBanding.7z?dl=0).
If there is a difference, it's really small.

Changing assumefps(25, 1) to (5, 1) might make it more visible.
Tweaked the source a bit, should have less banding to start with. Link updated.

[Motenai Yoda]

My bad, I though it was clear I was referring to MisterX

anyway Groucho's example without dithering (mode=-1) differences are almost 0, just some +/-1 noise on cplanes, even using sse2 or plain c

[MysteryX]

@MisteryX
I think that the biggest bottleneck of your plugin is Convert(To/From)Shader.
These 2 functions could be optimized with assembly code but I don't do assembly programming and nobody so far has done it. Perhaps it would help to integrate PlanarTools into it.

But they're not the bottleneck. Generally you're running a whole series of shaders and these 2 functions only have a minor impact on performance, especially since the CPU is under-utilized.

[MysteryX]

I revisited the ColorMatrix conversion methods. Also renamed the thread title to better reflect where the discussion has turned.

ColorMatrix

ColorBarsHD(960, 720).KillAudio()
ConvertToYV12()
ColorMatrix(mode="Rec.601->Rec.709")

https://s31.postimg.org/6hc5g92iv/Matrix_Color.png (https://postimg.org/image/6hc5g92iv/)

DitherTools

ColorBarsHD(960, 720).KillAudio().ConvertToYV24()
Dither_convert_8_to_16()
Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", output="YV24", lsb=true)
DitherPost(mode=6)

https://s31.postimg.org/ca4ahcrc7/Matrix_Dither_Tools.png (https://postimg.org/image/ca4ahcrc7/)

AviSynthShader

ColorBarsHD(960, 720).KillAudio().ConvertToYV24()
ResizeShader(Kernel="ColorMatrix", MatrixIn="601")

https://s31.postimg.org/z9iwzqa6f/Matrix_Shader.png (https://postimg.org/image/z9iwzqa6f/)

Indeed I'm not seeing much banding with ColorMatrix. However it causes weird distortion on borders!!

With the great performance improvements to AviSynthShader and the addition of dithering, it is now becoming a very interesting option for converting colorspace. I tested this with my dev version. I might not re-add the ColorMatrixShader function but rather add a Kernel="ColorMatrix" to the resize function so that it does only the color conversion without the resize.

I kind of like the ResizeShader version slightly better, EXCEPT that the colors in the dark grays are just very slightly more yellow. Any idea what could be the cause of this?

Edit: ColorMatrix distortion is most likely caused by ConvertToYV12. So ColorMatrix is fine if you're operating in YV12-land and don't want a 16-bit result.

[raffriff42]

Sorry to be hypercritical, but this is relevant to a discussion of colormatrix. People should learn what proper color bars look like.

Those color bars are incorrect (too "green" (http://avisynth.nl/index.php/Colorimetry#Using_color_bars)). It may be your screen shot and not your video; for instance, VirtualDub always decodes YUV sources for preview as Rec.601. For accurate preview of Rec.709 sources, you need to add the vdub filter AliasFormat (color space=Rec709).

[MysteryX]

It looks "incorrect" because I did a conversion from Rec601 into Rec709, and then VirtualDub reads it again as Rec601; but that doesn't really change anything.

[MysteryX]

Why topic starter use very old low quality plugins?!
The thread was started to update the documentation about colorimetry; but Groucho2004 doesn't see anything wrong with the standard function

[hello_hello]

I recall re-encoding an old black and white DVD video a while back. It was slightly "brown and white", so I used Tweak to make it black and white: Tweak(sat=0)
I was really surprised at how much banding it caused. I've never quite got my head around why, and haven't come across a B/W DVD that bad since, but I did try it on the first screen shot from post #9 and it seemed to increase the banding. Can that be attributed to rounding errors as such?
Or maybe it's just making the banding easier to spot?

ConvertToYv12()
https://s31.postimg.org/jwjs4j447/image.png (https://postimg.org/image/jwjs4j447/)

ConvertToYv12()
Tweak(sat=0)
https://s32.postimg.org/ux8qo4bf5/image.png (https://postimg.org/image/ux8qo4bf5/)

[Wilbert]

Tweak(sat=0) shouldn't cause banding. There is already banding in your colored image. Btw, you can use Tweak(sat=0, dither=true) to add some dithering.

Also make b/w using desaturate u and v is better to do using lossless operation:
ConvertToY8().ConvertToYV12()

It removes u and v channels without any changes in y.
Which is exactly the same as Tweak(sat=0).

[Reel.Deel]

Also make b/w using desaturate u and v is better to do using lossless operation:
ConvertToY8().ConvertToYV12()

It removes u and v channels without any changes in y.


Grayscale() does the same thing.

[hello_hello]

Thinking about it, I converted to black and white using greyscale() last time, not Tweak().

8-bit processing is good for 1998 year. Do not use it at all.

Trouble is, there's not many media players capable of decoding 10 bit video.... as yet. I like my encodes to look pristine, but they don't look like anything if I can't play them. ;)

As a general rule I don't have much problem with banding. I usually stick Gradfun3() at the end of a script (always if I'm denoising) and it's not much of an issue (I tend to use CRF16 or lower if I suspect banding might rear it's ugly head). Almost every time I see banding, when I check there's already banding in the source (although encoding tends to make it worse) but I think I spend more time trying to fix banding than prevent it.

I almost never encode animation though. Maybe banding's more of a problem there.

[geometer]

explanation is easy, was mentioned already at the beginning.

with using "brownish" colors, and some kind tolerance by the viewer, a greyscale picture/video can be expressed nicely in digital standard format.

but as soon as we go technically greyscale, literally, we have only 256 values for the whole thing!!
this won't go without ugly banding, or very visible dithering-noise.

and what bit-wise-better, or more tricky formats would an average video player support at all?

[hello_hello]

with using "brownish" colors, and some kind tolerance by the viewer, a greyscale picture/video can be expressed nicely in digital standard format.

but as soon as we go technically greyscale, literally, we have only 256 values for the whole thing!!
this won't go without ugly banding, or very visible dithering-noise.

Is that why so many old movies on DVD seem to be brownish and white, or blueish and white, rather than black and white?

I'm trying to get my head around that though. I take a black and white image and increase the level of blue so it's blueish and white, but does that really increase the number of luminance values that can be assigned? Unless the blue itself is dithered... maybe.

but as soon as we go technically greyscale, literally, we have only 256 values for the whole thing!!
this won't go without ugly banding, or very visible dithering-noise.

I generally include greyscale() in a script when encoding B/W video, and I've probably only noticed it causing banding a couple of times. Then again, old B/W movies tend to be somewhat noisy, so they have some anti-banding built-in.

and what bit-wise-better, or more tricky formats would an average video player support at all?

None, I suspect, although to be honest I'm not sure why you asked the question. That's why I encode with 8 bit x264.
Does 4K Bluray support 10 bit? I'd imagine the media players in TVs will eventually support 10 bit h265.. as a rule... but I'm not sure that necessarily means they'll also be endowed with 10 bit h264 support. That may never be mainstream. Does anybody know what the current state of affairs might be?

[bxyhxyh]

Trouble is, there's not many media players capable of decoding 10 bit video.... as yet. Yes, end result is almost always 8 bit.
But mid processing should be in higher precision. Especially for color adjusting filters like these.

If we can prevent banding before it appears. Isn't that good thing?
Because removing it later, destroys details.

[hello_hello]

Yes, end result is almost always 8 bit.
But mid processing should be in higher precision. Especially for color adjusting filters like these.

If we can prevent banding before it appears. Isn't that good thing?
Because removing it later, destroys details.

Yep. ;)

I usually stick Gradfun3() at the end of a script (always if I'm denoising) and it's not much of an issue (I tend to use CRF16 or lower if I suspect banding might rear it's ugly head).

[hello_hello]

I wouldn't argue with any of that, but the majority of AVISynth filters are 8 bit, so much of the time noise shaping at the end of a script is the only practical option.

[MysteryX]

There are quite a few filters that, when you read the documentation carefully, they have lsb_in/lsb_out parameters. SMDegrain, KNLMeans, SuperRes among others. And DitherTools provides many 16-bit functions.

There are already quite a few options for 16-bit processing, and I believe this will evolve a lot in the coming months with the upcoming version of AVS+ with native 16-bit (non-stack) support

I often like to do the processing in YV24, especially when upscaling, and you have to be careful with the YV12 to YV24 conversion. Use DitherTools to avoid banding and losing data right there.

[Wilbert]

I wouldn't argue with any of that, but the majority of AVISynth filters are 8 bit, so much of the time noise shaping at the end of a script is the only practical option.
Internally most (all?) filter at 16/32 bit, and they just output 8 bit. If that's what you meant by this, then i agree.


I often like to do the processing in YV24, especially when upscaling, and you have to be careful with the YV12 to YV24 conversion. Use DitherTools to avoid banding and losing data right there.
Usually you will get banding when you change the luma range. I doubt you will see much banding when converting from YV12 to YV24 (or back).

[MysteryX]

Internally most (all?) filter at 16/32 bit, and they just output 8 bit. If that's what you meant by this, then i agree
If most filters already process in 16-bit, it's insanity that during all these years it has been rounding to 8-bit after every filter.

Hopefully this will change soon.

This means that 16-bit processing won't have any performance cost.

[Wilbert]

If most filters already process in 16-bit, it's insanity that during all these years it has been rounding to 8-bit after every filter.

Hopefully this will change soon.

This means that 16-bit processing won't have any performance cost.
Depends on the filters. Several filters use 256x256 look up tables to speed up processing (filters like coloryuv, tweak, rgbadjust, levels, etc...). Using look up tables of 65536x65536 might be too much, so we need to think about what to do in such cases. If you don't use look up tables in will result in a performance drop.

[MysteryX]

Depends on the filters. Several filters use 256x256 look up tables to speed up processing (filters like coloryuv, tweak, rgbadjust, levels, etc...). Using look up tables of 65536x65536 might be too much, so we need to think about what to do in such cases. If you don't use look up tables in will result in a performance drop.
Query a 5GB file containing the values that ships alongside the DLL? Just kidding

65536 * 65536 * 2 = 8,589,934,592 = 8192MB. I wasn't far with my estimation.

[feisty2]

Wilbert, are you sure for performance drop? We mean a direct compute with one multiplication and 2 addition without a dropping CPU cache. I have 3000 fps for it.

EDIT: didn't read the previous posts clearly.
depends, if something like mt_lut which executes a user defined expression at runtime, that's gonna be super slow (without JIT).
vaporsynth got this filter called Expr, it executes expressions defined at runtime and at floating point precision, and it works on JIT

[Wilbert]

Wilbert, are you sure for performance drop? We mean a direct compute with one multiplication and 2 addition without a dropping CPU cache. I have 3000 fps for it.
What do you mean with one multiplication and 2 additions? You use look up tables in order to be able to do the computations using integers. If you can't use look up tables, you need to do the computations pixel-wise using float. For Levels() it's not even multiplications and or additions.

[TheFluff]

For some very simple expressions, the Vapoursynth expr() filter which reevaluates the expression for every pixel is actually faster than the equivalent lut(), even for 8-bit input, and for many not so simple expressions it's insignificantly slower than a LUT. It is pretty optimized (JIT asm compiler, constant folding, etc) but even in general you shouldn't assume that a LUT is always faster. Memory access isn't free.

[MysteryX]

expr and a lot of 16-bit support can be back-ported from VapourSynth

[MysteryX]

Now, if I develop AviSynth from scratch, I would prefer Microsoft AMP or video card compute like CUDA in 32-bit floating point only, because it is extremely fast.
AvisynthShader allows you to do 16-bit GPU processing in such a way