Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
14th August 2019, 21:37 | #1 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
HDRTools vs DitherTools vs ColorMatrix
Would any interested parties care to look at this comparison of color conversion methods? The ColorMatrix version used is this one.
Edit: For anyone reading this thread, HDRTools was getting the XYZ rec.2020 color conversions correct. Using that method it also converts the color primaries which produce a very different result. The banding problems I think I mentioned at some stage are due to an 8 bit video limitation. HDRTools doesn't cause banding if you use Avisynth+ and convert the video to 16 bit first. The rest of what I didn't understand when starting this thread is explained in post #19 and #20, as is why I think ColorMatrix 2.6 shouldn't be used for rec.2020 conversions. Aside from the fact it outputs the same colors as the source when converting between rec.2020 and rec.601, rather than outputting an error message because it doesn't support conversions between rec.2020 and rec.601, I'm still pretty sure it's rec.2020 to rec.709 conversion is incorrect, and it doesn't convert the color primaries either (neither does DitherTools). There's a bunch of screenshots to go with the tests below. I won't post them here as there's quite a few, but if you follow the link you'll find a gallery and the screenshots are properly named so as to easily see what they represent. Included is a screenshot of how much the images sizes vary after compression to PNG. The biggest changes are the DitherTools images which I suppose must be the result of dithering. The source is animation so there's large areas of single color to make the differences easy to see. https://postimg.cc/gallery/1r9z2a760/ All testing was done using classic Avisynth on XP, although a quick look indicates nothing changes for Avisynth+. The source video is HD, downscaled to SD, the various color conversions applied, and the screenshots taken by opening the script with MPC-HC. If I'm doing something wrong I'm happy to learn. Comments welcome! Rec.709 To Rec.601 All conversion methods seem to agree. HDRTools Linear looks better than HDRTools XYZ (no banding). HDRTools XYZ ConvertYUVtoXYZ(Color=2).ConvertXYZtoYUV(Color=3, pColor=2, OutputMode=2) HDRTools Linear ConvertYUVtoLinearRGB(Color=2).ConvertLinearRGBtoYUV(Color=3, OutputMode=2) DitherTools Dither_convert_yuv_to_rgb(matrix="709", slice=false).Dither_convert_rgb_to_yuv(matrix="601", output="YV12") ColorMatrix ColorMatrix(mode="Rec.709->Rec.601", clamp=0) Rec.2020 To Rec.601 DitherTools and HDRTools Linear apparently agree. ColorMatrix is different. HDRTools XYZ seems borked or I don't know what I'm doing. HDRTools XYZ ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=3, pColor=1, OutputMode=2) HDRTools Linear ConvertYUVtoLinearRGB(Color=1).ConvertLinearRGBtoYUV(Color=3, OutputMode=2) DitherTools Dither_convert_yuv_to_rgb(matrix="2020", slice=false).Dither_convert_rgb_to_yuv(matrix="601", output="YV12") ColorMatrix ColorMatrix(mode="Rec.2020->Rec.601", clamp=0) Rec.2020 To 709 I'd only be guessing as nobody agrees. DitherTools and HDRTools Linear possibly only disagree for "red". I think ColorMatrix is wrong. HDRTools XYZ seems borked or I don't know what I'm doing. HDRTools XYZ HDRToolsConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=1, OutputMode=2) HDRTools Linear ConvertYUVtoLinearRGB(Color=1).ConvertLinearRGBtoYUV(Color=2, OutputMode=2) DitherTools Dither_convert_yuv_to_rgb(matrix="2020", slice=false).Dither_convert_rgb_to_yuv(matrix="709", output="YV12") ColorMatrix ColorMatrix(mode="Rec.2020->Rec.709", clamp=0) Anyone know what the difference between the HDRTools two flavours of Rec.601 might be (BT601_525 & BT601_625)? When I do this, there's appears to be a difference, unless it's just the conversion and not a color difference, A = last B = ConvertYUVtoXYZ(Color=3).ConvertXYZtoYUV(Color=4, pColor=3, OutputMode=2) C = ConvertYUVtoLinearRGB(Color=3).ConvertLinearRGBtoYUV(Color=4, OutputMode=2) Compare(A, B) or Compare(A, C) (these are labelled "HDRTools Compare A&B" and "HDRTools Compare A&C" in the gallery. Last edited by hello_hello; 28th January 2020 at 21:11. |
15th August 2019, 01:38 | #2 | Link |
Registered User
Join Date: Jul 2018
Posts: 450
|
Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=1, OutputMode=2) Code:
z_ConvertFormat(colorspace_op="2020:709:2020=>2020:709:xyz").z_ConvertFormat(colorspace_op="2020:709:xyz=>709:709:709") and Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=2, OutputMode=2) Code:
z_ConvertFormat(colorspace_op="2020:709:709=>2020:709:xyz").z_ConvertFormat(colorspace_op="2020:709:xyz=>709:709:709") Code:
z_ConvertFormat(colorspace_op="709=>601",resample_filter_uv="spline36",dither_type="error_diffusion") "Anyone know what the difference between the HDRTools two flavours of Rec.601 might be (BT601_525 & BT601_625)?" - https://en.wikipedia.org/wiki/Rec._6...chromaticities ; https://en.wikipedia.org/wiki/PAL#PAL_broadcast_systems ; https://en.wikipedia.org/wiki/NTSC#Variants Take a look at "lines". Last edited by StvG; 15th August 2019 at 19:18. Reason: syntax |
15th August 2019, 10:51 | #3 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
StvG, Thanks for the reply.
I'm still not clear though as to whether HDRTools XYZ is getting it wrong, or if I'm doing something wrong, and why the other color conversion methods only have one flavour of rec.601 while HDRTools has two. Your link seems to indicate the two versions of rec.601 are for converting analogue NTSC & PAL to digital. Once it's digital, isn't the colorimetry the same, and if so, would there only be one method required for converting digital rec.601 to another colorimetry etc? Cheers. PS. I did try avsresize but couldn't get it to work. I couldn't see any mention of it in the docs, but I assume it's not compatible with XP? Last edited by hello_hello; 15th August 2019 at 13:41. |
15th August 2019, 19:33 | #4 | Link |
Registered User
Join Date: Jul 2018
Posts: 450
|
I guess HDRTools is wrong (bug?) because the output with both pColor=1/2 is the same.
Speaking of DVD content Rec.601 (BT470_BG/ST170_M) is the right setting. Speaking of broadcasting the setting could be different - https://forum.doom9.org/showthread.php?t=133982 The other conversion methods have "240" Dither (SMPTE 240M/BT601_525), "240m" avsresize... I didn't try avsresize on XP. |
15th August 2019, 21:16 | #5 | Link | |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
StvG,
Would I be able to ask you to compare a rec.2020 to rec.709 conversion between HDRTools "linear", DitherTools and AVSResize, if it's not too inconvenient? I'm assuming ColorMatrix is wrong but I'd be interested to find out if AVSResize agrees with DitherTools or HDRTools "linear" for rec.2020 to rec.709, as an indication which program is getting it right, given they're slightly different. I haven't compared them properly, but I think the result is the same when upconverting the colors as it is for downconverting ie rec.2020 to rec.601 vs rec.601 to rec.2020. I think if the downconversion is correct for each program the equivalent upconversion is correct too, or if the downconversion is wrong the upconversion is also wrong, but I should check that properly at some stage. Quote:
Cheers. Last edited by hello_hello; 15th August 2019 at 21:29. |
|
16th August 2019, 00:03 | #6 | Link | |
Registered User
Join Date: Jul 2018
Posts: 450
|
I used your "No-Color-Conversion.png". I converted it to yv24 in the same way for all three methods so all comparison have the same upscaled chroma before the conversion. As a final step I converted yv24 to yv12 so all comparisons have the closest possible matching chroma downscaling.
HDRTools: Code:
ImageSource("No-Color-Conversion.png") ConvertToPlanarRGB() z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion") ConvertYUVtoLinearRGB(Color=1, OutputMode=1) ConvertLinearRGBtoYUV() z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion") subtitle("HDRTools") Code:
ImageSource("No-Color-Conversion.png") ConvertToPlanarRGB() z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion") Dither_convert_yuv_to_rgb(matrix="2020") Dither_convert_rgb_to_yuv(matrix="709", output="YV24",lsb=true) DitherPost(mode=6) z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion") subtitle("Dither") Code:
ImageSource("No-Color-Conversion.png") ConvertToPlanarRGB() z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion") z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020=>rgb", dither_type="error_diffusion") z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb=>709",dither_type="error_diffusion") z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion") subtitle("avsresize") Code:
ImageSource("No-Color-Conversion.png") ConvertToPlanarRGB() z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion") z_ConvertFormat(pixel_type="yuv444p16",resample_filter_uv="spline36", colorspace_op="2020=>709",dither_type="error_diffusion") z_ConvertFormat(pixel_type="yv12", resample_filter_uv="spline36", dither_type="error_diffusion") Code:
ImageSource("No-Color-Conversion.png") ConvertToPlanarRGB() z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion") z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:709=>rgb:linear", dither_type="error_diffusion") z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:linear=>709:709",dither_type="error_diffusion") z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion") subtitle("avsresize linear") 709->601 is done in 16-bit for all methods but HDRTools and avsresize could use float too for that conversion. Quote:
Last edited by StvG; 16th August 2019 at 00:24. Reason: added "avsresize #1" |
|
16th August 2019, 03:21 | #7 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Thanks for doing that.
It was very disappointing to see your result was the same each time, or close enough for me to call them the same (I was hoping you'd get different results as I did). Obviously I can't replicate your method (avsresize no worky XP) so I tried doing it again a more traditional way, although I've looked at the various screenshots so many times today all I'm seeing is pink elephants at the moment.... I'm not sure I'm taking the same frame from the source as yesterday, and there's several frames in a row without movement and I didn't make a note of the frame number, but here's two screenshots I took today. Today the result is closer to yours, but the difference between converting to YV24 before using HDTools, or not, is noticeable. All I can assume is the difference is/was the YUV to RGB conversion, and for some reason it stood out more with rec.2020 to rec.709. I could duplicate this result, so I don't think it's a rendering problem. They're not massively different, but they are different. It's frustrating because today the result where I used ConvertToYV24 is very close to yesterday's screenshot where I didn't use it. Hopefully that's just because it's not the same frame from the source. Maybe I need to start over, although at least now I know HDRTools XYZ has problems and ColorMatrix definitely isn't doing rec.2020 conversions correctly, so I've learned something.... Thanks again. PS. I'm still don't really understand why HDRTools has two rec.601 flavours. Last edited by hello_hello; 16th August 2019 at 03:37. |
16th August 2019, 08:43 | #8 | Link |
Registered User
Join Date: Jul 2018
Posts: 450
|
Yes, the difference between HDRTools chroma upscale method and converttoyv24/z_convertformat is noticeable especially if the output is 8-bit (less noticeable with 16-bit output):
Code:
(#)converttoyv24() ConvertYUVtoLinearRGB(Color=1, OutputMode=1) ConvertLinearRGBtoYUV(outputmode=1) ConvertBits(8) subtitle("w/(o) converttoyv24") |
16th August 2019, 22:40 | #10 | Link | |
Registered User
Join Date: Oct 2002
Location: France
Posts: 2,316
|
Quote:
So Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=2, OutputMode=2) Code:
z_ConvertFormat(colorspace_op="2020:709:709=>2020:709:xyz").z_ConvertFormat(colorspace_op="2020:709:xyz=>709:709:709") It's described in the Warning page 3 of ColorConversion.pdf provided with the binaries. And, i clearly state in the filter doc : Code:
Note about YV12/YV16/YV24 input/ouput: For ConvertYUVtoxxx functions, convertion to YV24 if input is not YV24 is "quick". If you want a true correct precise/perfect convertion, i would avise you to use a resampler and feed directly to YV24. For ConvertxxxtoYUV functions, if you choose an ouput different from YV24, same remark, convertion is "quick". If you want a true correct precise/perfect convertion, i would avise you to ouput to YV24, and use a resampler to achieve the real output format you want.
__________________
My github. Last edited by jpsdr; 16th August 2019 at 22:44. |
|
16th August 2019, 23:47 | #11 | Link | ||
Registered User
Join Date: Jul 2018
Posts: 450
|
Quote:
Quote:
|
||
17th August 2019, 00:07 | #12 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
Have you tried using "opt = 0" with ColorMatrix?
__________________
Groucho's Avisynth Stuff |
17th August 2019, 00:24 | #13 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
No but it doesn't seem to need it for rec.709 <-> rec.601 conversions.
I'll try it later and report back if it changes anything, but the difference between ColorMatrix and the other programs is fairly substantial. Except for Rec.709 to Rec.2020. That's the one type of Rec.2020 conversion where it's the same. |
19th August 2019, 11:56 | #15 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
There's a 709 to 2020 up-conversion here. https://forum.doom9.org/showthread.p...24#post1881624
Everyone seems to get 601 <-> 709 right. I haven't gone back and redone any more tests to be sure, but I think the result between colorimetries was the same in both directions. If it's correct down-converting, up-converting is okay too. The one exception was ColorMatrix converting between 709 and 2020. It appears to down-convert incorrectly but the up-conversion is okay. It might pay not to use the ColorMatrix 2020 build though. There's tried and tested 32 and 64 bit builds for HD and SD color conversions. It appears HDTools XYZ is working correctly but I didn't know how to use it, or possible more accurately, why the results were different. There's info in this post and the replies that follow. Keep an eye on that thread. I'm still learning.... Last edited by hello_hello; 19th August 2019 at 14:44. |
23rd January 2020, 04:07 | #16 | Link | |
Registered User
Join Date: Jan 2018
Posts: 2,156
|
Quote:
|
|
27th January 2020, 02:23 | #17 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
I don't think version 2.6 has a 64 bit flavour, but there's a link for the 64 bit flavour of version 2.5 in the opening post here.
https://forum.doom9.org/showthread.p...05#post1374605 I think most people still use version 2.5 anyway. I do. |
27th January 2020, 13:00 | #18 | Link | ||
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Dont know where I got this archive, originally named "ColorMatrix26.7z".
Renamed archive to "ColorMatrix26(v2.6_avs+_x86&x64+Src_2020WithoutAsmOpts).7z". @ MediaFire in my sig, in HOSTED folder, or direct link here:- http://www.mediafire.com/file/5sshv9...s%2529.7z/file EDIT: Quote:
However, name of 7z archive is different, and two separate archives for dll and source, so looks like someone else added avs+ support, recompiled and issued combined archive. EDIT: Looks like was Groucho2004 that updated for avs+, same archive name[Grouchy 2004 suppository]:- http://www.iol.ie/~schubert/gas/gas.html From Groucho2004 Avisynth Stuff thread:- https://forum.doom9.org/showthread.php?t=173259 Quote:
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? Last edited by StainlessS; 27th January 2020 at 14:17. |
||
28th January 2020, 16:00 | #19 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
ColorMatrix 2.6 is borked
I ran some more tests to see if I'd still get the same results as last time, and now I understand part of the head-f*#k I was experiencing the last time around. I'm pretty sure the result is the same for the Grouchy versions as it is for the original ColorMatix 2.6.
I didn't notice, probably because I was concentrating on comparing the color conversion results between programs, that ColorMatrix 2.6 conversions between rec.2020 and rec.601 simply don't work in either direction (assuming they're supposed to). The output is the same as the input. In my defence though, I posted samples and nobody else noticed either. The result was the same as before when comparing rec.2020 and rec.709 conversions between ColorMatrix and DitherTools, For a conversion from rec.709 to rec.2020 the two programs produce the same result. When converting from rec.2020 to rec.709 they don't, and my guess is ColorMatrix 2.6 is the one getting it wrong. As I eventually learned, ColorMatrix and DitherTools both convert to/from Rec.2020 using a matrix-only method. If there's a point to converting to/from Rec.2020 that way I'd be keen to learn what it is. That's why the CropResize script no longer uses either ColorMatrix or DitherTools for rec.2020 conversions. Instead it only converts to/from Rec.2020 with HDRTools, as HDRTools can convert the color primaries too. You can get away with converting between rec.601 and rec.709 using a matrix-only method, because even though it's not technically perfect, the color primaries are so similar it doesn't matter too much. For conversions to/from Rec.2020, I have no idea when it'd make sense not to also convert the color primaries. There's examples of HDRTools doing rec.2020 conversions in the zip file below. For the 601/2020 conversions it's converting to/from the PAL color primaries. I haven't added images directly to this post because it's a chore to do at doom9 and anyone interested can download my test conversions as a zip file. Aside from the HDRTools conversions, where I used MeGUI to convert in 16 bit with Avisynth+, all the sample screenshots are taken from the Avisynth 2.6 script output using MPC-HC. They're the same resolution, so the same colorimetry would have been used to convert to RGB on playback. While obviously it couldn't be the correct method most of the time, it means the color differences you see in the screenshots are only due to the color conversion in the script, or in the case of conversions to/from Rec.2020 using ColorMatrix, the color differences you don't see. Sample Color Conversions.zip 3.6 MB (screenshots) Link good for 30 days. This is a more permanent link (until I run out of space). Sample Color Conversions.zip Last edited by hello_hello; 28th January 2020 at 18:14. |
28th January 2020, 18:07 | #20 | Link |
Registered User
Join Date: Mar 2011
Posts: 4,829
|
Looking back at the original ColorMatrix 2.6 thread it seems it was only intended to support conversions to/from rec.2020 and rec.709, so it's quite a shame the plugin doesn't produce an error when you try to convert between rec.2020 and rec.601.
That sort of information is something the help file should go out of it's way to make obvious, even if it's a poor substitute for producing an error message. Last edited by hello_hello; 28th January 2020 at 18:10. |
Thread Tools | Search this Thread |
Display Modes | |
|
|