Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th August 2019, 22:37   #1  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,003
HDRTools vs DitherTools vs ColorMatrix

Would any interested parties care to look at this comparison of color conversion methods? The ColorMatrix version used is this one.

There's a bunch of screenshots to go with the tests below. I won't post them here as there's quite a few, but if you follow the link you'll find a gallery and the screenshots are properly named so as to easily see what they represent. Included is a screenshot of how much the images sizes vary after compression to PNG. The biggest changes are the DitherTools images which I suppose must be the result of dithering. The source is animation so there's large areas of single color to make the differences easy to see.
https://postimg.cc/gallery/1r9z2a760/

All testing was done using classic Avisynth on XP, although a quick look indicates nothing changes for Avisynth+.
The source video is HD, downscaled to SD, the various color conversions applied, and the screenshots taken by opening the script with MPC-HC. If I'm doing something wrong I'm happy to learn. Comments welcome!

Rec.709 To Rec.601
All conversion methods seem to agree.
HDRTools Linear looks better than HDRTools XYZ (no banding).

HDRTools XYZ
ConvertYUVtoXYZ(Color=2).ConvertXYZtoYUV(Color=3, pColor=2, OutputMode=2)
HDRTools Linear
ConvertYUVtoLinearRGB(Color=2).ConvertLinearRGBtoYUV(Color=3, OutputMode=2)
DitherTools
Dither_convert_yuv_to_rgb(matrix="709", slice=false).Dither_convert_rgb_to_yuv(matrix="601", output="YV12")
ColorMatrix
ColorMatrix(mode="Rec.709->Rec.601", clamp=0)

Rec.2020 To Rec.601
DitherTools and HDRTools Linear apparently agree.
ColorMatrix is different.
HDRTools XYZ seems borked or I don't know what I'm doing.

HDRTools XYZ
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=3, pColor=1, OutputMode=2)
HDRTools Linear
ConvertYUVtoLinearRGB(Color=1).ConvertLinearRGBtoYUV(Color=3, OutputMode=2)
DitherTools
Dither_convert_yuv_to_rgb(matrix="2020", slice=false).Dither_convert_rgb_to_yuv(matrix="601", output="YV12")
ColorMatrix
ColorMatrix(mode="Rec.2020->Rec.601", clamp=0)

Rec.2020 To 709
I'd only be guessing as nobody agrees.
DitherTools and HDRTools Linear possibly only disagree for "red".
I think ColorMatrix is wrong.
HDRTools XYZ seems borked or I don't know what I'm doing.

HDRTools XYZ
HDRToolsConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=1, OutputMode=2)
HDRTools Linear
ConvertYUVtoLinearRGB(Color=1).ConvertLinearRGBtoYUV(Color=2, OutputMode=2)
DitherTools
Dither_convert_yuv_to_rgb(matrix="2020", slice=false).Dither_convert_rgb_to_yuv(matrix="709", output="YV12")
ColorMatrix
ColorMatrix(mode="Rec.2020->Rec.709", clamp=0)


Anyone know what the difference between the HDRTools two flavours of Rec.601 might be (BT601_525 & BT601_625)?
When I do this, there's appears to be a difference, unless it's just the conversion and not a color difference,

A = last
B = ConvertYUVtoXYZ(Color=3).ConvertXYZtoYUV(Color=4, pColor=3, OutputMode=2)
C = ConvertYUVtoLinearRGB(Color=3).ConvertLinearRGBtoYUV(Color=4, OutputMode=2)
Compare(A, B)
or
Compare(A, C)
(these are labelled "HDRTools Compare A&B" and "HDRTools Compare A&C" in the gallery.

Last edited by hello_hello; 14th August 2019 at 23:03.
hello_hello is offline   Reply With Quote
Old 15th August 2019, 02:38   #2  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 107
Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=1, OutputMode=2)
is equivalent of
Code:
z_ConvertFormat(colorspace_op="2020:709:2020=>2020:709:xyz").z_ConvertFormat(colorspace_op="2020:709:xyz=>709:709:709")



and
Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=2, OutputMode=2)
should be equivalent ? of
Code:
z_ConvertFormat(colorspace_op="2020:709:709=>2020:709:xyz").z_ConvertFormat(colorspace_op="2020:709:xyz=>709:709:709")
but it seems HDRTools assumes when matrix is 2020 (Color=1) then primaries of the input are 2020 (pColor=1) too.




Code:
z_ConvertFormat(colorspace_op="709=>601",resample_filter_uv="spline36",dither_type="error_diffusion")
it's avsresize example of 709->601 conversion.



"Anyone know what the difference between the HDRTools two flavours of Rec.601 might be (BT601_525 & BT601_625)?" - https://en.wikipedia.org/wiki/Rec._6...chromaticities ; https://en.wikipedia.org/wiki/PAL#PAL_broadcast_systems ; https://en.wikipedia.org/wiki/NTSC#Variants Take a look at "lines".

Last edited by StvG; 15th August 2019 at 20:18. Reason: syntax
StvG is offline   Reply With Quote
Old 15th August 2019, 11:51   #3  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,003
StvG, Thanks for the reply.

I'm still not clear though as to whether HDRTools XYZ is getting it wrong, or if I'm doing something wrong, and why the other color conversion methods only have one flavour of rec.601 while HDRTools has two.
Your link seems to indicate the two versions of rec.601 are for converting analogue NTSC & PAL to digital. Once it's digital, isn't the colorimetry the same, and if so, would there only be one method required for converting digital rec.601 to another colorimetry etc?

Cheers.

PS. I did try avsresize but couldn't get it to work. I couldn't see any mention of it in the docs, but I assume it's not compatible with XP?

Last edited by hello_hello; 15th August 2019 at 14:41.
hello_hello is offline   Reply With Quote
Old 15th August 2019, 20:33   #4  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 107
I guess HDRTools is wrong (bug?) because the output with both pColor=1/2 is the same.

Speaking of DVD content Rec.601 (BT470_BG/ST170_M) is the right setting. Speaking of broadcasting the setting could be different - https://forum.doom9.org/showthread.php?t=133982

The other conversion methods have "240" Dither (SMPTE 240M/BT601_525), "240m" avsresize...

I didn't try avsresize on XP.
StvG is offline   Reply With Quote
Old 15th August 2019, 22:16   #5  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,003
StvG,
Would I be able to ask you to compare a rec.2020 to rec.709 conversion between HDRTools "linear", DitherTools and AVSResize, if it's not too inconvenient?
I'm assuming ColorMatrix is wrong but I'd be interested to find out if AVSResize agrees with DitherTools or HDRTools "linear" for rec.2020 to rec.709, as an indication which program is getting it right, given they're slightly different.

I haven't compared them properly, but I think the result is the same when upconverting the colors as it is for downconverting ie
rec.2020 to rec.601 vs rec.601 to rec.2020.
I think if the downconversion is correct for each program the equivalent upconversion is correct too, or if the downconversion is wrong the upconversion is also wrong, but I should check that properly at some stage.

Quote:
Originally Posted by StvG View Post
Speaking of DVD content Rec.601 (BT470_BG/ST170_M) is the right setting. Speaking of broadcasting the setting could be different - https://forum.doom9.org/showthread.php?t=133982
That's why I'm still confused as to which HDRTools rec.601 I should be using when upscaling DVD video etc, assuming they're not the same.

Cheers.

Last edited by hello_hello; 15th August 2019 at 22:29.
hello_hello is offline   Reply With Quote
Old 16th August 2019, 01:03   #6  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 107
I used your "No-Color-Conversion.png". I converted it to yv24 in the same way for all three methods so all comparison have the same upscaled chroma before the conversion. As a final step I converted yv24 to yv12 so all comparisons have the closest possible matching chroma downscaling.

HDRTools:
Code:
ImageSource("No-Color-Conversion.png")
ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion")
ConvertYUVtoLinearRGB(Color=1, OutputMode=1)
ConvertLinearRGBtoYUV()
z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion")
subtitle("HDRTools")
Dither:
Code:
ImageSource("No-Color-Conversion.png")
ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion")
Dither_convert_yuv_to_rgb(matrix="2020")
Dither_convert_rgb_to_yuv(matrix="709", output="YV24",lsb=true)
DitherPost(mode=6)
z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion")
subtitle("Dither")
avsresize:
Code:
ImageSource("No-Color-Conversion.png")
ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion")
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020=>rgb", dither_type="error_diffusion")
z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb=>709",dither_type="error_diffusion")
z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion")
subtitle("avsresize")
avsresize #1
Code:
ImageSource("No-Color-Conversion.png")
ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion")
z_ConvertFormat(pixel_type="yuv444p16",resample_filter_uv="spline36", colorspace_op="2020=>709",dither_type="error_diffusion")
z_ConvertFormat(pixel_type="yv12", resample_filter_uv="spline36", dither_type="error_diffusion")
avsresize linear:
Code:
ImageSource("No-Color-Conversion.png")
ConvertToPlanarRGB()
z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:709:709:f=>709:709:709:l", resample_filter_uv="spline36", dither_type="error_diffusion")
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:709=>rgb:linear", dither_type="error_diffusion")
z_ConvertFormat(pixel_type="yv24", colorspace_op="rgb:linear=>709:709",dither_type="error_diffusion")
z_ConvertFormat(pixel_type="yv12",resample_filter_uv="spline36",dither_type="error_diffusion")
subtitle("avsresize linear")
gallery

709->601 is done in 16-bit for all methods but HDRTools and avsresize could use float too for that conversion.

Quote:
Originally Posted by hello_hello
That's why I'm still confused as to which HDRTools rec.601 I should be using when upscaling DVD video etc, assuming they're not the same.
Use BT601_625 for DVD content.

Last edited by StvG; 16th August 2019 at 01:24. Reason: added "avsresize #1"
StvG is offline   Reply With Quote
Old 16th August 2019, 04:21   #7  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,003
Thanks for doing that.

It was very disappointing to see your result was the same each time, or close enough for me to call them the same (I was hoping you'd get different results as I did).

Obviously I can't replicate your method (avsresize no worky XP) so I tried doing it again a more traditional way, although I've looked at the various screenshots so many times today all I'm seeing is pink elephants at the moment....

I'm not sure I'm taking the same frame from the source as yesterday, and there's several frames in a row without movement and I didn't make a note of the frame number, but here's two screenshots I took today. Today the result is closer to yours, but the difference between converting to YV24 before using HDTools, or not, is noticeable. All I can assume is the difference is/was the YUV to RGB conversion, and for some reason it stood out more with rec.2020 to rec.709. I could duplicate this result, so I don't think it's a rendering problem. They're not massively different, but they are different.

It's frustrating because today the result where I used ConvertToYV24 is very close to yesterday's screenshot where I didn't use it. Hopefully that's just because it's not the same frame from the source. Maybe I need to start over, although at least now I know HDRTools XYZ has problems and ColorMatrix definitely isn't doing rec.2020 conversions correctly, so I've learned something....





Thanks again.

PS. I'm still don't really understand why HDRTools has two rec.601 flavours.

Last edited by hello_hello; 16th August 2019 at 04:37.
hello_hello is offline   Reply With Quote
Old 16th August 2019, 09:43   #8  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 107
Yes, the difference between HDRTools chroma upscale method and converttoyv24/z_convertformat is noticeable especially if the output is 8-bit (less noticeable with 16-bit output):
Code:
(#)converttoyv24()
ConvertYUVtoLinearRGB(Color=1, OutputMode=1)
ConvertLinearRGBtoYUV(outputmode=1)
ConvertBits(8)
subtitle("w/(o) converttoyv24")
gallery
StvG is offline   Reply With Quote
Old 16th August 2019, 23:33   #9  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
Quote:
Originally Posted by hello_hello View Post
PS. I'm still don't really understand why HDRTools has two rec.601 flavours.
2.6 of R-REC-BT.601-7-201103-I!!PDF-E.
__________________
My github.
jpsdr is offline   Reply With Quote
Old 16th August 2019, 23:40   #10  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 1,765
Quote:
Originally Posted by StvG View Post
I guess HDRTools is wrong (bug?) because the output with both pColor=1/2 is the same.
Answer is in my thread, pColor is just for optimizing Min/Max range in integer mode.
So
Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=2, OutputMode=2)
is not
Code:
z_ConvertFormat(colorspace_op="2020:709:709=>2020:709:xyz").z_ConvertFormat(colorspace_op="2020:709:xyz=>709:709:709")
You're juste saying that XYZ integer input range are those of BT709 (when they should be those of BT2020).
It's described in the Warning page 3 of ColorConversion.pdf provided with the binaries.

And, i clearly state in the filter doc :
Code:
Note about YV12/YV16/YV24 input/ouput:
For ConvertYUVtoxxx functions, convertion to YV24 if input is not YV24 is "quick".
If you want a true correct precise/perfect convertion, i would avise you to use
a resampler and feed directly to YV24.
For ConvertxxxtoYUV functions, if you choose an ouput different from YV24, same
remark, convertion is "quick".
If you want a true correct precise/perfect convertion, i would avise you
to ouput to YV24, and use a resampler to achieve the real output format you want.
__________________
My github.

Last edited by jpsdr; 16th August 2019 at 23:44.
jpsdr is offline   Reply With Quote
Old 17th August 2019, 00:47   #11  |  Link
StvG
Registered User
 
Join Date: Jul 2018
Posts: 107
Quote:
Originally Posted by jpsdr View Post
Answer is in my thread, pColor is just for optimizing Min/Max range in integer mode.
So
Code:
ConvertYUVtoXYZ(Color=1).ConvertXYZtoYUV(Color=2, pColor=2, OutputMode=2)
is not
Code:
z_ConvertFormat(colorspace_op="2020:709:709=>2020:709:xyz").z_ConvertFormat(colorspace_op="2020:709:xyz=>709:709:709")
You're juste saying that XYZ integer input range are those of BT709 (when they should be those of BT2020).
It's described in the Warning page 3 of ColorConversion.pdf provided with the binaries.
Thanks for the clarification.

Quote:
Originally Posted by jpsdr View Post
And, i clearly state in the filter doc :
Code:
Note about YV12/YV16/YV24 input/ouput:
For ConvertYUVtoxxx functions, convertion to YV24 if input is not YV24 is "quick".
If you want a true correct precise/perfect convertion, i would avise you to use
a resampler and feed directly to YV24.
For ConvertxxxtoYUV functions, if you choose an ouput different from YV24, same
remark, convertion is "quick".
If you want a true correct precise/perfect convertion, i would avise you
to ouput to YV24, and use a resampler to achieve the real output format you want.
Right, that's why I used the same chroma upscaling/downscaling method for the comparison in #6.
StvG is offline   Reply With Quote
Old 17th August 2019, 01:07   #12  |  Link
Groucho2004
Fossil
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Location: A wretched hive of scum and villainy
Posts: 4,474
Quote:
Originally Posted by hello_hello View Post
ColorMatrix definitely isn't doing rec.2020 conversions correctly
Have you tried using "opt = 0" with ColorMatrix?
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline   Reply With Quote
Old 17th August 2019, 01:24   #13  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,003
Quote:
Originally Posted by Groucho2004 View Post
Have you tried using "opt = 0" with ColorMatrix?
No but it doesn't seem to need it for rec.709 <-> rec.601 conversions.
I'll try it later and report back if it changes anything, but the difference between ColorMatrix and the other programs is fairly substantial. Except for Rec.709 to Rec.2020. That's the one type of Rec.2020 conversion where it's the same.
hello_hello is offline   Reply With Quote
Old 17th August 2019, 15:59   #14  |  Link
Natty
Noob
 
Join Date: Mar 2017
Posts: 219
will u do up conversions? eg 601 to 709. 709 to 2020.?
Natty is offline   Reply With Quote
Old 19th August 2019, 12:56   #15  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,003
There's a 709 to 2020 up-conversion here. https://forum.doom9.org/showthread.p...24#post1881624
Everyone seems to get 601 <-> 709 right. I haven't gone back and redone any more tests to be sure, but I think the result between colorimetries was the same in both directions. If it's correct down-converting, up-converting is okay too.

The one exception was ColorMatrix converting between 709 and 2020. It appears to down-convert incorrectly but the up-conversion is okay. It might pay not to use the ColorMatrix 2020 build though. There's tried and tested 32 and 64 bit builds for HD and SD color conversions.

It appears HDTools XYZ is working correctly but I didn't know how to use it, or possible more accurately, why the results were different. There's info in this post and the replies that follow. Keep an eye on that thread. I'm still learning....

Last edited by hello_hello; 19th August 2019 at 15:44.
hello_hello is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:00.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.