Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th December 2021, 04:02   #61  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 413
Nope. Nevermind. It turns out:

1. Scaling linear RGB luminosity is a pretty good way to go for increasing brightness.
2. Color does not need to be converted to yxY before doing so (RGB0->yxY, scaling Y, and then yxY->RGB yields identical results to simply scaling RGB).

This effectively means that 0.3.0 was doing the right thing all along.

I have updated the tone mapper to work against R'G'B' instead of Y, however, as Y can produce out-of-gamut results. This should be rare, however.
wswartzendruber is offline   Reply With Quote
Old 3rd January 2022, 10:42   #62  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
For anybody reading this thread, I am pretty sure that this conversion I'm quoting here is wrong. In that it introduces a colorshift, which is especially visible in the yellows. This can be seen by just leaving out the cube statement, then this script shouldn't do anything to the colors, however it does.

Quote:
Originally Posted by frank View Post
And fast script using hlg-tools:
Code:
# Avisynth+

LWLibavVideoSource("video.uhd.rmx.mkv",cachefile="video.lwi",format="YUV420P16",decoder="hevc_qsv",prefer_hw=2) # hw

# (avsresize.dll)
z_Spline36Resize(1920,1080) # FHD

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="RGBP16",colorspace_op="2020:st2084:2020:l=>rgb:st2084:2020:f",dither_type="none")

# Apply LUT (vscube.dll)  input full / output full  HLG
Cube("hlg-tools65.cube",fullrange=true) # max-cll 1000

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="YUV420P10",dither_type="error_diffusion")

# Enable MT
Prefetch(2)
The proper conversion is reached when the colorspace change in the back-conversion is written out as well, emphasized in bold. So the correct script would be:

Code:
# Avisynth+

LWLibavVideoSource("video.uhd.rmx.mkv",cachefile="video.lwi",format="YUV420P16",decoder="hevc_qsv",prefer_hw=2) # hw

# (avsresize.dll)
z_Spline36Resize(1920,1080) # FHD

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="RGBP16",colorspace_op="2020:st2084:2020:l=>rgb:st2084:2020:f",dither_type="none")

# Apply LUT (vscube.dll)  input full / output full  HLG
Cube("hlg-tools65.cube",fullrange=true) # max-cll 1000

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="YUV420P16",colorspace_op="rgb:std-b67:2020:f=>2020ncl:std-b67:2020:l",dither_type="error_diffusion")

# Enable MT
Prefetch(2)
This does however also mean that DGPQtoHLG does something funny with colors! As DGPQtoHLG produces an output which is virtually identical in colors as the script I quoted using with the wrong colospace converison!

Last edited by ErazorTT; 6th January 2022 at 10:25.
ErazorTT is offline   Reply With Quote
Old 3rd January 2022, 18:45   #63  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
Oh and have you guys (FranceBB and wswartzendruber) settled on a value for the reference white? Is it 50% or 75%?
ErazorTT is offline   Reply With Quote
Old 3rd January 2022, 22:12   #64  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 413
I'm firmly settled that it should be 75%. FranceBB is the one in the broadcast industry, though.
wswartzendruber is offline   Reply With Quote
Old 4th January 2022, 10:28   #65  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,905
Quote:
Originally Posted by ErazorTT View Post
Oh and have you guys (FranceBB and wswartzendruber) settled on a value for the reference white? Is it 50% or 75%?
Yes.
75% for anything "recently" done and anything you will ever do in the future (current BBC standard), 50% on old legacy stuff done at the very beginning of HDR (old NHK proposal).

Quote:
Originally Posted by wswartzendruber View Post
I'm firmly settled that it should be 75%.
So am I now. Anything done nowadays should be 75%, no one should really be use the old NHK proposal. Not even the BBC uses it any longer and following their own specs with their own LUTs the output is always 75%, which checks out.

Quote:
Originally Posted by wswartzendruber View Post
FranceBB is the one in the broadcast industry, though.
for now, dunno what the future holds in those perilous times ehehehehe
(actually there's little to laugh about, the economic situation around the globe is tragic and there are cuts everywhere, but so far I've been one of the lucky ones who is still here and hasn't been fired)
FranceBB is offline   Reply With Quote
Old 4th January 2022, 12:07   #66  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
Quote:
Originally Posted by FranceBB View Post
Yes.
75% for anything "recently" done and anything you will ever do in the future (current BBC standard), 50% on old legacy stuff done at the very beginning of HDR (old NHK proposal).
So is this then reflected by the LUT's of your LinearTransformaion package? I'm asking because I'm having a hard time matching the LUT's of you two guys.

Last edited by ErazorTT; 4th January 2022 at 12:18.
ErazorTT is offline   Reply With Quote
Old 4th January 2022, 15:46   #67  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,905
Quote:
Originally Posted by ErazorTT View Post
So is this then reflected by the LUT's of your LinearTransformaion package? I'm asking because I'm having a hard time matching the LUT's of you two guys.
Yes, at least in the latest version it should be correct.
Please also note that my LUTs all work in RGB Full Range, so if you have a YUV Limited TV Range input you have to convert to Full Range RGB first, apply the lut and then convert back to Limited TV Range YUV. By default, the Avisynth built-in converters do this. My LUTs have also been implemented in the automation software we use at work and we're developing together with other TVs across the world here at Sky, namely FFAStrans https://ffastrans.com/wp/ - https://forum.doom9.org/showthread.php?t=176655
FranceBB is offline   Reply With Quote
Old 4th January 2022, 17:05   #68  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 413
Quote:
Originally Posted by ErazorTT View Post
I'm having a hard time matching the LUT's of you two guys.
What version of pq2hlg are you using, and what does your command line look like?
wswartzendruber is offline   Reply With Quote
Old 5th January 2022, 03:02   #69  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
Quote:
Originally Posted by FranceBB View Post
Please also note that my LUTs all work in RGB Full Range, so if you have a YUV Limited TV Range input you have to convert to Full Range RGB first, apply the lut and then convert back to Limited TV Range YUV.
Yes, sure.

Quote:
Originally Posted by wswartzendruber View Post
What version of pq2hlg are you using, and what does your command line look like?
I'm using version 0.3.0 this way:
Code:
# generate lut file by: .\pq2hlg.exe lut.cube --size 128 --max-cll 1000 --ref-white 203
ConvertBits(16).ConvertToPlanarRGB(matrix="2020ncl")
Cube("lut.cube",fullrange=true)
ConvertToYUV420(matrix="2020ncl")
I tried some combinations of max-cll and ref-white but I cannot match any of them to FranceBB's outcome. And I'm comparing against PQ_to_HLG_Nspec.cube from LinearTransformation 1.8.
Using FranceBB's LUT also appears to look great on my calibrated BT.1886 displays (which is weird since I would have expected that the deep blacks get slightly crushed there), but on my calibrated HDR display it appears that the blacks are too bright. I have a perfectly calibrated computer display which I set to BT.1886 and an LG OLED tv where I calibrated 1886, PQ and HLG separately. I'm calibrating using a spectrometer and a colorimeter.
Its already very late, so I will produce test clips tomorrow.

Last edited by ErazorTT; 5th January 2022 at 03:18.
ErazorTT is offline   Reply With Quote
Old 5th January 2022, 04:40   #70  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 413
I think FranceBB custom-tuned his LUTs.

Those parameters into pq2hlg will produce a stock conversion LUT according to BT.2408.

What are your calibration parameters? A HLG display in a reference environment should output reference white at 203 nits and max white at 1,000 nits.

Last edited by wswartzendruber; 5th January 2022 at 04:44.
wswartzendruber is offline   Reply With Quote
Old 5th January 2022, 05:23   #71  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,905
Oh no no no, don't use NSpec, that stands for NHK 50% reference white specification, no wonder the result is different. Use the other one, PQ_to_HLG.cube, which follows the BBC 75% standard.
FranceBB is offline   Reply With Quote
Old 5th January 2022, 10:54   #72  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
Quote:
Originally Posted by wswartzendruber View Post
Those parameters into pq2hlg will produce a stock conversion LUT according to BT.2408.
Yes, exactly. And my first supposition was that this should roughly match FranceBB's LUT.

Quote:
Originally Posted by wswartzendruber View Post
What are your calibration parameters? A HLG display in a reference environment should output reference white at 203 nits and max white at 1,000 nits.
Unfortunately my LG can only output 720nits peak in a 10% window. So I'm trying to calibrate it to a reference white of 159nits and a gamma of 1.14. (my calculation of these should be fine, since it exactly reproduces all the right values of tables 3 and 4 of the BT.2408-4 paper)
But I don't think that the perfromance on the peak white (or even reference white) matters at all since what I'm concerned about are the blacks.

Quote:
Originally Posted by FranceBB View Post
Oh no no no, don't use NSpec, that stands for NHK 50% reference white specification, no wonder the result is different. Use the other one, PQ_to_HLG.cube, which follows the BBC 75% standard.
Hm, ok. But that one behaves even more different, since now the whites are also different from wswartzendruber. With the Nspec LUT, at least the whites were equal.

I should be able to produce a half minute clip in the next hours, to which I will send you two a link via PM (because of DRM and such I will not be able to post it here, so sorry for anybody else who might be reading).

Last edited by ErazorTT; 5th January 2022 at 11:49.
ErazorTT is offline   Reply With Quote
Old 5th January 2022, 15:04   #73  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
Here are screenshot of the files being played in the mpv player which is running with the icc-profile parameter set to an exact profile of my pc display.

Original PQ:
PQ.jpg

hlg-tools LUT with max-cll of 10000 and a ref-white of 203:
HLG_hlgtools_10000_203.jpg

FranceBB's PQ_to_HLG LUT:
PQ_to_HLG.jpg

The blacks on the last screenshot are very much brighter and the whites are slightly darker than in the original PQ clip. So that clip has less contrast than it should have.

FYI: By comparison with with my well calibrated OLED tv I found that mpv does an extraordinary good job of tonemapping PQ and HLG clips on well calibrated computer displays with whatever gamma curve, as long as it is given the display profile and as long as the clip does not use too much of the very high brightnesses possible in HDR. It is helpfull if the computer display has a wide gammut of P3 so that at least the colors do not need to be tonemapped too much.

Last edited by ErazorTT; 5th January 2022 at 15:09.
ErazorTT is offline   Reply With Quote
Old 5th January 2022, 15:34   #74  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,905
I'm preparing a new version for you to check.
I'll send it over to you via PM for legal reasons.
FranceBB is offline   Reply With Quote
Old 5th January 2022, 18:25   #75  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
Out of interest, what size would you guys think is enough for the LUT for converisons from PQ to HLG?
ErazorTT is offline   Reply With Quote
Old 5th January 2022, 18:31   #76  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,905
Quote:
Originally Posted by ErazorTT View Post
Out of interest, what size would you guys think is enough for the LUT for converisons from PQ to HLG?
33x33x33 is the minimum required to be considered broadcast quality, 65x65x65 is recommended.

I would never use anything like 17x17x17.
FranceBB is offline   Reply With Quote
Old 5th January 2022, 20:17   #77  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 413
Unrelated

I put some numbers into a spreadsheet and effectively demonstrated a theorem: Multiplying gamma-corrected values by a smaller amount produces the same result as multiplying linear values by a larger amount.

This makes me wonder if multiplying PQ values by a smaller amount isn't what I'm supposed to be doing to adjust an image's brightness.
wswartzendruber is offline   Reply With Quote
Old 6th January 2022, 00:42   #78  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
well no, actually not.
Let's say z is the linear signal, n the factor on that linear signal, and Z is the non-linear signal with N is the factor on that non-linear signal then what you mean is:

N*Z = OETF(n*z) with N=f(n) for all z

That is not generally the case. Actually that is only the case if we're talking about a pure gamma curve where OETF(x)=x^k.
But not even BT.709 is a pure gamma curve, it has the linear part in the blacks. PQ is full of additive constants and HLG does not even remotely try to be a gamma curve. So you will never have a single pair (n,N) which statisfies the above equation for all z over the whole range. Thus a multiplication on the linear side is not equal to a multiplication on the non-linear side.

So if you want to change brightness via multiplication you must do this on the linear signal and not on the non-linear signal.

Last edited by ErazorTT; 6th January 2022 at 00:58.
ErazorTT is offline   Reply With Quote
Old 6th January 2022, 01:31   #79  |  Link
wswartzendruber
hlg-tools Maintainer
 
wswartzendruber's Avatar
 
Join Date: Feb 2008
Posts: 413
Quote:
Originally Posted by ErazorTT View Post
well no, actually not.
Let's say z is the linear signal, n the factor on that linear signal, and Z is the non-linear signal with N is the factor on that non-linear signal then what you mean is:

N*Z = OETF(n*z) with N=f(n) for all z

That is not generally the case. Actually that is only the case if we're talking about a pure gamma curve where OETF(x)=x^k.
But not even BT.709 is a pure gamma curve, it has the linear part in the blacks. PQ is full of additive constants and HLG does not even remotely try to be a gamma curve. So you will never have a single pair (n,N) which statisfies the above equation for all z over the whole range. Thus a multiplication on the linear side is not equal to a multiplication on the non-linear side.

So if you want to change brightness via multiplication you must do this on the linear signal and not on the non-linear signal.
Oh. I see.

Code:
git reset --hard && git clean -fdx
wswartzendruber is offline   Reply With Quote
Old 6th January 2022, 01:53   #80  |  Link
ErazorTT
Registered User
 
Join Date: Mar 2003
Location: Germany
Posts: 215
I am very pleased but the performance of your LUT generator. That first comparison I saw with the official LUTs showed me that what you implemented is actually spot on! You did a great job!
And actually some of the other conversion methods currenlty compatible to avisynth should really not be used, I'm talking about DGPQtoHLG or HDRTools. These absolutely do not do the right thing! (I have some awefull examples if there are any doubts about that statement)

What you should do is to add a reference to your software here: http://avisynth.nl/index.php/External_filters
(I can help there, if help is needed)

And we somehow need to get rid of the ones which do not do the converison properly. Because I lost at least a couple of days looking into these two.
ErazorTT is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:24.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.