I Have Published a HDR10 to HLG Converter

[FranceBB]

You should definitely support both, that would be ideal.
This problem arises from the fact that many broadcasters started producing contents early on and stick with BBC Specs.
We also have the official BBC paid LUTs over here which of course still follow the "old" BBC Specs.

[wswartzendruber]

This is comical, considering the BBC published a paper on PQ-to-HLG transcoding based on BT.2390, and BT.2390 blatantly states that SDR max white should be mapped to 75% HLG.

EDIT: It's the very first sentence on page 50.

EDIT: This paper (http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_HLG_LUT_Production_Guide.pdf) does nothing to put reference white at 50% and several things to put it at 75%.

EDIT: This BBC presentation (https://www.dutchguild.nl/31jan2017/HLG%20Overview%20-%20Dutch%20Guild%20published.pdf) establishes 75% as reference white, beginning on slide 44.

[FranceBB]

This is comical, considering the BBC published a paper on PQ-to-HLG transcoding based on BT.2390, and BT.2390 blatantly states that SDR max white should be mapped to 75% HLG.

EDIT: It's the very first sentence on page 50.

EDIT: This paper (http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_HLG_LUT_Production_Guide.pdf) does nothing to put reference white at 50% and several things to put it at 75%.

EDIT: This BBC presentation (https://www.dutchguild.nl/31jan2017/HLG%20Overview%20-%20Dutch%20Guild%20published.pdf) establishes 75% as reference white, beginning on slide 44.

Ok, this is comical indeed to say the very least.
Let me send you colorbars made with the BBC Specs ARIB STD-B67 according to the very latest matrices they sent.
I somehow suspect that they actually changed it to 75% reference white underneath everyone's nose eheheheh

[FranceBB]

Actually, it's color bars, I don't care.

https://we.tl/t-gADbIgHTle

[wswartzendruber]

I'm going to laugh if you guys are the only ones in the region broadcasting reference white at 0.5.

EDIT: My emojis didn't go through. Does this site not do Unicode?

TEST: 美國人입니다.

EDIT: It does Unicode; I wonder what happened to my emoji.

[FranceBB]

You might laugh but I might cry xD

Did you check the colorbars?

[wswartzendruber]

I did. I can't be sure what VLC is doing with the gamma there, but when played back via Direct3D, the top-left bar is grey with 71.0% sRGB luminosity. The white box below is also grey at 93.7% luminosity.

[Balling]

I did. I can't be sure what VLC is doing with the gamma there, but when played back via Direct3D, the top-left bar is grey with 71.0% sRGB luminosity. The white box below is also grey at 93.7% luminosity.

Use latest Stable Chrome, it supports scRGB for HDR. At least it works on images, like Display P3 ICC images. Latest mpv added support for hdr metadata finally. But only works if HDR in windows 11 activated desktop wide due to new WDDM 3.0 (in Chrome and in mpv). See for yourself: https://wsd.events/2019/07/13/pres/all-the-colors/samples/media-color-gamut/index.html

[wswartzendruber]

Well in the course of studying this HLG reference white issue, I've discovered that I'm not scaling linear display brightness correctly. So I'm adding a gamma correction step as called for in BT.2446 section 4.1.2.

SIDE NOTE: Last March, a bunch of stuff seemingly moved out of BT.2390 and into BT.2408, including PQ->HLQ transcoding and PQ tone mapping.

[wswartzendruber]

I really just...

http://littlefun.org/uploads/52309db3e691b236df7d6b76_736.jpg

BT.2446 says to apply gamma correction. I ended up having to do the reciprocal of that to make the picture come out right.

The next version will also feature a spec-compliant (as best I can tell) SDR downconversion system (for previewing the result).

Tone mapping is now R'G'B' instead of YRGB.

I've got some more testing to do then I'll post a build of 0.4.0.

[wswartzendruber]

Nope. Nevermind. It turns out:

1. Scaling linear RGB luminosity is a pretty good way to go for increasing brightness.
2. Color does not need to be converted to yxY before doing so (RGB0->yxY, scaling Y, and then yxY->RGB yields identical results to simply scaling RGB).

This effectively means that 0.3.0 was doing the right thing all along.

I have updated the tone mapper to work against R'G'B' instead of Y, however, as Y can produce out-of-gamut results. This should be rare, however.

[ErazorTT]

For anybody reading this thread, I am pretty sure that this conversion I'm quoting here is wrong. In that it introduces a colorshift, which is especially visible in the yellows. This can be seen by just leaving out the cube statement, then this script shouldn't do anything to the colors, however it does.

And fast script using hlg-tools:
# Avisynth+

LWLibavVideoSource("video.uhd.rmx.mkv",cachefile="video.lwi",format="YUV420P16",decoder="hevc_qsv",prefer_hw=2) # hw

# (avsresize.dll)
z_Spline36Resize(1920,1080) # FHD

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="RGBP16",colorspace_op="2020:st2084:2020:l=>rgb:st2084:2020:f",dither_type="none")

# Apply LUT (vscube.dll) input full / output full HLG
Cube("hlg-tools65.cube",fullrange=true) # max-cll 1000

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="YUV420P10",dither_type="error_diffusion")

# Enable MT
Prefetch(2)


The proper conversion is reached when the colorspace change in the back-conversion is written out as well, emphasized in bold. So the correct script would be:

# Avisynth+

LWLibavVideoSource("video.uhd.rmx.mkv",cachefile="video.lwi",format="YUV420P16",decoder="hevc_qsv",prefer_hw=2) # hw

# (avsresize.dll)
z_Spline36Resize(1920,1080) # FHD

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="RGBP16",colorspace_op="2020:st2084:2020:l=>rgb:st2084:2020:f",dither_type="none")

# Apply LUT (vscube.dll) input full / output full HLG
Cube("hlg-tools65.cube",fullrange=true) # max-cll 1000

# Converting format (avsresize.dll)
z_ConvertFormat(pixel_type="YUV420P16",colorspace_op="rgb:std-b67:2020:f=>2020ncl:std-b67:2020:l",dither_type="error_diffusion")

# Enable MT
Prefetch(2)


This does however also mean that DGPQtoHLG does something funny with colors! As DGPQtoHLG produces an output which is virtually identical in colors as the script I quoted using with the wrong colospace converison! :confused:

[ErazorTT]

Oh and have you guys (FranceBB and wswartzendruber) settled on a value for the reference white? Is it 50% or 75%?

[wswartzendruber]

I'm firmly settled that it should be 75%. FranceBB is the one in the broadcast industry, though.

[FranceBB]

Oh and have you guys (FranceBB and wswartzendruber) settled on a value for the reference white? Is it 50% or 75%?

Yes.
75% for anything "recently" done and anything you will ever do in the future (current BBC standard), 50% on old legacy stuff done at the very beginning of HDR (old NHK proposal).

I'm firmly settled that it should be 75%.

So am I now. Anything done nowadays should be 75%, no one should really be use the old NHK proposal. Not even the BBC uses it any longer and following their own specs with their own LUTs the output is always 75%, which checks out.


FranceBB is the one in the broadcast industry, though.

for now, dunno what the future holds in those perilous times ehehehehe
(actually there's little to laugh about, the economic situation around the globe is tragic and there are cuts everywhere, but so far I've been one of the lucky ones who is still here and hasn't been fired)

[ErazorTT]

Yes.
75% for anything "recently" done and anything you will ever do in the future (current BBC standard), 50% on old legacy stuff done at the very beginning of HDR (old NHK proposal).


So is this then reflected by the LUT's of your LinearTransformaion package? I'm asking because I'm having a hard time matching the LUT's of you two guys.

[FranceBB]

So is this then reflected by the LUT's of your LinearTransformaion package? I'm asking because I'm having a hard time matching the LUT's of you two guys.

Yes, at least in the latest version it should be correct.
Please also note that my LUTs all work in RGB Full Range, so if you have a YUV Limited TV Range input you have to convert to Full Range RGB first, apply the lut and then convert back to Limited TV Range YUV. By default, the Avisynth built-in converters do this. My LUTs have also been implemented in the automation software we use at work and we're developing together with other TVs across the world here at Sky, namely FFAStrans https://ffastrans.com/wp/ - https://forum.doom9.org/showthread.php?t=176655

[wswartzendruber]

I'm having a hard time matching the LUT's of you two guys.
What version of pq2hlg are you using, and what does your command line look like?

[ErazorTT]

Please also note that my LUTs all work in RGB Full Range, so if you have a YUV Limited TV Range input you have to convert to Full Range RGB first, apply the lut and then convert back to Limited TV Range YUV.

Yes, sure.

What version of pq2hlg are you using, and what does your command line look like?

I'm using version 0.3.0 this way:

# generate lut file by: .\pq2hlg.exe lut.cube --size 128 --max-cll 1000 --ref-white 203
ConvertBits(16).ConvertToPlanarRGB(matrix="2020ncl")
Cube("lut.cube",fullrange=true)
ConvertToYUV420(matrix="2020ncl")


I tried some combinations of max-cll and ref-white but I cannot match any of them to FranceBB's outcome. And I'm comparing against PQ_to_HLG_Nspec.cube from LinearTransformation 1.8.
Using FranceBB's LUT also appears to look great on my calibrated BT.1886 displays (which is weird since I would have expected that the deep blacks get slightly crushed there), but on my calibrated HDR display it appears that the blacks are too bright. I have a perfectly calibrated computer display which I set to BT.1886 and an LG OLED tv where I calibrated 1886, PQ and HLG separately. I'm calibrating using a spectrometer and a colorimeter.
Its already very late, so I will produce test clips tomorrow.

[wswartzendruber]

I think FranceBB custom-tuned his LUTs.

Those parameters into pq2hlg will produce a stock conversion LUT according to BT.2408.

What are your calibration parameters? A HLG display in a reference environment should output reference white at 203 nits and max white at 1,000 nits.

[FranceBB]

Oh no no no, don't use NSpec, that stands for NHK 50% reference white specification, no wonder the result is different. Use the other one, PQ_to_HLG.cube, which follows the BBC 75% standard.

[ErazorTT]

Those parameters into pq2hlg will produce a stock conversion LUT according to BT.2408.
Yes, exactly. And my first supposition was that this should roughly match FranceBB's LUT.


What are your calibration parameters? A HLG display in a reference environment should output reference white at 203 nits and max white at 1,000 nits.


Unfortunately my LG can only output 720nits peak in a 10% window. So I'm trying to calibrate it to a reference white of 159nits and a gamma of 1.14. (my calculation of these should be fine, since it exactly reproduces all the right values of tables 3 and 4 of the BT.2408-4 paper)
But I don't think that the perfromance on the peak white (or even reference white) matters at all since what I'm concerned about are the blacks.

Oh no no no, don't use NSpec, that stands for NHK 50% reference white specification, no wonder the result is different. Use the other one, PQ_to_HLG.cube, which follows the BBC 75% standard.
Hm, ok. But that one behaves even more different, since now the whites are also different from wswartzendruber. With the Nspec LUT, at least the whites were equal.

I should be able to produce a half minute clip in the next hours, to which I will send you two a link via PM (because of DRM and such I will not be able to post it here, so sorry for anybody else who might be reading).

[ErazorTT]

Here are screenshot of the files being played in the mpv player which is running with the icc-profile parameter set to an exact profile of my pc display.

Original PQ:
PQ.jpg (https://drive.google.com/file/d/1BwSjLdhQCNlvYSl-6U_sCWeL2oEKQjsX/view?usp=sharing)

hlg-tools LUT with max-cll of 10000 and a ref-white of 203:
HLG_hlgtools_10000_203.jpg (https://drive.google.com/file/d/1q3EO6VLYQm50jVytWdaPEGhF3lDyXNXh/view?usp=sharing)

FranceBB's PQ_to_HLG LUT:
PQ_to_HLG.jpg (https://drive.google.com/file/d/1pGFhBLk5fTuIhB_jC6JXwyHQ8iVD8LLq/view?usp=sharing)

The blacks on the last screenshot are very much brighter and the whites are slightly darker than in the original PQ clip. So that clip has less contrast than it should have.

FYI: By comparison with with my well calibrated OLED tv I found that mpv does an extraordinary good job of tonemapping PQ and HLG clips on well calibrated computer displays with whatever gamma curve, as long as it is given the display profile and as long as the clip does not use too much of the very high brightnesses possible in HDR. It is helpfull if the computer display has a wide gammut of P3 so that at least the colors do not need to be tonemapped too much.

[FranceBB]

I'm preparing a new version for you to check.
I'll send it over to you via PM for legal reasons.

[ErazorTT]

Out of interest, what size would you guys think is enough for the LUT for converisons from PQ to HLG?

[FranceBB]

Out of interest, what size would you guys think is enough for the LUT for converisons from PQ to HLG?

33x33x33 is the minimum required to be considered broadcast quality, 65x65x65 is recommended.

I would never use anything like 17x17x17.

[wswartzendruber]

Unrelated

I put some numbers into a spreadsheet and effectively demonstrated a theorem: Multiplying gamma-corrected values by a smaller amount produces the same result as multiplying linear values by a larger amount.

This makes me wonder if multiplying PQ values by a smaller amount isn't what I'm supposed to be doing to adjust an image's brightness.

[ErazorTT]

well no, actually not.
Let's say z is the linear signal, n the factor on that linear signal, and Z is the non-linear signal with N is the factor on that non-linear signal then what you mean is:

N*Z = OETF(n*z) with N=f(n) for all z

That is not generally the case. Actually that is only the case if we're talking about a pure gamma curve where OETF(x)=x^k.
But not even BT.709 is a pure gamma curve, it has the linear part in the blacks. PQ is full of additive constants and HLG does not even remotely try to be a gamma curve. So you will never have a single pair (n,N) which statisfies the above equation for all z over the whole range. Thus a multiplication on the linear side is not equal to a multiplication on the non-linear side.

So if you want to change brightness via multiplication you must do this on the linear signal and not on the non-linear signal.

[wswartzendruber]

well no, actually not.
Let's say z is the linear signal, n the factor on that linear signal, and Z is the non-linear signal with N is the factor on that non-linear signal then what you mean is:

N*Z = OETF(n*z) with N=f(n) for all z

That is not generally the case. Actually that is only the case if we're talking about a pure gamma curve where OETF(x)=x^k.
But not even BT.709 is a pure gamma curve, it has the linear part in the blacks. PQ is full of additive constants and HLG does not even remotely try to be a gamma curve. So you will never have a single pair (n,N) which statisfies the above equation for all z over the whole range. Thus a multiplication on the linear side is not equal to a multiplication on the non-linear side.

So if you want to change brightness via multiplication you must do this on the linear signal and not on the non-linear signal.
Oh. I see.

git reset --hard && git clean -fdx
:cool:

[ErazorTT]

I am very pleased but the performance of your LUT generator. That first comparison I saw with the official LUTs showed me that what you implemented is actually spot on! You did a great job!
And actually some of the other conversion methods currenlty compatible to avisynth should really not be used, I'm talking about DGPQtoHLG or HDRTools. These absolutely do not do the right thing! (I have some awefull examples if there are any doubts about that statement)

What you should do is to add a reference to your software here: http://avisynth.nl/index.php/External_filters
(I can help there, if help is needed)

And we somehow need to get rid of the ones which do not do the converison properly. Because I lost at least a couple of days looking into these two.

[wswartzendruber]

Thank you!

I'd like to wait until 0.4.0 for listing it. That's going to have arguably better tone mapping. (It will map the RGB channels independently, which doesn't allow out-of-gamut results.)

I need to get some scripts ported to Windows (PowerShell?) and then that can happen.

[ErazorTT]

No need to port scripts to windows. Just install the bash shell on windows, its in the package of git for windows:
https://gitforwindows.org/

Just install this, and you have a fully functional bash shell in windows.

[wswartzendruber]

Here's 0.4.0:

https://github.com/wswartzendruber/hlg-tools/releases/tag/release%2F0.4.0

I ported the BASH scripts over to PowerShell (for the Windows zip) so people could use them more easily.

[FranceBB]

That first comparison I saw with the official LUTs showed me that what wswartzendruber implemented is actually spot on! He did a great job!

Yep. @wswartzendruber you should have got the file I sent which has been encoded using the official BBC LUT through Avisynth, namely:

video=FFVideoSource("\\mibctvan000\Ingest\MEDIA\temp\PQ2HLG-20220105T125539Z-001\PQ2HLG\PQ.mkv")
audio=BlankClip(length=607, fps=24000, fps_denominator=1001, channels=1)
AudioDub(video, audio)

#From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Full Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion")

#From PQ to HLG with 16bit precision
Cube("C:\Program Files (x86)\AviSynth+\LUTs\1a_PQ1000_HLG_mode-nar_in-nar_out-nar_nocomp.cube", fullrange=true)

#From RGB 16bit planar Full Range to YUV422 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

and the result is close.

@ErazorTT I'm now encoding the test patters you sent me with the official BBC LUT and I'll post the 7 days link to the result here if you don't mind, so that everyone else can take a look at it too.

The idea, after that, would be to try to recreate a LUT as close as possible to the official one, so I'll send you back a new LUT file as well later in the afternoon.

[ErazorTT]

Here's 0.4.0
Awesome! I will test!

@ErazorTT I'm now encoding the test patters you sent me with the official BBC LUT and I'll post the 7 days link to the result here if you don't mind, so that everyone else can take a look at it too.

Sure, I have created those patterns with my own avisynth scripts, which are part of my calibration pattern suite. So I could actually also post those, if anybody needs PQ or HLG test patterns.

[FranceBB]

There we go:

Color Saturation Steps: https://we.tl/t-gwqgHChcyZ
Gray Steps: https://we.tl/t-q8f4O5AGpt

(link valid for 7 days, so hurry up eheheheh)


This should produce really really similar results to the official BBC LUT: https://github.com/FranceBB/LinearTransformation/blob/master/PQ_to_HLG.cube

https://i.imgur.com/e8yZ9dE.png

BBC LUT (left) - My LUT, new Implementation (right)

https://i.imgur.com/bWy2usA.png

https://i.imgur.com/j07xKjp.png

https://i.imgur.com/RJYpO6J.png

https://i.imgur.com/OK3O7uV.png

https://i.imgur.com/WWVeT3y.png

[ErazorTT]

There we go:

Awesome! After looking closely into the results I must again complement you guys!

So FranceBB's v1.9 PQ_to_HLG.cube LUT is alomst identical as the results with wswartzendruber's hlg-tools v0.3.0 and 0.4.0 with the settings of max-cll 1000 and ref-white 203. The differences are only single 8bit values apart, and that only in very view of all the test squares of the patterns.
And the best is that both are extremely simular to the output using the official LUT! So a big thumbs up there!

The big advantage of the hlg-tools generator is that the LUT can be adjusted to whatever max limunosity is needed, while in contrast the fixed LUT's all will cut abruptly at 1000nits, which might be undesirable. And if more tuning is needed, there is also the possibility of changing the ref-white (or the brightness factor).

I have to ask though, was that expected that 0.3.0 and 0.4.0 return exaclty identical results? Perhaps however the differences are below the 8bit threshold for which I tested.

[wswartzendruber]

I have to ask though, was that expected that 0.3.0 and 0.4.0 return exaclty identical results? Perhaps however the differences are below the 8bit threshold for which I tested.
For a reference white of 203 nits and a MaxCLL of 1,000 nits, they should produce identical results.

Functionally, 0.4.0 does tone mapping a bit differently, but that only matters for MaxCLL values above 1,000 nits (or if you boost the brightness so that MaxCLL gets pushed up above that internally).

Internally, there was a lot of code refactoring that went into 0.4.0 to better organize things.

EDIT: I kind of want to get a wiki going of different 4K UltraHD discs and the pq2hlg settings that should be put into it for each movie.

[FranceBB]

Awesome! After looking closely into the results I must again complement you guys!

So FranceBB's v1.9 PQ_to_HLG.cube LUT is alomst identical as the results with wswartzendruber's hlg-tools v0.3.0 and 0.4.0 with the settings of max-cll 1000 and ref-white 203. The differences are only single 8bit values apart, and that only in very view of all the test squares of the patterns.
And the best is that both are extremely simular to the output using the official LUT! So a big thumbs up there!


Nice! :D
Thank you for testing and making me work on this once more, I'm always happy to improve things. :)


The big advantage of the hlg-tools generator is that the LUT can be adjusted to whatever max limunosity is needed, while in contrast the fixed LUT's all will cut abruptly at 1000nits, which might be undesirable.

Yes, in particular, professionally wise, at Sky we have 3000 LUTs (I'm not joking) one for each nit, from 1000 nits to 4000 nits and we receive a masterfile like an Apple ProRes 4:4:4 10bit or 12bit or a Motion JPEG2000 4:4:4 12bit or a DNXHQX 4:2:2 12bit and an xml file in which the MaxCLL value is stored and from that we have a workflow that picks up that value and generates an Avisynth Script with the right LUT. Anything lower than 1000 nits uses the 1000 nits LUT, anything higher than 4000 nits uses the 4000 nits LUT 'cause we haven't got anything higher than that so far. Unfortunately I can't share those 'cause they're for internal use only (just like the BBC LUTs, they are licensed by the BBC to the individual broadcasters).
Still I gotta say, @wswartzendruber, impressive work, looks like you finally got it right this time! :D
Amazingly enough, something that you started as a hobby-like project, turned out to be so reliable that is able to produce professional-like results. Well done! ;)
I don't know why many people over the past months didn't bother to reply or help you, but I knew you would have got this on your own eventually. :D

[wswartzendruber]

I'm curious as to how pq2hlg's generated LUTs compete with the licensed ones you have.

Do you have the means+time+patience to compare pq2hlg at 4,000 nits MaxCLL to one of your licensed LUTs?

[FranceBB]

Do you have the means+time+patience to compare pq2hlg at 4,000 nits MaxCLL to one of your licensed LUTs?

Absolutely.
I think it's gonna be a useful exercise for us all. I stand on the free and open source software side, so the more open source softwares improve, the better.
Tomorrow I'm gonna be back at work and I'm gonna be busy hardsubbing the UHD HDR HLG Italian dubbed version of The Town with the Italian typesetting and it will take me a while 'cause it's always a bit of a pain in the... *Beep* to work with typesetting in HLG, especially given that Aegisub is stuck in the stone age (2014) and has no idea whatsoever of any colour curve and I have to use the colour picker to get exactly the same levels of the HLG video and then use a separate waveform monitor to cross check everything...

(You may think that official studios and professional use fancy paid HDR aware subtitling/typesetting software, but no, we're in the same tedious painful mess as the rest of the world eheheheh).

Anyway, I'm digressing, long story short, if I'm gonna have time tomorrow, I'll repeat the tests with the official 4000 nits LUT and see what happens. :)

[FranceBB]

p.s "What's the most common thing you do at work?"

Me: Wait for indexing

LMAO

https://i.imgur.com/5QTbeCy.png

(456 GB worth of mezzanine file, it's gonna take forever ehehehehe)

[wswartzendruber]

Of course, if this turns out too well, I'll probably have the BBC pissed off at me for taking away a source of revenue.

[wswartzendruber]

I've just now realized that GitHub has wikis!

https://github.com/wswartzendruber/hlg-tools/wiki/Movies-Database

I'll be filling this out as I work through my Marvel collection (really mad at DC right now).

[ErazorTT]

Anyway, I'm digressing, long story short, if I'm gonna have time tomorrow, I'll repeat the tests with the official 4000 nits LUT and see what happens. :)

Any news on that front? I'm still holding my breath :P

[FranceBB]

Any news on that front? I'm still holding my breath :P

Unfortunately I've had a bit of a rough series of days, as I've been busy with several issues (XDCAM frame properties and indexers reporting the wrong chroma location value, which screwed up some workflows I had. You can see the whole story on Avisynth Development, Avisynth+. As soon as that thing is solved/fixed, I'll spend time on the LUTs again, I promise).

[ErazorTT]

I ran tests with different lum-scales and I think there is something I do not understand.

Let's say I have a PQ clip with a maxcll of 1000, then I'm first going to try out a LUT setting maxcll of 1000 and a scale of 1.0.

Now lets say I find that a scale 2.0 would be more fitting. What happens is that everything is brightened, including the whites.
My tests showed that the whites are all pushed to the max brightness bin (1023 or 940 whatever you prefer)! So now there is loss of information since all whites are pushed together into the max brighteness.
Shouldn't maxcll be thus increased according to the scale? Thus when the scale is increased to 2 the maxcll should also be increased to 2000, to leave place for all the whites?
2000 is perhaps too much, but I'm pretty sure it should be increased to something above 1000.

Or is all that not the case and I somehow mess up the processing?

[wswartzendruber]

I ran tests with different lum-scales and I think there is something I do not understand.

Let's say I have a PQ clip with a maxcll of 1000, then I'm first going to try out a LUT setting maxcll of 1000 and a scale of 1.0.

Now lets say I find that a scale 2.0 would be more fitting. What happens is that everything is brightened, including the whites.
My tests showed that the whites are all pushed to the max brightness bin (1023 or 940 whatever you prefer)! So now there is loss of information since all whites are pushed together into the max brighteness.
Shouldn't maxcll be thus increased according to the scale? Thus when the scale is increased to 2 the maxcll should also be increased to 2000, to leave place for all the whites?
2000 is perhaps too much, but I'm pretty sure it should be increased to something above 1000.

Or is all that not the case and I somehow mess up the processing?
MaxCLL will be internally adjusted by the --lum-scale amount. So after specifying 1,000, the mapper will consider it to be 2,000 in light of --lum-scale 2.0. Tone mapping will then push everything down to 1,000 nits to prep for HLG conversion.

[FranceBB]

Ok, so, the issue doesn't seem to be closed and it's making us all talk a lot here: https://forum.doom9.org/showthread.php?t=181351 and yet the FFMpeg developers don't seem to care: https://trac.ffmpeg.org/ticket/9598 which is actually annoying me as this is a problem which affects our production environment.
That being said, I've actually temporarily worked around the thing by pretending to be using Avisynth 3.7.0 instead of 3.7.1 or 3.7.2 by using propClearAll() after every file I index, which is far from being perfect, but still...

Anyway, here are the samples encoded with the official BBC LUT 4000 nits

ColorSaturationSteps BT.2100 PQ - ColorSaturationSteps BT.2100 PQ_BBC_LUT_4000_nits: https://we.tl/t-KLyDHIyaeq

GraySteps All - GraySteps All_BBC_LUT_4000_nits: https://we.tl/t-dqPcNT4B3y

(Link valid for 7 days)

[ErazorTT]

Hm that is weird. The BBC lut produces two steps at almost exactly the same non-white brightness (red arrows in the grayBBC.png picture). These steps are apart 5% of the signal, which for PQ is really a lot in absolute terms. So that must be wrong!

Addionally, I took out my colorspace calculation sheet and found that a 4000nits white is at a 8bit value of 198 for a limited range signal. So actually in the picture the step with an orange arrow should already have been completly white, since that is at a value of 202. (Always assuming that I implemented my calcualtion sheet correctly which is quite a big if.)

The lut generator however produces anything as I expected. See picture gray_4000_1.0.png.

I also added the original grayPQ.png but with the signal values written as text so that anybody knows which step is which.

https://drive.google.com/drive/folders/1oDy8Z9iQ3fre14HLHfyNcGKpFVtHVYQT?usp=sharing