FranceBB LUT Collection [Archive] - Page 4

FranceBB

19th February 2022, 15:42

A fascinating read, but it does absolutely nothing to address why a 3D LUT won't suffice for colorspace conversion.

Exactly. @Balling, there's absolutely nothing wrong in using a LUT. You brought up the mpv implementation and I think we should differentiate the two things and then discuss Dolby Vision separately.

1) When we're dealing with dynamically changing metadata
2) when we're not
3) the special Dolby Vision Case

Now, let's start with the hard part. When we're dealing with dynamically changing metadata, the videos have been encoded with --repeat-headers but unlike HDR10, in HDR10+ we have info in the SEI message colour information changing on a scene by scene basis. (Of course, the thing is that you could technically change it with the GOP which incidentally is always 1 keyframe per second so --min-keyint 1 --keyint = framerate, so it's literally there every second and it could change every second of the authors really wanted to. For instance for 23,976 it's --keyint 24). Now, you might argue that if we want to convert it to whatever we might want to read those metadata and adjust our conversion every time according to the local maxcll info rather than using the global maxcll info. That approach would be tonemapping and it's sort of what MPV tries to accomplish, however it uses its own estimation rather than reading the info (although I haven't checked recently). Is it the de facto only correct possible way to do this? No. HDR10+ is backwards compatible with HDR10, so even a decoder which ignores the dynamically changing metadata is gonna be fine. This means that a conversion using global values rather than local values is gonna be just as fine. Hence, there's nothing wrong using a LUT. I also want to stress that none of this happens in HLG, in fact in HLG metadata is not required and it's always assumed in terms of both maxCll and reference white. What happens with HLG then? Well, there's literally no need to use the approach you depicted and using it to convert to, let's say, BT709 SDR 100 nits might even lead to unpleasant results. An example? Riviera. I've done many experiments with tonemapping and in HLG if you leave MPV with the default settings, it will try to use dynamically changing tonemapping on a pre-analysis basis. What happened with Riviera was that when a character was going walking under the light of the lightbulb, it was all incredibly lit and as it passed the light, it was all dark again. That is not ok and that's also a reason why that kind of technique shouldn't be always used, at least not for HLG which is backwards compatible with BT2020 SDR anyway and should be treated like that either by only converting the colormatrix or by using a static conversion with the proper maxcll and reference white info. Now, on to the Dolby Vision thing. First thing first, let's split the Dolby Vision in two as there are different profiles. The first one, the most common one, is the two layers Dolby Vision. Such a profile is essentially two video layers: one in HDR 10 so an H.265 3840x2160 10bit HDR PQ layer and then a metadata layer which is encoded as a 1920x1080 10bit H.265 stream. Such a stream cannot be decoded on its own as it's not really a stream, it's a metadata layer and when it's combined with the real stream, it creates a 12bit HDR PQ output with dynamically changing metadata. But guess what? When it's ignored by non Dolby Vision aware decoders, only layer 1 is decoded as a simple HDR PQ 10bit. So... Again, there's nothing wrong in using the global values and a LUT.
On to the last part, dvhe09, this is a completely new thing which is a new Dolby Vision profile. Such a profile has a completely different chroma and it has an RPU which gives information about the luma. Of course, a non Dolby Vision decoder won't be able to decode it correctly, but here's the thing: given that such a profile is supposed to use the resgaper to reshape the luma to have a common ground for HDR and SDR display, when you ignore the RPU and you get the right conversion, you can get the right chroma and... guess what? A common ground luma. Of course, it won't look pretty, but it's just as ok-ish on SDR displays, you're just gonna have the black value slightly higher on some scenes, big deal, nothing is stopping the user from modifying the thing himself.

The last thing I want to say is that you should encourage people to work in the community, not bring them down.
Your attitude is not helping anyone and I'm not talking about this topic, I'm talking about lots of other topics here on Doom9 that you replied to. Just let it go, man, live and let leave.

Balling

21st February 2022, 23:53

Mpv still does not support dynamic metadata of dolby vision. So...

>creates a 12bit HDR PQ output with dynamically changing metadata

That is called FEL. Also not supported.

>dvhe09

I think it is dvav.09 and that is avc, no?

>A fascinating read, but it does absolutely nothing to address why a 3D LUT won't suffice for colorspace conversion

It is not that it will not suffice, but it will be worse than complex SIMD by direct approach. And not bitperfect. And slower, because PQ is already optimised for 12 bit arithmetics, etc. Just like HLG. Anyway, the HW accel. in Vulkan and Directx 12/11 is even faster.

>you post here only to sow confusion and discord

Okay, sure. Whatever you say. You did read my answer here? https://forum.doom9.org/showthread.php?p=1953056#post1953056 are you still angry about it?

FranceBB

22nd February 2022, 00:14

And slower, because PQ is already optimised for 12 bit arithmetics, etc. Just like HLG.

What are you talking about?
PQ and HLG are colour curves, it doesn't make them any more feasible to 12bit than any other colour curve like LogC, Slog etc.
Besides, LUT-wise, everything is always done in RGB 16bit planar.

And not bitperfect.

You do realise that PQ to HLG is a non homomorphic transformation in linear algebra, right?
It's not 1-1 and onto, it will never be, so there's NOT such a thing as "bit perfect" no matter which algorithm you use to create your matrix. Please don't make me talk about how PQ is a real logarithmic curve so the black starts higher and HLG isn't so it starts lower similar to what you would have in an SDR curve and that when you're mapping from PQ to HLG you're gonna be mapping more input points into less output points from the input curve to output curve etc etc etc.
This topic is full of info about this, along with this one
https://forum.doom9.org/showthread.php?t=182499 and plenty others on Doom9...

Anyway, the HW accel. in Vulkan and Directx 12/11 is even faster.

So what? If Donald Graft will ever grant my wish to port Cube() to CUDA, it will be just as fast.
(Yes, it can be done and he's the perfect person to do it).

Balling

23rd February 2022, 15:32

>What are you talking about?

This: https://en.wikipedia.org/wiki/Perceptual_quantizer
4096 is everywhere as you can see (yes, it is not obvious but that is why I wrote the derivation in the wikipedia article).

Wow, ITU and SMPTE spend so many time on optimising it to 12 bit integer logic and you are not even aware of it.

HLG can also be optimised. And ICtCp HLG.

>Donald Graft will ever grant my wish to port Cube() to CUDA, it will be just as fast.

Yes, it is very well can be true. I think chrome already does it on a GPU. https://github.com/google/skia/commit/3d358fe2e533a6f25403397c36b626ec2dcc5116 (we are not using the code just yet). Also see the pull itself in skcms https://skia-review.googlesource.com/c/skcms/+/391136 and later https://skia-review.googlesource.com/c/skcms/+/388596/21

FranceBB

23rd February 2022, 18:26

Wow, ITU and SMPTE spend so many time on optimising it to 12 bit integer logic and you are not even aware of it.

HLG can also be optimised. And ICtCp HLG.

No. You're mixing oranges with apples.
12bit are used only because that's the recommendation to properly display a dynamic range of up to 10'000 nits. The reason it's mentioned is that the PQ implementation makes it so that instead of having to use 16bit to encode videos, it's possible to have 10'000 nits in as little as 12bit. On this regard, if you take HLG, 1000 nits can be easily reproduced in 10bits as you have enough precision. There's been an attempt by Sony to put HLG in 8bit by making use of the full range 255 but it was non standard and only allowed their cameras to have around 700 nits. Now here's the thing: it doesn't matter what the input bit depth is, nor what the output bit depth is, the important thing is how much precision you're using in the intermediate step.
You're talking about tonemapping. Fine. You're saying that since PQ is optimised for 12bit, we should use 12bit precision to make the calculations. This is BLATANTLY WRONG. Not only you can have more bits for better precision, but if you're doing the calculations in XYZ instead of YUV, you MUST have more than 12bits, otherwise you're gonna lose precision and your result is gonna be bad.
Generally tonemapping is done with 16bit planar or 32bit float precision and it's done like so: YUV 10bit (or 12bit) input -> 16bit YUV (or 32bit) -> Conversion to XYZ with 16bit (or 32bit) precision -> Tonemapping with 16bit (or 32bit) precision -> Conversion back to YUV with 16bit (or 32bit) precision -> Dithering to target bit depth.

Please read the PDF published by Jean Philippe Scotto di Rinaldi (also known as JPSDR) and you'll get why you need more than 12bit and also why the bit depth used to display something is very different from the one used in post processing.

wswartzendruber

23rd February 2022, 23:17

This: https://en.wikipedia.org/wiki/Perceptual_quantizer
4096 is everywhere as you can see (yes, it is not obvious but that is why I wrote the derivation in the wikipedia article).
The last person with any business editing Wikipedia articles is you. You are the reason I wasted hours trying to figure out how to convert between RGB and YCbCr. You, and your utterly wrong matrix implying that gamma correction could be applied through scalar multiplication.

Wow, ITU and SMPTE spend so many time on optimising it to 12 bit integer logic and you are not even aware of it.
Perceptual Quantizer is a curve that minimizes the human eye's ability to see banding at any bit depth. 10-bits is "good enough" for content distribution. If you need to do signal manipulation, well, you probably want 12-bits.

EDIT: If you are capturing a stream that will be manipulated, you will want a 12-bit capture at a minimum. To do the actual manipulation, you will want far more. FranceBB can correct me if I'm wrong.

HLG can also be optimised. And ICtCp HLG.
HLG is optimized (with BT.2408) to provide really good backwards compatibility with SDR such that the viewer is typically not even aware of the HDR encoding in the stream. That is what HLG is optimized for. And it can do this without metadata.

FranceBB

24th February 2022, 00:23

FranceBB can correct me if I'm wrong.

No need, you're absolutely correct. ;)

HLG is optimized (with BT.2408) to provide really good backwards compatibility with SDR such that the viewer is typically not even aware of the HDR encoding in the stream. That is what HLG is optimized for. And it can do this without metadata.

Precisely.

utterly wrong matrix implying that gamma correction could be applied through scalar multiplication.

@Balling, you see? Wswartzendruber is right. This is a problem. I noticed that you're trying to reply on almost every topic, but this is a serious community with experts in the field but also users that read those discussions and will read them for years to come. If you post something, make sure that you know what you're talking about, 'cause otherwise it's just misinformation. I read your posts several times and you write sentences as if they were absolutely true and you knew, but they're actually wrong and this can cause problems to users seeking help (and getting the wrong information) and also for developers like the poor Wswartzendruber who lost hours and hours of his time seeking your wrong hint before realising that it was wrong.
There's no shame in not fully knowing something and not replying or opening a post by saying that it should be taken with a pinch of salt etc. I myself don't know a lot of things and there are lots of topics I don't reply to. For instance, if I saw like DLT and Ferenc talking about porting a filter with inline assembly or perhaps having different results on different instructions, I wouldn't reply 'cause I wouldn't be of any use, I would just offer my help to test and that's it. That's because I can't read assembly proficiently and I would get lost in what the filter is doing, while they can. You on the other hand would probably read a few lines of assembly on Wikipedia and then try to read the code and reply to the topic. I'm not saying that it's wrong, it could be taken as a good initiative, but the fact that I know you would reply as if you understood all of it and you had the solution... well that's not ok and it can be confusing for lots of people.
Balling, I'm not telling you not to post ever again, but please, when you're about to write something, think twice or three times about how you're gonna phrase the answer, it's better for everyone, honestly.

FranceBB

25th February 2022, 20:34

Linear Transformation v2.0 Released! (https://github.com/FranceBB/LinearTransformation/releases)

- Introduced proper dvhe0509 (Dolby Vision Version 1.0 dvhe0509) support after 21 days of hard work together with algia (Livio Aloja), former Senior Sky Editor and Encoder.

Dolby Vision Version 1.0 dvhe0509 samples: https://we.tl/t-uyymdntXr6 (Link valid for 7 days)

Screenshot (left original - right BT709 SDR 100 nits):

https://i.imgur.com/2RSX7iQ.png
https://i.imgur.com/XaG01v4.png
https://i.imgur.com/d8wbwze.png
https://i.imgur.com/ewnF3yj.png
https://i.imgur.com/5z8ABGm.png
https://i.imgur.com/ZRZFgL4.png
https://i.imgur.com/WjBsrwG.png
https://i.imgur.com/1XuQzoU.png
https://i.imgur.com/IM1nuA2.png
https://i.imgur.com/PGtRZQm.png
https://i.imgur.com/cqRiSMw.png
https://i.imgur.com/EPurW3b.png
https://i.imgur.com/vZ5gIQV.png
https://i.imgur.com/4gFcdo8.png
https://i.imgur.com/ujP0HYp.png

Blue_MiSfit

25th February 2022, 22:37

How does this work? Does it basically use the RPUs to recover the original RGB from the dynamically shaped ITP and then tone map using the SDR trims in the metadata?

Also...

HLG is optimized (with BT.2408) to provide really good backwards compatibility with SDR such that the viewer is typically not even aware of the HDR encoding in the stream. That is what HLG is optimized for. And it can do this without metadata.

True, but in practice HLG with BT. 2020 will still look like crap on a legacy SDR display if the grading exceeds the gamut of BT. 709 in a meaningful way.

wswartzendruber

25th February 2022, 22:56

How does this work? Does it basically use the RPUs to recover the original RGB from the dynamically shaped ITP and then tone map using the SDR trims in the metadata?

Also...

True, but in practice HLG with BT. 2020 will still look like crap on a legacy SDR display if the grading exceeds the gamut of BT. 709 in a meaningful way.
I don't know what VLC on DirectX does, but it handles this amazingly well for everything I've thrown at it.

Blue_MiSfit

25th February 2022, 22:58

Well, sure when you've got a GPU doing smart stuff absolutely you can get pretty good results. That's also true for any PQ based HDR format.

You don't have that in the linear broadcast path where an HLG signal can and does get decoded and sent as-is to a legacy 1080p SDR BT. 709 display.

This has always been my biggest gripe with HLG. The "backwards compatible" branding is a bit disingenuous ;)

wswartzendruber

25th February 2022, 23:57

Well, it's doing wonders for me. I can watch my HLG encodes on Samsung TVs via thumbdrive. They play on my phone. My computers, family computers....

The project is doing exactly what i wanted it to do.

FranceBB

26th February 2022, 00:47

You don't have that in the linear broadcast path where an HLG signal can and does get decoded and sent as-is to a legacy 1080p SDR BT. 709 display.

To a legacy BT2020 SDR 100 nits display you mean eheheheh
I can't really send it to a BT709 one, it would be wrong.

This has always been my biggest gripe with HLG. The "backwards compatible" branding is a bit disingenuous ;)

Yes cause it's only backwards compatible with BT2020 capable TVs which can only interpret the colour matrix but not the colour curve.
As such, the FULL HD people who have a BT709 SDR display only will always get a real BT709 feed encoded from a different TX master ;)

Blue_MiSfit

26th February 2022, 01:12

Exactly. This is a critical nuance that a lot of people don't understand with HLG. I'm glad you guys do it right :)

wswartzendruber

26th February 2022, 01:58

To a legacy BT2020 SDR 100 nits display you mean eheheheh
I can't really send it to a BT709 one, it would be wrong.

Yes cause it's only backwards compatible with BT2020 capable TVs which can only interpret the colour matrix but not the colour curve.
As such, the FULL HD people who have a BT709 SDR display only will always get a real BT709 feed encoded from a different TX master ;)
Going by what VLC is able to do, I think the day will come when people with BT.709 TVs get a gamut-mapped HLG image from an external box connected via HDMI.

FranceBB

26th February 2022, 03:30

Going by what VLC is able to do, I think the day will come when people with BT.709 TVs get a gamut-mapped HLG image from an external box connected via HDMI.

Totally, that would be great 'cause encoding the same movie three times in UHD BT2020 HLG, FULL HD BT709 SDR and SD BT601 SDR really takes a lot of time. I hope one day boxes will be able to tonemap and downscale on the fly from the single UHD feed.

algia

2nd March 2022, 16:18

Hi all,
I'm working on a new version of the lut
In this version V_11 the blue and the green is better, the shades between colors are softer. but i'm not satisfied
I'll work on.
If you can try the new version, tell me about the issue.
Here the .cube and some image
https://we.tl/t-wOc9nnBeCF

rwill

2nd March 2022, 17:25

FranceBB

2nd March 2022, 17:57

algia, FranceBB, I am confused.

Do you do

<Video Decode> -> [Pixels] -> <DV Composer and Color Conversion> -> [HDR Image] -> <LUT> -> [Final Image]

or

<Video Decode> -> [Pixels] -> <LUT> -> [Final Image]

The latter.
Unfortunately there isn't an official specification and/or SDK unless you pay Dolby, therefore what we've been doing was reverse engineering to get a proper result.
By we I mean me and algia (aka Livio Aloja, here in CC).
There's nothing too interesting in the dvhe LUT, it's literally just a 65x65x65 LUT and as such it maps values to other values. The thing is that we noticed that even though luma is affected by the RPU and should be reshaped accordingly, by default it follows a pseudo hybrid curve. As such, the black always starts low at the bottom as in SDR and the white peaks at 75% with some scenes having highlights overshooting, so it's literally almost just as good as watching an HLG on a BT2020 SDR display. As such, we only mapped the chroma values, hence our LUT. We still have a really sharp (too sharp) transition between red and yellow as result of the fact that we don't have many control points in that region. What I mean by that is that the LUT is done via control points. A control point is a value which is mapped to another value and from there all the neighbors are mapped accordingly. Of course, the more control points you have, the better. Anyway, we still have the issue of a light violet component and a light blue component in some shades of white that shouldn't be there, which is why Livio is working on a new version like the one he shared in the post above.
The result is achieved by comparing side by side several sources and their official dolby decoded version to which we tried to get as close as possible.

rwill

2nd March 2022, 18:25

That is not going to work.

How [Pixels] is to be interpreted is defined in the RPU which can change each picture. [Pixels] can be really anything, its not a fixed format.

FranceBB

2nd March 2022, 22:57

That is not going to work.

How [Pixels] is to be interpreted is defined in the RPU which can change each picture. [Pixels] can be really anything, its not a fixed format.

It worked for every sample we've tried so far.
Sure, reshaping would change the luma which in turn also affects the chroma, but so far we've been focusing on reversing the chroma and we've got it right.
I shared the sample, you can try yourself too. ;)
If you have more samples, feel free to share :)

rwill

3rd March 2022, 00:42

It worked for every sample we've tried so far.
Sure, reshaping would change the luma which in turn also affects the chroma, but so far we've been focusing on reversing the chroma and we've got it right.
I shared the sample, you can try yourself too. ;)
If you have more samples, feel free to share :)

Yes most Profile 5 content in the wild I have seen has reshaped IPTc2 as [Pixels] so your LUT will
somehow match these more or less. One could create streams with ITP PQ, Bt.2020 PQ or just something
made up as a base format, reshape that (optional) and just flag these as Profile 5 though.

Regarding no specification being available...

The ETSI Composer spec is here:
https://www.etsi.org/deliver/etsi_gs/CCM/001_099/001/01.01.01_60/gs_CCM001v010101p.pdf

And there is a partial spec of the syntax here in Annex F:
https://dashif.org/docs/DASH-IF-IOP-v4.1-clean.pdf

With a Composer + Color Conversion you could re-create the HDR stored in [Pixels].
Then create HDR tone mapping LUTs to get SDR ?

Maybe quietvoid here on the forum can help. I cant, I don't want to risk my job.

quietvoid

3rd March 2022, 00:52

I've already shared the relevant info here: https://forum.doom9.org/showthread.php?p=1963203#post1963203
There's also an implementation in libplacebo, which I think was also mentioned.

Reshaping: https://code.videolan.org/videolan/libplacebo/-/blob/master/src/shaders/colorspace.c#L120
Color decoding: https://code.videolan.org/videolan/libplacebo/-/blob/master/src/shaders/colorspace.c#L411

If you still need help you can probably come around the libplacebo IRC channel on Libera.

Balling

5th March 2022, 15:53

>You, and your utterly wrong matrix implying that gamma correction could be applied through scalar multiplication.

Linear algebra postulates that nonlinear operation cannot be done like that. Thus you and only you and your knowledge in linear algebra is at fault. I will also point out that you even managed to make a mistake by a) looking into YUV instead of YCbCr article, b) look into BT.709 YUV matrix that was never in use EVER and will never be in use. c) transfer function can be 1, i.e. linear and then YCbCr is also linear.

>You, and your utterly wrong matrix implying that gamma correction could be applied through scalar multiplication.

Maybe you should have fixed that on Wikipedia. I fixed it. You did not. Because YOU KNOW NOTHING about how it all worked in 1989. I do. The fact that I copy pasted the matrix style from above from 12 years ago which you can simply see from commit history also suggests you know nothing about how wikipedia editing works and THUS YOU JUST CANNOT edit it, while I at least can.

>HLG is optimized (with BT.2408) to provide really good backwards compatibility with SDR such that the viewer is typically not even aware of the HDR encoding in the stream. That is what HLG is optimized for. And it can do this without metadata.

This is not optimisation, quite the opposite. Also it is naive to think SDR compatibility will be perfect. There are all kinds of problems, like ATR SEI is broken in ffmpeg and that scene color managment is very dumb and complex. What I meant is optimisations in code. For example, ICtCp HLG matrix has two values that are the same when you invert.

>You are the reason I wasted hours trying to figure out how to convert between RGB and YCbCr.

People waste years to do that. I did. And there is still a high chance that the scheme in H.273 is not compatible with that. I am still analysing it.

>Perceptual Quantizer is a curve that minimizes the human eye's ability to see banding at any bit depth. 10-bits is "good enough" for content distribution. If you need to do signal manipulation, well, you probably want 12-bits.

No, you need 12 bits for it. Full stop. The fact that you propose the opposite suggests you know nothing about the topic. Anyway, 16 bit PQ is also possible. What I was talking about was just math optimisations, not any derivation optimisation.

quietvoid

16th July 2022, 22:40

There shouldn't be a need for the "dvhe0509" LUT anymore for AVS users: https://github.com/Asd-g/avslibplacebo
At least if you have a GPU. It also gives correct results, contrary to the LUT.

StvG

6th August 2022, 19:02

FranceBB

6th August 2022, 21:36

You should set the correct matrix for rgb->yuv here (https://github.com/FranceBB/LinearTransformation/blob/master/LinearTransformation.avsi#L116-L126) otherwise 601 is used. Can be verified by putting propshow() after LinearTransformation.

Nicely spotted.
Fixed and I mentioned your suggestion in the release notes. :)

Linear Transformation v2.1 Released! (https://github.com/FranceBB/LinearTransformation/releases)

Changelog:
- Improved conversion back to input sampling by correctly setting the output matrix as per StvG suggestion

Balling

10th August 2022, 17:09

Well, you're welcome.
As a side note: last time I tried, ffmpeg wasn't able to read large matrices like 64x64x64 but was working fine with 32x32x32 ones and 16x16x16 ones as well. As result, some of the most accurate LUTs were not applied and resulted in an error. I don't know whether someone updated the lut3dfilter or not, but in case they didn't, just keep in mind that it might not work. If you wanna try, try to use the PQ to HLG matrix (which is definitely a 64x64x64) and check whether ffmpeg complains about it being too large or not: if it does, then they didn't update it, if it doesn't, then they updated it and I'm gonna add it as an example of how to use my LUTs. ;)

You know, you are saing it like there may be no bugs that lead to e.g. using simplied Little CMS mode that is only 49^3 max. See https://github.com/mpv-player/mpv/issues/10400

After this fix you can finally do even 512^3.

frank

10th September 2022, 17:26

Hello Francesco,
there is a problem with your 3D 65 lut BT709_to_HLG.cube.

# BT709 SDR to BT2020 HLG with reference white 75% (Full Range in - Full Range out)
You'll get limited spikes with color errors (black/blue) on bright reflections. After many tests I found it is not working on full range. I know you work with limited range in the studio. May be you mismatched something?
This Avisynth+ script worked on this lut and the resulting colors are good.
# (dgDecodeNV)
dgSource("movie.dgi") # SDR AVC

# (avsresize.dll)
z_ConvertFormat(pixel_type="RGBP16", \
colorspace_op="709:709:709:l=>rgb:709:709:l", \
chromaloc_op="left=>top_left")

# Apply LUT (vscube.dll) RGBP16
Cube("BT709_to_HLG.cube")

# (avsresize.dll)
z_ConvertFormat(pixel_type="YUV420P10", \
colorspace_op="rgb:std-b67:2020:l=>2020:std-b67:2020:l", \
chromaloc_op="top_left=>top_left")
Usually the red marked l shoud be f (full) for full range lut.
The movie was Those Magnificient Men in their Flying Machines.

You can also try STUDIOCANAL's AVC intro. The white text is inverted to black.
-------------
Dell XPS 15, Win10 Pro

frank

10th September 2022, 19:43

New test without Avisynth+.
I used NVEncC64 from rigaya. Best and fastest converter for NVIDIA.
You only can use size 65 LUTs in this program.
The importent cmd lines:
-i "movie.mkv" ^
--vpp-colorspace matrix=bt709:bt2020nc,lut3d=BT709_to_HLG.cube,range=limited:limited ^
-c hevc --profile main10 -u p5 --qvbr 25 ^
...
--chromaloc 2 --colorprim bt2020 --colormatrix bt2020nc ^
--transfer bt2020-10 --atc-sei arib-std-b67 ^
The program log shows:
Vpp Filters colorspace: cspconv(nv12 -> yuv444(16bit))
matrix:bt709->GBR
lut3d: table=BT709_to_HLG.cube
size=65, interp=tetrahedral
matrix:GBR->bt2020nc
cspconv(yuv444(16bit) -> yv12(16bit))
cspconv(yv12(16bit) -> p010)
Working perfect with your LUT. And faster, 240 fps on my notebook.
No more limited spikes, nice colors.
So I think there must be a problem with Avisynth+ and the plugins, and levels.

EDIT
Possible cause.
Depends how the 3dLUT is integrated. I think Avisynth cube reads elements as integer and not float, or has issues. So some elements for high levels destroy the colors. NVEncC64 correctly reads and converts elements as float.
All LUTs from wswartzendruber are working well with my script, but PQ to HLG only.

spoRv

11th September 2022, 17:02

Bravo Francesco, ottimo lavoro!

I'm thinking if you can make two further LUTs:
XYZ to 2020 HLG
XYZ to 2020 PQ
- unless double conversion XYZ->SDR 709 then SDR-709->HDR (HLG or PQ) is equivalent, without too much loss.

OT: how is possible to open a DCP (and other sources like HEVC 10 bit for example) and NOT getting only 8 bit?
I tried ffvideosource, directshowsource, DSS2, but always only 8 bit...

FranceBB

20th September 2022, 10:18

Hello Francesco,
there is a problem with your 3D 65 lut BT709_to_HLG.cube.

You'll get limited spikes with color errors (black/blue) on bright reflections. After many tests I found it is not working on full range. I know you work with limited range in the studio. May be you mismatched something?

I cannot reproduce.
It works just fine in full range here.

This is from the movie Lucy, a BT709 XDCAM-50 SDR 100 nits masterfile:

FFVideoSource("D:\Masterfiles\Lucy MPEG-2 FULL HD SDR BT709 YUV422 25i TFF 8bit.mxf")
propClearAll()

VideoTek(Mode="SDR")

https://i.imgur.com/67V5Hwr.png

As you can see, it peaks at around 700mV, so 235 in Limited TV range.

This is what happens when I go to HLG using my LUT which closely follows the BBC specification of their white paper as I've been reviewing it over and over and over again:

FFVideoSource("D:\Masterfiles\Lucy MPEG-2 FULL HD SDR BT709 YUV422 25i TFF 8bit.mxf")
propClearAll()

#From 4:2:0 16bit planar Narrow Range to RGB Planar 16bit Full Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full", resample_filter_uv="spline64", dither_type="error_diffusion")

#From BT709 SDR to BT2020 HDR HLG with 16bit precision
Cube("C:\Program Files (x86)\AviSynth+\LUTs\BT709_to_HLG.cube", fullrange=true)

#From RGB 16bit planar Full Range to YUV422 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

VideoTek(Mode="HLG")

https://i.imgur.com/FiMGK5P.png

As you can see, the value which was previously at 700mV (so 235, so 100% of the Limited TV Range SDR signal which corresponds to 100 nits in the BT709 SDR version) is now at 520mV, so we now have the reference white at 75% of the Limited TV Range HDR HLG signal.

In my view, there's nothing wrong with this.

I'm thinking if you can make two further LUTs:
XYZ to 2020 HLG
XYZ to 2020 PQ

No problem, but I'll need someone to test them too.
I'll put them on my "to do" list.

OT: how is possible to open a DCP (and other sources like HEVC 10 bit for example) and NOT getting only 8 bit?
I tried ffvideosource, directshowsource, DSS2, but always only 8 bit...

That is actually really weird.
Are you sure you're using the latest ffms2 and Avisynth+?
This is because on the old, legacy Avisynth 2.6, you would get 8bit planar all the time OR 16bit stacked or interleaved.
On the new Avisynth+, however, it will index high bit depth sources at their native bit depth, so if you have a 10bit planar source it will be 10bit etc.
There's one notable exception, though, and that is DCP.
If you have a DCP, it means that it's a MJPEG2000 4:4:4 XYZ 12bit, however XYZ isn't natively supported in Avisynth (despite my desperate requests to support it dating back to 2017 and which I included in my meme collection https://forum.doom9.org/showthread.php?p=1953469).
There are some people like Jean Philippe who had to work in XYZ, so what they did was to create a fake RGB64 output on his ConvertYUVtoXYZ() function with XYZ inside. Some other people like Hydra/HolyWu/Asd just recognized that XYZ isn't supported in Avisynth and therefore indexers like LWLibavVideoSource() won't output anything and report an error:

https://i.imgur.com/sq24AWJ.png

Other people like Myrsloyk instead added automatic conversion, which is the case for ffms2, which brings me to the final reply of your question: if you index an XYZ 12bit file with FFVideoSource(), it will automatically bring it to 16bit, convert it to YUV and output a YUV 16bit stream.

As you can see here, my input was a MJPEG2000 in UHD XYZ 4:4:4 12bit and after indexing with FFVideoSource() it has become a YUV 4:4:4 16bit planar:

https://i.imgur.com/6j88Bqt.png

This is the Mediainfo of the original file:

Video
ID : 2
Format : JPEG 2000
Format profile : D-Cinema 4k
Format settings, wrapping mode : Frame
Codec ID : 0D010301020C0100-0401020203010104
Duration : 1 h 15 min
Bit rate : 248 Mb/s
Width : 3 996 pixels
Height : 2 160 pixels
Display aspect ratio : 1.85:1
Frame rate : 25.000 FPS
Color space : XYZ
Chroma subsampling : 4:4:4
Bit depth : 12 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 1.147
Stream size : 130 GiB (100%)
Title : Picture Track
Color range : Full

XYZ seen as is through MPV: https://i.imgur.com/46XuSBi.jpg
XYZ indexed by FFVideoSource() in Avisynth which automatically converts to YUV 16bit: https://i.imgur.com/xi8pHH4.jpg

wswartzendruber

20th September 2022, 14:03

I cannot reproduce.

As you can see, the value which was previously at 700mV (so 235, so 100% of the Limited TV Range SDR signal which corresponds to 100 nits in the BT709 SDR version) is now at 520mV, so we now have the reference white at 75% of the Limited TV Range HDR HLG signal.

In my view, there's nothing wrong with this.
This is in agreement with BT.2408, so long as you are including SDR in a larger HLG presentation. If you are converting SDR to HLG outright, that is covered in BT.2446, which offers three methods.

frank

21st September 2022, 07:48

I used hardware decoding. I'll test again when I return from my trip. Weekend.

FranceBB

21st September 2022, 09:42

when I return from my trip. Weekend.

Vacation? Where are you chillaxing? :)
As to me, right now I wanna go nowhere. I've had enough of travelling after Amsterdam to attend IBC. Not just 'cause it was all over-expensive, but also 'cause I got caught in the Schipol airport strike and lost my flight after a 4h queue... :(
I had to get back and book another flight (with no refund) and it costed me 400€... Madness.

I've had enough.

This is in agreement with BT.2408, so long as you are including SDR in a larger HLG presentation. If you are converting SDR to HLG outright, that is covered in BT.2446, which offers three methods.

Yep, that's exactly right. The original BBC White Paper from 2017 specified mapping in which BT.709 SDR is "mapped" to HLG by making HLG act as its container. In other words, there's no "creative" intent whatsoever and it's not using any of the methods specified in ITU BT.2446 which will inevitably "try to be smart" to try to give an authentic HLG feel and will change the overall perceived contrast. In the original White Paper, in fact, the BBC suggested to use the hybrid nature of HLG to make it act as a container for BT709. In other words

"Conversions whereby SDR content is placed into an HDR container without changing the dynamic range"

in accordance with ITU BT.2390.
The reason for this method is strictly related to rights.
While when we're both producers and broadcaster we can do what we want (like with sports in which we can use scene-referred HLG conversion if we have some non HLG camera, like the small ones for replays inside the net in football games), for movies and tv series whose rights are not ours, we gotta use display-referred mapping with the method I mentioned above 'cause otherwise if we apply the methods highlighted in ITU BT.2446 the content provider might reject the conversion as it will inevitably change the content. (We had a few occasions in the past where this happened and our contents got rejected, which is why I reviewed this LUT several times in the past). :)

spoRv

21st September 2022, 21:19

So, I finally found a way to load >8bit sources with:

ffvideosource("filename.extension",enable10bithack=true)

it correctly see DCPs as 12 bit; it automatically converts XYZ color space, though.

FranceBB

21st September 2022, 22:01

I'll ask you again: which version of Avisynth are you on? Is it Avisynth 2.6.1 from 2016 or 3.7.2 from 2022?
If you're not on 3.7.2, you might wanna update https://github.com/AviSynth/AviSynthPlus/releases

And don't worry, the only OS that were dropped in the meantime are Windows 98SE and Windows 2000, so given that it's highly unlikely that you're dealing with MJPEG2000 12bit using any of them, you should really update.

The reason why I'm saying this is the line you wrote right here:

"enable10bithack=true"

This parameter here doesn't exist any longer in ffms2.
As a matter of fact it has been deprecated years ago.
The reason is that back in the days of 2.6.1, Avisynth was only working in 8bit planar, while the only high bit depth supported were 16bit stacked (Dither Tools) and interleaved (HDR Core).

Back in the days, FFVideoSource() outputted 10bit+ sources as 16bit stacked, while LWLibavVideoSource () outputted 10+ sources as 16bit interleaved.
Nowadays, they both output in planar, so 10bit planar sources are kept as 10bit planar etc.
Although there's nothing wrong in working in 16bit stacked as you're doing, you need to be careful about MSB and LSB and which filters support it etc.
Let me be very clear: when you work in stacked, you must tell each and every function you're gonna call later that you're working with that.

This is an updated build of ffms2

ffms2_6ad7738 (https://ppp.woelkli.com/s/NMk6MyD6LZR7nYH) (pass: u7Uezbh3fTHM):
- ffms2@90975ec;
- ffmpeg n5.1@1764a6887b;
- zlib 1.2.12;
- dav1d 1.0.0;
- AviSynth: set Dolby Vision RPU data in frame props.

I'm gonna quote myself:

HBD stands for "High Bit Depth" and it refers to the bit depth in which you're working with.
There are 3 possible ways to work with high bit depth in Avisynth:

- Planar
- Stacked
- Interleaved

Planar means regular high bit depth, namely you can work in 8bit, 10bit, 12bit, 14bit, 16bit planar and 32bit float. The picture is the way it is and has regular high bit depth.
In other words, 1920x1080 16bit planar is 1920x1080.

https://i.imgur.com/tsDIBBZ.png

Stacked means that MSB (most significant bit) and LSB (less significant bit) are "stacked" one on top of the other, so you end up with a picture which has double its height.
In other words, 1920x1080 16bit stacked becomes 1920x2160 as it has 8bit MSB + 8bit LSB stacked one on top of the other.

https://i.imgur.com/s4yznAb.png

Interleaved means that MSB and LSB are "interleaved" together one next to the other, so you end up with a picture which has twice its Width.
In other words, 1920x1080 16bit interleaved becomes 3840x1080 as it has 8bit MSB + 8bit LSB interleaved one next to the other.

https://i.imgur.com/s9g8ilr.png

spoRv

26th September 2022, 21:05

I'll ask you again: which version of Avisynth are you on? Is it Avisynth 2.6.1 from 2016 or 3.7.2 from 2022?
If you're not on 3.7.2, you might wanna update https://github.com/AviSynth/AviSynthPlus/releases

...

This is an updated build of ffms2
...

I run Avisynth+ 3.7.0 - by the way, "enable10bithack=true" works ONLY inside MP_Pipeline.

So, I've downloaded the updated ffms2 build: still 8 bit only, both with 10bit HEVC and 12bit DCP.

Then I've downloaded 3.7.2; the same; but now ffvideosource doesn't work inside MP_Pipeline anymore; dss2 doesn't work at all, directshowsource load everything at 640x480...

Reverting to 3.7.0 everything works as previously!

Back on topic:
shouldn't "BT2020_HLG" be "BT2100_HLG"?
And, if so, couldn't "BT2100_PQ" be "BT2084_PQ" to differentiate it more from "BT2100_HLG"?

FranceBB

26th September 2022, 22:34

I run Avisynth+ 3.7.0 - by the way, "enable10bithack=true" works ONLY inside MP_Pipeline.

Are you using Ferenc's version of MPPipeline()?

https://github.com/pinterf/MP_Pipeline/releases

So, I've downloaded the updated ffms2 build: still 8 bit only, both with 10bit HEVC and 12bit DCP.

But how...? O_O
I clearly get 4:4:4 16bit planar converted to YUV when using an updated build of ffms2 and Avisynth+...

Then I've downloaded 3.7.2; the same; but now ffvideosource doesn't work inside MP_Pipeline anymore; dss2 doesn't work at all, directshowsource load everything at 640x480...

Reverting to 3.7.0 everything works as previously!

Wait, so... changing Avisynth version affects the indexing? O_O
Are you using propclearall() right after indexing in modern version of Avisynth? (that's needed 'cause otherwise other plugins will misbehave)
This is a really really really strange behavior.
I'll test tomorrow again with MPPipeline()

shouldn't "BT2020_HLG" be "BT2100_HLG"?
And, if so, couldn't "BT2100_PQ" be "BT2084_PQ" to differentiate it more from "BT2100_HLG"?

Nah, with BT2020 I mean the SDR version of the BT2020 which is signalled in HLG streams.
Given that HLG is hybrid and backwards compatible with BT2020 SDR TVs, that is signalled as BT2020 and arib-std-b67, hence TVs which can read HLG will interpret the color curve in the transfer characteristics and will reproduce accordingly, while the ones that can't will ignore the color curve and just interpret it as BT2020 SDR, hence showing everything a bit dimmed (as the reference white is 75% instead of 100%) but still making use of the BT2020. When it comes to SMPTE2084, so PQ, on the other hand, such a color curve isn't backwards compatible with BT2020 SDR TVs 'cause it's totally logarithmic, therefore blacks are pushed higher and the waveform stays in the middle, which is why it's identified by the "BT2100" which is used for HDR.

Anyway, none of this affects anything, it's just a naming convention to make some rationale around those LUTs and inside the parameters of the function I made, but yeah, they can be changed eventually, so I'll stay open to suggestions. :)

wswartzendruber

27th September 2022, 00:07

I would offer that the BT.2020 SDR fallback capability of BT.2100 HLG is largely a mechanic of HEVC signaling. With AV1, for example, I can find no way to specify a fallback transfer function. Ergo, with AV1, BT.2100 HLG streams are signaled as that and only that.

After reviewing these specifications and how they're organized, I would have expected three different naming patterns to occur:

1. BT2020_SDR
2. BT2100_HLG
3. BT2100_PQ

frank

27th September 2022, 17:32

Vacation? Where are you chillaxing? I was driving to visit relatives in Leipzig.

You did the lut3d test with a file from your office.
Lucy MPEG-2 FULL HD SDR BT709 YUV422 25i TFF 8bit.mxfEh, you should use a normal Blu-ray with AVC YUV420.

The two known cube filters avsCube and dgCube have in Avisynth+ different issues with BT709_to_HLG.cube. The cause was the black level! If the source goes to black (0) or lower then avsCube inverts borders to white, and dgCube destroys colors, makes spikes. Also clearly visible in the vectorscope of VideoTek.

https://i.imgur.com/09Zj4z1.jpeg avsCube with BT709 to HLG

I was able to eliminate it by limiting the levels of the source.
Levels(16,1,235,17,235,coring=false)This line eliminates black 16 (0) in limited YUV space. So I think your test source .mxf never reached real black or super black.

Then I looked at your BT709_to_HLG.cube and dada...
the first filter line has small negative values. It corresponds to black 0. This generates errors in Avisynth+ in the multiplication. It must be a filter problem with floating/integer conversion. In ffmpeg and NVEncC the lut3D filter is working without issues.

I replaced this filter line with three zeroes and BT709_to_HLG.cube is working perfectly.
I don't know where the negative shift is coming from. First line must be zero, we don't want super blacks.
Please recalculate this LUT.
_________________________________________
Dell XPS 15 9510, Win10 Pro, Avisynth+ 3.7.2 (64)

FranceBB

30th September 2022, 15:12

You did the lut3d test with a file from your office.

Guilty :P

Eh, you should use a normal Blu-ray with AVC YUV420.
The two known cube filters avsCube and dgCube have in Avisynth+ different issues with BT709_to_HLG.cube. The cause was the black level! If the source goes to black (0) or lower then avsCube inverts borders to white, and dgCube destroys colors, makes spikes. Also clearly visible in the vectorscope of VideoTek.

Ouch.
I know what this is about: compression artifacts lead to overshooting.
Essentially, in consumer-tier stuff, you have values outside the limited tv range which were "created" from scratch. I.e they shouldn't be there, but they are as a result of compression.
In other words, once those values are taken into account, unexpected things can happen.
Generally in professional master files and TX Masters, we try to avoid those as much as possible 'cause having them would result in QC Fail as operators always play the files with a Tektronix Hardware Waveform Monitor at their left hand side, taking the input signal from the hardware playback port (carried via SDI).

I was able to eliminate it by limiting the levels of the source.
Levels(16,1,235,17,235,coring=false)This line eliminates black 16 (0) in limited YUV space. So I think your test source .mxf never reached real black or super black.

That's exactly right. By making sure all values are within the legal region, you basically created the exact same environment we work on as you got rid of compression overshooting. :)

p.s just FYI, what you did is really good and should be done all the time as illegal values won't be displayed anyway as all monitors are RGB Full Range and therefore if you feed a monitor with a 16-235 YUV limited signal, it will expand it to 0-255, therefore anything lower than 16 will be negative, thus not displayed (same goes for out of range whites).

Then I looked at your BT709_to_HLG.cube and dada...
the first filter line has small negative values. It corresponds to black 0. This generates errors in Avisynth+ in the multiplication. It must be a filter problem with floating/integer conversion. In ffmpeg and NVEncC the lut3D filter is working without issues.

I think the best thing to do would be to inform Donald Graft about this.
Since he rarely visits Doom9 any longer and he's staying on his own solitary ice castle ehm I mean his forum, I hope my fellow compatriot @Tormento will point him to this topic here. :)

frank

30th September 2022, 20:32

The best way is to zero out that first line.

You know a lut3d is mathematically a linear matrix, or table:
input vector x matrix = output vector
Input is full range 0..max, and output is is full 0... max.
All vector elements are greater or equal zero.
The filter works internally with a fast integer format. So negative table elements can generate overflow, depending on word length. You'll get overshuts that don't exist in the input, epecially around point 0.

FranceBB

1st October 2022, 11:36

The best way is to zero out that first line.

I finally had a bit of spare time to look at this and I can't believe what I found...

The values from the first three rows are:

-0.00086043 -0.00046653 -0.00063309
0.00905835 0.00289278 0.00107100
0.01858871 0.00607219 0.00265007

And you're absolutely right, the first row is negative.
How did it happen?
Easy.
I always create the LUTs at work in Studio RGB due to our requirements, then I perform a direct trilinear interpolation to convert the values to Full Range RGB and then post here publicly so that they can be used everywhere.

Simply put: trilinear interpolation got it wrong and mapped the Studio RGB values to non existing values in Full Range RGB.
This led to negative coefficients which mean absolutely nothing in this case and are wrong.
By zeroing the first line you've done the right thing 'cause you've removed the negative mapping. With a simple calculation, the actual value of the first row should be:

0.00819792 0.00242625 0.00043791

in other words, the first three rows are supposed to be:

0.00819792 0.00242625 0.00043791
0.00905835 0.00289278 0.00107100
0.01858871 0.00607219 0.00265007

You can replace them safely and make some tests, but they're gonna be right.
I will update the LUT on GitHub too and credit you in the pull request.

EDIT: Done - https://github.com/FranceBB/LinearTransformation/commit/8d54afc5eef07867d34b91366cfece62b7ea4c23

Thank you for pointing this out, very very very well done as I completely overlooked it.