FranceBB LUT Collection

[FranceBB]

Originally Posted by momocampo;

Hello Frank,

Thanks a lot for this! It will be useful for sure.
I wonder if you work on Sony Slog 2? I must admit it will be useful for me
Slog 2 to rec 709.
Thanks again.
Cheers.

Benjamin

I don't use Slog2 anymore, but I do have some old contents shot using that curve, so it's gonna be useful to me as well.
Slog2 is the second revision of the old version of the Sony log, which is significantly different if you compare it to Slog3 as we can see from here:




(The scale used in the graph is the same, so it's easier to spot the differences).

As you can see from the graph, Slog2 curve is pretty much the same (although slightly lower) on the low terms with a more gradual raise, while as it goes up it intersects with Slog3 and it ends up raising faster. If you take a look at this Slog2 shot, you can see the differences between the two curves:

This is the orignal Slog2 footage:



This is how it would have been if it was recorded in Slog3:



Keep in mind that this specific example is kinda "bogus" 'cause the content has been originally shot in Slog2, so a linear transformation matrix had to be applied in order to convert it to Slog3; luckily the transformation was 1 to 1, but it's not onto. The reason is that every element in Slog2 can be mapped to Slog3, however the contrary is not true: for instance, Slog2 is a 14 stop maximum curve, so the peak level is recorded right at the top of the recording range, while the S-Log3 curve is designed to go beyond 14 stops. In other words, an element that resides within the >14 area can't be mapped back, so the linear transformation is not onto which basically means that it's not an isomorphism, which is basically why I said that the example above is just to give you something else other than the graph.

Enough digression for Slog2 vs Slog3 and linear algebra lovers, so let's crack on and take a look at the graph that goes from Slog2 and then we'll take a look at some examples.

This is what I've got so far for Slog2 to BT709 SDR:



Please note that it's not just the curve, but also the Gamut as we have S-Gamut with Slog2 (it would have been S-Gamut3 with Slog3), which is not exactly so straightforward to convert to BT709 SDR as the color science inside it's slightly different from the one we use to deal with.

Let's take a look at our first example here:

Slog2:

BT709 SDR:


This is exactly what I mean with "it's not exactly so straightforward". It's very easy to get the different tonalities of the leaves/ground wrong, especially on a general purpose LUT, which is why I tried to keep it as low as possible. For instance, when I tweaked it to a scene, I've got a red-ish colors on brown tonalities, so this is the compromise I managed to get.

Let's take a look at a normal shot you are usually going to get:

Slog2:

BT709 SDR:


As you can see, skin tones are natural and chroma grading is in general kinda conservative, as well as levels.

Let's take a look at the last example here:

Slog2:

BT709 SDR:


In this case, although the overall picture appears fine, in the BT709 SDR output we've got combing in the hat. Don't get me wrong, it wasn't introduced as an artifact by the conversion, but it was on the Slog2 S-Gamut already, however while it looked like a fine gradient, it became a noticeable "artifact" once it has been converted to BT709, so in this case it might be worth doing it manually.


I'm gonna finish a few things and perhaps tweak it a little bit more and then I'm gonna upload it next week.

Cheers,
Frank.

[FranceBB]

When logarithmic curves are life, linear algebra is your passion and you can't start the day without using a matrix of linear transformation :P

[ChaosKing]

Nice and tasty results

[FranceBB]

Over the last few days I've been taking pictures of pretty much everything in LogC.
I'm gonna upload the LUT in a matter of hours, but right now I uploaded a few images here 'cause it's "educative" to show how linear transformations performed by the matrix created (my LUT) are ok, however there's always margin for manual adjustment: the so called "artistic-choice" which cannot be included in a LUT by default as it would produce a good result on some occasions and bad results on some others.

(Examples are in BT709 SDR as the sensor I used is very bad and with a ridiculously low amount of stops).

Example1:

Log-C:


Linear Transformation BT709 SDR:


Artistic Choice BT709 SDR:



Log-C:


Linear Transformation BT709 SDR:


Artistic Choice BT709 SDR:



Log-C:


Linear Transformation BT709 SDR:


Artistic Choice BT709 SDR:



As you can see, the matrix cannot tell whether it's ok to raise the highlights or not 'cause in some occasions it is, in some others it's not, so it just sticks with a rather conservative linear transformation; after all, it is my personal choice to raise certain highlights.
For instance, the cat should be black 'cause it was completely black in the reality, but I wanted to raise the grass a little bit and give a bit more life to the fur at the expense of making it "dirty" a little bit.
Anyway, those are just examples, they are shot with a ridiculously crappy sensor and I literally spent 2 minutes to grade them, however I do believe that they were good examples of what I mean by "artistic adjustment".

This is the LUT I'm gonna upload this evening:

[FranceBB]

Have you ever noticed that many of the latest music videos have been originally shot in Log? I just noticed that Rita Ora - Lonely Together, Ed Sheeran - Beautiful People and Ellie Goulding - Sixteen were all Log; you can tell by the way they have been graded and brought to BT709 linear with a matrix of linear transformation (LUT).

This is a frame from "Ed Sheeran - Beautiful People":

[FranceBB]

Here I am again, ready to play with matrices of linear transformation as I have a little bit more spare time.
A few days ago, a user, Gser received a DCP package in M-JPEG2000 which contained an XYZ 4:4:4 12bit video.
He was trying to convert it to RGB and then YUV to encode it and I got interested in the subject as the very first thing I thought about was "we can do it with a matrix of linear transformation!"
Unfortunately, though, I didn't really have time to spend on that as I was both working and studying for my Calculus III exam (which I just passed; hurray! ).
Anyway, in the mean time another encoder popped up to help Gser and made his own matrix using Davinci Resolve; the encoder who made the matrix is Bradley Greer, founder of Kyotocolor and a very famous colorist who worked for several movie productions and who has been invited to speak on behalf of Kodak, Sony, and Avid multiple times...
I know what you are thinking: "C'mon Frank, I know you like matrices, but can't you just give us a link to his matrix and stop?"
Well, if you just wanna stop reading and you wanna use his LUT, you can download it from this website: Link
However, there are some interesting findings I made for what follows...
As an engineer who truly loves Linear Algebra and who likes math, I can't stop thinking about matrices of linear transformation and although there was already a matrix made by a really talented encoder, I really wanted to make my own matrix in order to see how far I was going to get, so first things first.
First of all, a DCI it's actually relatively easier to handle as we're gonna make our conversion in linear gamma, specifically from 2.60 to 1.90, as we're gonna have to bring it down to make use of it in our final BT709 target. The second think we need to do of course is to convert the XYZ colorspace. X, Y and Z are extrapolations of RGB created mathematically to avoid negative numbers and are called Tristimulus values: Y means luma (as we all know), Z is somewhat equal to blue, and X is a mix of cone response curves chosen to be orthogonal to the luma and non-negative.
In my attempt to make a decent-looking matrix, I used the monitor I have here at work to test it and I'm not very satisfied with the result as I think that the black level is too low and the white level has been compressed so much to avoid clipping that the final look actually greatly differs from what the original intent was, however it doesn't look bad and I think it can be used in production.

Anyway, before trying to spot the conversion differences, let's take a look at the graph of the curves first:




If you remember how logaritmic curves behave, you can see that they're completely different.
Logaritmic curves generally use to start higher than the linear BT709 one and then they generally curve 'till they get almost straight as if they were y=x+3 in the first quadrant, crossing the linear BT709 curve two times, at the beginning after the non-so-black values are displayed and at the end right when the linear BT709 one gets a correction in order to avoid to get out of range values, therefore applying a sort of soft-clipping, which someone likes to call "knee". In the DCI case, however, there are no points in common whatsoever between the two curves and the DCI one is always above the BT709 one. We kinda expected this as we knew that it was going to be higher, but the interesting thing is that the difference between the two curves isn't constant as we expected in a linear scenario, on the contrary, it varies as there's less difference in the mid-tones than there's in the black-ish tones. As to the white tones, the DCI curve quickly peaks to high values while the BT709 one is smoothed to make sure to avoid out of range values (i.e clipped whites and so on), so it diverges but we expected this and it makes sense.
Alright, let's take a look at the comparison, shall we?


Original XYZ footage 1:



BT709 Bradley Greer matrix of linear transformation:



BT709 FranceBB matrix of linear transformation:



Original XYZ footage 2:



BT709 Bradley Greer matrix of linear transformation:



BT709 FranceBB matrix of linear transformation:




We can see how the Bradley Greer matrix has both luma and chroma right to the right level and it offers a more natural representation of how the scene should look like. As to the matrix I used, blacks are slightly too low I think and in order to have whites in range I ended up scaling everything down and getting perhaps too smoothed values. I'm gonna post the matrix by tomorrow, so feel free to use it anyway if you prefer it over the Bradley Greer one and you like a darker, more contrast-looking output with kinda saturated colors, or just use the Bradley one as many other people are.
When it comes to these things, it's really up to you and your eyes as both matrices are mathematically correct.

Temporary Link (I'll add it to the main link inside the folder soon): Link

Cheers,
Frank.

[FranceBB]

A few days ago I was explaining how HLG is useful but it looks different for people who do have a BT2020 SDR capable TV but not an HLG HDR capable one.

I strongly invite you to read it here: Link

As a follow up, I decided to do a few more comparisons to show you how it looks different across different TVs.

On the left hand side, you'll find the BT709 SDR footage.
In the middle, you'll find the HLG version as it should look like for people who have an HLG Capable TV.
On the right hand side, you'll find the HLG version as it should look like for people who have a BT2020 SDR capable TV.















Link to sample files:

BT709 SDR & BT2020 HLG HDR: https://we.tl/t-cXvmayyONR
Comparison Video (BT709 SDR - HLG BT2020 - BT2020 SDR): https://we.tl/t-NkikWq8AwO

[FranceBB]

Hi there,
it's been a while.
Today I had to encode a few contents shot with a Nikon camera in Z-Log.
I gotta say that I was surprised 'cause I totally ignored the fact that Z-Log even existed, however it seems that it's really a thing and it has been developed around mid-2018 or so.
Anyway, since I didn't really want to get into HDR as I have no flipping clue about how many stops (nits) Nikon cameras have, I decided to go to Linear BT709 SDR 100 nits as target for my matrix of linear transformation.
To be fair, even though I don't know the exact amount of stops that those cameras have, from the tests I made, I don't really think that there are enough stops to get a decent amount of nits that would justify going to HDR (either HLG or PQ), so it's better to stick with BT709 SDR.

I'm about to show you some Z-Log 10bit footages before and after applying my matrix of linear transformation (LUT):

Z-Log Original 1



Linear Transformation to BT709 1



Z-Log Original 2



Linear Transformation to BT709 2



Z-Log Original 3



Linear Transformation to BT709 3




I know what you're thinking "you generally plot the input and output curves in a graph and show the differences between the input and the output, why didn't you do it this time?".
Well, mathematical laziness at its finest I say, so I'm asking you to fire up your brain and picture a 2D plane in your mind.



You should be familiar by now with the classic BT709 curve, so let's focus now on the Z-Log one.
Well, Z-Log basically has the black level that starts higher than normal, however it quickly goes up with its highlights to the point it doesn't even look like a logarithmic curve anymore at the very top. This is because its main focus is to prioritize blacks and offer a better way to deal with situations that would otherwise be difficult to deal with and would get blacks crashed. So, if you're ever going to shoot in Z-Log, keep in mind that although it does a fine job in retaining the details of the blacks, it won't help you with highlights. In other words, if you are using Z-Log, don't expect to have skies not clipped out and a very wide dynamic range, it's not what it's meant for! The reason for that, I suppose, is that sensors used by Nikon cameras don't have many stops and although they do a relatively fine job with dark areas, they are not able to get bright areas and highlights.

I'm gonna add the .LUT file called "Z-Log_to_BT709.cube" tomorrow.
As a plus, this time it's gonna be compatible not just with Avisynth, but also with ffmpeg.

Cheers,
Frank.

[anatol2002]

Hi Frank
I tried to access the LUT's link, but it did not work for. Is it broken?

Actually I am looking for HLG to PQ conversion. Do u have that LUT?

Thanks,
Anatol

[FranceBB]

Originally Posted by anatol2002

Hi Frank
I tried to access the LUT's link, but it did not work for. Is it broken?

Actually I am looking for HLG to PQ conversion. Do u have that LUT?

Thanks,
Anatol

It works fine on my end. https://i.imgur.com/ZqYl5I9.png
Please note that Mega is blocked by some ISP via DNS, so you might wanna try to change your DNS to the google ones or open dns ones or cloudflare ones.

FYI
Google DNS are 8.8.8.8, 8.8.4.4
Cloudflare one is 1.1.1.1

As to the LUTs, I did make a PQ to HLG but I didn't make an HLG to PQ. If you need it, I might take a look at that and make it. Ideally I would make a 1000 nits HLG to 1000 nits PQ, if that's alright. Would you be willing to test it if I send you a LUT?

[anatol2002]

Hi Frank
Thanks for your advice - I succeeded to download the LUT's files from another connection.

And thanks again - I will surely test the HLG to PQ LUT! Much appreciated!

Anatol

[FranceBB]

Ok, perfect. The reason why I use Mega as a host is because it's fast and I have 50 GB of space free of charge, but it's blocked in some countries.

Anyway I'm gonna be out of office 'till Christmas, so I won't be able to work on the LUT (heck I don't even have a 1000 nits capable display at home T_T), but I will as soon as I get back to work.

For those curious, I'm actually eating a delicious half roast chicken with a pig-in-blanket, roast potatoes, gingerbread stuff, red wine jus and of course two Yorkshire puddings.
As you probably figured out, I'm currently in the UK as there's hardly anything more British than a Sunday Roast. https://i.imgur.com/k0N36Cg.jpg

[anatol2002]

Merry Christmas Frank!
Have a great time in UK!

P.S. - is it a roast chicken or broadcast chicken

[FranceBB]

Alright, before getting to HLG to PQ, I wanted to review my HDR PQ to HLG matrix and add a new one.
I gotta say though that making a PQ to HLG matrix is a bit tricky 'cause it's NOT an homomorphism 'cause the PQ input is so much wider than the HLG output, so I can't match each and every point of the input to a single point of the output (not 1-1) but every member of the input is mapped to all the output (so it is onto) but definitely not an homomorphism. In other words, you can't go from PQ to HLG without losing something.
A matrix of linear transformation is just... you know... a matrix, which basically means that can map some points of the input to some other points of the output, which is exactly what I'm doing here for PQ to HLG, however PQ can span from 0 to 10'000 nits while HLG can only span 0 to 1000 nits which is far less than PQ. The thing is that I can't make the LUT change on a scene-by-scene basis or anything, it's static, it's just a "mapping", therefore there's no magical matrix that works for any content in the world.
I started with the BBC White Paper as a good reference point for doing just this as I did last time for the other LUT, however I was trying to adapt this in order to be good with any kind of contents with many more nits than 1000 nits without brutally clipping them in my attempt to bring them down to 1000 in HLG, but I wasn't satisfied. The results were poor and my matrix only behaved correctly on not-so-bright contents. Fortunately, I noticed that NHK published a similar reference paper for HDR PQ to HLG which has been translated in English (thanks God as I don't really understand Japanese other than a few words I learned from watching anime in my younger age) and it was far different from the BBC one. After trying a few options, the curve I got has a much greater slope compared to what the BBC states and I gave a lot of importance to the black level. I sacrificed whites, but that's an acceptable compromise; after all, I gotta map everything to just 1000 nits, so I don't really have much choice, but I tried not to brutally clip whites, but to smooth them out a bit instead, although I do also sort of "clip" the very very high ones.



As you can see, blacks are preserved, but whites are smoothed out before eventually clipping them.



If you take a look at the nits of this HDR PQ test image, for instance, you can see that there's a lot in the low area with very low nits: the background is at about 200 nits, the sky is pretty low as it's evening so it's slightly over 500 nits at like 520 nits and the skin is between 240 to 500 nits, depending on the part of the face we're actually analyzing as some parts gets more photons from the light emitting source. Lastly, the power source peaks at 1000 nits and it's therefore clipped out as the PQ test file I used has just 1000 nits. Please note that the picture you're seeing is a PQ on an HDR Waveform Scope, which basically means that you're already seeing how it's supposed to look like, but for the sake of references, here's how you would see it if you open the raw pixel file without taking into account the curve (i.e NOT how is supposed to look):



From here we can see that the windows at the center pops up as it peaks at 1000 nits as well as each and every "candle" (heck, there are no candles anymore, just AC light bulbs getting a mono-phase current and emitting photons ).
Walls, chairs made out of wood and other things are at about 400 nits and so on.




Alright, enough waveforms, now I'm gonna post a few screenshot comparisons about HDR PQ (on the left) and HDR HLG (on the right).
As you can see, I couldn't quite match the same tones of the input, but I preserved as many things as possible while mapping.
I gotta say that all the test files I used have about 1300 nits but nothing more, so it was relatively easier to re-map to 1000 nits, but it should be fine for contents slightly higher than 1300 nits (although I didn't test it).

HDR PQ (left) vs HDR HLG (right):

[FranceBB]

I added it to the list with the name "PQ_to_HLG_Nspec.cube" which stands for "NHK specifications based" + of course my modifications.
Now I'm finally ready to work on your LUT, which is the inverse: from HLG to PQ, which should be fairly easier as this time I can really map each and every point of the input space (HLG) to each and every point of the output space (PQ) without losing quality as HLG is less wide than PQ.
I decided to set them to scale in a linear fashion: in other words, if you have an HLG at 250 nits you'll get a PQ at 250 nits, if you have an HLG at 400 nits you'll get a PQ at 400 nits and if you have an HLG of 1000 nits you'll get a PQ at 1000 nits.
One of the common mistakes people make is to map the maximum of a certain input curve to the maximum of the output curve, 'cause generally when you're dealing with matrices of linear transformation and you have an input space and an output space, you tend to do it that way.
Think about BT709 to HLG: many people think that the correct way to do it is to map the whole linear BT709 to HLG thus getting the peak brightness of BT709 SDR to 1000 nits in HLG HDR, but that's a very doggy approach. As a matter of fact, in HLG to PQ I've done the same: they're both mapped in a linear fashioned so that instead of having the peak HLG be mapped to the peak PQ, they're mapped to get equal results on the output; in other words, instead of mapping the 1000 nits HLG to 10'000 nits PQ, I mapped it to 1000 nits PQ.
That's the way it should be done anyway in my opinion.
I'm doing it in this very moment, it should be ready soon!

Originally Posted by anatol2002

Merry Christmas Frank!
Have a great time in UK!

P.S. - is it a roast chicken or broadcast chicken

I'm afraid that the canteen we have at work is way worse.

[FranceBB]

Alright, your LUT is ready now.
I mapped it out to 1000 nits in PQ in order to have the same amount of nits in both the resulting HDR, rather than remapping the maximum of HLG to the maximum of PQ.
This is the resulting curve:





I know what you're thinking "hold on a second! You said that you were going to map it to 1000 nits but the resulting PQ curve is left unclipped 'till 10000 nits, why is that?"
Well, that's not gonna affect the results at all.
Yes, it's true, I left the curve unclipped 'till 10'000 nits, but since I mapped every point of the HLG to its reciprocal of PQ 'till 1000 nits, those other 9000 nits that are there left are NOT gonna be used at all.
I.e there isn't going to be any data in there.
As an example ('cause mine are just words), I'm gonna show you a frame.



This is how the waveform would look like if I clip the resulting curve of the matrix to 1000 nits by forcing it to be 1000 nits directly from the matrix:




Now, look at what happens when I use the very same curve but I leave it unclipped to 10'000 nits:




Nothing, exactly. This is because although I left the curve unclipped to 10'000 nits, I mapped the resulting points in a way that NOTHING is gonna get mapped over 1000 nits.

Now, for the sake of knowledge, this is how it would be if I had to map the maximum of the HLG to the maximum of the PQ (i.e 10'000 nits):



See the curve? See the waveform? Now I have a completely different slope and a completely different peak as points that were lower will be mapped to an higher point, but I don't really like it, I think it's wrong, 'cause essentially you're making something out of nowhere for apparently no reason at all, so...
It would be like when I see broadcasters mapping the 100 nits of the linear BT709 to the maximum of the HLG so that I get a "nice" 1000 nits of white and completely clipped out sky -.-
So, that's why I decided to take this other route.
I didn't actually manage to get it 100% the same, but I think I'm fairly close and it's gonna be fine for your needs.

A few screenshots (on the left the original HDR HLG 1000 nits and on the right the HDR PQ 1000 nits output from the matrix of linear transformation):













As you can see, the very low dark levels are slightly different, but that it's really gonna depend on how low your monitor can go, so it's gonna be tricky to check, especially if you have an LCD, while you'll be perfectly able to see it if you have an OLED. This is because in HLG the low end of the curve resembles a sort of linear SDR one and the mid-high end part is more log-prone (so more inclined to the HDR), hence the reason why it's called Hybrid Log Gamma. When it's mapped to PQ, though, which is purely meant for HDR, the low end part is gonna be mapped to several completely different points that are way different from the HLG one and if your monitor isn't very good at displaying low lights it may seem as if they're not there, but really they are, 'cause I doubled checked with my waveform monitor. This is particularly "funny" 'cause I can see details in the shadows in the HLG one that get mapped to points in PQ that make them invisible to my monitor at home but they're still there and I can see them in the waveform (and I probably will be able to see them at work in one of the monitor we have there).

You should find the new LUT in the folder (Mega) in the afternoon.
I look forward for your reply.

Cheers,
Frank.

[anatol2002]

Hi Frank
Thanks for prompt response!
Unfortunately I did not time to get to test your new LUT. Just overloaded with lots of other stuff.
Hope that will get some time in couple of days.

Happy New Year!

Anatol

[FranceBB]

Originally Posted by anatol2002

Unfortunately I did not time to get to test your new LUT. Just overloaded with lots of other stuff.
Hope that will get some time in couple of days.

Happy New Year!

Sure, no problem, happy new year!

[FranceBB]

Alright, so I've come across some Panasonic Cameras that shoot in V-Log.
I gotta say that I do not like Panasonic Cameras nor V-Log, however I had to quickly do something to get something out of this.
Please note that this is a very rough linear transformation from V-Log to Linear BT709 SDR.
Unfortunately although I've got the Luma right, I don't quite seem to understand the internal color science of Panasonic that much, that's why I didn't quite get the tone I wanted, anyway if you wanna use it feel free to use it (it will be added to the link in the first page soon). Temporary Link: Link

[hydra3333]

thank you