FranceBB LUT Collection

[DavidK_]

Originally Posted by FranceBB

I did, but it's been 18 days since I left the office due to coronavirus and it seems like there's no plan for me to get back there AT LEAST 'till April. Right now I'm sitting at home waiting for this whole mess to calm down. Making a LUT with a crappy commercial 4K TV and monitor wouldn't be wise when I have a 50'000$ Sony monitor at work. Anyway, since this whole think is like a surreal situation, if you want me to try on my commercial TV and monitor, I will, it's just that I don't know how good the output will be.

WOW! Glad you are well but wow. I'm in US and we're still figuring out how to react but having some closures and cancellations too.

If you could go ahead and try from home if you can. It would be helpful and much appreciated. And it would give you something to occupy your time - ha ha? No matter what stay well.

[frank]

Did you try both the two .cube files for PQ to BT709? Do they have the same problem? 'cause that's really weird...

Yes, I tested both. But the v2 works better.
Is there another Cube plugin available?

[frank]

Ok, I've got it!!

The color must not be linearized. The example in Donald's Cube plugin is wrong.
Here the correct Avisynth+ script:

LWLibavVideoSource(...)
z_ConvertFormat(pixel_type="RGBP16",colorspace_op="2020ncl:st2084:2020:l=>rgb:st2084:2020:f",dither_type="none")
Cube("C:\Program Files (x86)\AviSynth+\LUT\PQ_to_BT709_v1.cube", cpu=2, fullrange=true)
z_ConvertFormat(pixel_type="YV12",colorspace_op="rgb:709:709:f=>709:709:709:l",dither_type="ordered")

v1 has the right contrast.
v2 has lower contrast.

[FranceBB]

I'm glad it finally worked out for you.

WOW! Glad you are well but wow. I'm in US and we're still figuring out how to react but having some closures and cancellations too.

It's gonna be a mess, mark my word. People have been underestimating this so have some governments and they won't realize that until it's too late.
Look what happened in Italy! They are on their knees! And it's not even one of the worse European Health Service, on the contrary, it's kinda "ok" but it's been overwhelmed...
In the UK the government pushed 50 million pound in the research of a vaccine and their plan seems to be to contain and then delay the epidemic 'till this summer when hopefully there will be a cure (the vaccine) and hospitals will be slightly less busy because of the end of seasonal flu. Yet, it's March, it's a long way 'till June, so I truly hope they'll find the vaccine way sooner.
Please, please please, people, don't underestimate this, don't go out if it's not necessary.

[DavidK_]

Originally Posted by FranceBB

I'm glad it finally worked out for you.



It's gonna be a mess, mark my word. People have been underestimating this so have some governments and they won't realize that until it's too late.
Look what happened in Italy! They are on their knees! And it's not even one of the worse European Health Service, on the contrary, it's kinda "ok" but it's been overwhelmed...
In the UK the government pushed 50 million pound in the research of a vaccine and their plan seems to be to contain and then delay the epidemic 'till this summer when hopefully there will be a cure (the vaccine) and hospitals will be slightly less busy because of the end of seasonal flu. Yet, it's March, it's a long way 'till June, so I truly hope they'll find the vaccine way sooner.
Please, please please, people, don't underestimate this, don't go out if it's not necessary.

Did you have a chance to make the adjustments to the LUT? I didn't see a new link but I'm still not use to this forum fully.




As for Corona - we are now on partial lockdown here in the US! Store shelfs are bare with everyone hoarding food - especially toilet paper.

But if you had a chance to do that last small adjustment on the LUT let me know and send me a link here!

[wonkey_monkey]

with everyone hoarding food - especially toilet paper.

You eat weird stuff.

[Andouille]

Originally Posted by wonkey_monkey

You eat weird stuff.

Fibers are good for digestive system.

[DavidK_]

Originally Posted by Andouille

Fibers are good for digestive system.

And really tasty with a little hot sauce.

[masterkivat]

Hey FranceBB, thank you very much for your LUT collection! I'll give a try someday later on these
I wanna ask you something: while I understand the current situation with HDR and such, do you by any change know where to download or have plans to make an LUT file that converts BT.601 to BT.709? I didn't find anything related to this kind of conversion using an LUT file...

[FranceBB]

Well, to be fair I've been thinking about it for a while 'cause if I introduce linear BT601 and I get that right I could potentially get all the matrices that already have linear BT709 as either input or output and modify them to make them BT601 compatible.
The only "bad" thing with Linear BT601 is that there are actually two of them: one for PAL and one for NTSC which would double the work.
I would also like to cross check my results since I don't really trust myself getting the calculations right all the time...
I know that I can't use color matrix 'cause it only converts the matrix and not the primaries... Perhaps I can compare it with the one from "plugin_JPSDR" made by Jean Philippe as he probably got it right.

I might try in the next few days...

[masterkivat]

Originally Posted by FranceBB

Well, to be fair I've been thinking about it for a while 'cause if I introduce linear BT601 and I get that right I could potentially get all the matrices that already have linear BT709 as either input or output and modify them to make them BT601 compatible.
The only "bad" thing with Linear BT601 is that there are actually two of them: one for PAL and one for NTSC which would double the work.
I would also like to cross check my results since I don't really trust myself getting the calculations right all the time...
I know that I can't use color matrix 'cause it only converts the matrix and not the primaries... Perhaps I can compare it with the one from "plugin_JPSDR" made by Jean Philippe as he probably got it right.

I might try in the next few days...

hmmm I see... But yeah, man... Take your time on these, I really hope you could provide these LUT files in the future, this kind of conversion would help me a lot with some personal projects that require such conversions
Thanks for answering me!

[FranceBB]

Ok, so I made a few tests.
I started with the PAL version of BT601 as you can see:


The curve is exactly the same and there's a reason: the difference between those two is so subtle that it can't really be noticed on this scale...

To test this, I indexed a BT709 file and I converted it to BT601 in three different ways:

1st Indexing only (my source file):

FFVideoSource("I:\temp\sample.ts")
Spline64Resize(848, 480)

2nd my LUT:

FFVideoSource("I:\temp\sample.ts")
Spline64Resize(848, 480)
ConvertBits(16)
ConvertToPlanarRGB()
Cube("C:\Programmi\AviSynth+\LUTs\test.cube", fullrange=true)

3rd the old good colormatrix:

FFVideoSource("I:\temp\sample.ts")
Spline64Resize(848, 480)
ColorMatrix(mode="Rec.709->Rec.601", interlaced=false, threads=0, thrdmthd=0)

4th Matrix from HDR Core:

FFVideoSource("I:\temp\sample.ts")
Spline64Resize(848, 480)
ConvertBits(16)
ConvertToDoubleWidth()
Matrix(from=709, to=601, rg=1.0, gg=1.0, bg=1.0, a=16, b=235, ao=16, bo=235, bitdepth=16)
ConvertFromDoubleWidth()

This is the original frame in Linear BT709:


This is the result of my linear Transformation:


This is ColorMatrix:


This is Matrix from HDR Core:


A few frame-comparison, but honestly the difference is so small that it's barely noticeable:












Could you test it, please, and let me know?


Link: FranceBB LUT


This one instead is made by Adam Stanislav and his results are pretty much the same as mine with very little difference. Can you check this one as well, please?
Link: Stanislav LUT

[masterkivat]

Your conversions looks pretty damn good!
But for my purpose, it would look "technically" wrong... My source file is listed as:

Originally Posted by Source file

Color range : Limited
Color primaries : BT.601 NTSC
Transfer characteristics : BT.601
Matrix coefficients : BT.601

And since I'm working in this project (upscaling SD content to HD with the proper color conversion), my goal is to do a proper color conversion from BT.601 NTSC to BT.709.
I know this is just the beginning of your work on LUT files, so I'll be patiently waiting your LUT.
As for Adam Stanislav LUT files, I think his "conversion" looks way stronger... Check out this comparison chart.

[FranceBB]

Ok, so I've done it.
This is the BT601 PAL to BT709:



And this is the NTSC one:



I then used the CIECAT02 module and its equations for calculating mathematical correlates for the six technically defined dimensions of color appearance: brightness (luminance), lightness, colorfulness, chroma, saturation, and hue.
I've got two matrices 274628 X 3, which are way too precise for such a simple calculation.
Nonetheless, here they are.
Take a look at them and let me know.

p.s of course I made the inverse as well.

Updated link in the 1st post.

[FranceBB]

Introducing LinearTransformation.

With the increasing number of matrices to do any kind of linear transformations, I kinda felt the need to make something more accessible for users, which is why I'm introducing now "LinearTransformation".
This plugin performs Linear Transformations based on a matrix conversion. Matrices are made by me and can be used independently from Avisynth as well.
The function has two parameters: Input and Output.
Currently, only those linear transformations are supported:

From "Linear_BT601_NTSC" to "Linear_BT709"
From "Linear_BT601_PAL" to "Linear_BT709"
From "Linear_BT709" to "Linear_BT601_NTSC"
From "Linear_BT709" to "Linear_BT601_PAL"
From "Linear_BT709" to "BT2020_HLG"
From "Linear_BT709" to "BT2100_PQ"
From "Linear_BT709" to "DCI_XYZ"
From "Linear_BT709" to "ZLog"
From "BT2100_PQ" to "Linear_BT2020"
From "BT2100_PQ" to "Linear_BT709"
From "BT2100_PQ" to "BT2020_HLG"
From "CLog3" to "Linear_BT709"
From "CLog3" to "BT2020_HLG"
From "CLog3" to "BT2100_PQ"
From "SLog2" to "Linear_BT709"
From "SLog3" to "Linear_BT709"
From "BT2020_HLG" to "Linear_BT709"
From "BT2020_HLG" to "BT2100_PQ"
From "DCI_XYZ" to "Linear_BT709"
From "LogC" to "Linear_BT709"
From "VLog" to "Linear_BT709"

A Linear Transformation is essentially a matrix that maps all points of a certain space to another, which includes of course points belonging to a certain curve to other in order to get a different curve.
Of course, a linear transformation can be used in encoding to map some values to some other values and therefore have conversions from curves like PQ to HLG and so on.

The transformation is performed with 16bit precision, which means that if your input source is lower, let's say, 8bit planar yv12, it will be brought to 16bit planar RGB internally, the linear transformation will be applied with 16bit planar precision and then the result will be brought down to 8bit planar yv12.

Planar RGB 16bit is strongly suggested as your source as it's gonna be faster, in fact 4:2:0, 4:2:2, 4:4:4 planar up to 16bit will be converted back and forth internally.

Inside the plugin, the path specified for the matrices is "mypath" and by default is: mypath = "C:\Program Files (x86)\AviSynth+\LUTs" which means that it's gonna look for my LUTs in a folder in such a location. It's not mandatory to have my LUTs there, you can have them in any location you want, provided that you do update that string.

A huge thanks to StainlessS and to the whole community for greatly improving the script and helping me writing it.

Plugin: https://github.com/FranceBB/LinearTr...formation.avsi
Matrices: https://github.com/FranceBB/LinearTransformation

Usage Example:

Example 1: From Slog3 to Linear BT709

FFVideoSource("mySlog3source.mxf")
LinearTransformation(Input="SLog3", Output="Linear_BT709")

Example 2: From PQ to HLG

FFVideoSource("myPQsource.mxf")
LinearTransformation(Input="BT2100_PQ", Output="BT2020_HLG")

[Dogway]

I'm curious about your LUTs, the reds are deeper using yours for matrix conversion, is this expected for the CIECAT chromatic adaptation? Can you choose Bradford or any other for comparison?
EDIT: Nevermind, you were comparing PAL. I got confused because those coefficients are more in line with SMPTE

Also I think that adding an option for the chroma resample kernel should be a good idea, I always switch to spline36 since bicubic is a bit soft, not that I can't do myself but as a suggestion.

[FranceBB]

Originally Posted by Dogway

I'm curious about your LUTs, the reds are deeper using yours for matrix conversion, is this expected for the CIECAT chromatic adaptation? Can you choose Bradford or any other for comparison?
EDIT: Nevermind, you were comparing PAL. I got confused

Yeah, I was indeed comparing with BT601 PAL; anyway, technically I could switch and use the Bradford Chromatic Adaptation



but unless there's a real need, I would personally stick with CIE 'cause the CAT02 provided is the international one that is expected to be used everywhere by everyone, which uses the following matrix:

Originally Posted by Dogway

Also I think that adding an option for the chroma resample kernel should be a good idea, I always switch to spline36 since bicubic is a bit soft, not that I can't do myself but as a suggestion.

You mean in LinearTransformation()? Sure, I can do that, no problem, but in the meantime, if you want you can use the matrices manually like this:

ConvertBits(16)

ConvertToPlanarRGB()

Cube("C:\Programmi\AviSynth+\LUTs\whatever.cube")

Converttoyuv420(chromaresample="Spline64")

so this way you can specify the chroma resample yourself. Since all calculations are done in RGB 16bit Planar and then brought back to whatever you had in input.

That would be the same as:

ConvertBits(16)

ConvertToPlanarRGB()

LinearTransformation(Input="something", Output="something_else")

Converttoyuv420(chromaresample="Spline64")

as you're feeding it with RGBP16 and it will output RGBP16 thus allowing you to resize the chroma with your favorite kernel.

[Dogway]

Yes, CAT02 is meant to be an improved version but I was curious about why the red is so much deeper. I wanted to test this but couldn't find Linear_BT709 and 601 in your repo(?). I'm missing these LUTs:

BT601_NTSC_to_BT709
BT601_PAL_to_BT709
BT709_to_BT601_NTSC
BT709_to_BT601_PAL

I don't know how it behaves currently but the example in that post is wrong I think. Rec709 and Rec601_PAL only differ 0.010 in red. This is a test I did in Nuke.

Rec709/sRGB

 0.640, 0.330, 0.030
 0.300, 0.600, 0.100
 0.150, 0.060, 0.790

Rec.601 PAL

 0.640, 0.330, 0.030
 0.290, 0.600, 0.110
 0.150, 0.060, 0.790

P22-EBU - ColorMatch RGB (D50) (Radius PressView Monitors)

 0.630, 0.340, 0.030
 0.295, 0.605, 0.100
 0.150, 0.075, 0.775

Now when I inverted the matrices M-1 (which is wrong) I get a match of your example.

[FranceBB]

Originally Posted by Dogway

Yes, CAT02 is meant to be an improved version but I was curious about why the red is so much deeper. I wanted to test this but couldn't find Linear_BT709 and 601 in your repo(?). I'm missing these LUTs:

BT601_NTSC_to_BT709
BT601_PAL_to_BT709
BT709_to_BT601_NTSC
BT709_to_BT601_PAL

Oh, dang... I forgot to upload them, now they're there.

Originally Posted by Dogway

Now when I inverted the matrices M-1 (which is wrong) I get a match of your example.

I see! I made a boo boo, but it's my fault, it's not the CAT02. I'll make a new one for PAL. NTSC should be ok I think, but now that they're uploaded, feel free to test it as well.

[Dogway]

Thanks, just tested also with SMPTE coefficients and it's working fine : D