FranceBB LUT Collection

[FranceBB]

Originally Posted by ProWo

Hi FranceBB

can you tell me a working FFmpeg command line to apply your LUTs?
If I apply this:
ffmpeg -i input_HDR -vf lut3d=your_lut ...
I always get wrong colors (to much contrast, to much red).

All my LUTs work in RGB Full Range, which means that you should always convert to RGB Full Range, apply the LUT and convert back.
Such a conversion is trivial in Avisynth (and probably VapourSynth too) but it's slightly less so in FFMpeg.
Now, in theory, lut3d *should* do that for you anyway, so something as simple as this should, in theory, suffice:

ffmpeg.exe -hide_banner -i "Test.mov" -vf lut3d="test.cube" -vcodec v210 -color_range 1 -color_primaries bt2020 -color_trc arib-std-b67 -colorspace bt2020nc -c:a pcm_s32le -ar 48000 -y "Test_v210_HLG_BT2020.mov"

When I ran the test:

ffmpeg.exe -hide_banner -i "Test.mov" -vf lut3d="test.cube" -vcodec v210 -color_range 1 -color_primaries bt2020 -color_trc arib-std-b67 -colorspace bt2020nc -c:a pcm_s32le -ar 48000 -y "Test_v210_HLG_BT2020.mov"
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Test.mov':
Metadata:
major_brand : qt
minor_version : 512
compatible_brands: qt
encoder : Lavf58.76.100
Duration: 00:00:01.28, start: 0.000000, bitrate: 4423685 kb/s
Stream #0:0[0x1]: Video: v210 (v210 / 0x30313276), yuv422p10le, 3840x2160, 4423680 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : FFMP
Stream mapping:
Stream #0:0 -> #0:0 (v210 (native) -> v210 (native))
Press [q] to stop, [?] for help
Output #0, mov, to 'Test_v210_HLG_BT2020.mov':
Metadata:
major_brand : qt
minor_version : 512
compatible_brands: qt
encoder : Lavf59.35.100
Stream #0:0: Video: v210 (v210 / 0x30313276), yuv422p10le(tv, bt2020nc/bt2020/arib-std-b67, progressive), 3840x2160, q=2-31, 4423680 kb/s, 25 fps, 12800 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : FFMP
encoder : Lavc59.56.100 v210
frame= 32 fps=2.2 q=-0.0 Lsize= 691201kB time=00:00:01.24 bitrate=4566385.2kbits/s speed=0.0852x
video:691200kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000128%

In this case I'm going from BT709 SDR to BT2020 HLG HDR and indeed mediainfo shows the right info in the container:

General
Complete name : Test_v210_HLG_BT2020.mov
Format profile : QuickTime
Codec ID : qt 0000.02 (qt )
File size : 675 MiB
Duration : 1 s 280 ms
Overall bit rate mode : Constant
Overall bit rate : 4 424 Mb/s
Frame rate : 25.000 FPS
Writing application : Lavf59.35.100

Video
ID : 1
Format : YUV
Codec ID : v210
Codec ID/Hint : AJA Video Systems Xena
Duration : 1 s 280 ms
Bit rate mode : Constant
Bit rate : 4 424 Mb/s
Width : 3 840 pixels
Clean aperture width : 3 840 pixels
Height : 2 160 pixels
Clean aperture height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Scan type : Progressive
Compression mode : Lossless
Bits/(Pixel*Frame) : 21.333
Stream size : 675 MiB (100%)
Color primaries : BT.2020
Transfer characteristics : HLG
Matrix coefficients : BT.2020 non-constant

When I index the newly created version with ffmpeg and I compare it to the Avisynth one I can see that lut3d in FFMpeg did convert to RGB Full Range, applied the LUT and converted back to YUV Limited TV Range correctly:

LWLibavVideoSource("T3_LUT_IB_UHD_BT709_v210.mov")

#From 4:2:0 16bit planar Narrow Range to RGB Planar 16bit Full Range
z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

#From BT709 SDR to BT2020 HDR HLG with 16bit precision
Cube("C:\Program Files (x86)\AviSynth+\LUTs\5a_BT709_HLG_UPCONVERT_DISPLAY_mode-nar_in-nar_out-nar_nocomp.cube", fullrange=1, interp=1)

#From RGB 16bit planar Full Range to YUV422 10bit planar Narrow Range with dithering
z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion", use_props=0)

VideoTek()

avisynth=last


ffmpeg=LWLibavVideoSource("Test_v210_HLG_BT2020.mov").VideoTek()


StackVertical(avisynth, ffmpeg)

Avisynth top, FFMpeg bottom:

Originally Posted by ProWo

Found it.
-colorspace -color_primaries -color_trc must be specified explicitly after the lut.

Yes of course, when you perform any kind of conversion you need to tell the encoder and container what changed so that the right header is gonna be written.
I also recommend -map_metadata -1 if you're doing it in FFMpeg directly to completely get rid of the original info before replacing it with the new one.
By the way, this needs to be done regardless of the encoder you're using; I mean, even with Avisynth + x264 for instance you still need to specify the output to make decoders interpret it correctly, for instance in x264 it would have been something like:

x264-10b.exe "Test.avs" --preset medium --profile high422 --level 5.2 --keyint 1 --no-cabac --slices 8 --bitrate 500000 --vbv-maxrate 500000 --vbv-bufsize 100000 --deblock -1:-1 --overscan show --colormatrix bt2020nc --range tv --log-level info --thread-input --transfer arib-std-b67 --colorprim bt2020 --videoformat component --nal-hrd cbr --aud --output-csp i422 --output-depth 10 --output "raw_video.h264"

[ProWo]

Thanks FranceBB for the detailed info.
ProWo

[FranceBB]

Linear Transformation v2.6 Released!

- Added preliminary RGB32 support (clips with alpha channels will be subject to the very same transformation on both the 24bit video and the 8bit alpha channel before being recomposed back to the output)

Now, as an example, let's take a look at the following fake RGB32 input:

ColorBars(848, 480, pixel_type="RGB32")
pippo=last

pluto=pippo.ConverttoY()

AddAlphaPlane(pippo, pluto)

This creates an RGB32 stream in which the first 24bit are RGB (i.e 8+8+8=24):



and the last 8 are the alpha channel (+8 = 32):





This is in BT709 SDR.
When we apply:

LinearTransformation(Input="Linear_BT709", Output="BT2100_PQ", Interpolation="Tetrahedral")

what's gonna happen is that the stream will be divided in two parts: the normal part and the alpha channel and both will be subject to the very same transformation (in this case from BT709 to BT2020 PQ) before being added back to create a proper RGB32 output:





as per mojie126's request.

[coolgit]

LinearTransformation(Input="dvhe0509", Output="Linear_BT709")

Is there a way to convert from 709 to dvhe0509?

[FranceBB]

Currently no, we can only go from dvhe0509 to BT709 SDR and even that took quite a lot of work for me and algia (Livio Aloja) one of my former colleagues, also an encoder, a senior editor and great friend of mine at Sky before he retired.
Nowadays he mostly goes fishing on the river in summer and fetching mushrooms in winter.
That being said, going from dvhe0509 was extremely difficult due to the lack of openness from Dolby about the proprietary metadata in the pixel shader, so much so that the result of that conversion comes from using the official Harmonic reference encodes in both dvhe0509 and BT709 and reverse engineering the transformation matrix used, which is why it's less than perfect as it's not based on an official spec, but it's "good enough".
Specifically, it applies to IPTc2 dvhe0509 inputs specifically, so it won't work with other kind of inputs.
This is because, on a normal pipeline, you should have:

Source dvhe0509 -> IPTc2 -> Proprietary Metadata interpreter (RPU) -> standard HDR image -> LUT -> BT709 SDR

however what we've done is this:

Source dvhe0509 -> IPTc2 -> LUT -> BT709 SDR

The fact is that nothing tells us that the input is always going to be IPTc2, for instance if the input is dvhe0509 with ITP PQ, that LUT can't be used.
Going the other way round, so from BT709 to dvhe0509 is impossible 'cause although the transformation could, in theory, be inverted, I would have no way to insert the proprietary metadata in the file so you would be left with something that no decoder could decode properly.
In other words, you could, in theory, invert and do:

BT709 SDR -> LUT -> IPTc2

then what?
You're left with IPTc2 that needs to be encoded and flagged as such with the relative metadata (RPU) so that an official decoder will be able to decode it, but we have no way to do that I'm afraid, so all you're left with is an uncompressed A/V stream living in RAM coming out from Avisynth with no metadata at all and that any decoder would interpret wrongly.

Out of curiosity, why are you trying to do this?

[coolgit]

Is dvhe0509 (dvhe.05.09) the HDR format as my video has dvhe.08.06, so doesn't work?

Out of curiosity, why are you trying to do this?

I am just curious to why you have dvhe to 709 but not the other way around. I am sure some computer geek will crack the Dolby metadata one day.

I have just bought a new 4k tv with HLG, HDR10, HDR10+, dolby vision after the last tv went kaput last week just before Black Friday... perfect timing. I have been using different HDR versions of a film to see one is best. There isn't much different between HDR10, +, dolby. Dolby is slighty better when dealing with white levels as both HDR10 and + brightness tend to stick out like a sore thumb sometimes. Dolby is more subtle.

[FranceBB]

Originally Posted by coolgit

Is dvhe0509 (dvhe.05.09) the HDR format as my video has dvhe.08.06, so doesn't work?

That's correct, you can't use the dvhe0509 LUT on dvhe0806, it's not gonna work.

Originally Posted by coolgit

There isn't much different between HDR10, +, dolby. Dolby is slighty better when dealing with white levels as both HDR10 and + brightness tend to stick out like a sore thumb sometimes. Dolby is more subtle.

Well the difference between HDR10 and 10+ is literally just dynamically changing metadata in the header so you would benefit from it only if your TV has less nits than the actual ones the content was mastered for.
This is because if the content was mastered on more nits than the ones of your actual TV, your TV would have to read the maxCLL value declared in the header and remap it to its own maximum (HDR10) however if we have something extreme like with lots of computer generated graphics that skyrockets to 5000 nits (I'm saying this 'cause afaik there's no camera with enough stops to capture more than 3300 nits currently, so it has to be computer generated content) and your TV is, let's say, an 880 nits OLED TV, it would have a really hard time with all the other scenes as they would have to be remapped much lower. This is where HDR10+ comes in to help with dynamically changing metadata in the header, the maxCLL value now refers to the scene so that you don't have the global one only. There's still the global one to be backwards compatible with normal HDR10 devices, however for those that can read the metadata in the header they're gonna be able to adapt and remap the peak dynamically thus achieving a far better result.

Last but not least is Dolby Vision which has so many flavors right now unfortunately that it's hard to compare them all.

IPTc2 dvhe0509 has no technological advancement over HDR10 and HDR10+ 'cause those two are PQ based while IPTc2 is more on the SDR side. It's still considered HDR but the luma is shaped to peak at around 80% turning pseduo-logarithmically to preserve the highlights while the blacks start lower as in SDR. The idea being that the shader interpreting the IPTc2 and the metadata can produce either an HDR version close to what HLG would look like on an HDR display and output a very good SDR version on an SDR display.
PQ on the other hand is a totally logarithmic curve which yields far better results on the HDR side as it preserves both blacks and highlights (but HDR10/10+ can't be displayed on an SDR display as a drawback).


Now going to what Dolby Vision should have actually sticked to, the dual layer profile, that one makes sense and does have some technological advancement over HDR10 and 10+. This is because it's literally two layers, so one layer with a normal UHD H.265 10bit HDR PQ encoded video and the second layer carrying dynamically changing metadata (you're gonna see it as a "fake" FULL HD stream that is completely green if you try to play it on a normal device). That thing there is the metadata layer and with that not only you have dynamically changing metadata like in HDR10+ but a proper Dolby Vision Dual Layer decoder would use it to display a perfect 12bit output and that's where Dolby Vision makes sense.

Originally Posted by coolgit

I have been using different HDR versions of a film to see one is best. There isn't much difference

Charles Poynton, the inventor of luma, once told me: "once the director signs a version off, no art should occur in the encoding process".

In other words, the goal of an encoder at that point is to make the different versions a movie is encoded to look as close as possible to each other while retaining the director's view and exploiting the advancement of one technology while compensating for the lack on another and so on.
The fact that you haven't been able to see major differences between the different versions of a movie means that such a job was done right by the person who encoded the movies.
For instance, when you target a DCI P3 XYZ theatrical release, you have to keep in mind that the peak is gonna be 48 nits but that the cinema room is gonna be completely dark and given that human vision separates in Photopic, Mesopic, Scotopic according to whether you're using rods, cones or both and that the less photons there are the less chroma you see, to make it look more "natural" you have to compensate for that on the chroma side. This is just one of the several things that encoders take into account when producing the final encoded output.
I'm just a guy who works on TV, so I haven't made many movies that ended up in the cinemas in my life (just a couple of documentaries, really) but, to a very lesser extent, I face those challenges too when producing the different TX Ready versions starting from the same master that need to then go on air.

[coolgit]

Charles Poynton, the inventor of luma, once told me: "once the director signs a version off, no art should occur in the encoding process".

That would obviously be the standard policy. Makes sense too. I think the slight differences is that Dolby, hdr10+, adjusts the brightness and tone mapping per scene, so in some darkish scene the shadows on the face are subtle and better than hdr10, 709. In normal day brightness there is hardly anything. But if i watch it normally, instead of pausing, one doesn't see any differences unless one knows where to look and concentrate on finding the differences. In the real world no one going to do that, they just accept the marvellous quality output of colours. Having said that is there really any need for Dolby when hdr10+ does the same but better? When streaming and if the tv doesn't have hdr10+, it can default to hdr10. With Dolby it would default to SDR.

second layer carrying dynamically changing metadata

This is similar to overlay isn't it?

Here the thing that got me thinking for ages now. When you watch the Lord of the Ring, there is a lot of action/violence but given 12 rating, but a film with less violence is given 18 rating, simply because there is blood, whereas there isn't any in the Lord of the Ring. But in terms of cinema and dvds sales, the 18 rated film would earn less than a 12 rated film. What if the blood parts of any films is on the 2nd layer. There would be 2 versions, 12 and 18. In the cinema, especially most are multiplex, one room would be 12 rated so adult can dump their children in and squeamish people in, in another room 18 rated for adults only. The same for dvds. The menu can at the beginning would show 12 or 18 and the parents can decide which version they want. Surely this way could maximise the income/sales/ by not excluding a large share of the market. For me i wouldn't mind seeing blood squirting out of the chopped head in the Return of the King. The same would go for swear words not appropriate for under 18 by having 12 rated sounds and 18 rated sounds.

[FranceBB]

Originally Posted by coolgit

What if the blood parts of any films is on the 2nd layer. There would be 2 versions, 12 and 18

That ain't gonna work, unfortunately. One thing is adding an LSB kind of metadata to create a 12bit output, another is having a completely different version of the same movie (TL;DR you can't just add VFX).

Anyway it's an interesting idea, but it can't be done the way it currently is implemented.

Originally Posted by coolgit

is there really any need for Dolby when hdr10+ does the same but better?

dvhe? Nope, no advantage at all.
Dolby Vision dual layer? Yes, it would still default to 10bit HDR10 when watched on a standard decoder, but when watched on a proper Dolby Vision decoder it would not just have dynamically changing metadata like HDR10+ but it would also deliver 12bit and that's very important 'cause by standard on a totally logarithmic curve like PQ if you use only 10bit limited tv range code values you don't have enough bits to go all the way up to 10'000 nits, while with 12bit you do. Of course currently is a bit overkill 'cause there are no movies whatsoever graded at 10k nits, there are no monitors that can get to 10k nits and there are no cameras that can record enough stops to get to 10k nits, but eventually, in the future, we'll get there and that's where 12bit is gonna make a difference.

[coolgit]

Originally Posted by FranceBB

That ain't gonna work, unfortunately. One thing is adding an LSB kind of metadata to create a 12bit output, another is having a completely different version of the same movie (TL;DR you can't just add VFX).

Anyway it's an interesting idea, but it can't be done the way it currently is implemented.

Maybe not 2nd layer but some sort of overlay like subtitles do in a separate file. If there can be many sub files and audio files then why not many video files?

Theoretically it should be possible.

[StainlessS]

That ain't gonna work, unfortunately. One thing is adding an LSB kind of metadata to create a 12bit output, another is having a completely different version of the same movie (TL;DR you can't just add VFX).

Maybe alternate 'angle' like DVD (I assume BD also has alternate angle capability).