Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
![]() |
#301 | Link | |||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
Quote:
Quote:
The "good news" however is that VapourSynth - unlike Avisynth - expects the user to set them in the function so that they can also be set in the frame properties. This means that up until this point as long as you set them correctly in the YUV->RGB->YUV roundtrip, then you're good to go. ![]() Nope, you're doing everything right already, the "issue" is limited to Avisynth in which we have to use PropSet(). In general, we can use this rule of thumb after going back to YUV. For BT601 PAL outputs: Code:
propSet("_ColorRange", 1) #Limited TV Range propSet("_Matrix", 5) #BT601 PAL propSet("_Transfer", 6) #BT601 PAL propSet("_Primaries", 5) #BT601 PAL Quote:
Code:
propSet("_ColorRange", 1) #Limited TV Range propSet("_Matrix", 1) #BT709 propSet("_Transfer", 1) #BT709 propSet("_Primaries", 1) #BT709 Code:
propSet("_ColorRange", 1) #Limited TV Range propSet("_Matrix", 9) #BT2020 propSet("_Transfer", 14) #BT2020 propSet("_Primaries", 9) #BT2020 Code:
propSet("_ColorRange", 1) #Limited TV Range propSet("_Matrix", 9) #BT2020 propSet("_Transfer", 18) #HLG propSet("_Primaries", 9) #BT2020 Code:
propSet("_ColorRange", 1) #Limited TV Range propSet("_Matrix", 9) #BT2020 propSet("_Transfer", 16) #PQ propSet("_Primaries", 9) #BT2020 Code:
propSet("_ColorRange", 1) #Limited TV Range propSet("_Matrix", 2) propSet("_Transfer", 17) #DCI P3 propSet("_Primaries", 11) #DCI P3 Tested and working as expected in AVSPmod mod: ![]() ![]() ![]() ![]() ![]() ![]() Yeah, that's actually something I wanted to do for quite some time now. I did actually make a first implementation of D-Log to BT709, however I didn't particularly like the results that came out of it. On that note, most of the log conversions I made were targeting BT709, however nowadays log is mostly used to produce BT2020 HLG outputs, at least internally, so I should really start updating those. Anyway, I have a few samples of D-Log footage, so perhaps if I get enough samples of D-Log M footage as well I can make both D-Log to BT709 and D-Log M to BT709. Hopefully come next year I'll have a bit more time to make things like Slog3 to HLG and to PQ which is kinda important right now as more and more productions shot in Slog3 are then graded and converted in either HLG or PQ for the final release (in our case, it's always HLG but we're a broadcasting company, so we're biased). |
|||
![]() |
![]() |
![]() |
#302 | Link | |
Registered User
Join Date: May 2005
Posts: 1,561
|
Quote:
I cannot thank you enough for your, once again, very extensive and helpful post! ![]()
__________________
Gorgeous, delicious, deculture! |
|
![]() |
![]() |
![]() |
#303 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
Linear Transformation v2.7 Released!
Changelog: Quote:
Merry Christmas! |
|
![]() |
![]() |
![]() |
#304 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
In the next release we're gonna have two new conversions that are gonna be added:
- BT2020 SDR to BT709 SDR - BT709 SDR to BT2020 SDR Let's start from the first one and let's start by saying that other people have been working on this before me, in fact William (the creator and maintainer of HLG Tools) released a version based on Oklab here: Link. There's absolutely nothing wrong with his LUTs and you can go ahead and use them. What I'm doing is something much more simplistic, in fact the BT2020 SDR to BT709 SDR conversion is actually based on the most simple calculation illustrated in the ITU recommendation 2407. The conversion is performed as follows: Code:
[0.4124 0.3576 0.1805]^-1 [0.6370 0.1446 0.1689] [0.2126 0.7152 0.0722] [0.2627 0.6780 0.0593] = [0.0193 0.1192 0.9505] [0.0000 0.0281 1.0610] Let's take the matrix for which we have to calculate the inverse and let's create the augmented matrix Code:
[0.4124 0.3576 0.1805 1.0000 0.0000 0.0000] [0.2126 0.7152 0.0722 0.0000 1.0000 0.0000] [0.0193 0.1192 0.9505 0.0000 0.0000 1.0000] We make the pivot in the 1° column by dividing the first row by 0.4124 Code:
[1.0000 0.8671 0.4376 2.4248 0.0000 0.0000] [0.2126 0.7152 0.0722 0.0000 1.0000 0.0000] [0.0193 0.1192 0.9505 0.0000 0.0000 1.0000] Code:
[1.0000 0.8671 0.4376 2.4248 0.0000 0.0000] [0.0000 0.5308 -0.0208 -0.5155 1.0000 0.0000] [0.0000 0.1024 0.9420 -0.0467 0.0000 1.0000] Code:
[1.0000 0.8671 0.4376 2.4248 0.0000 0.0000] [0.0000 1.0000 -0.0392 -0.9711 1.8837 0.0000] [0.0000 0.1024 0.9420 -0.0467 0.0000 1.0000] Code:
[1.0000 0.0000 0.4717 3.2669 -1.6334 0.0000] [0.0000 1.0000 -0.0392 -0.9711 1.8837 0.0000] [0.0000 0.0000 0.9460 0.0527 -0.1930 1.0000] Code:
[1.0000 0.0000 0.4717 3.2669 -1.6334 0.0000] [0.0000 1.0000 -0.0392 -0.9711 1.8837 0.0000] [0.0000 0.0000 1.0000 0.0557 -0.2040 1.0569] Code:
[1.0000 0.0000 0.0000 3.2408 -1.5372 -0.4986] [0.0000 1.0000 0.0000 -0.9689 1.8757 0.0415] [0.0000 0.0000 1.0000 0.0557 -0.2040 1.0569] Code:
[3.2408 -1.5372 -0.4986] [-0.9689 1.8757 0.0415] [0.0557 -0.2040 1.0569] Code:
[3.2408 -1.5372 -0.4986] [0.6370 0.1446 0.1689] [-0.9689 1.8757 0.0415] [0.2627 0.6780 0.0593] = [0.0557 -0.2040 1.0569] [0.0000 0.0281 1.0610] Code:
[A B C] [D E F] [G H I] A= 3.2408 x 0.637 + (-1.5372) x 0.2627 + (-0.4986) x 0 = 1.6605 B= 3.2408 x 0.1446 + (-1.5372) x 0.678 + (-0.4986) x 0.0281 = -0.5876 C= 3.2408 x 0.1689 + (-1.5372) x 0.0593 + (-0.4986) x 1.061 = -0.0727 D= -0.9689 x 0.637 + 1.8757 x 0.2627 + 0.0415 x 0 = -0.1244 E= -0.9689 x 0.1446 + 1.8757 x 0.678 + 0.0415 x 0.0281 = 1.1327 F= -0.9689 x 0.1689 + 1.8757 x 0.0593 + 0.0415 x 1.061 = -0.0083 G= 0.0557 x 0.637 + (-0.204) x 0.2627 + 1.0569 x 0 = -0.0181 H= 0.0557 x 0.1446 + (-0.204) x 0.678 + 1.0569 x 0.0281 = -0.1005 I= 0.0557 x 0.1689 + (-0.204) x 0.0593 + 1.0569 x 1.061 = 1.1186 so our final matrix is: Code:
[ 1.6605 -0.5876 -0.0727] [-0.1244 1.1327 -0.0083] [-0.0181 -0.1005 1.1186] Let's put this theory to the test and compare it against HDRTools: Code:
test1=LWLibavVideoSource("Source1.mxf") test2=LWLibavVideoSource("Source2.mxf") test3=LWLibavVideoSource("Source3.mxf") test4=LWLibavVideoSource("Source4.mxf") test1++test2++test3++test4 propClearAll() ConvertBits(16) bt2020=Subtitle("BT2020 SDR") bt2020_sdr=last ConvertToPlanarRGB(bt2020_sdr) Cube("A:\Ingest\MEDIA\temp\BT2020_to_BT709.cube", interp=1, fullrange=1) ConverttoYUV444(matrix="Rec709") Subtitle("BT709 SDR LUT") bt709_cube=last ConvertYUVtoXYZ(bt2020_sdr, Color=1) ConvertXYZtoYUV(pColor=1) Subtitle("BT709 SDR HDRTools") bt709_hdrtools=last StackHorizontal(bt2020, bt709_cube, bt709_hdrtools) ![]() Looks fine. To spot the actual differences between this method and the XYZ roundtrip made by HDRTools we have to check solid colors like in those commercials in which the clipping of values outside the BT709 realm done by the LUT becomes very apparent: ![]() ![]() The difference is pretty clear and it can be seen in both the Colgate and EE backgrounds, with HDRTools actually performing a much better approximation, while the LUT is clipping several values out as it's a simple onto conversion (as every element in the range has at least one corresponding element in the domain) but clearly not one to one (because more elements map to the same destinations). This is also due to the normalizations of value above 1 and below 0. Speaking of values below 0, inside the LUT you're gonna be able to see a bunch of zeroes, that's to avoid issues with non legal values, in fact everything negative has been normalized to 0 to avoid a repetition of frank's issue and more specifically the lut3d implementation. When we test with real life examples, however, the difference becomes much more slim: ![]() ![]() ![]() ![]() ![]() ![]() For those who wanna start testing: BT2020_to_BT709.cube |
![]() |
![]() |
![]() |
#305 | Link |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 452
|
Regarding my BT.2020-to-BT.709 conversion LUT he mentioned, I have a new one I just generated:
https://wswartzendruber.net/uploads/bt2020to709-2.cube It should be very similar, but the math is a little tighter. It still uses Oklab/Oklch, so it's as perceptually accurate as that is. EDIT: I mainly use it for viewing HLG on BT.709 SDR. I may as well include screenshots and peddle my wares! ![]() ![]() ![]() ![]() ![]() ![]() Last edited by wswartzendruber; 21st February 2025 at 04:55. |
![]() |
![]() |
![]() |
#307 | Link | |||||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
It feels like a good time to make a release, so... here we are.
Linear Transformation v2.8 Released! Changelog: Quote:
Quote:
Quote:
![]() The roundtrip was also tested and it should work. Oh and before anyone says anything, yes, I did actually check to make sure we're populating frame properties correctly eheheheh Quote:
![]() Quote:
![]() (and I've just realized that I screwed up the screenshot but whatever). Last edited by FranceBB; 8th March 2025 at 18:48. |
|||||
![]() |
![]() |
![]() |
#308 | Link | |||||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
wh0phd pointed out some less than optimal results when performing the conversions to and from BT601 (both PAL and NTSC). Being little used nowadays, the original conversions from 4 years ago didn't quite achieve the expected results, especially for the conversions to and from BT2020 as well as the HLG (arib-std-b67) and PQ (smpte2084) conversions. A review of those was long overdue and as I'm trying to tidy things up I decided to finally spend time on those.
I recreated the matrices by inserting the BT601 PAL, BT601 NTSC, BT709 and BT2020 SDR primaries: ![]() ![]() ![]() ![]() The white point stays the same for all of them, so BT601 PAL and BT601 NTSC as well as BT709 and BT2020 SDR, namely x white = 0.3127 y white = 0.3290 The values for red, green and blue however change. BT601 PAL has x red = 0.640 y red = 0.330 x green = 0.290 y green = 0.600 x blue = 0.150 y blue = 0.060 BT601 NTSC has x red = 0.630 y red = 0.340 x green = 0.310 y green = 0.595 x blue = 0.155 y blue = 0.070 BT709 has x red = 0.640 y red = 0.330 x green = 0.300 y green = 0.600 x blue = 0.150 y blue = 0.060 BT2020 has x red = 0.708 y red = 0.292 x green = 0.170 y green = 0.797 x blue = 0.131 y blue = 0.046 The idea here is to repurpose the BT709 conversions that have been created a long time ago and that have been tested over and over again by the community to produce the BT601 ones by just changing the primaries. Let's take for instance the old BT2020 HLG to BT601 PAL conversion: it was remapping the 75% reference white in the luma to 100% and clipping the rest. As to the chroma, it was converting the primaries from the BT2020 ones to the BT601 PAL ones. Now, this is NOT what we're doing in the BT2020 HLG to BT709 SDR conversion as we're keeping the luma in place and only converting the primaries (which is the whole purpose behind HLG given that it would be what a user would see if he bought a BT2020 UHD TV that can't interpret the transfer). So, in the new BT2020 HLG to BT601 PAL conversion we're using the same logic as the BT2020 HLG to BT709 SDR conversion, namely keeping the luma in place and converting the primaries. This is the old BT2020 HLG to BT601 PAL conversion: ![]() and this is the new one: ![]() Using the same logic, I could now fill the gap and add the missing BT601 conversions for Slog3, Vlog, LogC and Clog3 as well. Quote:
![]() Quote:
![]() Quote:
![]() I also fixed an issue in the old BT601 NTSC to BT601 PAL conversion as well as BT601 PAL to BT601 NTSC. Both LUTs have been regenerated using CIECAT02 instead of the Bradford Chromatic Adaptation. Taking z_ConvertFormat() as a safe reference point (i.e avsresize), I started testing the two conversions. Quote:
![]() and the other way round as well Quote:
![]() You can already test the new versions on Github, but I'm planning to make a new release next week. |
|||||
![]() |
![]() |
![]() |
#309 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
Linear Transformation v2.9 Released!
Changelog: Quote:
|
|
![]() |
![]() |
![]() |
#310 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
Linear Transformation v3.0 Released!!
Changelog: Quote:
This feature was requested by ossnorry. As a bit of history, in 1953 the NTSC version of BT601 was born, however the tube phosphors originally used by the TVs of the time had issues displaying those values and the standard was later changed (reduced) in 1987 to make life easier for the hardware of the time, however one country never stopped using the original NTSC version and that country is Japan, which is why the original NTSC version has become known as "NTSC-J" which stands for "Japan". Anyway, the old BT601 NTSC Japan (1953) is based on the following values: x white = 0.3100 y white = 0.3160 x red = 0.670 y red = 0.330 x green = 0.210 y green = 0.710 x blue = 0.140 y blue = 0.080 and is indicated with --colormatrix smpte170m --transfer bt470m --colorprim bt470m while the normal BT601 NTSC SMPTE C (1987) that all other NTSC countries adopted has the following values: x white = 0.3127 y white = 0.3290 x red = 0.630 y red = 0.340 x green = 0.310 y green = 0.595 x blue = 0.155 y blue = 0.070 and is indicated with: --colormatrix smpte170m --transfer smpte170m --colorprim smpte170m As you can see, all values are different, including the white point. All the LUTs created up until this point that referred to BT601 NTSC were targeting the normal BT601 NTSC SMPTE C (1987), namely smpte170m. This release allows the conversion between the two BT601 NTSC standards: the old BT601 NTSC Japan (1953), namely bt470m, and the normal BT601 NTSC SMPTE C (1987), namely smpte170m. Left is BT601 NTSC SMPTE C (1987) - Right is BT601 NTSC-Japan (1953): ![]() |
|
![]() |
![]() |
![]() |
#313 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 3,227
|
The one you're referring to is the dual layer, the most common version of Dolby Vision. You're right, in the dual layer you have the first layer as a normal H.265 3840x2160 4:2:0 HDR PQ 10bit planar track and the second layer as a fake H.265 1920x1080 4:2:0 10bit which is actually carrying metadata (it would look green if you tried to decode it on its own). Normal players ignore the second layer and just play the first one as a normal HDR10. Dolby Vision players can read the metadata track and combine it with the first to output 12bit HDR PQ with dynamically changing metadata.
Now, obviously making a LUT for such a thing wouldn't make sense, which is why I'm pretty sure that mojie126 is actually referring to dvhe0509. You see, dvhe0509 is actually a proprietary Dolby space (IPTc2) and the videos encoded with it are not supposed to be played back by normal devices. If you were to playback such a file, you would see the luma resemble a pseudo SDR curve and the chroma being with totally different values. See here: https://forum.doom9.org/showthread.php?p=1964646 And here in this thread: https://forum.doom9.org/showthread.p...42#post1964642 Quote:
LinearTransformation(Input="dvhe0509", Output="Linear_BT709") ![]() Now, surely we could just reverse it so that we could potentially use the same logic to go from BT709 to a "fake" dvhe0509 IPTc2 and we could probably even get the values somewhat right but we would have no way of writing the correct metadata as we still wouldn't be able to generate the appropriate RPU. I'm pretty sure that no Dolby Vision decoder would accept a stream without the relevant metadata, so I guess we're stuck with this... ![]() |
|
![]() |
![]() |
![]() |
#315 | Link | |
Registered User
Join Date: Aug 2024
Posts: 525
|
Quote:
Which probably look like some noise. The track has low bitrate still, so it will look terrible, but at least it still does something I guess... Is that dvhe0506 conversion works as converting it to RGB using Rec.709 matrix then "correct" the wrong RGB? |
|
![]() |
![]() |
![]() |
#317 | Link | ||
Registered User
Join Date: Oct 2012
Posts: 8,395
|
Quote:
the DV encoder got bt 2020 PQ to eat and the display is presenting bt 2020 PQ at the end. itp instead of ycbcr a meta data lever or what ever does not change that. how many types of DV are there 12? Quote:
|
||
![]() |
![]() |
![]() |
#318 | Link | |
Registered User
Join Date: Dec 2013
Location: Berlin, Germany
Posts: 447
|
Quote:
__________________
My github... |
|
![]() |
![]() |
![]() |
#319 | Link | |
Registered User
Join Date: Oct 2012
Posts: 8,395
|
Quote:
even SDR screen are doing that for decades. |
|
![]() |
![]() |
![]() |
Thread Tools | Search this Thread |
Display Modes | |
|
|