Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
6th January 2022, 01:56 | #81 | Link |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 412
|
Thank you!
I'd like to wait until 0.4.0 for listing it. That's going to have arguably better tone mapping. (It will map the RGB channels independently, which doesn't allow out-of-gamut results.) I need to get some scripts ported to Windows (PowerShell?) and then that can happen. |
6th January 2022, 02:14 | #82 | Link |
Registered User
Join Date: Mar 2003
Location: Germany
Posts: 215
|
No need to port scripts to windows. Just install the bash shell on windows, its in the package of git for windows:
https://gitforwindows.org/ Just install this, and you have a fully functional bash shell in windows. |
6th January 2022, 07:09 | #83 | Link |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 412
|
Here's 0.4.0:
https://github.com/wswartzendruber/h...elease%2F0.4.0 I ported the BASH scripts over to PowerShell (for the Windows zip) so people could use them more easily. |
6th January 2022, 08:58 | #84 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Quote:
Code:
video=FFVideoSource("\\mibctvan000\Ingest\MEDIA\temp\PQ2HLG-20220105T125539Z-001\PQ2HLG\PQ.mkv") audio=BlankClip(length=607, fps=24000, fps_denominator=1001, channels=1) AudioDub(video, audio) #From 4:2:2 16bit planar Narrow Range to RGB Planar 16bit Full Range z_ConvertFormat(pixel_type="RGBP16", colorspace_op="2020:st2084:2020:limited=>rgb:st2084:2020:full", resample_filter_uv="spline64", dither_type="error_diffusion") #From PQ to HLG with 16bit precision Cube("C:\Program Files (x86)\AviSynth+\LUTs\1a_PQ1000_HLG_mode-nar_in-nar_out-nar_nocomp.cube", fullrange=true) #From RGB 16bit planar Full Range to YUV422 10bit planar Narrow Range with dithering z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion") @ErazorTT I'm now encoding the test patters you sent me with the official BBC LUT and I'll post the 7 days link to the result here if you don't mind, so that everyone else can take a look at it too. The idea, after that, would be to try to recreate a LUT as close as possible to the official one, so I'll send you back a new LUT file as well later in the afternoon. Last edited by FranceBB; 6th January 2022 at 09:36. |
|
6th January 2022, 10:10 | #85 | Link |
Registered User
Join Date: Mar 2003
Location: Germany
Posts: 215
|
Awesome! I will test!
Sure, I have created those patterns with my own avisynth scripts, which are part of my calibration pattern suite. So I could actually also post those, if anybody needs PQ or HLG test patterns. |
6th January 2022, 11:56 | #86 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
There we go:
Color Saturation Steps: https://we.tl/t-gwqgHChcyZ Gray Steps: https://we.tl/t-q8f4O5AGpt (link valid for 7 days, so hurry up eheheheh) This should produce really really similar results to the official BBC LUT: https://github.com/FranceBB/LinearTr...PQ_to_HLG.cube BBC LUT (left) - My LUT, new Implementation (right) |
6th January 2022, 15:38 | #87 | Link |
Registered User
Join Date: Mar 2003
Location: Germany
Posts: 215
|
Awesome! After looking closely into the results I must again complement you guys!
So FranceBB's v1.9 PQ_to_HLG.cube LUT is alomst identical as the results with wswartzendruber's hlg-tools v0.3.0 and 0.4.0 with the settings of max-cll 1000 and ref-white 203. The differences are only single 8bit values apart, and that only in very view of all the test squares of the patterns. And the best is that both are extremely simular to the output using the official LUT! So a big thumbs up there! The big advantage of the hlg-tools generator is that the LUT can be adjusted to whatever max limunosity is needed, while in contrast the fixed LUT's all will cut abruptly at 1000nits, which might be undesirable. And if more tuning is needed, there is also the possibility of changing the ref-white (or the brightness factor). I have to ask though, was that expected that 0.3.0 and 0.4.0 return exaclty identical results? Perhaps however the differences are below the 8bit threshold for which I tested. |
6th January 2022, 16:43 | #88 | Link | |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 412
|
Quote:
Functionally, 0.4.0 does tone mapping a bit differently, but that only matters for MaxCLL values above 1,000 nits (or if you boost the brightness so that MaxCLL gets pushed up above that internally). Internally, there was a lot of code refactoring that went into 0.4.0 to better organize things. EDIT: I kind of want to get a wiki going of different 4K UltraHD discs and the pq2hlg settings that should be put into it for each movie. Last edited by wswartzendruber; 6th January 2022 at 17:05. |
|
6th January 2022, 17:28 | #89 | Link | ||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Quote:
Thank you for testing and making me work on this once more, I'm always happy to improve things. Quote:
Still I gotta say, @wswartzendruber, impressive work, looks like you finally got it right this time! Amazingly enough, something that you started as a hobby-like project, turned out to be so reliable that is able to produce professional-like results. Well done! I don't know why many people over the past months didn't bother to reply or help you, but I knew you would have got this on your own eventually. Last edited by FranceBB; 6th January 2022 at 17:30. |
||
6th January 2022, 20:50 | #91 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Quote:
I think it's gonna be a useful exercise for us all. I stand on the free and open source software side, so the more open source softwares improve, the better. Tomorrow I'm gonna be back at work and I'm gonna be busy hardsubbing the UHD HDR HLG Italian dubbed version of The Town with the Italian typesetting and it will take me a while 'cause it's always a bit of a pain in the... *Beep* to work with typesetting in HLG, especially given that Aegisub is stuck in the stone age (2014) and has no idea whatsoever of any colour curve and I have to use the colour picker to get exactly the same levels of the HLG video and then use a separate waveform monitor to cross check everything... (You may think that official studios and professional use fancy paid HDR aware subtitling/typesetting software, but no, we're in the same tedious painful mess as the rest of the world eheheheh). Anyway, I'm digressing, long story short, if I'm gonna have time tomorrow, I'll repeat the tests with the official 4000 nits LUT and see what happens. |
|
7th January 2022, 11:46 | #92 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
p.s "What's the most common thing you do at work?"
Me: Wait for indexing LMAO https://i.imgur.com/5QTbeCy.png (456 GB worth of mezzanine file, it's gonna take forever ehehehehe) Last edited by FranceBB; 7th January 2022 at 11:53. |
14th January 2022, 02:33 | #94 | Link |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 412
|
I've just now realized that GitHub has wikis!
https://github.com/wswartzendruber/h...ovies-Database I'll be filling this out as I work through my Marvel collection (really mad at DC right now). |
15th January 2022, 18:16 | #96 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Unfortunately I've had a bit of a rough series of days, as I've been busy with several issues (XDCAM frame properties and indexers reporting the wrong chroma location value, which screwed up some workflows I had. You can see the whole story on Avisynth Development, Avisynth+. As soon as that thing is solved/fixed, I'll spend time on the LUTs again, I promise).
|
17th January 2022, 13:25 | #97 | Link |
Registered User
Join Date: Mar 2003
Location: Germany
Posts: 215
|
I ran tests with different lum-scales and I think there is something I do not understand.
Let's say I have a PQ clip with a maxcll of 1000, then I'm first going to try out a LUT setting maxcll of 1000 and a scale of 1.0. Now lets say I find that a scale 2.0 would be more fitting. What happens is that everything is brightened, including the whites. My tests showed that the whites are all pushed to the max brightness bin (1023 or 940 whatever you prefer)! So now there is loss of information since all whites are pushed together into the max brighteness. Shouldn't maxcll be thus increased according to the scale? Thus when the scale is increased to 2 the maxcll should also be increased to 2000, to leave place for all the whites? 2000 is perhaps too much, but I'm pretty sure it should be increased to something above 1000. Or is all that not the case and I somehow mess up the processing? Last edited by ErazorTT; 17th January 2022 at 15:01. |
17th January 2022, 17:13 | #98 | Link | |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 412
|
Quote:
|
|
17th January 2022, 17:18 | #99 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Ok, so, the issue doesn't seem to be closed and it's making us all talk a lot here: https://forum.doom9.org/showthread.php?t=181351 and yet the FFMpeg developers don't seem to care: https://trac.ffmpeg.org/ticket/9598 which is actually annoying me as this is a problem which affects our production environment.
That being said, I've actually temporarily worked around the thing by pretending to be using Avisynth 3.7.0 instead of 3.7.1 or 3.7.2 by using propClearAll() after every file I index, which is far from being perfect, but still... Anyway, here are the samples encoded with the official BBC LUT 4000 nits ColorSaturationSteps BT.2100 PQ - ColorSaturationSteps BT.2100 PQ_BBC_LUT_4000_nits: https://we.tl/t-KLyDHIyaeq GraySteps All - GraySteps All_BBC_LUT_4000_nits: https://we.tl/t-dqPcNT4B3y (Link valid for 7 days) |
17th January 2022, 19:06 | #100 | Link |
Registered User
Join Date: Mar 2003
Location: Germany
Posts: 215
|
Hm that is weird. The BBC lut produces two steps at almost exactly the same non-white brightness (red arrows in the grayBBC.png picture). These steps are apart 5% of the signal, which for PQ is really a lot in absolute terms. So that must be wrong!
Addionally, I took out my colorspace calculation sheet and found that a 4000nits white is at a 8bit value of 198 for a limited range signal. So actually in the picture the step with an orange arrow should already have been completly white, since that is at a value of 202. (Always assuming that I implemented my calcualtion sheet correctly which is quite a big if.) The lut generator however produces anything as I expected. See picture gray_4000_1.0.png. I also added the original grayPQ.png but with the signal values written as text so that anybody knows which step is which. https://drive.google.com/drive/folde...QT?usp=sharing Last edited by ErazorTT; 17th January 2022 at 19:26. |
Thread Tools | Search this Thread |
Display Modes | |
|
|