Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
16th July 2022, 22:40 | #181 | Link |
Registered User
Join Date: Jan 2019
Location: Canada
Posts: 568
|
There shouldn't be a need for the "dvhe0509" LUT anymore for AVS users: https://github.com/Asd-g/avslibplacebo
At least if you have a GPU. It also gives correct results, contrary to the LUT.
__________________
LG C2 OLED | GitHub Projects |
6th August 2022, 21:36 | #183 | Link | ||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,869
|
Quote:
Fixed and I mentioned your suggestion in the release notes. Linear Transformation v2.1 Released! Changelog: Quote:
|
||
10th August 2022, 17:09 | #184 | Link | |
Registered User
Join Date: Feb 2020
Posts: 538
|
Quote:
After this fix you can finally do even 512^3. |
|
10th September 2022, 17:26 | #185 | Link | |
Banned
Join Date: Oct 2001
Location: https://t.me/pump_upp
Posts: 811
|
Hello Francesco,
there is a problem with your 3D 65 lut BT709_to_HLG.cube. Quote:
This Avisynth+ script worked on this lut and the resulting colors are good. Code:
# (dgDecodeNV) dgSource("movie.dgi") # SDR AVC # (avsresize.dll) z_ConvertFormat(pixel_type="RGBP16", \ colorspace_op="709:709:709:l=>rgb:709:709:l", \ chromaloc_op="left=>top_left") # Apply LUT (vscube.dll) RGBP16 Cube("BT709_to_HLG.cube") # (avsresize.dll) z_ConvertFormat(pixel_type="YUV420P10", \ colorspace_op="rgb:std-b67:2020:l=>2020:std-b67:2020:l", \ chromaloc_op="top_left=>top_left") The movie was Those Magnificient Men in their Flying Machines. You can also try STUDIOCANAL's AVC intro. The white text is inverted to black. ------------- Dell XPS 15, Win10 Pro Last edited by frank; 15th September 2022 at 09:43. |
|
10th September 2022, 19:43 | #186 | Link | |
Banned
Join Date: Oct 2001
Location: https://t.me/pump_upp
Posts: 811
|
New test without Avisynth+.
I used NVEncC64 from rigaya. Best and fastest converter for NVIDIA. You only can use size 65 LUTs in this program. The importent cmd lines: Code:
-i "movie.mkv" ^ --vpp-colorspace matrix=bt709:bt2020nc,lut3d=BT709_to_HLG.cube,range=limited:limited ^ -c hevc --profile main10 -u p5 --qvbr 25 ^ ... --chromaloc 2 --colorprim bt2020 --colormatrix bt2020nc ^ --transfer bt2020-10 --atc-sei arib-std-b67 ^ Quote:
No more limited spikes, nice colors. So I think there must be a problem with Avisynth+ and the plugins, and levels. EDIT Possible cause. Depends how the 3dLUT is integrated. I think Avisynth cube reads elements as integer and not float, or has issues. So some elements for high levels destroy the colors. NVEncC64 correctly reads and converts elements as float. All LUTs from wswartzendruber are working well with my script, but PQ to HLG only. Last edited by frank; 16th September 2022 at 17:41. |
|
11th September 2022, 17:02 | #187 | Link |
Registered User
Join Date: Nov 2016
Posts: 151
|
Bravo Francesco, ottimo lavoro!
I'm thinking if you can make two further LUTs: XYZ to 2020 HLG XYZ to 2020 PQ - unless double conversion XYZ->SDR 709 then SDR-709->HDR (HLG or PQ) is equivalent, without too much loss. OT: how is possible to open a DCP (and other sources like HEVC 10 bit for example) and NOT getting only 8 bit? I tried ffvideosource, directshowsource, DSS2, but always only 8 bit... |
20th September 2022, 10:18 | #188 | Link | |||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,869
|
Quote:
It works just fine in full range here. This is from the movie Lucy, a BT709 XDCAM-50 SDR 100 nits masterfile: Code:
FFVideoSource("D:\Masterfiles\Lucy MPEG-2 FULL HD SDR BT709 YUV422 25i TFF 8bit.mxf") propClearAll() VideoTek(Mode="SDR") As you can see, it peaks at around 700mV, so 235 in Limited TV range. This is what happens when I go to HLG using my LUT which closely follows the BBC specification of their white paper as I've been reviewing it over and over and over again: Code:
FFVideoSource("D:\Masterfiles\Lucy MPEG-2 FULL HD SDR BT709 YUV422 25i TFF 8bit.mxf") propClearAll() #From 4:2:0 16bit planar Narrow Range to RGB Planar 16bit Full Range z_ConvertFormat(pixel_type="RGBP16", colorspace_op="709:709:709:limited=>rgb:709:709:full", resample_filter_uv="spline64", dither_type="error_diffusion") #From BT709 SDR to BT2020 HDR HLG with 16bit precision Cube("C:\Program Files (x86)\AviSynth+\LUTs\BT709_to_HLG.cube", fullrange=true) #From RGB 16bit planar Full Range to YUV422 10bit planar Narrow Range with dithering z_ConvertFormat(pixel_type="YUV422P10", colorspace_op="rgb:std-b67:2020:full=>2020:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion") VideoTek(Mode="HLG") As you can see, the value which was previously at 700mV (so 235, so 100% of the Limited TV Range SDR signal which corresponds to 100 nits in the BT709 SDR version) is now at 520mV, so we now have the reference white at 75% of the Limited TV Range HDR HLG signal. In my view, there's nothing wrong with this. Quote:
I'll put them on my "to do" list. Quote:
Are you sure you're using the latest ffms2 and Avisynth+? This is because on the old, legacy Avisynth 2.6, you would get 8bit planar all the time OR 16bit stacked or interleaved. On the new Avisynth+, however, it will index high bit depth sources at their native bit depth, so if you have a 10bit planar source it will be 10bit etc. There's one notable exception, though, and that is DCP. If you have a DCP, it means that it's a MJPEG2000 4:4:4 XYZ 12bit, however XYZ isn't natively supported in Avisynth (despite my desperate requests to support it dating back to 2017 and which I included in my meme collection https://forum.doom9.org/showthread.php?p=1953469). There are some people like Jean Philippe who had to work in XYZ, so what they did was to create a fake RGB64 output on his ConvertYUVtoXYZ() function with XYZ inside. Some other people like Hydra/HolyWu/Asd just recognized that XYZ isn't supported in Avisynth and therefore indexers like LWLibavVideoSource() won't output anything and report an error: Other people like Myrsloyk instead added automatic conversion, which is the case for ffms2, which brings me to the final reply of your question: if you index an XYZ 12bit file with FFVideoSource(), it will automatically bring it to 16bit, convert it to YUV and output a YUV 16bit stream. As you can see here, my input was a MJPEG2000 in UHD XYZ 4:4:4 12bit and after indexing with FFVideoSource() it has become a YUV 4:4:4 16bit planar: This is the Mediainfo of the original file: Code:
Video ID : 2 Format : JPEG 2000 Format profile : D-Cinema 4k Format settings, wrapping mode : Frame Codec ID : 0D010301020C0100-0401020203010104 Duration : 1 h 15 min Bit rate : 248 Mb/s Width : 3 996 pixels Height : 2 160 pixels Display aspect ratio : 1.85:1 Frame rate : 25.000 FPS Color space : XYZ Chroma subsampling : 4:4:4 Bit depth : 12 bits Scan type : Progressive Bits/(Pixel*Frame) : 1.147 Stream size : 130 GiB (100%) Title : Picture Track Color range : Full XYZ seen as is through MPV: https://i.imgur.com/46XuSBi.jpg XYZ indexed by FFVideoSource() in Avisynth which automatically converts to YUV 16bit: https://i.imgur.com/xi8pHH4.jpg |
|||
20th September 2022, 14:03 | #189 | Link | |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 410
|
Quote:
|
|
21st September 2022, 09:42 | #191 | Link | ||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,869
|
Vacation? Where are you chillaxing?
As to me, right now I wanna go nowhere. I've had enough of travelling after Amsterdam to attend IBC. Not just 'cause it was all over-expensive, but also 'cause I got caught in the Schipol airport strike and lost my flight after a 4h queue... I had to get back and book another flight (with no refund) and it costed me 400€... Madness. I've had enough. Quote:
Quote:
The reason for this method is strictly related to rights. While when we're both producers and broadcaster we can do what we want (like with sports in which we can use scene-referred HLG conversion if we have some non HLG camera, like the small ones for replays inside the net in football games), for movies and tv series whose rights are not ours, we gotta use display-referred mapping with the method I mentioned above 'cause otherwise if we apply the methods highlighted in ITU BT.2446 the content provider might reject the conversion as it will inevitably change the content. (We had a few occasions in the past where this happened and our contents got rejected, which is why I reviewed this LUT several times in the past). Last edited by FranceBB; 21st September 2022 at 09:44. |
||
21st September 2022, 22:01 | #193 | Link | |||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,869
|
I'll ask you again: which version of Avisynth are you on? Is it Avisynth 2.6.1 from 2016 or 3.7.2 from 2022?
If you're not on 3.7.2, you might wanna update https://github.com/AviSynth/AviSynthPlus/releases And don't worry, the only OS that were dropped in the meantime are Windows 98SE and Windows 2000, so given that it's highly unlikely that you're dealing with MJPEG2000 12bit using any of them, you should really update. The reason why I'm saying this is the line you wrote right here: Quote:
As a matter of fact it has been deprecated years ago. The reason is that back in the days of 2.6.1, Avisynth was only working in 8bit planar, while the only high bit depth supported were 16bit stacked (Dither Tools) and interleaved (HDR Core). Back in the days, FFVideoSource() outputted 10bit+ sources as 16bit stacked, while LWLibavVideoSource () outputted 10+ sources as 16bit interleaved. Nowadays, they both output in planar, so 10bit planar sources are kept as 10bit planar etc. Although there's nothing wrong in working in 16bit stacked as you're doing, you need to be careful about MSB and LSB and which filters support it etc. Let me be very clear: when you work in stacked, you must tell each and every function you're gonna call later that you're working with that. This is an updated build of ffms2 Quote:
I'm gonna quote myself: Quote:
Last edited by FranceBB; 21st September 2022 at 22:06. |
|||
26th September 2022, 21:05 | #194 | Link | |
Registered User
Join Date: Nov 2016
Posts: 151
|
Quote:
So, I've downloaded the updated ffms2 build: still 8 bit only, both with 10bit HEVC and 12bit DCP. Then I've downloaded 3.7.2; the same; but now ffvideosource doesn't work inside MP_Pipeline anymore; dss2 doesn't work at all, directshowsource load everything at 640x480... Reverting to 3.7.0 everything works as previously! Back on topic: shouldn't "BT2020_HLG" be "BT2100_HLG"? And, if so, couldn't "BT2100_PQ" be "BT2084_PQ" to differentiate it more from "BT2100_HLG"? |
|
26th September 2022, 22:34 | #195 | Link | ||||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,869
|
Quote:
https://github.com/pinterf/MP_Pipeline/releases Quote:
I clearly get 4:4:4 16bit planar converted to YUV when using an updated build of ffms2 and Avisynth+... Quote:
Are you using propclearall() right after indexing in modern version of Avisynth? (that's needed 'cause otherwise other plugins will misbehave) This is a really really really strange behavior. I'll test tomorrow again with MPPipeline() Quote:
Given that HLG is hybrid and backwards compatible with BT2020 SDR TVs, that is signalled as BT2020 and arib-std-b67, hence TVs which can read HLG will interpret the color curve in the transfer characteristics and will reproduce accordingly, while the ones that can't will ignore the color curve and just interpret it as BT2020 SDR, hence showing everything a bit dimmed (as the reference white is 75% instead of 100%) but still making use of the BT2020. When it comes to SMPTE2084, so PQ, on the other hand, such a color curve isn't backwards compatible with BT2020 SDR TVs 'cause it's totally logarithmic, therefore blacks are pushed higher and the waveform stays in the middle, which is why it's identified by the "BT2100" which is used for HDR. Anyway, none of this affects anything, it's just a naming convention to make some rationale around those LUTs and inside the parameters of the function I made, but yeah, they can be changed eventually, so I'll stay open to suggestions. |
||||
27th September 2022, 00:07 | #196 | Link |
hlg-tools Maintainer
Join Date: Feb 2008
Posts: 410
|
I would offer that the BT.2020 SDR fallback capability of BT.2100 HLG is largely a mechanic of HEVC signaling. With AV1, for example, I can find no way to specify a fallback transfer function. Ergo, with AV1, BT.2100 HLG streams are signaled as that and only that.
After reviewing these specifications and how they're organized, I would have expected three different naming patterns to occur: 1. BT2020_SDR 2. BT2100_HLG 3. BT2100_PQ |
27th September 2022, 17:32 | #197 | Link | ||
Banned
Join Date: Oct 2001
Location: https://t.me/pump_upp
Posts: 811
|
Quote:
You did the lut3d test with a file from your office. Quote:
The two known cube filters avsCube and dgCube have in Avisynth+ different issues with BT709_to_HLG.cube. The cause was the black level! If the source goes to black (0) or lower then avsCube inverts borders to white, and dgCube destroys colors, makes spikes. Also clearly visible in the vectorscope of VideoTek. avsCube with BT709 to HLG I was able to eliminate it by limiting the levels of the source. Code:
Levels(16,1,235,17,235,coring=false) Then I looked at your BT709_to_HLG.cube and dada... the first filter line has small negative values. It corresponds to black 0. This generates errors in Avisynth+ in the multiplication. It must be a filter problem with floating/integer conversion. In ffmpeg and NVEncC the lut3D filter is working without issues. I replaced this filter line with three zeroes and BT709_to_HLG.cube is working perfectly. I don't know where the negative shift is coming from. First line must be zero, we don't want super blacks. Please recalculate this LUT. _________________________________________ Dell XPS 15 9510, Win10 Pro, Avisynth+ 3.7.2 (64) Last edited by frank; 30th September 2022 at 11:31. |
||
30th September 2022, 15:12 | #198 | Link | |||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,869
|
Guilty :P
Quote:
I know what this is about: compression artifacts lead to overshooting. Essentially, in consumer-tier stuff, you have values outside the limited tv range which were "created" from scratch. I.e they shouldn't be there, but they are as a result of compression. In other words, once those values are taken into account, unexpected things can happen. Generally in professional master files and TX Masters, we try to avoid those as much as possible 'cause having them would result in QC Fail as operators always play the files with a Tektronix Hardware Waveform Monitor at their left hand side, taking the input signal from the hardware playback port (carried via SDI). Quote:
p.s just FYI, what you did is really good and should be done all the time as illegal values won't be displayed anyway as all monitors are RGB Full Range and therefore if you feed a monitor with a 16-235 YUV limited signal, it will expand it to 0-255, therefore anything lower than 16 will be negative, thus not displayed (same goes for out of range whites). Quote:
Since he rarely visits Doom9 any longer and he's staying on his own solitary ice castle ehm I mean his forum, I hope my fellow compatriot @Tormento will point him to this topic here. Last edited by FranceBB; 30th September 2022 at 15:15. |
|||
30th September 2022, 20:32 | #199 | Link |
Banned
Join Date: Oct 2001
Location: https://t.me/pump_upp
Posts: 811
|
The best way is to zero out that first line.
You know a lut3d is mathematically a linear matrix, or table: Code:
input vector x matrix = output vector All vector elements are greater or equal zero. The filter works internally with a fast integer format. So negative table elements can generate overflow, depending on word length. You'll get overshuts that don't exist in the input, epecially around point 0. Last edited by frank; 1st October 2022 at 09:20. |
1st October 2022, 11:36 | #200 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,869
|
I finally had a bit of spare time to look at this and I can't believe what I found...
The values from the first three rows are: Code:
-0.00086043 -0.00046653 -0.00063309 0.00905835 0.00289278 0.00107100 0.01858871 0.00607219 0.00265007 How did it happen? Easy. I always create the LUTs at work in Studio RGB due to our requirements, then I perform a direct trilinear interpolation to convert the values to Full Range RGB and then post here publicly so that they can be used everywhere. Simply put: trilinear interpolation got it wrong and mapped the Studio RGB values to non existing values in Full Range RGB. This led to negative coefficients which mean absolutely nothing in this case and are wrong. By zeroing the first line you've done the right thing 'cause you've removed the negative mapping. With a simple calculation, the actual value of the first row should be: 0.00819792 0.00242625 0.00043791 in other words, the first three rows are supposed to be: Code:
0.00819792 0.00242625 0.00043791 0.00905835 0.00289278 0.00107100 0.01858871 0.00607219 0.00265007 You can replace them safely and make some tests, but they're gonna be right. I will update the LUT on GitHub too and credit you in the pull request. EDIT: Done - https://github.com/FranceBB/LinearTr...fece62b7ea4c23 Thank you for pointing this out, very very very well done as I completely overlooked it. Last edited by FranceBB; 1st October 2022 at 11:51. |
Thread Tools | Search this Thread |
Display Modes | |
|
|