Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 26th July 2018, 22:58   #1  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Help creating a LUT: BT709 to PQ

Hi,
I have a very particular request.
I've got the raw files of an interview of a famous football player. The interview has been shot in 3840x2160 1.77 FF 10bit BT2020 HDR, 4:2:0 Type 2 with the following specs:

Code:
Standard : Component 
Color space : YUV 
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits 
Color range : Limited 
Color primaries : BT.2020 
Transfer characteristics : SMPTE ST 2084 
Matrix coefficients : BT.2020 non-constant 
Mastering display color primaries : R: x=0.680000 y=0.320000, G: x=0.265000 y=0.690000, B: x=0.150000 y=0.060000, White point: x=0.312700 y=0.329000 
Mastering display luminance : min: 0.0000 cd/m2, max: 1000.0000 cd/m2 
Maximum Content Light Level : 1000 cd/m2 
Maximum Frame-Average Light Level : 400 cd/m2

So far so good.
Here comes the tricky part.
While the player is talking, the interviewer is gonna show some footages of his old games. Some of them are old XDCAM50 in BT709 and some other were old BetaCAM that I had to acquire from scratch. My target file is PQ, in other words BT2020 HDR10 and the specs must match the ones of the HDR file provided, which means that I gotta:

1) Deinterlace
Code:
tdeint(mode=2, order=-1, field=-1, mthreshL=6, mthreshC=6, map=0, type=2, debug=false, mtnmode=1, sharp=true, cthresh=6, blockx=16, blocky=16, chroma=true, MI=64, tryWeave=true, link=1, denoise=true, slow=2, opt=4)
2) Upscale to 3840x2160
Code:
nnedi3_resize16(target_width=1920, target_height=1080, mixed=true, thr=1.0, elast=1.5, nns=4, qual=2, etype=0, pscrn=4, threads=0, tv_range=true, kernel_d="Spline", kernel_u="Spline", taps=6, f_d=1.0, f_u=2.0, sharp=0, lsb_in=true, lsb=true)
3) Deband and convert 16bit stacked to 16bit interleaved
Code:
flash3kyuu_deband_f3kdb(range=15, Y=64, Cb=64, Cr=64, grainY=64, grainC=64, sample_mode=2, blur_first=True, dynamic_grain=False, opt=-1, mt=True, dither_algo=3, keep_tv_range=False, input_mode=1, input_depth=16, output_mode=2, output_depth=16)
4) Convert from 16bit interleaved to real 16bit
Code:
avs2yuv.exe "H:\AVS Script.avs" -depth 16 -csp auto - |
5) Convert from BT709 to BT2020 with 16bit precision and 4:2:0 to 4:2:0 Type2
Code:
ffmpeg.exe -i - -vf lut3d="Custom_LUT_2.cube" -pix_fmt yuv420p16le -vf scale=out_color_matrix=bt2020nc:out_h_chr_pos=0:out_v_chr_pos=0 -pix_fmt yuv420p16le
6) Convert from BT2020 4:2:0 Type2 SDR to PQ / HDR10
Code:
-vf lut3d="Custom_LUT.cube"
(The red part is the one I don't know how to do).

I was thinking about creating a LUT, but I've never done anything like this before... I mean, I never graded an SDR to HDR10 (PQ). Looking for solutions on google I found this site: https://www.mysterybox.us/store/sdr-...conversionsv10 but I would rather not spend 200 bucks for that.

Any thoughts? Thank you in advance.
__________________
Broadcast Encoder
LinkedIn

Last edited by FranceBB; 26th July 2018 at 23:01.
FranceBB is offline   Reply With Quote
Old 28th July 2018, 00:18   #2  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,512
Maybe use Resolve or some other NLE for this? I think they'll probably do it pretty well.
Blue_MiSfit is offline   Reply With Quote
Old 28th July 2018, 03:17   #3  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 758
Does it have to be a LUT? I could crank out a CUDA Avisynth+ filter (100+ fps for 3840x2160) in no time.
videoh is offline   Reply With Quote
Old 28th July 2018, 05:46   #4  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Quote:
Originally Posted by videoh View Post
Does it have to be a LUT? I could crank out a CUDA Avisynth+ filter (100+ fps for 3840x2160) in no time.
It doesn't have to be a LUT. I was thinking about a LUT 'cause everybody seems to be using LUTs for this kind of things.

I ended up making one and I tweaked levels and so on, but it I can't get colors right, especially red.

This is the LUT I created. Very "light" chroma, but still... https://mega.nz/#!3N0TSSLT!KkhslQPud...YxUcIV2BZzxuw0
__________________
Broadcast Encoder
LinkedIn

Last edited by FranceBB; 3rd August 2018 at 03:39.
FranceBB is offline   Reply With Quote
Old 28th July 2018, 08:21   #5  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 159
What camera shoots directly PQ?
Are you sure these are the raw files, seems more like graded material.

I would use Resolve (or any other tool that does color space transforms) take Rec2100 ST2084 as the input colorspace, use your own timeline colorspace and make the output colorspace Rec.709.
Obviously you would have to map the wider space onto the smaller Rec709 gamma and you also may have to do some gamut mapping to the smaller Rec.709 gamut.
Cary Knoop is offline   Reply With Quote
Old 28th July 2018, 14:41   #6  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Quote:
Originally Posted by Cary Knoop View Post
Are you sure these are the raw files, seems more like graded material.
They are graded, but we relied on an external service to record the interview and that's what they sent us. Not really sure what they did, but what they sent us is our "starting file" (that's what I meant with "raw").
__________________
Broadcast Encoder
LinkedIn

Last edited by FranceBB; 3rd August 2018 at 04:47.
FranceBB is offline   Reply With Quote
Old 3rd August 2018, 05:11   #7  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
For everybody reading (in the future as well) I solved the problem by creating my own LUTs.
In order to convert Rec709 to Rec2020 PQ, I tried to scale the Rec709 curve "as it is", but it ended up being too dark (of course).
Then I came across this White Paper by BBC https://pdfs.semanticscholar.org/daf...0884f61cd1.pdf and it opened my mind.
So, I made two LUTs, with Rec709 scaled by 400% and 800% and this somehow worked out as expected with results really close to the original Rec709 version and only slightly darker, but in PQ Rec2020:

BT709_to_PQ_400: https://mega.nz/#!3N0TSSLT!KkhslQPud...YxUcIV2BZzxuw0
BT709_to_PQ_800: https://mega.nz/#!DdMhUCgC!djQpewXxZ...T7FbHnVoYKV2l0

Please note that I limited the output to 1000nit.
Once again, guys, please note that the intent of these two LUTs is to make Rec709 SDR look almost identical in Rec2020 PQ HDR 'cause I'm gonna encode everything in Rec2020 PQ HDR due to the fact that I have mixed contents (SDR and HDR).
__________________
Broadcast Encoder
LinkedIn

Last edited by FranceBB; 3rd August 2018 at 05:15.
FranceBB is offline   Reply With Quote
Old 3rd August 2018, 06:49   #8  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 159
You can't make footage that is recorded in Rec.709 into HDR. It could be possible to make footage that is recorded in high dynamic range and graded in 10 bit (or higher) Rec709 turn into HDR but it depends.
And obviously, while you can put a Rec.709 color space into a Rec.2020 container you cannot invent colors beyond the Rec709 gamut.
Cary Knoop is offline   Reply With Quote
Old 3rd August 2018, 17:09   #9  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
@Cary Knoop... of course, making Rec.709 in PQ won't make it look better; what I needed was different. I had mixed sources: a legacy SDR with Rec709 and an HDR with Rec2020 PQ.
I needed to encode them together in a single video file as Rec2020 PQ, so I had to scale the SDR Rec709 parts to be displayed correctly in Rec2020 color space using the PQ curve.
This way, they are gonna look exactly identical to the original Rec.709 SDR file, but they will be encoded in a Rec2020 PQ file.
In other words, I can encode the scenes of the journalist interviewing the player (real HDR Rec2020 PQ), and the footages of the old games that were in SDR Rec709 all together in Rec2020 PQ.
Scaling them is not gonna add any additional information, but the "goal" is to encode legacy contents together with new contents in an HDR Rec2020 PQ file.
__________________
Broadcast Encoder
LinkedIn

Last edited by FranceBB; 3rd August 2018 at 17:11.
FranceBB is offline   Reply With Quote
Old 3rd August 2018, 17:21   #10  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 159
Quote:
Originally Posted by FranceBB View Post
@Cary Knoop... of course, making Rec.709 in PQ won't make it look better; what I needed was different. I had mixed sources: a legacy SDR with Rec709 and an HDR with Rec2020 PQ.
I needed to encode them together in a single video file as Rec2020 PQ, so I had to scale the SDR Rec709 parts to be displayed correctly in Rec2020 color space using the PQ curve.
This way, they are gonna look exactly identical to the original Rec.709 SDR file, but they will be encoded in a Rec2020 PQ file.
In other words, I can encode the scenes of the journalist interviewing the player (real HDR Rec2020 PQ), and the footages of the old games that were in SDR Rec709 all together in Rec2020 PQ.
Scaling them is not gonna add any additional information, but the "goal" is to encode legacy contents together with new contents in an HDR Rec2020 PQ file.
You are absolutely right with respect to the need for mapping the Rec709 to the Rec2020 gamut otherwise the colors will become oversaturated.

With respect to mapping the gamma curves what diffused white point do you use for the Rec709 mapped onto PQ and how do you treat the blacks?
Cary Knoop is offline   Reply With Quote
Old 3rd August 2018, 20:32   #11  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 620
Nothing particularly complicated, really.
On the left there's the graph representing the LUT I used (thanks to the BBC White Paper I linked above) and on the right there's the official ITU Proposal:



Red: Rec709 scaled to 800% (BBC White Paper - ITU Proposal).
Blue: PQ Curve (limited to 1000nit).

I didn't specify a "knee", otherwise it would have been different.
__________________
Broadcast Encoder
LinkedIn
FranceBB is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.