Encoding 4K HDR 4:2:0 10bit BT.2020

[benwaggoner]

Only if you used that when downsampling to 4:2:0. Type 0 is the same as current MPEG Progressive 4:2:0 used on DVD, Blu-ray and streaming services. Type 1 is JPG and type 2 is new. Seems like something everyone will get wrong if you stream 4:2:0 to the display for UHD 60p. I would stick with type 0.
I know some early HDR displays required 2. I'm not sure how universal that requirement is in practice. But good point about making sure that the chroma subsampling is done correctly to that mode. Slight chroma alignment errors aren't likely to be visible at UHD resolutions, but for low bitrate, low resolution HDR streams, it could become an issue.

[sneaker_ger]

I know some early HDR displays required 2. I'm not sure how universal that requirement is in practice.
UltraHD BluRay requires type 2 for BT.2020.

[benwaggoner]

UltraHD BluRay requires type 2 for BT.2020.
There we go. It's pretty much standard for all HDR-10 encodes AFAIK, which is what UHD BD uses for HDR.

[benwaggoner]

UltraHD BluRay requires type 2 for BT.2020.
Further speaking of which, I note that ffmpeg doesn't appear to have any way to set chromaloc. So getting the right chroma subsample location is going to be the domain of high-end specialized tools at the moment.

Unless someone else knows of anything generally available that can subsample to chromaloc 2.

[James Freeman]

Hi fellows,
Is there a quick way to convert a lossless screen capture RGB, mp4, 8bit to The new UHD spec (10bit, ST.2084, ST.2086 REC.2020)?
Don't mind about the range conversion (PC to TV) and color space.

Surami, I see you accomplished that, would you care to share the command line and what the source video can be?
I have used x264 by VideoLan and it simply works.

[kolak]

ffmpeg has these:

-chroma_sample_location <int> ED.V.... chroma sample location (from 0 to 6) (default unspecified)
unspecified ED.V.... Unspecified
left ED.V.... Left
center ED.V.... Center
topleft ED.V.... Top-left
top ED.V.... Top
bottomleft ED.V.... Bottom-left
bottom ED.V.... Bottom

-src_v_chr_pos <int> E..V.... source vertical chroma position in luma grid/256 (from -513 to 512) (default -513)
-src_h_chr_pos <int> E..V.... source horizontal chroma position in luma grid/256 (from -513 to 512) (default -513)
-dst_v_chr_pos <int> E..V.... destination vertical chroma position in luma grid/256 (from -513 to 512) (default -513)
-dst_h_chr_pos <int> E..V.... destination horizontal chroma position in luma grid/256 (from -513 to 512) (default -513)


but no idea is this is useful.

[surami]

Hi fellows,
Is there a quick way to convert a lossless screen capture RGB, mp4, 8bit to The new UHD spec (10bit, ST.2084, ST.2086 REC.2020)?
Don't mind about the range conversion (PC to TV) and color space.

Surami, I see you accomplished that, would you care to share the command line and what the source video can be?
I have used x264 by VideoLan and it simply works.

I don't know any simple way to convert a lossless 8bit, x264, RGB source to 10bit, ST.2084, x265, 4:2:0.

What I did in my experiment is to try to achieve a flat, unsaturated und unhighlighted source, what kind we can see if we watch an UHD HDR sample on a "normal", not for HDR content prepared playback setup (There is already a kind of solution to watch HDR sample as "normal" looking /not flat, unsaturated, etc./ video with the latest MPC-HC, madVR and LAVSplitter, I just tested it tomorrow).

So in AE I did some tricks with my source (ProRes, 4:4:4, 10bit, UHD) before I pushed out with Advanced FrameServer. The tricks were; shadow: +30, highlight: +5, master saturation: -25, contrast: -35 and there is an effect called "HDR Highlight Compression": 100%.

I did this kind of "reverse engineering", because as I experienced the ST.2084 command switches some settings on the UHD TV set, it bumps up the saturation, contrast, brightness and who knows what more settings, I didn't checked all of them.

So with the Advanced FrameServer plugin I pushed out a new looking AVI RGB24 video file (it's not rendering, just frameserving) and fed the AviSynth with it, where I used the command, what Asmodian suggested:

AVISource("D:\rendering\signpost.avi", audio=false).AssumeFPS(25,1)

Dither_convert_rgb_to_yuv(matrix="2020")

At this part of the pipe you have already an AVS file, what can be passed to the x265 by using the avs4x26x.

My command line was this:
avs4x26x_64bit.exe -L "x265_10bit.exe" --seek-mode safe --input-depth 8 --preset slow --subme 7 --crf 13
--profile main10 --level-idc 5.1 --no-high-tier --bframes 12 --keyint 25 --min-keyint 1 --no-open-gop
--colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc
--master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(8000000,1)"
--max-cll "800,400" --chromaloc 0 --repeat-headers
--output "D:\rendering\output.hevc" "D:\rendering\signpost.avs"
pause

[James Freeman]

Thank you for the reply.

I have a script error when running just the script in MPC-HC:
http://www.mediafire.com/convkey/e11a/3raelcy59pqjqq6zg.jpg

I made sure I have all the plug-ins. :(

[benwaggoner]

I don't know any simple way to convert a lossless 8bit, x264, RGB source to 10bit, ST.2084, x265, 4:2:0.
Yes, the challenge is that the source is sRGB (Rec. 709 primaries) with gamma. That can be mapped into a subset of PQ-space, where R'G'B'=255 could be mapped to a nominal 100 nits and color primaries mapped to the 709 subset of Rec. 2020. That should look just like the original on a properly calibrated display. But a sRGB-709 conversion would do the exact same thing and look at least as good. Doing a SDR 10-bit would avoid the limited range rounding issues and all that.

Mapping a SDR source to HDR output is a highly creative and subjective art. I wouldn't want to do it without HDR-capable reference monitors and grading software. It takes some high-end gear and software to even see HDR pixels values of a PC.

[benwaggoner]

ffmpeg has these:

-chroma_sample_location <int> ED.V.... chroma sample location (from 0 to 6) (default unspecified)
unspecified ED.V.... Unspecified
left ED.V.... Left
center ED.V.... Center
topleft ED.V.... Top-left
top ED.V.... Top
bottomleft ED.V.... Bottom-left
bottom ED.V.... Bottom

-src_v_chr_pos <int> E..V.... source vertical chroma position in luma grid/256 (from -513 to 512) (default -513)
-src_h_chr_pos <int> E..V.... source horizontal chroma position in luma grid/256 (from -513 to 512) (default -513)
-dst_v_chr_pos <int> E..V.... destination vertical chroma position in luma grid/256 (from -513 to 512) (default -513)
-dst_h_chr_pos <int> E..V.... destination horizontal chroma position in luma grid/256 (from -513 to 512) (default -513)


but no idea is this is useful.
The references I find to the first parameter suggest this is just metadata set in the bitstream. The latter parameters sound like they would actually impact the image processing, but I have no idea what the correct values would be.

ffmpeg documentation always seems to be lacking in direct proportion to how much documentation would be useful...

[James Freeman]

Thanks surami, all is well now.
Here are the resulting files: Small HDR Screenshot Test Videos (http://forum.blu-ray.com/showpost.php?p=12112147&postcount=439)

PS.
I've struggled the past two days with seeking problems in the resulting HEVC file, till I found that I had to Mux them. :facepalm:

[surami]

That's nice you did some experiments and you are partly happy with it! :) Would you be so kind to share the workflow (with code lines please, as we did it), that how did you produce that samples?

I experimented with this things last year, but as I remember well this was my hevc to mp4 mux code:
mp4box -add output.hevc:FMT=HEVC:fps=25 -new output.mp4
pause

[James Freeman]

Before the encoder I convert (not attach) the standard 709 gamma 2.2 image to BT.2020 ST.2084 using an ICC profile in Gimp (freeware photoshop), Image->Mode->Convert to Color Profile.
http://www.mediafire.com/download/hssxy3gwojyxnlj/ST.2084_Profiles.zip
Then I lower the output Levels to 190 (out of 255) to limit the HDR image to 1000nit.
Or better yet, use this curve for best results that look exactly like an HDR image from the UHD disc.
http://www.mediafire.com/convkey/7e52/d9l46dw6p66wf61zg.jpg
Process this way as many images as you like, make sure you name them in order 1,2,3,4,etc...

Then I create a lossless AVI video with VirtualDub.
Open first photo (1) and it will load all the rest automatically.
Change frame rate to 1/4 (its an images slideshow not a movie), color depth to 4:2:0, no audio.
Codec Lossless.
Export to 1.avi (20 lossless images = 60MB).

AVISource("E:\x265\1.avi", audio=false)
The video/image should already be in 2020 st.2084 as though it was graded on such monitor.
We already did that in Photoshop/Gimp.

avs4x26x.exe -L "x265.exe" --seek-mode safe --input-depth 8 --preset slow --subme 7 --crf 0 --profile main10 --level-idc 5.1 --no-high-tier --b-adapt 0 --bframes 0 --keyint 1 --min-keyint 1 --no-open-gop --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --output "E:\x265\output.hevc" "E:\x265\Script.avs"

muxer -i output.hevc?[fps=1/4] -o output.mp4
del output.hevc

L-Smash muxer as suggested here by sneaker_ger: http://forum.doom9.org/showthread.php?p=1742938#post1742938
https://www.dropbox.com/sh/7jjrz5i4wcxhf49/AADZlN98OhKAvLHVwfaXYK4da?dl=0
Works great.

This is not a movie but an HDR images slideshow with 1/4 fps (each image 4 second on screen) therefor: keyint 1, no B frames, crf 0.

Now you just click Run.bat and you have a beautiful HDR slideshow Output.mp4.
Looks great on HDR TV's as reported by testers. :)

Download the latest Avatar HDR update I've posted there. ;)

[surami]

Yes, the challenge is that the source is sRGB (Rec. 709 primaries) with gamma. That can be mapped into a subset of PQ-space, where R'G'B'=255 could be mapped to a nominal 100 nits and color primaries mapped to the 709 subset of Rec. 2020. That should look just like the original on a properly calibrated display. But a sRGB-709 conversion would do the exact same thing and look at least as good. Doing a SDR 10-bit would avoid the limited range rounding issues and all that.

Mapping a SDR source to HDR output is a highly creative and subjective art. I wouldn't want to do it without HDR-capable reference monitors and grading software. It takes some high-end gear and software to even see HDR pixels values of a PC.

Everything starts with proper exposure settings on a camera, which has a very good quality sensor at around (at least) ~12stops dynamic range or not? If you expose to the right (ETTR) and you bring back shadows, push down the highlights and set the gamma, contrast, saturation to the right values on a RAW footage, you are somewhere at a flat tonemapped HDR kind looking picture. You can use the HDR Compander and HDR Highlight Compression effects (https://helpx.adobe.com/after-effects/using/utility-effects.html#hdrcompandereffect) and you are ready to encode the footage as 4K UHD 10bit BT.2020 ST.2084 video and that's all or not? I will test this two effects as I have more time, I only tested that HDR Highlight Compression thing on my footage.

[surami]

I just uploaded a new experiment file (https://drive.google.com/file/d/0B9p82xjTYmAxcmNOSGtIMnNMWW8/view?usp=sharing).

1. ETTR, full resolution CR2 RAW files from a Canon 550D camera (http://www.dxomark.com/Cameras/Canon/EOS-550D) ->
2. 4K UHD Cineform 12bit RGBA intermediate footage, rendered straight from the RAW sequence ->
3/a. AE: UHD, sRGB, 32bit composition
3/b. Effects on the intermediate footage:
+ Exposure; Gamma Correction: 1,60
+ Hue/Saturation; Master Saturation: -30
+ Compress; Gain: 1, Gamma: 1
+ Shadow/Highlight; Auto Amounts
+ Color Balance; Shadow Blue Balance: -5
+ Expand; Gain: 1, Gamma: 1
+ HDR Highlight Compression: 100%
+ Unsharp Mask; Amount: 20, Radius: 50
3/c. Advanced FrameServer: RGB24 ->
4. AviSynth conversation ->
AVISource("D:\rendering\signpost.avi", audio=false).AssumeFPS(25,1)
Dither_convert_rgb_to_yuv(matrix="2020")
5. x265 rendering through avs4x26x:
avs4x26x_64bit.exe -L "x265_10bit.exe" --seek-mode safe --input-depth 8 --preset slow --subme 7 --crf 13
--profile main10 --level-idc 5.1 --no-high-tier --bframes 12 --keyint 25 --min-keyint 1 --no-open-gop
--colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)"
--max-cll "1000,500" --chromaloc 0 --repeat-headers
--output "D:\rendering\output.hevc" "D:\rendering\signpost.avs"
pause

6. muxing with mp4box:
mp4box -add output.hevc:FMT=HEVC:fps=25 -new output.mp4
pause

James, could you please share the footage on the Blu-ray forum for checking the file by UHD HDR TV set or player owners and report back?

[James Freeman]

Yes, here are my HDR videos: http://forum.blu-ray.com/showpost.php?p=12112147&postcount=439

You are doing it the hard way!
Simply convert your video to 2084 using the ICC profile I attached in my previous post.
Then lower the output level to 75% (1000nit).

You have an error in this clip, metadata says 80 nit peak instead of 800 (I assume).

[surami]

I just encoded it again, please redownload. I corrected the master display value and the max content light level, now the metadatas are fine:
--master-display "G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)"
--max-cll "1000,500"

On my Samsung UE40JU6000 UHD TV it looks quite nice.

I will try the method, what you suggested. Your last two samples (Revenant, Avatar) have clipping problems in the highlight area if I play on PC with latest MPC-HC + LAVSplitter + madVR, but on my TV is not so visible, maybe this is because of the poor source quaility. You did a very good job man. :)

I just asked you to share my footage or this topic on the blu-ray forum for checking by other people, who has 4K HDR TV set. (I don't want to register on that site, here is already a good discussion about the topic.)

[James Freeman]

Your video looks great.

Nothing is clipped in SDR to HDR clips if you select the 1,300 (or higher) nit value in madVR because it is higher than the encoded 1000 nit value.
The source of this images: Avatar 1 (http://www.blu-ray.com/movies/Avatar-Blu-ray/10629/#Screenshots) Avatar 2 (http://www.blu-ray.com/movies/Avatar-Blu-ray/7847/#Screenshots) Revenant (http://www.blu-ray.com/movies/The-Revenant-Blu-ray/123436/#Screenshots)

BTW, I don't think madVR uses the max-cll metadata to do anything (I tested that), nor any TV for that matter.
A TV just maps the 10bit code words of the ST.2084 curve values to the luminace, if it supports all the encoded ones.
If not, it compresses (or clips?) the code words like madVR does to it's highest luminance value.

I use HCFR to determine what code word in 8 bit would be 1000 nit on the static st.2084 curve because that is the peak of what most TV support today.
In 8bit (0-255 PC range) with the ST.2084 EOTF, 1000 bit would be at 75% input, so 256*0.75 = 190 (or 768 in 10bit full range).
That is why I always make sure your peak white before HEVC encoding is no higher than 75% of input range.
Looks like you already know that by checking your video without madVR and taking a screenshot.

The trick is how to make a good looking HDR video WITHOUT owning an HDR monitor right? :)
That answer is Histogram.

I will share your test video Blu-Ray forum.

[James Freeman]

From what I read in your post, you convert your Canon 550D RAW footage to something that will look good after encoding to HDR.
Are you converting the entire color space of the RAW footage from the Cannon 550D to RGB without using a Cannon 550D profile?

Then you de-saturate (-30) and lift the gamma (1.6) of the image so it will look right AFTER you encode it to BT.2020 ST.2084 HDR.
No wonder it took you a million tries till it looked right, you are doing it the hard way.

Can you please share a single screenshot from the Cineform 12bit RGBA intermediate footage (before you apply effects on the intermediate footage)?

[surami]

That is why I always make sure your peak white before HEVC encoding is no higher than 75% of input range.

Now I put a last effect on my footage named "Levels", I can see there the histogram. If I switch on and off the HDR Highlight Compression effect (set on 100%), then I can see exactly that thing what you wrote, it shrinks down the range to ~75%, so the highlights of the footage will be flat. But this is not enough to prepare the footage for encoding, you have to desaturate it and push up the gamma. I just tested it and if I don't do that, then the encoded file is too contrasty and oversaturated.

From what I read in your post, you convert your Canon 550D RAW footage to something that will look good after encoding to HDR.
Are you converting the entire color space of the RAW footage from the Cannon 550D to RGB without using a Cannon 550D profile?

Then you de-saturate (-30) and lift the gamma (1.6) of the image so it will look right AFTER you encode it to BT.2020 ST.2084 HDR.
No wonder it took you a million tries till it looked right, you are doing it the hard way.

Can you please share a single screenshot from the Cineform 12bit RGBA intermediate footage (before you apply effects on the intermediate footage)?

I'm encoding my Auto-ETTR exposed (by Magic Lantern) (https://en.wikipedia.org/wiki/Exposing_to_the_right) debayered (by Adobe Camera RAW) 14bit CR2 RAW files (http://www.canon-europe.com/for_home/product_finder/cameras/digital_slr/eos_550d/#p-specification13) to a Cineform 12bit RGBA 4:4:4:4 (https://helpx.adobe.com/premiere-pro/using/gopro-cineform-codec.html) (highest quality, max bitdepth) timelapse MOV footage using sRGB profile on the composition.

I uploaded the 4K UHD Cineform footage (https://drive.google.com/file/d/0B9p82xjTYmAxMHlGWl9YTWwtUk0/view?usp=sharing) (882 MB), if you or somebody want to play with it. Decoder (http://cineform.com/gopro-cineform-decoder) is needed to play the file, but use a previous LAVSplitter, it won't work with the latest one and OSD screen also says, that's a YUY2 4:2:2 8bit footage, but that's wrong.

[James Freeman]

sRGB video makes it a lot easier but If the video was RAW this makes things a little more complicated because I don't have your camera profile, but if I did, I would preserve a wider gamut to contain in 2020.

Here is your video using my method in After Effects: 4K Sun HDR Test.mp4 (http://www.mediafire.com/download/qp12w5fm5ng3tcm/4K+sun+HDR+test.mp4)

Exactly the same process on Martian SDR Trailer: Martian SDR to HDR Trailer.mp4 (http://www.mediafire.com/download/oeawrppgi3hzfv3/Martian+SDR+to+HDR+Trailer.mp4)
Source: https://www.youtube.com/watch?v=ej3ioOneTy8

I just applied a BT.2020 ST.2084 ICC profile (I linked it) to the video and a slight Curve to limit output to 75% in 32bit and lower the average picture level but keep the highlights at peak, THAT'S IT!
The saturation and brightness is already as it should be because the profile knows how to convert from 709 gamma 2.2 to 2020 ST.2084 automatically.

I also posted this on Blu-Ray.com
http://forum.blu-ray.com/showthread.php?t=273716&page=25

EDIT:
Martian trailer updated from 40s to full 2m:40s time.

[James Freeman]

I don't know your favorite video editing software but I use After Effects.
Converting from SDR to HDR is actually very easy in two simple steps.

1. Convert 709 to 2020 ST.2084 using an ICC profile.
2. Use Curves to limit to 75% output (1000 nit on 2084 curve), and expand the highlights while maintaining straight line for lower tones.
This curve is NOT a gamma curve, it is an Expander curve, basically a reverse S-Curve (you know what that is right?).
Export and encode with HEVC (2020, ST.2084, P3).
That's it!

If you ask me, the studios are doing just that for the UHD blu-rays.
They use a high bit depth 2K SDR master which was graded on a P3 Grade 1 monitor (SDR).
For Digital Cinema or Blu-Ray they simply apply a Color Profile at export and done.
For the HDR blu-ray they also expand (un-compress) the range back (reverse S-Curve), and apply a color profile.

http://www.mediafire.com/convkey/854e/n6i8b28aw1ihf63zg.jpg

[surami]

sRGB video makes it a lot easier but If the video was RAW this makes things a little more complicated because I don't have your camera profile, but if I did, I would preserve a wider gamut to contain in 2020.
If dcp files are good for you to do something, then you can download them (https://drive.google.com/file/d/0B9p82xjTYmAxSFRKSkhrLUQzcjA/view?usp=sharing). Here is an opensource dcp to icc converter (https://sourceforge.net/projects/dcp2icc/?source=typ_redirect).
The intermediate Cineform file is encoded from 125 piece of ETTR exposed Canon 550D CR2 RAW files (they are photos not videos).

Here is your video using my method in After Effects:
4K Sun HDR Test.mp4
It has been encoded correctly by your method, but there isn't any mastering on it. There should be a correction to get back the shadow details and push down the highlight details a bit.

Exactly the same process on Martian SDR Trailer:
Martian SDR to HDR Trailer.mp4
It looks fine on my TV set and madVR playback.

The saturation and brightness is already as it should be because the profile knows how to convert from 709 gamma 2.2 to 2020 ST.2084 automatically.
Thanks, now I understand and thanks for the ICC profiles too.

I encoded a new file (https://drive.google.com/file/d/0B9p82xjTYmAxQy0tMVJWZUxaZUU/view?usp=sharing) using a hybrid method (yours and my). The playback of my videos isn't so smooth then yours. Maybe this is because my CPU has weak power to serve this pipe (AE, ME, Advanced FrameServer, AviSynth, avs4x26x, x265), there are some missed frames as I see.

Anyway, my settings are:
sRGB, 32bit composition
+ Compress; Gain: 1, Gamma: 0,9
+ Shadow/Highlight; 10/5
+ Color Balance; Shadow Blue Balance: -10
+ Expand; Gain: 1, Gamma: 0,9
+ Color Profile Converter; In:sRGB, Out: Rec2020 ST.2084
+ HDR Highlight Compression: 100%
+ Unsharp Mask; Amount: 20, Radius: 50

+ I used --preset slower at x265.

[James Freeman]

Anyway, my settings are:
sRGB, 32bit composition
+ Compress; Gain: 1, Gamma: 0,9
+ Shadow/Highlight; 10/5
+ Color Balance; Shadow Blue Balance: -10
+ Expand; Gain: 1, Gamma: 0,9

Looks beautiful in SDR.. BUT you made a mistake with "HDR Highlight Compression".
Do not use "HDR Highlight Compression" it is not the same!!
You compress (s-curve) the highlight while my Curves EXPAND (anti S-curve), two exactly opposite things!
This is not an optional step, it is mandatory to lower average picture level while maintain peak white.

Your average picture level is too high, apply the Curves effect after Color Profile Converter, as in my image.
SDR to HDR (no black clip).ACV (http://www.mediafire.com/download/9w1tf37n60owx94/SDR_to_HDR_%28no_black_clip%29.ACV)
* Curves preset.

When the average picture level is high it's like an SDR screen with 1000nit peak.
The APL should be around 10nit for SDR and HDR, but the peaks are the difference between SDR and HDR.
Meaning, skin tones should be roughly the same luminance level in SDR or HDR, the extra luminance is there for things that actually need it.

[James Freeman]

Using HCFR I checked what input (0-255) would be what luminance (Nit) value up to 50 nit on Gamma 2.2 (black compensated) and ST.2084, and made a small table.

Most of SDR content resides below 50 nit (step 186), actually, most of it on average is around 6 nit.
From 50nit to 100nit (190 to 255) is the compressed highlights that the studio compressed from the high dynamic film or camera to SDR 1000:1 monitor.

What I did is create a Curve after the Profile Conversion so that the luminance values from 0.1 to 50 nit would be the same on SDR and HDR, actually, it supposed to be that way!

From 50 nit and up is the big difference!
Where the studios compress the highlights to 100nit on SDR, now is compressed (uncompressed actually) to 1000 nit on HDR.

To check this I applied this method on an SDR photo and compared to a official unprocessed HDR photo (from blu-ray.com site).
The processed SDR photo looks identical to the official ST.2084 HDR photo using this method.
You already can see that on the SDR to HDR Martian trailer I converted.

BT.2020 ST.2084 Profiles: http://www.mediafire.com/download/hssxy3gwojyxnlj/ST.2084_Profiles.zip
After Effects Curves Preset: http://www.mediafire.com/download/9w1tf37n60owx94/SDR_to_HDR_%28no_black_clip%29.ACV
Now that there is a preset you can work peacefully in 32bit floating and not worry about 8bit input/output values.

Have fun. :D

http://www.mediafire.com/convkey/c920/9tu677scngx6222zg.jpg

[surami]

We are making two totally different things.

You are encoding compressed 4:2:0 8bit SDR file to compressed 4:2:0 HDR 10bit file (but it's 8bit encoded as 10bit at the end, because you cannot get back the extra 2bits of information, it's not there as I know and of course you have to correct the things in post) with your method and you judge it with madVR on an 8bit display (what's your brand and type?). Because of your corrections (curve, level) the SDR 8bit footage looks the same as the HDR 10bit.

I'm encoding almost uncompressed 4:4:4:4 12bit RGBA files (ETTR) to compressed 4:2:0 HDR 10bit (I have to lift the shadows and shrink the highlights + compress it, because if I don't put HDR Highlight Compression effect to the end, the encoded footage is totally blown out, etc.) and I judge it with my calibrated Samsung UE40JU6000 4K TV set through USB pendrive.

Of course I also see strange colors on my DELL 2311H display, because it has 8bit panel, it can't show me the full 10bit spectrum, but on my TV it's a very nice HDR footage, a bit overexposed (it can be corrected before encoding, maybe with highlight 10-20). That's why 10bit HDR mastering display is suggested for grading and that's why I asked you to share the footage on blu-ray forum, maybe there are people with HDR displays.

But man these are experimentations and I learnt very much on this days, thanks for you too.

I'll be back as I have time, have a nice weekend.

[James Freeman]

I'm using a i1 Display Pro calibrated and profiled Dell 2410 IPS 10bit input (8bit+FRC), 3DLUT in madVR in 10bit (if needed).

You are right indeed that my source files are in 8bit and there is no way to reconstruct data that is not there.
Still, 1 nit is 1 nit in Gamma or 2084 EOTF, although a higher bit depth file has more bit words from 0 to 1 nit and has more room to work with the shadow detail without destroying it.

I agree that it is much better to work in 10bit if your monitor supports it, bit if you are using madVR it dithers down to 8bit and not even superman can see the difference between 125 and 126 in one pixel size.

[surami]

higher bit depth file has more bit words from 0 to 1 nit and has more room to work with the shadow detail without destroying it.
At first I was amazed, that how much dynamic range is in this 12bit ETTR intermediate footage after correction. A new file (https://drive.google.com/file/d/0B9p82xjTYmAxRmN5d3h6N0pVLVk/view?usp=sharing) is uploaded, highlights: 20 and I added two more values into my encoding command --pools 2 --frame-threads 1 because of the missing frames in the previous encodes. Now I think the CPU power is well shared, I have a i5 760 processor with 4 cores.

[James Freeman]

It looks the same, over exposed and the APL is much too high, you did not apply the Curves again.

I'll just copy my post here:
What we must understand is that most film shot on camera still use 18% output as middle grey in mind.
Middle grey is 50% input (128 in 8bit) that will give you 18 nit (cd/m2) output.
Who determined that 50% input is 18 nit output? 100 nit peak Gamma 2.4 CRT TV!
So the whole system is still adapted to SDR CRT technology with Gamma of around 2.4.

The camera (digital or film) may capture huge dynamic range but the captured middle grey card should still produce around 18 nit on your SDR or HDR display!

A calibrated SDR TV (100nit peak) and HDR TV (1000nit peak) have the same picture luminance and information from 0 to around 50 nit and SHOULD look the same.
75% input (190 in 8bit) for that matter is 50 nit on a CRT or any SDR TV.
It's from 50 nit and up what makes the difference between SDR and HDR TV.

On an SDR TV the huge dynamic range highlights are compressed from 50nit to 100nit with something called Log curve in the camera (film does that naturally).
While on HDR TV the highlight have much more room from 50nit to 1000nit so the highlights are not as compressed, in fact they have to be expanded from the camera Log curve.

The Light Illusion site has a terrible example.
His error is that he thinks the luminance is totally clipped above 100nit from the camera capture (like the HDR to SDR images blu-ray.com posted).
In reality all the F-Stops the camera captured are compressed from 50nit to 100nit (75% input to 100% input) or 190 to 255 in 8bit.
For HDR, 18 nit and 50 nit are STILL the same as SDR but above 50 nit, luminance is simply expanded and mapped.

In other words, from 50 to 1,000nit the captured data above 50nit has to be expanded linearly and mapped to input levels 44% to 75% in ST.2084.
Whether on SDR, from 50 to 100nit output is 75% to 100% input.

The Curve preset I've posted will do just that, it will maintain the same luminance values up to 50nit on HDR TV as the original SDR grade on an SDR TV, but expand the highlight to 1000 nit.
Why do you insist on using "HDR Highlight Compression" when it maps the luminance values absolutely wrong?

[surami]

James, could you please show me a properly corrected (shadows,highlights,etc.) and encoded footage with your own settings? I'll check that with my TV and I'll report back what happens.

If I apply the "PQ 4,000 ACES 800 BT.2020 to BT.709 2.2 Gamma" 3dlut on my Dell display + I use pure power curve 1.8, then my footage looks the same as on my TV, but the brightness is weaker, because the display's max nit is 300.

[James Freeman]

Just do what looks good to you in SDR and add Profile and Curves (my preset) after that.

sRGB, 32bit composition
+ Compress; Gain: 1, Gamma: 0,9
+ Shadow/Highlight; 10/5
+ Color Balance; Shadow Blue Balance: -10
+ Expand; Gain: 1, Gamma: 0,9
+Color Profile Converter
+Curves (my preset)


I don't want to grade your video, it's your work. ;)

[surami]

If I do what you wrote (+ level correction, it have to be done too), then my encoded footage has less dynamic range, less color, less shadow details, blown out highlights. So simple your curve is not for this workflow, it's for SDR to HDR conversation. But I wrote it already, that we do two totally different things.

That's ok, that my encoded footage looks overexposed, but that is because of my grading.

If you play my last uploaded (https://drive.google.com/file/d/0B9p82xjTYmAxRmN5d3h6N0pVLVk/view?usp=sharing) (#05) experiment with madVR + you add this 3dlut (http://madshi.net/HDRto800nits.madVR.rar) (4,000nits BT.2020 HDR encoding to 800nits BT.709 SDR, gamma 2.2) as calibration + you set the gamma correction/pure power curve to 1.8, then you will see the correct footage and it's circa how it looks on my 4K TV, the difference is brightness.

I think something is wrong with the encoded metadatas, that's why madVR doesn't corrects to the proper settings or I don't know.

[benwaggoner]

I'm not sure what the point is here. Making HDR from SDR is pretty fraught, and a decent effort requires significant shot-by-shot tweaking by a professional colorist. If the goal is to experiment with HDR encoding and playback, great, but I don't recommend anyone actually use such a simple automated approach to generate anything they think represents what "real" HDR looks like.

It would be good to get some HDR sources posted for people to play with. I'm not aware of any. The 4K 3D version of Big Buck Bunny had OpenEXR files, and if those have >1.0 values in them, an HDR regrade may be pretty straightforward for someone with access to the right equipment and HDR color grading experience.

Even shooting with an "HDR" camera generally results in log-space RAW files of some sort, and it's not a trivial task converting those to Rec. 2020. It's akin to DNG-to-sRGB in Lightroom, but 24 times a second, and with >10x the dynamic range to work with.

[surami]

It would be good to get some HDR sources posted for people to play with. I'm not aware of any.
I only know about a RED Epic Dragon 6K footage (https://www.youtube.com/watch?v=qZwPoFwlZJE) provided by Luke Neumann (Neumann Films) and RED One/RED Epic 4K/5K footages (https://www.youtube.com/channel/UCjJee-JAzoRRH5T0wqe7_tw/search?query=4k+clip) provided by Frederick Tschernutter (UHDmotion).
Even shooting with an "HDR" camera generally results in log-space RAW files of some sort, and it's not a trivial task converting those to Rec. 2020. It's akin to DNG-to-sRGB in Lightroom, but 24 times a second, and with >10x the dynamic range to work with.
I convert those slightly corrected (with Lighroom: temperature, lens profile correction, chromatic abberation, denoise, sharpening) Adobe Camera Raw debayered CR2 RAW frames (+ XMP info for each frame) to a 12bit 4:4:4:4 RGBA Rec.709 Cineform footage with the help of After Effects (32bit composition) and Media Encoder. There isn't any correction in AE at this step, I just put the RAW sequence on the timeline and encode. After that I import the Cineform footage and I put it on an another composition trying to achieve a similar looking footage, what you can see here (http://zsolt.gyapay.hu/images/portfolio/still-photography/still-photography-46.jpg), it was graded in Lightroom and this picture is a frame from the same ETTR shooted RAW sequence. I make that intermediate file, because continous fluid CR2 RAW sequence playback isn't possbile in AE or PP. I thought, that if I make shadow/highlight/color corrections on the intermediate footage + I add a Rec.709 to Rec.2020 ST.2084 profile correction (linked by James) + highlight compression, then I will be ready to make a 10bit x265 compressed HDR footage through Advanced FrameServer + AviSynth. My starting point was, that if I can achieve a good looking picture with Lightroom (of course with corrections), then I can do this with AE too on my intermediate file and I can play back my corrected 10bit compressed footage on a 4K TV set, which TV can play 10bit BT.2020 ST.2084 HDR files very nicely.

As you see I'm not professional, I'm just experimenting. I don't have 10bit HDR grading display and high end stuffs, that's why I'm asking people for checking my encoded footage. Is that correct or not? I don't know what madVR makes with 10bit encoded HDR BT.2020 ST.2084 footages, but my video looks quite good on my 8bit display. Yes it's not perfect, could be better of course (better corrections, grading, etc.), but it was a very quick way to do this experimentation.

At the weekend I went into a shop and checked my last encoded footage on a Samsung UE65JS9500 TV set, the playback was smooth, colors were nice (it could be better), gradients in the sky was also good.

[benwaggoner]

Going from RAW to 709 you lost all the HDR-ness. Ideally you could go from RAW to Rec. 2020 to at least get bigger chroma primaries. However all HDR uses the PQ curve instead of gamma, and I don't know of a way to do that in the Adobe tools yet, or really anything that'll work in a <$100K setup.

I expect things will get MUCH easier once the Windows 10 summer update is out.

[James Freeman]

Okay, I have managed to maintain a 10bit 4K workflow from start to finish.

First I export to PNG image sequence in 16bit, smallest file format compared to TIFF, DPX, EXR because PNG varies in size depending on the content.
EDIT: Okay, EXR is actually better and faster.
Next I use ffmpeg PNG image sequence to Y4M raw in yuv420p10, 16bit RGB full range PNG converted and dithered down to 10bit YUV 4:2:0 Limited range by ffmpeg with 1:1 color accuracy.

For anyone interested, here are my steps:
ffmpeg.exe -framerate 24 -i "E:\Sequence\img-%%05d.png" -strict -1 -pix_fmt yuv420p10 -vf colormatrix=bt601:bt709 "E:\x265\RAW.Y4M"

x265.exe --input-depth 10 --uhd-bd --preset slow --crf 18 --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --chromaloc 2 --output "E:\x265\output.hevc" "E:\x265\RAW.y4m"

mp4box.exe -add E:\X265\output.hevc -new E:\X265\output.mp4

After that sneaker_ger suggested that I don't need to generate the HUGE y4m file before pushing it to x265.
But simply I could pipe ffmpeg straight to x265 in a single process without wasting time or HDD space, so I did.
Now I run this "single click" batch file to go from PNG to MP4 automatically:

ffmpeg.exe -framerate 24 -i "E:\x265\Sequence\%%05d.png" -strict -1 -pix_fmt yuv420p10 -vf colormatrix=bt601:bt709 -f YUV4MPEGPIPE - | x265.exe --input-depth 10 --uhd-bd --preset slow --crf 18 --colorprim bt2020 --transfer smpte-st-2084 --colormatrix bt2020nc --master-display "G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,1)" --max-cll "0,0" --chromaloc 2 --output "E:\x265\output.hevc" - --y4m
mp4box.exe -add E:\X265\output.hevc -new E:\X265\output.mp4
pause

EDIT Important!
The "-vf colormatrix=bt601:bt709" should be changed to "-vf scale=out_color_matrix=bt2020" because the colormatrix command is only 8bit and it dithers down to 8bit.
While the scale=out_color_matrix=bt2020 command bypasses any conversion while retains correct chroma values between RGB to YUV conversion.

[surami]

Did you export a black to white gradient PNG sequence from AE? Could you please share what composition settings, effects on the footage, etc.? I could export a Cineform RGBA 12bit to test, it should be smaller than PNG as I think, but I don't know how ffmpeg supports the Cineform at this time.

[James Freeman]

Yes I created (in AE) and exported 16 bit black to white gradient in PNG and indeed the resulting 10bit file from ffmpeg is dithered perfectly from 16bit RGB full range to 10bit YUV 4:2:0 limited range.
Yes I can share my AE effect setting, right after this exporting will be done...

[James Freeman]

OpenEXR is smaller and faster than PNG and in Full Floating point.

[benwaggoner]

OpenEXR is smaller and faster than PNG and in Full Floating point.
And is what most HDR content is graded in.

An OpenEXR with values >1.0 is pretty much by definition HDR. Hollywood is moving towards having intermediate rendering stages preserving >1.0 data for later HDR regrading.

[nevcairiel]

Next I use ffmpeg PNG image sequence to Y4M raw in yuv420p10, 16bit RGB full range PNG converted and dithered down to 10bit YUV 4:2:0 Limited range by ffmpeg with 1:1 color accuracy

ffmpeg colormatrix filter does not support anything above 8-bit, so your conversion is losing a bunch of information along the way.

If you have a very recent version of ffmpeg, you can use the brandnew colorspace filter which adds full high bitdepth and bt2020 support.

On the other hand doing a 601->709 conversion and then encoding it as 2020 seems rather oddball either way, so something is crazy anyway.

[James Freeman]

The "colorspace" filter simply doesn't work for me, it always stops ffmpeg with a red error "Unsupported input primaries 2 <unknown>"
-vf colorspace=bt709

the 601 to 709 conversion is for ffmpeg so it retain correct chroma values so the resulting YUV has the same RGB values when decoded (I would skip that if the colors will not shift).
I don't need ffmpeg to do a color conversion to 2020, After Effects already done that much better.
It does not matter if you "label" the video 709 before you encode it, the image content IS in 2020 already done in After Effects and the only converter that has to know the color data inside the video/image is x265.
Similar if you take a screen shot of 2020 image, it will be in 709 when you paste it in a photo editor.

Thanks for the heads-up about the colormatrix filter being 8bit, I would really like to retain a high bitdepth across all the chain.
It looks like the colormatrix filer dithers down to 8bit but at this point I can't tell because the RGB to YUV conversion also dithers, maybe it is also in 8bit?
Dither on dither on dither, go figure at what point and what element did the down sampling...

[nevcairiel]

The "colorspace" filter simply doesn't work for me, it always stops ffmpeg with a red error "Unsupported input primaries 2 <unknown>"
-vf colorspace=bt709

You may need to specify the input colorspace, if the source file does not provide that information.
In fact, the source is RGB, is it not? If you specify the source color space, you should be able to influence the RGB->YUV conversion to use the correct colorspace from the get go, and avoid any need for a second conversion.

Something like this may work (untested):
ffmpeg.exe -framerate 24 -i "E:\Sequence\img-%%05d.png" -strict -1 -vf scale=out_color_matrix=bt709,format=yuv420p10 "E:\x265\RAW.Y4M"



the 601 to 709 conversion is for ffmpeg so it retain correct chroma values so the resulting YUV has the same RGB values when decoded (I would skip that if the colors will not shift).
I don't need ffmpeg to do a color conversion to 2020, After Effects already done that much better.

Well the problem is that any such conversion might cut off out of gamut colors, so not sure I understand whats going on, but it sounds like its working around another problem that should be resolved at the root, instead of this way.
In general you should really strive to limit the number of conversions to the essentials, as any extra step could re-dither the signal or even just cut off data, like with the colormatrix filter.

[James Freeman]

Done some testing;

ffmpeg yuv420p10 | x265 input-depth 10
http://www.mediafire.com/convkey/965b/fetf777a2b72ykyzg.jpg

ffmpeg yuv420p10 colormatrix=bt601:bt709 | x265 input-depth 10
http://www.mediafire.com/convkey/ab59/3evx4du6xrsnch3zg.jpg

ffmpeg yuv420p | x265 input-depth 8
http://www.mediafire.com/convkey/583e/5uz1k47sfl2cykazg.jpg

ffmpeg yuv420p colormatrix=bt601:bt709 | x265 input-depth 8
http://www.mediafire.com/convkey/583e/57zq9dnsrf3ku4gzg.jpg

Manipulated with Levels to clearly see the dithering, they all look smooth when displayed in madVR in 10bit without madVR dithering.
The only one that is not dithered and true 10bit is the first one.
BUT it appears that the colormatrix=bt601:bt709 dithering in the lowest bitdepth of the workflow and not always 8bit, 10bit in this case so I would assume it is safe to use this command.
The 8bit images are clearly dithered down to 8bit with or without the colormatrix=bt601:bt709 command.

What do you say Nev?

[nevcairiel]

The colormatrix filter supports nothing but 8-bit, so if you use it, the input gets converted to 8-bit before the colormatrix filter (with dithering), and potentially converted back to 10-bit after since thats what your output format asks for.
I trust reading the code over some image comparison any day. ;) Looking at the output of the ffmpeg command should also tell you which conversions it applies.

Like I mentioned above, it would be much better to try to control the RGB->YUV comparison to output the correct YUV material right away instead of chaining a 8-bit converter afterwards.

[James Freeman]

Something like this may work (untested):
ffmpeg.exe -framerate 24 -i "E:\Sequence\img-%%05d.png" -strict -1 -vf scale=out_color_matrix=bt709 format=yuv420p10 "E:\x265\RAW.Y4M"

It works! The Chroma is spot on and no conversion down to 8bit!
Now the chroma is correct and the image is in 10bit undithered by ffmpeg or x265.
Perfect for test patterns.
Thank you.

For future readers:
The "-vf colormatrix=bt601:bt709" should be changed to "-vf scale=out_color_matrix=bt2020" because the colormatrix command is only 8bit and it dithers down to 8bit.
While the scale=out_color_matrix=bt2020 command bypasses any conversion while retains correct chroma values between RGB to YUV conversion.

[Jamaika]

How don't I add this command "-vf scale=out_color_matrix=bt709" to output colormatrix is always BT601?
Strange, I was convinced that it is 'undef'.
Like I mentioned above, it would be much better to try to control the RGB->YUV comparison to output the correct YUV material right away instead of chaining a 8-bit converter afterwards.
Too bad that there is no preview. I understand that as I add "-vf scale=out_color_matrix=bt2020nc" and so will in the meantime conversion from RGB to BT709.

[James Freeman]

If anyone interested here is what the colormatrix=bt601:bt709 command does:
yuv444p is indeed only 8bit.

[auto-inserted scaler 0 @ 00000000006ed7e0] picking yuv444p out of 4 ref:rgb48le alpha:0
[auto-inserted scaler 0 @ 00000000006ed7e0] w:3840 h:2160 fmt:rgb48le sar:1/1 -> w:3840 h:2160 fmt:yuv444p sar:1/1 flags:0x4
[Parsed_colormatrix_0 @ 000000000033ada0] bt601 -> bt709
[auto-inserted scaler 1 @ 00000000003630a0] w:3840 h:2160 fmt:yuv444p sar:1/1 -> w:3840 h:2160 fmt:yuv420p10le sar:1/1 flags:0x4


scale=out_color_matrix=bt709:

[Parsed_scale_0 @ 000000000057a540] w:3840 h:2160 fmt:rgb48le sar:1/1 -> w:3840 h:2160 fmt:yuv420p10le sar:1/1 flags:0x4

[nevcairiel]

For future readers:
The "-vf colormatrix=bt601:bt709" should be changed to "-vf scale=out_color_matrix=bt709" because the colormatrix command is only 8bit and it dithers down to 8bit.
While the scale=out_color_matrix=bt709 command bypasses any conversion while retains correct chroma values between RGB to YUV conversion.

The important part is to also include the format after it, so its told what to convert to, ie "-vf scale=out_color_matrix=bt709,format=yuv420p10"
Although depending on the order ffmpeg processes things, it might work by using -pix_fmt as well, although I personally would rather explicitly tell the converter what to do.

But always look at the output of the command what its doing. :)

[James Freeman]

But always look at the output of the command what its doing. :)
Yes indeed, I had to use "-v debug" to see what's going on.
From the debug log it appears that -pix_fmt is obeyed and format=yuv420p10 can be omitted.

BTW, to test bitdepth, chroma and dithering accuracy I export a smooth grey gradient and color steps 16bit RGB to EXR, then run it through the converison and encoding chain (ffmpeg piped into x265).
I run "--input-depth 10 --colorprim bt709 --transfer bt709 --colormatrix bt709" in x265 so nothing extraordinaire is going on, just a straight forward and quality 16bit RGB Full to 10bit 4:2:0 Limited.
Played the video with madVR and carefully tested with various techniques to see what's going on.
The final image is indeed Dithered 10bit from 16bit with correct chroma.