VDub2 Canopus HQX 16-bit negotiation issue

[poisondeathray]

Side note- ProRes444 modes do pass those reserved bits (in official Apple encoder).

Good to know;. But you would expect it to store 0-1023 for 10bit and 0-4095 for 12bit variants because the 4444 variants need to store the alpha channel. You can't have clipping in an alpha channel, or it would be completely useless

[kolak]

Well, alpha is a separate "stream" inside ProRes444. It's just losslessly compressed bits, more like a data than video.
444 is always encoded as 12bit. Alpha can be processed as 8 or 16bit.

[poisondeathray]

kolak did you use resolve to encode Pr444 or something else ? Did you use that test pattern ? Can you upload the Pr444 output ?


ffmpeg pr444 variant encode results in 4-4091 in 12bit, with the expected gaps at 12bit

If you down convert to 10bit, you get 1-1023, but smooth, basically perfect except for the missing zero value. PSNR y:90.300512

And if you feed native 12bit src, the ffmpeg pr4444 variant is still 4-4091 (and with gaps at 12bpc , not smooth)

[kolak]

It was done in Resolve on Mac.

ffmpeg ProRes444 encoder is 'broken'- 10bit only. They fixed only decoder.

[poisondeathray]

Thanks kolak, I don't have handy access to a Mac

It's ok if resolve processes it - and when downconverted and exported from resolve as v210 it's the correct 0-1023

But if you take the pr444 converted from ffmpeg imported into resolve then export as v210 it's 0-1022. So there are differences in either the up conversion and encoding of pr444 as detected by resolve. Yes, ffmpeg pr444 is only 10bit, but theoretically you should get same results since input source was 10bit anyways

But both mac/resolve pr444 and ffmpeg pr444 samples are 4-4091 according to libavcodec decoder. Both 1-1023 in downconverted 10bit. There must be differences in the way they decode and/or downconvert to 10bit as well. I'm looking into it



(BTW - There is a bug in the ffmpeg -c:v prores version with profile:v 4 alpha channel . It's been patched for prores_ks , but not prores (ticket 8509) . When decoded by libavcodec at 12bit, it's supposed to be 4095 in 12bit, but it's 4092. In other compositing programs, it's not 1.0, it's 0.99. This results in partial transparency)



12bit tests
Here is a 12bit444 HEVC perfect gradient. 4K DCI Y 0-4095 . Older Resolve version decodes it ok, so v17.x should too. Can you run it through a 4K/DCI timeline and export a Pr4444 frame and upload it when you get a chance?

https://www.mediafire.com/file/73cx5yq6kc1lzlk/hevc_12bit444_perfect_greyscale_ramp_Y(0-4095)_4K_DCI.mp4/file

[kolak]

Free Resolve doesn't decode it (neither you can do DCI frame size in free Resolve).
If you want to pass 12bit then I assume best format for Resolve will be DPX 12 bit or eg. 16bit TIFF. Use UHD size not DCI 4K.

[poisondeathray]

I thought you had studio version ? It decodes ok. I just wanted to test Prores444 12bit Y levels

You can check if resolve is decoding 12bit correctly by temporarily setting full range attributes, exporting 12bit DPX, then checking levels in vapoursynth or ffmpeg . Full range Y to RGB export is 1:1 mapping for Y to R,G,B

The reason for 4K choice is perfect gradient 4096 values. Same reason for 1024 width for 10bit test. Easier to identify workflow problems. If you scale or change width resolution you will create duplicate values or gaps.

For example, if you check dnxhr 12bit 444 or 422, in YUV, the range is correct 0-4095, but there are slight gap abnormalities in the Y waveform, with some intermediate regular values missing. How "solid" is official pr444 ?

[kolak]

I done ProRes444 test long time ago in AE with steps for 12bit inside 16bit TIFF. Cineform/DNxHR (RGB mode) were fine and ProRes was generating some strange artefacts in lowest levels (dither alike). It could be just AE etc. No idea. RGB based codecs were absolutely fine.
ProRes is always YUV internally, so RGB data which may go to it as RGB64A gets converted to YUV. If you look at ProRes private headers then there is an info what pixel format was on input. In many cases it's RGB64A, but Apple is using 32bit float YUV as well (FCPX does it I think). Problem is that this header may not be set at all or simply lying.

[kolak]

Apparently 12bit mode on SDI doesn't use those reserved bits, so maybe this is why ProRes444 which is 12bit passes all values.

[poisondeathray]

Apparently 12bit mode on SDI doesn't use those reserved bits, so maybe this is why ProRes444 which is 12bit passes all values.

According to SMPTE ST 372 and 403 for dual SDI - values 0-15 and 4080-4095 are reserved for 12bit

What is the equivalent v210, v410 fourcc for 12bit? v212, v412 ? I don't see 12bit uncompressed 4:2:2 or 4:4:4 options anywhere

[kolak]

Never seen "v210" for 12bit.
Here is explanation which I've seen:

RGB 4:4:4 (and RGBA 4:4:4:4) 10-bit can't use the full 10-bit range when carried on SDI interfaces. If they do, there is a risk of certain patterns of pixels causing the receiver to lose synchronization.

This is because RGB 4:4:4 10-bit uses the same bit layout in transmission as RGBA 4:4:4:4 10-bit, and so uses all of the available 40 bits per pixel. This means it would be possible to send illegal synchronization bit sequences if the full 0-1023 range of levels is allowed in the pixel values.

RGB 4:4:4 12-bit does not have this problem, as it only uses 36 of the available 40 bits per pixel, and they are arranged such that no 36-bit value can represent a synchronization bit sequence. So the full 0-4095 levels can be used.

[poisondeathray]

Not for SDI...


Full range, SDI Range, SMPTE / Legal range


https://professionalsupport.dolby.com/s/article/Dolby-Vision-Legal-Range-workflows-for-home-distribution?language=en_US


Video Range standards



10-bit Video Range Options (code values)

Full Range 0-1023

SDI Range 4-1019

SMPTE/Video/Legal Range 64-940



12-bit Video Range Options (code values)

Full Range 0-4095

SDI Range 16-4079

SMPTE/Video/Legal Range 256-3760



https://www.dcimovies.com/drafts/DCI-DRAFT-HDR-D-Cinema-Addendum_v09_2018-1116.pdf

Note: If the data is transported over SMPTE ST 372 (SDI dual link), code values 0-15 and 4080-4095 are reserved
(illegal) code values and these code values will be clipped.

[kolak]

No idea.
We would need SDI spec. I may be able to get confirmation from the actual spec, but at the end it's not that big deal.

[Hotte]

Hi,

I am coming back to the TV/Full-Range discussion, because still I am not 100% sure how to handle this correctly. I have got much more 8-bit full-footage, but also TV-range stuff.

So a maybe silly question at the beginning: Why does full-range footage on a tv-range player look so much more contrasty ?

I thought, that full uses the lower and upper ends of the range and TV doesn't.

I'd understand if these ends are just simply being cut off and you miss some blacks and whites.

I'd also understand that - if it is smarter - the full contrast range is being mapped proportionally to fit into a tighter range and you end up with the same contrast appearance but less contrast variation within the range.

What I do not understand is why the whole contrast range looks completely different as if sbd boosted the gamma curve. Is this a compatibility issue and so interpreted the wrong way ?

I was told in this thread that it is ok to record and filter with full (since this gives more headroom especially in 8-bit) but the final result should be in tv-range. However I do not want the contrast to be completely spoiled and I do want to make use of the full contrast range that I recorded.

How is that to be achieved in AVS+ ?

[poisondeathray]

Hi,

I am coming back to the TV/Full-Range discussion, because still I am not 100% sure how to handle this correctly. I have got much more 8-bit full-footage, but also TV-range stuff.

So a maybe silly question at the beginning: Why does full-range footage on a tv-range player look so much more contrasty ?

I thought, that full uses the lower and upper ends of the range and TV doesn't.

I'd understand if these ends are just simply being cut off and you miss some blacks and whites.

I'd also understand that - if it is smarter - the full contrast range is being mapped proportionally to fit into a tighter range and you end up with the same contrast appearance but less contrast variation within the range.

What I do not understand is why the whole contrast range looks completely different as if sbd boosted the gamma curve. Is this a compatibility issue and so interpreted the wrong way ?

I was told in this thread that it is ok to record and filter with full (since this gives more headroom especially in 8-bit) but the final result should be in tv-range. However I do not want the contrast to be completely spoiled and I do want to make use of the full contrast range that I recorded.

How is that to be achieved in AVS+ ?


Normal range video takes Y 16-235 (Y 16-235 black to white) and applies a contrast stretch to RGB 0-255 (black to white) on a computer monitor. Every thing looks normal. Black and white point match and are correct

If you take full range 8bit video (Y 0-255 black to white) , and display it incorrectly by "mapping" the same Y=16-235 to RGB 0-255, the contrast increases, and you lose the ends Y 0-15 and 236-255. Black and white point do not match.

Correct full range display would be Y 0-255 mapping to RGB 0-255. Full range video is not common for end delivery. Things like DVD, Blu-ray, web video, are all typically standard range

But modern TV's and players can handle full range video that is created and flagged correctly - but there are millions of devices, older TV sets , setups that do not handle full range correctly - Hence the earlier recommendation to deliver standard range

[Hotte]

Thanks poisondeathray.

I can identify all my clips that are full range.

(How) can I convert them from full to TV range with AVS ?

[poisondeathray]

(How) can I convert them from full to TV range with AVS ?


One way is to use levels or similar . This "squishes" full range to limited range

e.g. 8bit full to limited
levels(0,1,255,16,235,false)

10bit full to limited
levels(0,1,1023,64,940,false)

You can use ExtractY, ExtractU, ExtractV if you wanted to control Y,Cb,Cr separately along with CombinePlanes

There are other ways - such as coloryuv pc->tv presets, avsresize, smoothlevels (preset pc2tv) , a few others... but fundamentally it can be a problem to express 256 levels in 219 "slots", or 1024 levels in 876 "slots". If there are problems like banding, you can grade and work at higher bit depths and dither the down conversion to help reduce the issue

There is nothing wrong with full range video, if you know your playback scenario handles it correctly. But full range video is not recommended for general use.

[Hotte]

poisondeathray, I picked ColorYUV() from your list of recommendations as it is an avs+ native implementation, HBD-friendly and easy to use. My tests hat very good conversion results (full>tv). No visible color or contrast shifts, banding or whatever even with my very critical 10-bit blue sky gradient.

I tried out different older full-video sources of mine and found that each source type behaves differently: Some need tv-conversion to look the same, some not (although export- and codec paths where the same in all cases). Don`t know really. I think I have to test each source and implement source-specific conversion paths.

If you were asked for a player software for Windows which represents some standard oder "reference" concerning correct playback of color range , which one would you pick ? VLC ? Mediaplayer classic ?...

I read through most of this thread again - so much knowledge... but still I am not able to say wether I now should keep on recording my videos in full mode or wether I should switch the camera to tv-mode after all you said about it.

I still have in mind, that "full" must in some way give more headroom for contrast which I find important, especially for 8-bit footage. Or would you say this adantage is less relevant than the probable loss of quality I might envoke by the obvious need to remap using tools like ColorYUV() in most of the playback use cases today ?

[poisondeathray]

I tried out different older full-video sources of mine and found that each source type behaves differently: Some need tv-conversion to look the same, some not (although export- and codec paths where the same in all cases). Don`t know really. I think I have to test each source and implement source-specific conversion paths.

Some might not be flagged correctly, or incorrect metadata

Often intermediate file types won't have correct metadata eg. uncompressed v210 won't have any, and it's up to you to interpret it correctly in the editor

Final formats should always have correct metadata, so the player/hardware/software knows how to display it correctly


If you were asked for a player software for Windows which represents some standard oder "reference" concerning correct playback of color range , which one would you pick ? VLC ? Mediaplayer classic ?...


Probably MPV , definitely not VLC . MPC can be ok if it's configured correctly



I still have in mind, that "full" must in some way give more headroom for contrast which I find important, especially for 8-bit footage. Or would you say this adantage is less relevant than the probable loss of quality I might envoke by the obvious need to remap using tools like ColorYUV() in most of the playback use cases today ?

Full range is generally better if you're doing something to it in other programs , such as grading manipulations, because the range is spread out

If you're not doing much of anything else, and the final goal was limited range - then limited range would generally be the better acquisition choice

[Hotte]

This was a helpful excursion that allows me to draw the following exclusions:

Older 8-bit "familiy and people" footage which was recorded in full range (because I just simply didn't know better) should have been better recorded in TV range, because this is sth you do not want to grade a lot but just simply "quick cut and watch" on different PC-Monitors, TV sets or projectors.

Lately high quality landscape filming comes to the fore. I am recording in 10-bit with flat profiles (maybe Log-profiles later or HDR). The goal is to pull the very best quality out of it so grading in most cases is mandatory.

I understood that full range is probably the better choice for the latter. However the price might be to add ColorYUV(levels="PC->TV") at the end of the process if the video is intended for non-full range players/monitors and in this case to take care that ColorYUV() is not doing any harm to the colors and creates no banding.

And generally the final encoder should be one that tells the color range to the player. I will be trying this out with mpv.

Do this sound coherent to you ?

Thanks!

[poisondeathray]

This was a helpful excursion that allows me to draw the following exclusions:

Older 8-bit "familiy and people" footage which was recorded in full range (because I just simply didn't know better) should have been better recorded in TV range, because this is sth you do not want to grade a lot but just simply "quick cut and watch" on different PC-Monitors, TV sets or projectors.

Lately high quality landscape filming comes to the fore. I am recording in 10-bit with flat profiles (maybe Log-profiles later or HDR). The goal is to pull the very best quality out of it so grading in most cases is mandatory.

I understood that full range is probably the better choice for the latter. However the price might be to add ColorYUV(levels="PC->TV") at the end of the process if the video is intended for non-full range players/monitors and in this case to take care that ColorYUV() is not doing any harm to the colors and creates no banding.

And generally the final encoder should be one that tells the color range to the player. I will be trying this out with mpv.

Do this sound coherent to you ?

Thanks!

Yes, but for the RGB to YUV step, (from a full range YUV source initially), if you're grading , or working in resolve, you're working in RGB

It would be faster (and better quality) to go directly to limited range YUV from RGB; instead of RGB to full range YUV then to limited range YUV in 8 or 10 bit (using a coloryuv or similar step) - because the latter involves an extra quantization step. You can demonstrate that 2 steps introduces more banding. Basically, fewer steps in integer math (resolve works internally in float), results in higher quality, fewer rounding losses

[Hotte]

Well, I do not grade with Resolve. I do the grading in Edius usually in YUV (I might be using the Edius-internal Primary Color Correction Filter which is suspected to work in RGB, but let's leave that out).

If this is not what you were refering to, then I could understand that it might be better to record tv-range rather than full-range (which applies to sensor-RBG > YUV) with having to apply ColorYUV() for tv-range at the end of the yuv-chain because quality loss of the additional ColorYUV() quantisation is likely to be more relevant than the narrower range. To put it short: Only record Full if you are sure to also display Full.

This way ?

[poisondeathray]

In general, you'd only use full if you're going to be grading it more than some minor adjustments (if you're still going to limited, full range acquisition can help when grading) , or if you final output is full

How are you adjusting the log footage ? If you're using LUTs , there is going to be an RGB step somewhere (at least internally or hidden)

[Hotte]

Ok, so I need to weigh the decision according due to the level of grading needed. Since I always fight with high contrast with my landscape footage I will probably stay with Full while recording and maybe I will have a projector being capable of displaying Full as well. Seems more future-proof to me and ColorYUV() did quite a good job I found.

Currently I am not shooting LOG because it has to be paid extra with the Panasonic G9. I am using a profile called Cinelike-D which is pretty soft. I grade it to personal gusto using standard YUV-Tools in Edius. But if the GH6 comes out I will probably go for either YUV422 10-bit 50p with HLG2100 or - if it is available - VLOG but with V-Gamma (very wide gamut).

I want to move to wide gamut about which I had an interesting discussion with Frank.

So I start to shoot HLG.2010/2100 which is also available on my G9 (10-bit but only 25p). There is a grading option in the Primary Color Correction Filter in Edius, where you can also apply LUTs to transate HLG2100 to Rec709 until I am equipped with an HLG capable display or projector (might take some time).
The conversion results are not very satisfying. The reds and greens do not look very natural. Just started...

[poisondeathray]

Ok, so I need to weigh the decision according due to the level of grading needed. Since I always fight with high contrast with my landscape footage I will probably stay with Full while recording and maybe I will have a projector being capable of displaying Full as well. Seems more future-proof to me and ColorYUV() did quite a good job I found.


It's just a general suggestion, not some fixed rule - try it out and see what works for you

Full range is "better" in the sense that shadows, midtones, highlights are less compressed. ie. A given scene is described more accurate or better. You have 1024 "slots". It's easier/better to make adjustments or tease out details in a given range