Why ProRes/DNxHR 4444 should always be legal/limited range?...

[groucho86]

I know integer-based YCbCr codecs should always be legal range in order for no clipping to occur when returning to RGB. I've proved this with SMPTE bars out of Resolve.

...But I can't articulate exactly *why* that is.

I attempted to reproduce the concept in Vapoursynth: Test 1 is pretending someone rendered ProRes 4444 12bit Full Range, Test 2 is pretending someone rendered ProRes 4444 12bit Legal/Limited range.

The results look like this:
Starting point is RGB 12bit Full: 0 4095 4095

===========

TEST 1 - Full to Full to Full
Converted to YCbCr 12bit Full: 3224 2517 0
Converted to RGB 12bit Full: 0 4095 4094

------------

TEST 2 - Full to Limited to Full

Converted to YCbCr 12bit Full: 3015 2459 256
Converted to RGB 12bit Full: 0 4095 4095

Can someone explain, in not-quite-laymen's terms, why that's the case?

The Vapoursynth python code:

import vapoursynth as vs
core = vs.core

cyan_rgb = core.std.BlankClip(width=1920, height=1080, format=vs.RGB36, color=[0,4095,4095] )

r = core.std.PlaneStats(cyan_rgb, plane=0, prop='Gamut')
g = core.std.PlaneStats(cyan_rgb, plane=1, prop='Gamut')
b = core.std.PlaneStats(cyan_rgb, plane=2, prop='Gamut')

print('Starting point is RGB 12bit Full: {} {} {}'.format(r.get_frame(0).props['GamutMax'],
g.get_frame(0).props['GamutMax'],
b.get_frame(0).props['GamutMax']))

print('\n===========\n')

print('TEST 1 - Full to Full to Full')

cyan_ycbcr = core.resize.Bicubic(cyan_rgb, format=vs.YUV444P12, matrix_s="709",
range_in_s='full', range_s='full')

y = core.std.PlaneStats(cyan_ycbcr, plane=0, prop='Gamut')
cb = core.std.PlaneStats(cyan_ycbcr, plane=1, prop='Gamut')
cr = core.std.PlaneStats(cyan_ycbcr, plane=2, prop='Gamut')

print('Converted to YCbCr 12bit Limited: {} {} {} '.format(str(y.get_frame(0).props['GamutMax']),
str(cb.get_frame(0).props['GamutMax']),
str(cr.get_frame(0).props['GamutMax'])))


cyan_rgb2 = core.resize.Bicubic(cyan_ycbcr, format=vs.RGB36, matrix_in_s="709",
range_in_s='full', range_s='full')

r2 = core.std.PlaneStats(cyan_rgb2, plane=0, prop='Gamut')
g2 = core.std.PlaneStats(cyan_rgb2, plane=1, prop='Gamut')
b2 = core.std.PlaneStats(cyan_rgb2, plane=2, prop='Gamut')

print('Converted to RGB 12bit Full: {} {} {}'.format(r2.get_frame(0).props['GamutMax'],
g2.get_frame(0).props['GamutMax'],
b2.get_frame(0).props['GamutMax']))
print('\n------------\n')
print('TEST 2 - Full to Limited to Full\n')


cyan_ycbcr = core.resize.Bicubic(cyan_rgb, format=vs.YUV444P12, matrix_s="709",
range_in_s='full', range_s='limited')

y = core.std.PlaneStats(cyan_ycbcr, plane=0, prop='Gamut')
cb = core.std.PlaneStats(cyan_ycbcr, plane=1, prop='Gamut')
cr = core.std.PlaneStats(cyan_ycbcr, plane=2, prop='Gamut')

print('Converted to YCbCr 12bit Limited: {} {} {} '.format(str(y.get_frame(0).props['GamutMax']),
str(cb.get_frame(0).props['GamutMax']),
str(cr.get_frame(0).props['GamutMax'])))


cyan_rgb2 = core.resize.Bicubic(cyan_ycbcr, format=vs.RGB36, matrix_in_s="709",
range_in_s='limited', range_s='full')


r2 = core.std.PlaneStats(cyan_rgb2, plane=0, prop='Gamut')
g2 = core.std.PlaneStats(cyan_rgb2, plane=1, prop='Gamut')
b2 = core.std.PlaneStats(cyan_rgb2, plane=2, prop='Gamut')

print('Converted to RGB 12bit Full: {} {} {}'.format(r2.get_frame(0).props['GamutMax'],
g2.get_frame(0).props['GamutMax'],
b2.get_frame(0).props['GamutMax']))

[poisondeathray]

The 1st case looks like a bug for 12bit . If the YCbCr full intermediate is 10, 14 or 16 bit, you get 0,4095,4095 for the end RGB. But 12bit should work too.

Your text description does not match the code for case 1; it should say full instead of limited: "Converted to YCbCr 12bit Full: 3224 2517 0 "

[groucho86]

The 1st case looks like a bug for 12bit . If the YCbCr full intermediate is 10, 14 or 16 bit, you get 0,4095,4095 for the end RGB. But 12bit should work too.

Your text description does not match the code for case 1; it should say full instead of limited: "Converted to YCbCr 12bit Full: 3224 2517 0 "

Thanks for catching that, fixed.

...So why does ProRes 444 Full range clip for cyan? (reminder that it's theoretically a 12bit codec).

[poisondeathray]

...So why does ProRes 444 Full range clip for cyan? (reminder that it's theoretically a 12bit codec).


Actually, it's not a bug - you need +2 bits for the lossless roundtrip for all possible RGB values (not just min/max), because there are too many YCbCr values (several YCbCr values can "map" the the same RGB value when at the same bit depth) . It's analogous to the 8bit RGB => and using 10bit YCbCr intermediate => 8bit RGB lossless roundtrip scenario. So for 12bit RGB, you need 14bit YCbCr or greater intermediate for all values

[groucho86]

Actually, it's not a bug - you need +2 bits for the lossless roundtrip for all possible RGB values (not just min/max), because there are too many YCbCr values (several YCbCr values can "map" the the same RGB value when at the same bit depth) . It's analogous to the 8bit RGB => and using 10bit YCbCr intermediate => 8bit RGB lossless roundtrip scenario. So for 12bit RGB, you need 14bit YCbCr or greater intermediate for all values

Riiight, right... So the workaround/hack in the world of ProRes is to artificially provide those ~+2bits by scaling the signal to legal range?

[poisondeathray]

Riiight, right... So the workaround/hack in the world of ProRes is to artificially provide those ~+2bits by scaling the signal to legal range?

It's not necessarily an accurate workaround. Other values will be slightly off, not necessarily just min/max in a channel

eg. limited doesn't work here
RGB 4095,4095,0 => 12bit full YCbCr => RGB 4095,4095,4095

RGB 4095,4095,0 => 12bit limited YCbCr => RGB 4094,4095,4095

You need +2 bits if you want/need accuracy

12bit prores 4444XQ is meant for 10bit sources

[groucho86]

It's not necessarily an accurate workaround. Other values will be slightly off, not necessarily just min/max in a channel

eg. limited doesn't work here
RGB 4095,4095,0 => 12bit full YCbCr => RGB 4095,4095,4095

RGB 4095,4095,0 => 12bit limited YCbCr => RGB 4094,4095,4095

So legal range for ProRes/DNx/H.264/etc YCbCr is more of a legacy broadcast thing and offers no real benefit compared to full range?

[poisondeathray]

So legal range for ProRes/DNx/H.264/etc YCbCr is more of a legacy broadcast thing and offers no real benefit compared to full range?

This is a can of worms...

Yes, it has more to do with equipment, broadcast, compatibility

[groucho86]

...If I have SMPTE bars in Resolve and render out ProRes 4444.

Legal range MOV brought back in will match the original RGB image (visually, scopes wise)

Full range MOV (force interpreted as full since MOV has no tag to denote range) has minor visually noticeable deviance in the scopes.

Is this a limitation of Resolve's ProRes encoder, of YCbCr, or of Resolve's YCbCr to RGB conversion? Of Something else?

[poisondeathray]

...If I have SMPTE bars in Resolve and render out ProRes 4444.

Legal range MOV brought back in will match the original RGB image (visually, scopes wise)

Full range MOV (force interpreted as full since MOV has no tag to denote range) has minor visually noticeable deviance in the scopes.

Is this a limitation of Resolve's ProRes encoder, of YCbCr, or of Resolve's YCbCr to RGB conversion? Of Something else?

Not sure, have to look into it in more detail...

If you look at the ProRes whitepaper, ProRes 4444XQ actually supports encoding and decoding of YCbCr and RGB . But not all applications or equipment are set up to handle it properly.

In theory, starting from 12bit RGB, you could use 12bit ProRes 4444XQ RGB and avoid any additional pixel format / bit depth losses (just minimal lossy compression losses)

IIRC, FFmpeg/libavodec decode all variants as YCbCr only. Have to double check on that too, there might have been some progress

[groucho86]

This is a can of worms...

Yes, it has more to do with equipment, broadcast, compatibility

I created a 16bit TIFF SMPTE bars out of Resolve. ProRes 4444 full range exported out of Resolve and out of Baselight exhibit similar issues (when imported back into Resolve/Baselight), whereas legal range matches much more closely the RGB TIFF source.

Beyond legacy/compatibility, there's *something* that makes legal range ProRes 4444 technically correct. But I can't fully articulate it.

I plan on trying other Finishing/Broadcast softwares that allow setting data levels. I know Premiere/Media Encoder doesn't. Not sure about Apple Compressor. I will test Transkoder.

[poisondeathray]

I created a 16bit TIFF SMPTE bars out of Resolve. ProRes 4444 full range exported out of Resolve and out of Baselight exhibit similar issues (when imported back into Resolve/Baselight), whereas legal range matches much more closely the RGB TIFF source.

Beyond legacy/compatibility, there's *something* that makes legal range ProRes 4444 technically correct. But I can't fully articulate it.

I plan on trying other Finishing/Broadcast softwares that allow setting data levels. I know Premiere/Media Encoder doesn't. Not sure about Apple Compressor. I will test Transkoder.




Prores is almost always limited range, most programs are setup to handle it like that, so I'd use it like that - for compatibility - if you're "forced" to use prores

But if color was vitally important, and I was starting with 12bit RGB, I'd use an RGB format, such as EXR or 16bit PNG, or some +2 or higher bit depth YUV format (not sure of any compatible in resolve) You're only testing a handful of values with colorbars - what about the 68 trillion other colors ? :D (2^12 * 2^12 * 2^12 = )

Math wise , RGB to YUV limited range round trip (vs YUV full range) will have more expected errors (on all colors, not just a few bars). Full range would preserve more unique colors of the source, if it's handled correctly (maybe try HEVC, since it supports 12bit, full range). In the 8bit full vs limited case it was about ~1.35 million more colors or ~1.49x more accurate unique colors for full range YUV . To put it another way, you 're potentially losing about 83% of the unique colors of the source by limited range YUV at the same bit depth. But that was only 8bit ~16.7M colors. Too large to test 12bit. If you assume a similar trend at 12bit and ratio for scaling, it's still a significant difference in color accuracy in terms of unique colors

You can see some of the tests here, I don't think I posted the full range one
https://forum.doom9.org/showthread.php?t=183816

"identify" -format "%k" 8bitroundtrip.png
2760365
~14M unique colors for typical 8bit RGB=>YUV=>RGB round trip (no subsampling) are lost !!
ConvertToYV24(matrix="rec709")
ConvertToRGB24(matrix="rec709")

full range round trip preserves more colors
ConvertToYV24(matrix="pc.709")
ConvertToRGB24(matrix="pc.709")
"identify" -format "%k" 8bitroundtrip_fullrange.png
4114580

[groucho86]

Hey PDR, I really appreciate you taking the time to entertain my questions...

I'm sort of stuck on making sense specifically of ProRes 4444 because some broadcast/streaming services use the codec as their final deliverable.

I'm loving understanding the theoretical best practices when it comes to RGB > YUV > RGB. I find it interesting that real-word scenarios dealing with lossy codecs and whatnot may paint (perhaps?) a different picture.

I'm on Mac and don't have the bandwidth to get Avisynth installed to replicate the testing done in that other thread. But I did find this PNG (http://www.brucelindbloom.com/index.html?RGB16Million.html) to conduct similar tests.

I used Resolve to convert the 8bit PNG to be a 16bit TIFF. I then rendered it out as legal range ProRes 4444 and full range ProRes 4444. I brought those two back in Resolve, correctly interpreted their data levels and rendered out 12bit DPX (the only 12bit RGB uncompressed format available). Then use oiiotool to calculate diffs:

% oiiotool /private/tmp/8bit_to_16bit.tif /private/tmp/full-legal-full.dpx --diff
Computing diff of "/private/tmp/8bit_to_16bit.tif" vs "/private/tmp/full-legal-full.dpx"
Mean error = 6.10615e-05
RMS error = 7.10204e-05
Peak SNR = 82.9723
Max error = 0.000732433 @ (13, 9, B) values are 0.0508736, 0.0350652, 0 vs 0.050782, 0.0349126, 0.000732433
16772673 pixels (100%) over 1e-06
16772673 pixels (100%) over 1e-06
FAILURE

% oiiotool /private/tmp/8bit_to_16bit.tif /private/tmp/full-full-full.dpx --diff
Computing diff of "/private/tmp/8bit_to_16bit.tif" vs "/private/tmp/full-full-full.dpx"
Mean error = 0.000133395
RMS error = 0.000176972
Peak SNR = 75.0419
Max error = 0.0209811 @ (3840, 3840, B) values are 0.000381476, 0.000167849, 0.999985 vs 0.000488289, 0.0021973, 0.979004
16776203 pixels (100%) over 1e-06
16776203 pixels (100%) over 1e-06
FAILURE

Is this a pointless test because I'm not testing all the other colors 12bit has to offer?

[poisondeathray]

I'm sort of stuck on making sense specifically of ProRes 4444 because some broadcast/streaming services use the codec as their final deliverable.


Then use it - you have no other options. Broadcast/ streaming services don't care about this stuff and will expect limited range anyways. This is just academic geeky testing

I don't have quick access to a mac right now, but Pr4444 was touted as supporting RGB and YCbCr (whitepaper) . It should be (almost) perfect in RGB mode if it's handled correctly.



I'm on Mac and don't have the bandwidth to get Avisynth installed to replicate the testing done in that other thread. But I did find this PNG (http://www.brucelindbloom.com/index.html?RGB16Million.html) to conduct similar tests.


ffmpeg has one too - allrgb
https://ffmpeg.org/ffmpeg-filters.html#allrgb_002c-allyuv_002c-color_002c-colorchart_002c-colorspectrum_002c-haldclutsrc_002c-nullsrc_002c-pal75bars_002c-pal100bars_002c-rgbtestsrc_002c-smptebars_002c-smptehdbars_002c-testsrc_002c-testsrc2_002c-yuvtestsrc


The allrgb source returns frames of size 4096x4096 of all rgb colors.


And you can use vapoursynth to do the conversions (you get similar results but a few values are different with zimg library. Identical results to each other when avisynth AVSResize is used (both use zimg))


I used Resolve to convert the 8bit PNG to be a 16bit TIFF. I then rendered it out as legal range ProRes 4444 and full range ProRes 4444. I brought those two back in Resolve, correctly interpreted their data levels and rendered out 12bit DPX (the only 12bit RGB uncompressed format available). Then use oiiotool to calculate diffs:


16bit input is ok, but one issue I noticed with Resolve 12bit DPX input is it clips 4095 (but that was on Resolve 17 on Windows; maybe it's been fixed in 18) . But 12bit to 16bit (e.g. PNG or TIFF) input into Resolve is ok for Resolve exporting 12bit DPX. Or 16bit export, and downconverting to 12bit with other tools. You can do in/out tests and can check with multiple tools to see if they have the same results


% oiiotool /private/tmp/8bit_to_16bit.tif /private/tmp/full-legal-full.dpx --diff
Computing diff of "/private/tmp/8bit_to_16bit.tif" vs "/private/tmp/full-legal-full.dpx"
Mean error = 6.10615e-05
RMS error = 7.10204e-05
Peak SNR = 82.9723
Max error = 0.000732433 @ (13, 9, B) values are 0.0508736, 0.0350652, 0 vs 0.050782, 0.0349126, 0.000732433
16772673 pixels (100%) over 1e-06
16772673 pixels (100%) over 1e-06
FAILURE

% oiiotool /private/tmp/8bit_to_16bit.tif /private/tmp/full-full-full.dpx --diff
Computing diff of "/private/tmp/8bit_to_16bit.tif" vs "/private/tmp/full-full-full.dpx"
Mean error = 0.000133395
RMS error = 0.000176972
Peak SNR = 75.0419
Max error = 0.0209811 @ (3840, 3840, B) values are 0.000381476, 0.000167849, 0.999985 vs 0.000488289, 0.0021973, 0.979004
16776203 pixels (100%) over 1e-06
16776203 pixels (100%) over 1e-06
FAILURE


Is this a pointless test because I'm not testing all the other colors 12bit has to offer?

Not necessarily "pointless" , it offers some limited value and suggests problems with the tools - either coversion or measuring. Those observations are unexpected. Full (if handled correctly) should always have higher PSNR and more correct values. You can't reduce the range of values and yet have higher precision - common sense says something is fishy with those results

Some measuring tools might have problems with 16 to 8bit or 12 to 8bit. Do simple in/out tests (without prores or full vs limited at first) to check the validity of the test chain. Some down conversions might dither and add essentially noise. Verify results with other tools, check each step.

You can perform other mini tests, ramps, patterns with 12bit colors . It's very difficult to test all 12bit colors, or you'd have to divide it up into sections.



Here is a 12bit test ramp with primaries,secondaries,greyscale in 12bit DPX, and one version converted 16bit PNG in vapoursynth
4096x896 (7 ramps, 128px height)
https://www.mediafire.com/file/h36giv3709pf9hw/4k_12bit_ramp_greyscale_primaries_secondaries.zip/file

ImageMagick reports 28666 unique colors for both the dpx and png . 4096*7=28672 colors, but subtract 6 because 0,0,0 occurs 7 times at the start of the ramps (6 duplicates), so it's a correct reading

"identify" -format "%k" 4k_12bit_ramp_greyscale_primaries_secondaries_00.dpx
28666

It's a far cry from 68 trillion colors, but it's one starting point

Can you check on a mac if the 12bit DPX import/export clips 4095 ? I got 28659 colors, so that clued me in that 12bit Resolve in/out dpx had problems and found the 7 missing values

In vapoursynth , RGB36 => YUV444P12 full => RGB36 => imwri.Write 12bit DPX export
23423
ffmpeg psnr
PSNR r:78.524691 g:83.142373 b:77.818719 average:79.278724 min:79.278724 max:79.278724

RGB36 => YUV444P12 limited => RGB36 => imwri.Write 12bit DPX export
21280
ffmpeg psnr
PSNR r:77.223044 g:80.433767 b:76.491457 average:77.743932 min:77.743932 max:77.743932

[FranceBB]

Ah, Groucho! You're still here after all! :D
It's nice to see you back. :)

I know integer-based YCbCr codecs should always be legal range in order for no clipping to occur when returning to RGB. I've proved this with SMPTE bars out of Resolve.


Correct.


...But I can't articulate exactly *why* that is.


It's actually pretty simple, really.
Honestly, it's easier and more straightforward than you think and frankly you already gave yourself the answer :P.
Anyway, suppose you're shooting a video and it's in YUV limited tv range.
In your case 12bit, so 256-3760 which correspond to 0.0-0.7V (actually it doesn't, 'cause it would be a dual link config to get 12bit in FULL HD, so 700 millivolt x2, but even that it's not 100% exact, so c'mon pass me this), Limited TV Range in YUV.
Now, all TVs are actually RGB and as we know RGB is always Full PC Range, which means that when such a signal is passed to a TV, it will read the Limited TV Range flag and convert it to RGB Full Range by taking the value at 256 and bringing it to 0 and the value at 3760 and bringing it to 4080, scaling everything accordingly.

https://i.imgur.com/aZtIN8V.png

So far so good, the TV performs a conversion and everything is fine.

In other words, when you submit this in Limited TV Range 12bit:

https://i.imgur.com/pN6coJ9.png

it will be "expanded" so that the values at 3760 (700mV) will go to 4080 etc:

https://i.imgur.com/fRvC5wI.png

Now, if you were to submit something flagged as Limited TV Range BUT with overshooting, so with values that exceed the limited tv range threshold, let's say, something like this:

https://i.imgur.com/ErNt7WM.png

see that the white goes way above 3760 even though it's supposed to be in Limited TV Range.
When the TV converts it to RGB Full PC Range, guess what happens?
Well, whatever is at 3760 is brought to the maximum, so 4080 and anything that was already above 3760 would be shifted upwards, but since there's nothing over 4080 in 12bit, those values are truncated (i.e lost) and they become just... "white":

https://i.imgur.com/OYyCLOn.png



This is why it's important to ALWAYS deliver files to broadcasters in Limited TV Range AND make sure that they actually are with a Waveform Monitor.
Don't worry, they will tell you if it's not 'cause the file will probably go through an AutoQC suite like Tektronix Cerify, Tektronix Aurora, Baton etc that will check scene by scene whether it actually is in limited tv range or not and if it's not, it will report it to the QC operator who will promptly reject the content with a QC Fail (well, most of the time it's actually QC TBC unless it's really bad and everywhere out of range).


As to the question of "why can't we work in Full PC Range all the time?" the answer is: we can't because in real world scenarios signals are electrical and have to go through cables. It would be fine if the whole cable bandwidth was used for signal only, but it's not, in SDI cables you have stuff dedicated to the sync/timing/phase of the signal coming and going, you have the reference timecode etc and all those things take some millivolt (so in a digital world, they take some bits, which, in a Full PC Range representation, wouldn't be carried through as you can't physically transmit those).




I'm on Mac

Bad... bad Groucho... That's not what I expected from the creator of AVSMeter eheheheheh

https://i.imgur.com/9I4zHzt.png

[poisondeathray]

Now, all TVs are actually RGB and as we know RGB is always Full PC Range, which means that when such a signal is passed to a TV, it will read the Limited TV Range flag and convert it to RGB Full Range by taking the value at 256 and bringing it to 0 and the value at 3760 and bringing it to 4080, scaling everything accordingly.


Not quite, because most consumer TV's are actually limited range RGB out of the box

Computer monitors use computer range (full) RGB (0-255), but most TV's are actually limited range RGB out of the box (RGB 16-235 in 8bit)
https://www.benq.com/en-us/knowledge-center/knowledge/full-rgb-vs-limited-rgb-is-there-a-difference.html

Almost every broadcaster (like your company) uses r103, which uses the Studio range RGB equations (in avisynth language it would be similar the full range or PC matrices) . In 8bit 0 to 255 code values are strictly reserved for SDI timing (0 to 3, 1020 to 1023 for 10bit)

The r103 R,G,B out of gamut check actually uses studio range equations, not the computer range equations you typically see (ie. studio range RGB, RGB 16-235 black to white in 8bit, 64-940 RGB in 10bit - Not RGB 0-255, Not RGB 0-1023


Bad... bad Groucho... That's not what I expected from the creator of AVSMeter eheheheheh


Wrong groucho ...

[kolak]

No one said YUV has to be legal. There is no such a hard requirement. It's just how it was used in the past, so it became a norm, but it's really obsolete "requirement". Today's digital world doesn't need it anymore. If not worse compression efficiency we could just stick to RGB snd don't even bother with YUV. YUV is for saving bandwidth (by 'cheating' on sampling) and also to get better compressibility.
When it comes to ProRes itself then things are bit different, because by official spec ProRes should always be legal. There is no way to flag range in ProRes private headers (nor in eg. MOV container), but there is in MXF. Saying this you can export full range ProRes, just be careful when importing as app has to be "somehow" aware it's full range. Resolve lets you interpret both ways, where eg. Premiere has (stupid) hard coded rules- 422 is always treated as limited where 444 as full!. ProRes is always YUV based internally.
In this aspect DNxHR is better as it has private headers range flag and 444 can also be YUV or RGB based, so it covers all cases (if I remember well Resolve uses RGB version).

[kolak]

...If I have SMPTE bars in Resolve and render out ProRes 4444.

Legal range MOV brought back in will match the original RGB image (visually, scopes wise)

Full range MOV (force interpreted as full since MOV has no tag to denote range) has minor visually noticeable deviance in the scopes.

Is this a limitation of Resolve's ProRes encoder, of YCbCr, or of Resolve's YCbCr to RGB conversion? Of Something else?

I assume you did export full range and then on import interpreted manually as full range?
It all may be down to fact that Resolve is RGB internally and ProRes is YUV. You have quite a lot RGB<->YUV going on.
Try the same with DNxHR. I think 444 one from Resolve is actually RGB based so you won't get any RGB<->YUV conversion and results should be "perfect".
Other fact- codecs even like ProRes do overshoots during encoding, so any hard contrast will produce values which are slightly different than source ones.
If I remember well you can expect up to 4 levels difference even on bars, so eg. 64 may end up as 60. This 'imperfection' is not really relevant in any real world scenarios, unless you do some scientific projects. In this case don't use any compression and stick to eg. EXR, TIFF or 16 bit RGB MOV.

[FranceBB]

Wrong groucho ...

Oops.
I wonder what happened to the other Groucho then, he's been quiet for a while now... :(

[kolak]

ProRes444 out of Resolve will have min Y as 254 and max Y of 3765 for SMPTE bars. Both slightly off ideal values.
ProResHQ will have 63 and 941.
(at least those are reported by ffmpeg, which I don't trust 100% either).

DNxHR has 245 and 3777, but because ffmpeg decodes it to YUV (even if file is RGB). I requested YUV stats, so actually it has to convert.

There seems to be also some bug in Resolve when you force full levels. Something is not right I think. Well- never trust those tools :)
Resolve use to properly read DNxHR range flag. BM had mess with it, I reported they fixed it. And few versions later it looks like it's broken again :p Range is set properly, but on import not read.
Some things never change- it's that much you can trust those pro tools :)

[poisondeathray]

ProRes is always YUV based internally.


Perhaps it's stored that way, like other codecs that have YUV and RGB, but do you know of any implementation that handles Pr4444 with RGB options properly ?

The Apple Prores whitepaper claims that Pr4444: "also offers direct encoding of, and decoding to, both RGB and Y’CBCR pixel formats."


ProRes444 out of Resolve will have min Y as 254 and max Y of 3765 for SMPTE bars. Both slightly off ideal values.
ProResHQ will have 63 and 941.
(at least those are reported by ffmpeg, which I don't trust 100% either).



I don't 100% trust them either, and I always try verify with other tools/workflows. You can export 12bit dpx out of Resolve, using various Resolve decoder(s) and measure that (if the import wasn't 12bit dpx, resolve 17 clips 4095 on windows according to the test above) also check export 16bit PNG/TIFF and downcovert to 12bit without dithering and measure



DNxHR has 245 and 3777, but because ffmpeg decodes it to YUV (even if file is RGB). I requested YUV stats, so actually it has to convert.




Yes, reported before on ffmpeg trac for DNxHR 444 12bit RGB. The dev answer was: "This is because files use ACT, adaptive color transform, when some blocks are encoded in RGB instead of YUV and vice versa." - and it's not implemented in libavcodec for DNxHR decoding . If you take the test ramp posted above, and encode to DNxHR 444 12bit in resolve, you can see the errors in ffmpeg/libavcodec based decoders on the RGB blocks.
(e.g. vapoursynth/avisynth lsmash, ffmpeg/ffplay/mpv) . I also posted examples in other threads



Some things never change- it's that much you can trust those pro tools :)

Certainly the lessons I've learned in the past are are don't blindly trust any tool (pro or opensource) . Version upgrades can break stuff previously working. Don't upgrade in the middle of projects. Verify your workflow before you commit to anything

[groucho86]

Broadcast/ streaming services don't care about this stuff and will expect limited range anyways
There's one service that explicitly asks for full range and I've been academically debating it with them....

This is just academic geeky testing
100%. And loving it :)


Can you check on a mac if the 12bit DPX import/export clips 4095 ?
Confirmed still clipping.


No one said YUV has to be legal. There is no such a hard requirement. It's just how it was used in the past, so it became a norm, but it's really obsolete "requirement". Today's digital world doesn't need it anymore. If not worse compression efficiency we could just stick to RGB snd don't even bother with YUV. YUV is for saving bandwidth (by 'cheating' on sampling) and also to get better compressibility.
When it comes to ProRes itself then things are bit different, because by official spec ProRes should always be legal. There is no way to flag range in ProRes private headers (nor in eg. MOV container), but there is in MXF. Saying this you can export full range ProRes, just be careful when importing as app has to be "somehow" aware it's full range. Resolve lets you interpret both ways, where eg. Premiere has (stupid) hard coded rules- 422 is always treated as limited where 444 as full!. ProRes is always YUV based internally.
In this aspect DNxHR is better as it has private headers range flag and 444 can also be YUV or RGB based, so it covers all cases (if I remember well Resolve uses RGB version).

I did not know Premiere assumed ProRes 4444 (MXF and MOV?) was full range. That is strange.

...

So I decided to take a break from pattern testing - clearly the SMPTE bars were giving me an incredibly incomplete picture of of RGB > ProRes 4444 > RGB.
I took 10min worth of 16bit DPX source of Rec2020 (P3 constrained) PQ 1000nit and used Colorfront Transkoder to render out ProRes 4444 legal and ProRes 44444 full and brought them back in to run PSNIR metrics against the original source. Results:


Legal ProRes 4444
min: 52.4257
max: 90.0
avg: 59.19808741851951

Full ProRes 4444
min: 53.2485
max: 162.556
avg: 59.91924627198857

As predicted by poisondeathray and others, full range provided a slightly better match. My SMPTE bars test was flawed.

In conclusion...?
Pros for full range ProRes 444 MOV

Higher fidelity

Cons for full range ProRes 444 MOV

No tag that specifies full range. Most softwares will misinterpret it as legal

Pros for legal range ProRes 444 MOV

Expected spec
Most softwares will correctly interpret it

Cons for legal range ProRes 444 MOV

Slightly lower fidelity

[poisondeathray]

There's one service that explicitly asks for full range and I've been academically debating it with them....

Not common at all...

Which service out of curiosity ? One would have to assume they were handling full range correctly

And did they ask for Pr4444 full range specifically ? or just full range in general ?

[groucho86]

Not common at all...

Which service out of curiosity ? One would have to assume they were handling full range correctly

And did they ask for Pr4444 full range specifically ? or just full range in general ?

I'd rather not say on a public forum. But yes, we need to assume they're handling it correctly...

UHD deliverables (SDR and HDR) are to be ProRes 4444(XQ) full range

HD SDR deliverables are to be ProRes 422 HQ legal.

[poisondeathray]

UHD deliverables (SDR and HDR) are to be ProRes 4444(XQ) full range

Interesting, full range unexpected for SDR in this scenario

[kolak]

Dolby created mess saying DV has to be full range. It all went bad as most people use ProRes for DV mastering (one level down from 16bit TIFF masters) without knowing that there is nothing describing range in ProRes MOVs.
Now it's all been recalled :) You can and maybe should use limited range for DV ProRes masters. It's practically harmless (lost precision is meaningless), where full range will in 50% cases handled badly!

If delivery place says full range ProRes I would assume they know what they are saying and will handle it properly. 99% is limited range and better stay this way until specifically requested otherwise.

[kolak]

Perhaps it's stored that way, like other codecs that have YUV and RGB, but do you know of any implementation that handles Pr4444 with RGB options properly ?

The Apple Prores whitepaper claims that Pr4444: "also offers direct encoding of, and decoding to, both RGB and Y’CBCR pixel formats."


This is confirmed now by many different sources including a guy who can write bit-perfect to Apple ProRes encoder.

Apple paper says encoder/decoder can take RGB data directly (it does internal conversion). Basically that's all what it says in this magic line. Internal data is always YUV (probably for efficiency).

There is no ProRes which stores data as RGB as this is not in Apple spec.
If you want that use DNxHR (just make sure app uses RGB variant for 444) or Cineform.

[poisondeathray]

There is no ProRes which stores data as RGB as this is not in Apple spec.

I was asking about output from the decoder (less interested in how it's stored; but YCgCo would technically be better) - Are you aware of any Pr4444 implementation that has YCbCr vs. RGB output options or controls? Whitepaper implies 2 decoding cases, RGB and YCbCr. For example, you feed RGB in , you get RGB out. Or you feed YCbCr in you get YCbCr out . It seems to be YCbCr in most cases, and the host application possibly converts to RGB .

[groucho86]

Are you aware of any Pr4444 implementation that has YCbCr vs. RGB output options or controls?.

Nope, always been under the hood.

[kolak]

I was asking about output from the decoder (less interested in how it's stored; but YCgCo would technically be better) - Are you aware of any Pr4444 implementation that has YCbCr vs. RGB output options or controls? Whitepaper implies 2 decoding cases, RGB and YCbCr. For example, you feed RGB in , you get RGB out. Or you feed YCbCr in you get YCbCr out . It seems to be YCbCr in most cases, and the host application possibly converts to RGB .

As far as I understand official decoder provides many pixel formats on output (you can feed many on input as well).
It' up to the app to call what it needs/wants. I think Resolve in most cases uses b64a (16bit RGBA), FCPX 32bit float YUV4444 (r4fl). If app is good it writes properly which pixel format was used on input and you can check it with advanced view in mediainfo. Some apps may "lie" though :)
So basically it all depends how app is written. Good ones try "most direct" pixel format between input/output, eg. if you converting ProResHQ to Cineform it will most likely go over v210. If there is no need you should avoid going to RGB.
RGB in, RGB out would make sense if encoder had native support for YUV and RGB (DNxHR, Cineform). If it does conversion then it's already "less perfect", but you may still avoid double conversion.

Nuke may let you choose requested pixel format. Use to have this.

You have list of supported formats here:
https://wiki.multimedia.cx/index.php/Apple_ProRes

[kolak]

I did not know Premiere assumed ProRes 4444 (MXF and MOV?) was full range. That is strange.



I think last version I tested was 2020. It's was hard coded and Premiere has no "easy" way to overwrite range (like eg. Resolve does). It's very dangerous behaviour.
MOV, no idea about MXF.
Try it. You will see bad levels straight away if you feed limited ProRes444. Then do full and it should be fine.
Resolve (and many tools) always (for 422 and 444) uses limited by default (this is more per Apple spec I think).