Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
![]() |
|
Thread Tools | Search this Thread | Display Modes |
![]() |
#1 | Link |
Registered User
Join Date: Apr 2016
Posts: 82
|
Why ProRes/DNxHR 4444 should always be legal/limited range?...
I know integer-based YCbCr codecs should always be legal range in order for no clipping to occur when returning to RGB. I've proved this with SMPTE bars out of Resolve.
...But I can't articulate exactly *why* that is. I attempted to reproduce the concept in Vapoursynth: Test 1 is pretending someone rendered ProRes 4444 12bit Full Range, Test 2 is pretending someone rendered ProRes 4444 12bit Legal/Limited range. The results look like this: Code:
Starting point is RGB 12bit Full: 0 4095 4095 =========== TEST 1 - Full to Full to Full Converted to YCbCr 12bit Full: 3224 2517 0 Converted to RGB 12bit Full: 0 4095 4094 ------------ TEST 2 - Full to Limited to Full Converted to YCbCr 12bit Full: 3015 2459 256 Converted to RGB 12bit Full: 0 4095 4095 The Vapoursynth python code: Code:
import vapoursynth as vs core = vs.core cyan_rgb = core.std.BlankClip(width=1920, height=1080, format=vs.RGB36, color=[0,4095,4095] ) r = core.std.PlaneStats(cyan_rgb, plane=0, prop='Gamut') g = core.std.PlaneStats(cyan_rgb, plane=1, prop='Gamut') b = core.std.PlaneStats(cyan_rgb, plane=2, prop='Gamut') print('Starting point is RGB 12bit Full: {} {} {}'.format(r.get_frame(0).props['GamutMax'], g.get_frame(0).props['GamutMax'], b.get_frame(0).props['GamutMax'])) print('\n===========\n') print('TEST 1 - Full to Full to Full') cyan_ycbcr = core.resize.Bicubic(cyan_rgb, format=vs.YUV444P12, matrix_s="709", range_in_s='full', range_s='full') y = core.std.PlaneStats(cyan_ycbcr, plane=0, prop='Gamut') cb = core.std.PlaneStats(cyan_ycbcr, plane=1, prop='Gamut') cr = core.std.PlaneStats(cyan_ycbcr, plane=2, prop='Gamut') print('Converted to YCbCr 12bit Limited: {} {} {} '.format(str(y.get_frame(0).props['GamutMax']), str(cb.get_frame(0).props['GamutMax']), str(cr.get_frame(0).props['GamutMax']))) cyan_rgb2 = core.resize.Bicubic(cyan_ycbcr, format=vs.RGB36, matrix_in_s="709", range_in_s='full', range_s='full') r2 = core.std.PlaneStats(cyan_rgb2, plane=0, prop='Gamut') g2 = core.std.PlaneStats(cyan_rgb2, plane=1, prop='Gamut') b2 = core.std.PlaneStats(cyan_rgb2, plane=2, prop='Gamut') print('Converted to RGB 12bit Full: {} {} {}'.format(r2.get_frame(0).props['GamutMax'], g2.get_frame(0).props['GamutMax'], b2.get_frame(0).props['GamutMax'])) print('\n------------\n') print('TEST 2 - Full to Limited to Full\n') cyan_ycbcr = core.resize.Bicubic(cyan_rgb, format=vs.YUV444P12, matrix_s="709", range_in_s='full', range_s='limited') y = core.std.PlaneStats(cyan_ycbcr, plane=0, prop='Gamut') cb = core.std.PlaneStats(cyan_ycbcr, plane=1, prop='Gamut') cr = core.std.PlaneStats(cyan_ycbcr, plane=2, prop='Gamut') print('Converted to YCbCr 12bit Limited: {} {} {} '.format(str(y.get_frame(0).props['GamutMax']), str(cb.get_frame(0).props['GamutMax']), str(cr.get_frame(0).props['GamutMax']))) cyan_rgb2 = core.resize.Bicubic(cyan_ycbcr, format=vs.RGB36, matrix_in_s="709", range_in_s='limited', range_s='full') r2 = core.std.PlaneStats(cyan_rgb2, plane=0, prop='Gamut') g2 = core.std.PlaneStats(cyan_rgb2, plane=1, prop='Gamut') b2 = core.std.PlaneStats(cyan_rgb2, plane=2, prop='Gamut') print('Converted to RGB 12bit Full: {} {} {}'.format(r2.get_frame(0).props['GamutMax'], g2.get_frame(0).props['GamutMax'], b2.get_frame(0).props['GamutMax'])) Last edited by groucho86; 21st December 2022 at 20:22. Reason: fixing based on poisondeathray's comment |
![]() |
![]() |
![]() |
#2 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,285
|
The 1st case looks like a bug for 12bit . If the YCbCr full intermediate is 10, 14 or 16 bit, you get 0,4095,4095 for the end RGB. But 12bit should work too.
Your text description does not match the code for case 1; it should say full instead of limited: "Converted to YCbCr 12bit Full: 3224 2517 0 " Last edited by poisondeathray; 21st December 2022 at 18:48. |
![]() |
![]() |
![]() |
#3 | Link | |
Registered User
Join Date: Apr 2016
Posts: 82
|
Quote:
...So why does ProRes 444 Full range clip for cyan? (reminder that it's theoretically a 12bit codec). |
|
![]() |
![]() |
![]() |
#4 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,285
|
Quote:
Actually, it's not a bug - you need +2 bits for the lossless roundtrip for all possible RGB values (not just min/max), because there are too many YCbCr values (several YCbCr values can "map" the the same RGB value when at the same bit depth) . It's analogous to the 8bit RGB => and using 10bit YCbCr intermediate => 8bit RGB lossless roundtrip scenario. So for 12bit RGB, you need 14bit YCbCr or greater intermediate for all values Last edited by poisondeathray; 21st December 2022 at 21:47. |
|
![]() |
![]() |
![]() |
#5 | Link | |
Registered User
Join Date: Apr 2016
Posts: 82
|
Quote:
|
|
![]() |
![]() |
![]() |
#6 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,285
|
Quote:
eg. limited doesn't work here RGB 4095,4095,0 => 12bit full YCbCr => RGB 4095,4095,4095 RGB 4095,4095,0 => 12bit limited YCbCr => RGB 4094,4095,4095 You need +2 bits if you want/need accuracy 12bit prores 4444XQ is meant for 10bit sources |
|
![]() |
![]() |
![]() |
#7 | Link | |
Registered User
Join Date: Apr 2016
Posts: 82
|
Quote:
|
|
![]() |
![]() |
![]() |
#9 | Link |
Registered User
Join Date: Apr 2016
Posts: 82
|
...If I have SMPTE bars in Resolve and render out ProRes 4444.
Legal range MOV brought back in will match the original RGB image (visually, scopes wise) Full range MOV (force interpreted as full since MOV has no tag to denote range) has minor visually noticeable deviance in the scopes. Is this a limitation of Resolve's ProRes encoder, of YCbCr, or of Resolve's YCbCr to RGB conversion? Of Something else? |
![]() |
![]() |
![]() |
#10 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,285
|
Quote:
If you look at the ProRes whitepaper, ProRes 4444XQ actually supports encoding and decoding of YCbCr and RGB . But not all applications or equipment are set up to handle it properly. In theory, starting from 12bit RGB, you could use 12bit ProRes 4444XQ RGB and avoid any additional pixel format / bit depth losses (just minimal lossy compression losses) IIRC, FFmpeg/libavodec decode all variants as YCbCr only. Have to double check on that too, there might have been some progress |
|
![]() |
![]() |
![]() |
#11 | Link | |
Registered User
Join Date: Apr 2016
Posts: 82
|
Quote:
Beyond legacy/compatibility, there's *something* that makes legal range ProRes 4444 technically correct. But I can't fully articulate it. I plan on trying other Finishing/Broadcast softwares that allow setting data levels. I know Premiere/Media Encoder doesn't. Not sure about Apple Compressor. I will test Transkoder. |
|
![]() |
![]() |
![]() |
#12 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,285
|
Quote:
Prores is almost always limited range, most programs are setup to handle it like that, so I'd use it like that - for compatibility - if you're "forced" to use prores But if color was vitally important, and I was starting with 12bit RGB, I'd use an RGB format, such as EXR or 16bit PNG, or some +2 or higher bit depth YUV format (not sure of any compatible in resolve) You're only testing a handful of values with colorbars - what about the 68 trillion other colors ? ![]() Math wise , RGB to YUV limited range round trip (vs YUV full range) will have more expected errors (on all colors, not just a few bars). Full range would preserve more unique colors of the source, if it's handled correctly (maybe try HEVC, since it supports 12bit, full range). In the 8bit full vs limited case it was about ~1.35 million more colors or ~1.49x more accurate unique colors for full range YUV . To put it another way, you 're potentially losing about 83% of the unique colors of the source by limited range YUV at the same bit depth. But that was only 8bit ~16.7M colors. Too large to test 12bit. If you assume a similar trend at 12bit and ratio for scaling, it's still a significant difference in color accuracy in terms of unique colors You can see some of the tests here, I don't think I posted the full range one https://forum.doom9.org/showthread.php?t=183816 "identify" -format "%k" 8bitroundtrip.png 2760365 ~14M unique colors for typical 8bit RGB=>YUV=>RGB round trip (no subsampling) are lost !! ConvertToYV24(matrix="rec709") ConvertToRGB24(matrix="rec709") full range round trip preserves more colors ConvertToYV24(matrix="pc.709") ConvertToRGB24(matrix="pc.709") "identify" -format "%k" 8bitroundtrip_fullrange.png 4114580 Last edited by poisondeathray; 22nd December 2022 at 02:23. |
|
![]() |
![]() |
![]() |
#13 | Link |
Registered User
Join Date: Apr 2016
Posts: 82
|
Hey PDR, I really appreciate you taking the time to entertain my questions...
I'm sort of stuck on making sense specifically of ProRes 4444 because some broadcast/streaming services use the codec as their final deliverable. I'm loving understanding the theoretical best practices when it comes to RGB > YUV > RGB. I find it interesting that real-word scenarios dealing with lossy codecs and whatnot may paint (perhaps?) a different picture. I'm on Mac and don't have the bandwidth to get Avisynth installed to replicate the testing done in that other thread. But I did find this PNG to conduct similar tests. I used Resolve to convert the 8bit PNG to be a 16bit TIFF. I then rendered it out as legal range ProRes 4444 and full range ProRes 4444. I brought those two back in Resolve, correctly interpreted their data levels and rendered out 12bit DPX (the only 12bit RGB uncompressed format available). Then use oiiotool to calculate diffs: Code:
% oiiotool /private/tmp/8bit_to_16bit.tif /private/tmp/full-legal-full.dpx --diff Computing diff of "/private/tmp/8bit_to_16bit.tif" vs "/private/tmp/full-legal-full.dpx" Mean error = 6.10615e-05 RMS error = 7.10204e-05 Peak SNR = 82.9723 Max error = 0.000732433 @ (13, 9, B) values are 0.0508736, 0.0350652, 0 vs 0.050782, 0.0349126, 0.000732433 16772673 pixels (100%) over 1e-06 16772673 pixels (100%) over 1e-06 FAILURE % oiiotool /private/tmp/8bit_to_16bit.tif /private/tmp/full-full-full.dpx --diff Computing diff of "/private/tmp/8bit_to_16bit.tif" vs "/private/tmp/full-full-full.dpx" Mean error = 0.000133395 RMS error = 0.000176972 Peak SNR = 75.0419 Max error = 0.0209811 @ (3840, 3840, B) values are 0.000381476, 0.000167849, 0.999985 vs 0.000488289, 0.0021973, 0.979004 16776203 pixels (100%) over 1e-06 16776203 pixels (100%) over 1e-06 FAILURE |
![]() |
![]() |
![]() |
#14 | Link | |||||
Registered User
Join Date: Sep 2007
Posts: 5,285
|
Quote:
I don't have quick access to a mac right now, but Pr4444 was touted as supporting RGB and YCbCr (whitepaper) . It should be (almost) perfect in RGB mode if it's handled correctly. Quote:
https://ffmpeg.org/ffmpeg-filters.ht...02c-yuvtestsrc Quote:
Quote:
Quote:
Some measuring tools might have problems with 16 to 8bit or 12 to 8bit. Do simple in/out tests (without prores or full vs limited at first) to check the validity of the test chain. Some down conversions might dither and add essentially noise. Verify results with other tools, check each step. You can perform other mini tests, ramps, patterns with 12bit colors . It's very difficult to test all 12bit colors, or you'd have to divide it up into sections. Here is a 12bit test ramp with primaries,secondaries,greyscale in 12bit DPX, and one version converted 16bit PNG in vapoursynth 4096x896 (7 ramps, 128px height) https://www.mediafire.com/file/h36gi...aries.zip/file ImageMagick reports 28666 unique colors for both the dpx and png . 4096*7=28672 colors, but subtract 6 because 0,0,0 occurs 7 times at the start of the ramps (6 duplicates), so it's a correct reading "identify" -format "%k" 4k_12bit_ramp_greyscale_primaries_secondaries_00.dpx 28666 It's a far cry from 68 trillion colors, but it's one starting point Can you check on a mac if the 12bit DPX import/export clips 4095 ? I got 28659 colors, so that clued me in that 12bit Resolve in/out dpx had problems and found the 7 missing values In vapoursynth , RGB36 => YUV444P12 full => RGB36 => imwri.Write 12bit DPX export 23423 ffmpeg psnr PSNR r:78.524691 g:83.142373 b:77.818719 average:79.278724 min:79.278724 max:79.278724 RGB36 => YUV444P12 limited => RGB36 => imwri.Write 12bit DPX export 21280 ffmpeg psnr PSNR r:77.223044 g:80.433767 b:76.491457 average:77.743932 min:77.743932 max:77.743932 Last edited by poisondeathray; 22nd December 2022 at 18:14. |
|||||
![]() |
![]() |
![]() |
#15 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,723
|
Ah, Groucho! You're still here after all!
![]() It's nice to see you back. ![]() Quote:
It's actually pretty simple, really. Honestly, it's easier and more straightforward than you think and frankly you already gave yourself the answer :P. Anyway, suppose you're shooting a video and it's in YUV limited tv range. In your case 12bit, so 256-3760 which correspond to 0.0-0.7V (actually it doesn't, 'cause it would be a dual link config to get 12bit in FULL HD, so 700 millivolt x2, but even that it's not 100% exact, so c'mon pass me this), Limited TV Range in YUV. Now, all TVs are actually RGB and as we know RGB is always Full PC Range, which means that when such a signal is passed to a TV, it will read the Limited TV Range flag and convert it to RGB Full Range by taking the value at 256 and bringing it to 0 and the value at 3760 and bringing it to 4080, scaling everything accordingly. ![]() So far so good, the TV performs a conversion and everything is fine. In other words, when you submit this in Limited TV Range 12bit: ![]() it will be "expanded" so that the values at 3760 (700mV) will go to 4080 etc: ![]() Now, if you were to submit something flagged as Limited TV Range BUT with overshooting, so with values that exceed the limited tv range threshold, let's say, something like this: ![]() see that the white goes way above 3760 even though it's supposed to be in Limited TV Range. When the TV converts it to RGB Full PC Range, guess what happens? Well, whatever is at 3760 is brought to the maximum, so 4080 and anything that was already above 3760 would be shifted upwards, but since there's nothing over 4080 in 12bit, those values are truncated (i.e lost) and they become just... "white": ![]() This is why it's important to ALWAYS deliver files to broadcasters in Limited TV Range AND make sure that they actually are with a Waveform Monitor. Don't worry, they will tell you if it's not 'cause the file will probably go through an AutoQC suite like Tektronix Cerify, Tektronix Aurora, Baton etc that will check scene by scene whether it actually is in limited tv range or not and if it's not, it will report it to the QC operator who will promptly reject the content with a QC Fail (well, most of the time it's actually QC TBC unless it's really bad and everywhere out of range). As to the question of "why can't we work in Full PC Range all the time?" the answer is: we can't because in real world scenarios signals are electrical and have to go through cables. It would be fine if the whole cable bandwidth was used for signal only, but it's not, in SDI cables you have stuff dedicated to the sync/timing/phase of the signal coming and going, you have the reference timecode etc and all those things take some millivolt (so in a digital world, they take some bits, which, in a Full PC Range representation, wouldn't be carried through as you can't physically transmit those). Bad... bad Groucho... That's not what I expected from the creator of AVSMeter eheheheheh ![]() Last edited by FranceBB; 22nd December 2022 at 20:19. |
|
![]() |
![]() |
![]() |
#16 | Link | ||
Registered User
Join Date: Sep 2007
Posts: 5,285
|
Quote:
Computer monitors use computer range (full) RGB (0-255), but most TV's are actually limited range RGB out of the box (RGB 16-235 in 8bit) https://www.benq.com/en-us/knowledge...ifference.html Almost every broadcaster (like your company) uses r103, which uses the Studio range RGB equations (in avisynth language it would be similar the full range or PC matrices) . In 8bit 0 to 255 code values are strictly reserved for SDI timing (0 to 3, 1020 to 1023 for 10bit) The r103 R,G,B out of gamut check actually uses studio range equations, not the computer range equations you typically see (ie. studio range RGB, RGB 16-235 black to white in 8bit, 64-940 RGB in 10bit - Not RGB 0-255, Not RGB 0-1023 Quote:
|
||
![]() |
![]() |
![]() |
#17 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,834
|
No one said YUV has to be legal. There is no such a hard requirement. It's just how it was used in the past, so it became a norm, but it's really obsolete "requirement". Today's digital world doesn't need it anymore. If not worse compression efficiency we could just stick to RGB snd don't even bother with YUV. YUV is for saving bandwidth (by 'cheating' on sampling) and also to get better compressibility.
When it comes to ProRes itself then things are bit different, because by official spec ProRes should always be legal. There is no way to flag range in ProRes private headers (nor in eg. MOV container), but there is in MXF. Saying this you can export full range ProRes, just be careful when importing as app has to be "somehow" aware it's full range. Resolve lets you interpret both ways, where eg. Premiere has (stupid) hard coded rules- 422 is always treated as limited where 444 as full!. ProRes is always YUV based internally. In this aspect DNxHR is better as it has private headers range flag and 444 can also be YUV or RGB based, so it covers all cases (if I remember well Resolve uses RGB version). Last edited by kolak; 22nd December 2022 at 20:54. |
![]() |
![]() |
![]() |
#18 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,834
|
Quote:
It all may be down to fact that Resolve is RGB internally and ProRes is YUV. You have quite a lot RGB<->YUV going on. Try the same with DNxHR. I think 444 one from Resolve is actually RGB based so you won't get any RGB<->YUV conversion and results should be "perfect". Other fact- codecs even like ProRes do overshoots during encoding, so any hard contrast will produce values which are slightly different than source ones. If I remember well you can expect up to 4 levels difference even on bars, so eg. 64 may end up as 60. This 'imperfection' is not really relevant in any real world scenarios, unless you do some scientific projects. In this case don't use any compression and stick to eg. EXR, TIFF or 16 bit RGB MOV. Last edited by kolak; 22nd December 2022 at 20:54. |
|
![]() |
![]() |
![]() |
#20 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,834
|
ProRes444 out of Resolve will have min Y as 254 and max Y of 3765 for SMPTE bars. Both slightly off ideal values.
ProResHQ will have 63 and 941. (at least those are reported by ffmpeg, which I don't trust 100% either). DNxHR has 245 and 3777, but because ffmpeg decodes it to YUV (even if file is RGB). I requested YUV stats, so actually it has to convert. There seems to be also some bug in Resolve when you force full levels. Something is not right I think. Well- never trust those tools ![]() Resolve use to properly read DNxHR range flag. BM had mess with it, I reported they fixed it. And few versions later it looks like it's broken again ![]() Some things never change- it's that much you can trust those pro tools ![]() Last edited by kolak; 22nd December 2022 at 21:46. |
![]() |
![]() |
![]() |
Tags |
full range, limited range, prores, ycbcr |
Thread Tools | Search this Thread |
Display Modes | |
|
|