VDub2 Canopus HQX 16-bit negotiation issue

[Hotte]

edius v210 and canopus hqx are similar in one group in terms of color
All pics were created manually with VDub2 in the following way: Load Clip, Direct stream copy, Decode Format=AutoSelect,Interpret YCbCr none/none, export single TIFF
These two are AVI-export directly from Edius. All Edius exports in 4.2.2 CSS like the original, project properties are YCbCr 10-bit BT.709


original, v210 to (ffmpeg) dnxhr_hqx, and edius direct dnxhr_hqx are similar in the other group
Same procedure and settings with VDub2:
- Original loaded unchanged
- Edius dnxhr was direct MXF export from built-in codec
- dnxhd was converted from the uncompressed V210 of above with ffmpeg latest release

How can I do better ?

[poisondeathray]

All pics were created manually with VDub2 in the following way: Load Clip, Direct stream copy, Decode Format=AutoSelect,Interpret YCbCr none/none, export single TIFF
These two are AVI-export directly from Edius. All Edius exports in 4.2.2 CSS like the original, project properties are YCbCr 10-bit BT.709


Same procedure and settings with VDub2:
- Original loaded unchanged
- Edius dnxhr was direct MXF export from built-in codec
- dnxhd was converted from the uncompressed V210 of above with ffmpeg latest release

How can I do better ?



In terms of the 601/709 mismatch - it doesn't add up.

All video versions are YUV. You used same procedure in vdub for screenshot creation for all of them. If one has the "wrong" color, they should all have same the "wrong" color . None of the jpg's have color profiles or metadata .

Even stranger is the ffmpeg dnxhd_dnxhr was created from the v210 version and yet has different color...

Something else is going on, there might be some flags or metadata interfering , but afaik vdub ignores them

Since the "AVI" files are in one group, there might be some local configuration difference, some VFW filter or vdub config difference






Another way is to control the conversion in avisynth explicitly

#load YUV source, use LWLibavVideoSource to be consistent
LWLibavVideoSource("video.ext")
ConvertToRGB24(matrix="rec709")





Or more important to your workflow, would be to take the screenshots in Edius, assuming what procedure Edius uses for screenshots is what it actually uses on the timeline





For example if you wanted to change the edius "v210" to "look" like the "original" colors

ImageSource("jK34YI9_edius_v210.jpg")
ConvertToYV24(matrix="rec601")
ConvertToRGB24(matrix="rec709")


or "original" to "v210"

ImageSource("LCWcIHe_orig.jpg")
ConvertToYV24(matrix="rec709")
ConvertToRGB24(matrix="rec601")

So that "prooves" that it's a "pure" 601/709 mismatch somewhere in the workflow

[Hotte]

I have redone all my samples to be sure I have not made any mistake - because there are lots of opportunities for errors when you fiddle with different codecs. And this time I also exported TIFFs out of Edius (which are 8bit only but that does not matter for this comparison).

The Edius-TIFFs show a very different but consistent result:

1. All exports and the original brought back into Edius have the exact same color and contrast if you compare them. This is what I would have expected from professional software.

2. Resolution of the codecs does not differ a lot. The same is true for sharpness except for ffmpeg DNxHD-HQX (derived from V210) which has some slight softness, so the haze is back. But the haze is not as bad as before. The reason could be that I was on an old 2017-version of ffmpeg which I updated now - Thanks Kolak!

3. And now comes what really suprises me: The contrast profile of all Edius-Tiffs (like in the preview) is visibly flatter then any of the VDub-TIFFs. I had a look into the viewfinder of my camera and found it is flatter than in the camera also. But there is no primary color-mapping applied in Edius.

Samples in better quality now:

Edius-TIFF: V210 uncompressed
https://imgur.com/ILFz7hv

Edius TIFF: V210> ffmpeg DNxHD HR_HQX. Same color/contrast now but slightly less sharp:
https://imgur.com/fUjkNXh

VDUB TIFF: V210 uncompressed: VDub color space is different, more contrast
https://imgur.com/HhBw4OH

I am a bit confused now: What is the correct contrast profile ? Even more confusion is this: I tried your avisynth/lwlibavvideosource tip but...
lwlibavvideosource("F:\Temp\Export_V210.avi")
info()
Return last
...returns interleaved (?) rubbish. How is this to be read with avs+? It is misinterpreted. It defintely is 10bit.
https://imgur.com/dulokJH

I can`t correctly interpret any of the codec exports with libav, only with ffm2.

Btw ffplayer shows the same false colors as VDub does.

Thanks for any advice.

[poisondeathray]

The contrast difference is full vs. limited range

The color difference is 601 vs. 709 matrix

If you adjust for them, you can make one group look like the other or vice-versa



10bit - Possibly you're using an old lsmash build, or old avisynth build

https://github.com/HolyWu/L-SMASH-Works/releases

v210 should open up as yuv422p10

MOV and MP4 container does not require indexing if you use LSmashVideoSource - ISO base media formats, (unlike ffms2 or LWLibavVideoSource)

[shekh]

Yeah! ***** The DNxHD would be a great addition to the compression menu as it seems to play some not unimportant role in professional video exchange and looks like being an excellent codec, also encoding is fully supported by ffmpeg.

I'd be ready to support that in any way that makes sense. I`d be happy to hear Shekh`s evaluation of the matter. I have no idea if that is a lot of work to do.


When I evaluated what codecs are good as intermediate I also saw DNxHD. However it is quite special as it does not work with arbitrary resolutions. I think it is not big problem to make some basic implementation, I just didnt like it.

[Hotte]

:thanks:
My god, 6 tips that turn my video conversion world upside down! I write my findings down here for Edius-X and Avisynth/VDub2 user generations to come (who are as ingenuous as myself). Be nice and honour Mr./Mrs. Poisondeathray for his/her generous help.

1. Luminance range: YES, this the first time I understand why Edius marks my original Input clip as BT.709 [Full] instead of BT.709 only. In my camera I activated Rec.709 full luminance range (0-1023 with 10-bit) instead of TV range (64-1023). Edius obeys that and shows flat full range. The camera viewfinder/monitor doesn't care and always displays TV (bad!). VDub2 picks TV as well because it doesn't know or care and the AVI-container doesn't tell either.

2. Version of LSMASH: YES, I was on a 2017 built. This one cannot read "YUV422P10" correctly. Upgrading to the latest version solves it. And this is the first time when info() shows the correct colorspace and bit-depth and that is:
Color Space: YUV422P10 Bits per Component: 10. ffmpeg will per default convert to YUV422P16 which is not bad but also not fully correct. You should go for this:
LSmashVideoSource("F:\Temp\Export_HQX.mov",format="YUV422P10")
info()
Return last

3. Pick the right video-container: YES. Edius-X (at least the workgroup version) allows Canopus HQX-super-fine 10bit Exports with AVI and MOV (see Quicktime-Export Menu). I've always tended to avoid Apple proprietary stuff. But in times of ffmpeg and libav this seems nothing to be concerned about. And this time it pays off to pick MOV instead of AVI, because then you can use LSmash and be rewarded with faster load times, no interim index files and a less stupid container that shows and transports more media information. Hence you can see the code above to import a Canopus HQX10bit MOV which is a dignified and adequate replacement for cumbersome Uncompressed AVI as far as my resolution tests yielded.

4. Color space: YES, I managed to tell VDub2 how to display this. At least I am getting very close (not 100% but must be around 99,8%) with the following extension LSmashVideoSource("F:\Temp\Export_HQX.mov",format="YUV422P10")
ConvertToRGB24(matrix="pc.709")
info()
Return last
To display a YUV color space on a monitor in sth that makes sense for our eyes it needs to be converted to RGB anywhere. To make sure that colorspace is not subject to a random default (like oldish Rec.601) in VDub, avs+ or anywhere else it is a good idea to talk Turkey: This is Rec.709. And to make clear that we don't want the PC (aka full) luminance range to be changed we choose "PC.709" instead of forcing gamma down to the more contrasty TV-Range which would be "Rec709".

@Poisondeathray, did I get it get it right or do you see anything to be improved with this little code snippet ?

I am getting almost correct colors and contrast with this. There are very subtle color differences, only viewable at very large zoomlevels:

Edius TIFF: HQX-10bit superfine
https://imgur.com/3bWpRUR

VDub2 TIFF: HQX-10bit superfine converted to RGB24 pc.709 as above
https://imgur.com/YZiFJAv

To round up the whole matter, there are a few questions left and I´d be very, very happy if you could answer these too:

1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?

2. I am not sure I fully understood the luminance range matter.

2a. Which luminance range would you recommend...
- for recording (in the camera)
- during video post processing for color grading with a calibrated sRGB monitor
- for the final version to be watched using a good video projector or modern TV

2b. Does the change of the luminance range have any lossy impact on my video...
- as long as I stay in the YUV world (e.g. avs+ filtering) or is it just a gamma curve that does not spoil anything ?
- when I change it being already in RGB ?

3. From all I learned from you and other experts in this forum I suspect that it is a good idea to stay in the YUV-world as long as possible while exporting and filtering with avs+ to have a minimum of loss. You only go RGB if a filter needs it or at the very end to display.
Is this a correct conclusion ?

If 3 is correct:
I heard that the latest "Primary color correction" filter of Edius-X works in RGB. Does this potentially come with IQ loss (there are YUV alternatives on board) ? I mean Edius would internally convert to RGB48 or something and convert back to YUV before it exports YUV e.g. with HQX.

[Hotte]

When I evaluated what codecs are good as intermediate I also saw DNxHD. However it is quite special as it does not work with arbitrary resolutions. I think it is not big problem to make some basic implementation, I just didnt like it.

Shekh, that sounds ever so great!

DNxHR is used in the AVID broadcast world (and some other manufacturers) as intraframe interchange/intermediate codec. It is well supported by Edius and is pretty fast on its timeline.
DNxHD was the name of its predecessor which was only used for up to 1920x1080 10bits 4.2.2

There is an ffmpeg implementation of DNxHD which can run in some hqx_hr profile to manage e.g. 4K in 10 bits 4.2.2 and there is a hqx_444 profile for even more.

The implementation of these two profiles would be of great use. The ffmpeg version is a very good codec. I only noticed a very tiny hint of softness - or let`s say defensive micro contrast. Others obviously did not notice that. Anyway this could be cured easily in post if ever necessary since you will never notice it at regular zoom levels in 4K.

For Edius-X users it seems to me the best alternative for 10-bit content to be filtered with avs+ and VDub2 to bring back the filtered clip into sth which Edius can handle. The reasons are:

- Edius' native codec Canopus HQX is only 8-bit in ffmpeg hence VDub2 (it is 10 bit in Edius)
- reimported ProRes is 10bit and has great IQ but it is very slow on an EdiusX-Timeline
- reimported CineForm is 10 bit and has great IQ as well, but is extremely slow on an EdiusX-Timeline - can even make the Editor crash

I know, you are not Edius support but I am sure it would well appreciate VDub2 to offer
ffmpeg DNxHD profile dnxhr_hqx
ffmpeg DNxHD profile dnxhr_444

as new compression options.

[kolak]

DNxHD an DNxHR in ffmpeg is same thing. They are all under dnxhd codec, just different profiles. You should today use DNxHR and forget DNxHD. All current ffmpeg builds have DNxHR.


Edius' native codec Canopus HQX is only 8-bit in ffmpeg


Wrong. HQX is fine ffmpeg and works at 10bit.
HQX is only 8bit in its VFW/Directshow codec installed by Edius (not in ffmpeg).
In Vdub2 you are using ffmpeg module to decode HQX (which I fine and 10bit), but VFW codec to encode, which only supports 8bit. Plain simple. Your only other tool which can encode HQX at 10bit is Resolve.

[Hotte]

DNxHD an DNxHR in ffmpeg is same thing. They are all under dnxhd codec, just different profiles. You should today use DNxHR and forget DNxHD. All current ffmpeg builds have DNxHR.
Are you are saying (ffmpeg DNxHD profile=dnxhr_hqx) = DNxHR ? So that's even better!
Still I can see a difference (the slightly reduced microcontrast I was talking about) between ffmpeg DNxHR profile=dnxhr_hqx and native DNxHR Exports from Edius. But that could be a difference between the professional and the ffmpeg implementation - couldn`t it ?

Wrong. HQX is fine ffmpeg and works at 10bit.
HQX is only 8bit in its VFW/Directshow codec installed by Edius (not in ffmpeg).
In Vdub2 you are using ffmpeg module to decode HQX (which I fine and 10bit), but VFW codec to encode, which only supports 8bit. Plain simple. Your only other tool which can encode HQX at 10bit is Resolve.
Ah! Does that mean that if VDub2 Compress-Dialog would implement HQX as "ffmpeg HQX" (rather than "HQX" without ffmpeg as today) it would compress 10 bit ? Now I understood, that HQX is not native VDub2 but came with Edius... Sometimes it takes time to understand - sorry. But then ffmpeg HQX would be another great implementation candidate...

[kolak]

1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?

2. I am not sure I fully understood the luminance range matter.

2a. Which luminance range would you recommend...
- for recording (in the camera)
- during video post processing for color grading with a calibrated sRGB monitor
- for the final version to be watched using a good video projector or modern TV

2b. Does the change of the luminance range have any lossy impact on my video...
- as long as I stay in the YUV world (e.g. avs+ filtering) or is it just a gamma curve that does not spoil anything ?
- when I change it being already in RGB ?

3. From all I learned from you and other experts in this forum I suspect that it is a good idea to stay in the YUV-world as long as possible while exporting and filtering with avs+ to have a minimum of loss. You only go RGB if a filter needs it or at the very end to display.
Is this a correct conclusion ?

If 3 is correct:
I heard that the latest "Primary color correction" filter of Edius-X works in RGB. Does this potentially come with IQ loss (there are YUV alternatives on board) ? I mean Edius would internally convert to RGB48 or something and convert back to YUV before it exports YUV e.g. with HQX.

If you have HD file then you use Rec.709. Anything which goes through 601 is plain wrong and needs fixing. This conversion is probably near lossless (don't think it can be lossless)- point is that it's totally not needed and all should be handled over 709.

Luminance range is basically meaningless (specially for 10bit + formats) if it's handled properly by your tools. Conversion from one to another is nothing to worry much about unless it's done badly.
It's end delivery point/spec (before this it can be both) which may mandate certain levels, eg. YUV for broadcast is limited range. Although nothing stops you in your own environment use full range for YUV before final export. It all depends on workflows, your tools, monitoring etc. If you don't use properly calibrated monitor over video card (but Edius GUI) then you are not very accurate anyway. Edius GUI preview is not a reference preview and should not be used for anything color critical. You need to buy BM card and feed calibrated monitor with signal from it. For Edius this is a proper way of monitoring video.



3. From all I learned from you and other experts in this forum I suspect that it is a good idea to stay in the YUV-world as long as possible while exporting and filtering with avs+ to have a minimum of loss. You only go RGB if a filter needs it or at the very end to display.
Is this a correct conclusion ?

Yes, overall this is the best practice. Once your video hits YUV at some point then it's good to stay in YUV (if possible), specially when you are going to deliver only to formats which require YUV. This is exactly why Edius uses YUV engine as it's tool for broadcast not really high-end finishing/grading. Staying with YUV has its own issues as well- you can easily produce out of gamut files (so such a YUV combinations which converting to final display RGB produce illegal (negating, or above max level) values). We know this is real issue in Edius and it has no native gamut legaliser (Premiere does now). If you work in RGB and convert to final YUV master out of gamut errors can't happen (can only as codecs overshoots).

From other hand when you start with high-end recording formats (RAW, film scans etc) then you rather want to work in RGB on GPU (with 32-bit floating point processing) and only during final exports you may convert to YUV.

This is a typical master chain for high quality production: RAW recording, grading/finishing in RGB (in tools like Resolve, Baselight, Mistika, Flame etc), export- quite often at this point master goes to YUV domain. This is the point where you may receive such a master, eg. ProRes, DNxHR and do editing, versioning etc. Here you ideally want to stay in YUV (as we already left RGB during "main" export), but in today's app this is not that essential anymore as RGB<->YUV conversion if not lossless (apparently with 32bit float it can be) is so high quality that this is your last worry. There are 1000x more important aspect of your video- editing, grading etc which impact its final perception.

Stop chasing some ideal paths- focus on artistic value as as long as you stay technically correct with main principals then this is easily good enough. For example your lack of proper monitoring in Edius is 100x more important than whole YUV<->RGB conversion "issue".

With tools like avs etc you have to test and validate path to make sure bit depth, levels, etc are preserved properly. Most NLes are rather closed apps and things work there fine, but don't take this as 100% ideal either. There are problems in Resolve, Premiere as well. I don't think I've seen some critical issues in Edius engine though (except maybe whole new primary color filter, which works in RGB). It seems to be well tested. Problems can still happen so any new export (new major app version) needs new validation and checks.

[kolak]

Are you are saying (ffmpeg DNxHD profile=dnxhr_hqx) = DNxHR ? So that's even better!
Still I can see a difference (the slightly reduced microcontrast I was talking about) between ffmpeg DNxHR profile=dnxhr_hqx and native DNxHR Exports from Edius. But that could be a difference between the professional and the ffmpeg implementation - couldn`t it ?


Ah! Does that mean that if VDub2 Compress-Dialog would implement HQX as "ffmpeg HQX" (rather than "HQX" without ffmpeg as today) it would compress 10 bit ? Now I understood, that HQX is not native VDub2 but came with Edius... Sometimes it takes time to understand - sorry. But then ffmpeg HQX would be another great implementation candidate...

DNxHR differences can be down to implementations. FFmpeg code is different than AVID reference one (it's the same as you have 10 different h264 encoders- some codec but you get slightly\very different results). You also have to make sure your avs etc path is correct.
For me ffmpeg is actually slightly better than AVID encoder. And because I use plain simple test (v210 source directly encoded ) I have no place for any error (except actually ffmpeg issue if exist). And because ffmpeg gives slightly higher PSNR and also when I visually compare in Resolve ffmpeg DNxHR show less difference I simply don't believe in your tests (your path is full of possible issues- it involves so many "engines").

Yes, but atm. ffmpeg has no HQX encoder- just decoder, so it won't happen.

[Hotte]

Thanks a lot Kolak for your extensive and well understandable explanations!

With your and poisondeathray's help I was able to understand the traps of the workflow and how I can handle them. With respect to my intended workflow Camera > Edius > AVS+ > Edius would you be so kind to check if these conclusion sounds basically consistent to you:

- My camera delivers H.264 YUV422 10bit. I should always record in rec709 with full luminance range
- I learned my NLE has a YUV engine. I can use the Primary Color Correction filter, no need to be concerned about that as possible conversion loss is not relevant in the amateur field
- Export HQX superfine 10bit as MOV is ok. I am staying in YUV
- Import into avs+ with LSMASH in YUV422P10. Check that rec709 arrives. If not, convert it.
- whenever possible use 16-bit AVS+ YUV422 capable filters
- to monitor interim results in VDub2 convertToRGB24(matrix="pc.709"). (monitor only, not for export)
- Compress the result with appropriate YUV422 10bit codec to import back into Edius (both of us seem to like ffmpeg DNxHD :))

And if you finally could tell what PSNR means and how you can measure it (as amateur) - would be great.

Thank you so much.

[poisondeathray]

@Poisondeathray, did I get it get it right or do you see anything to be improved with this little code snippet ?

I am getting almost correct colors and contrast with this. There are very subtle color differences, only viewable at very large zoomlevels:

Edius TIFF: HQX-10bit superfine
https://imgur.com/3bWpRUR

VDub2 TIFF: HQX-10bit superfine converted to RGB24 pc.709 as above
https://imgur.com/YZiFJAv


Yes, minor color chanage.

-Compare v210 instead of HQX. There are also detail differences. Is that from jpeg, or decoder differences for GV HQX ?

-There are slightly different algorithms to convert YUV422P10 to RGB . For example, chroma upsampling algorithm. (one might use bicubic, one might use slightly different coefficients of bicubic, or bilinear etc...). Chroma center location interpretation. Avisynth internal resizers use "center" instead of "mpeg2". You can use z_convert_format (avsresize) instead of ConvertToRGB(blah).

You're just using it for a rough "preview" instead of vdub's 601 preview, so it shouldn't matter too much

If you want to include other accurate testing - use 10bit colorbars . 10bit colorbars should reproduce exact 8bit RGB colors as per rp219. Also include some other test material like gradients in your workflow to debug things end to end




1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?


Not sure what you mean ?

If you perform a colormatrix transform in YUV integer (8bit, 10bit, 16bit) - it's not lossless, not 100% reversible. If you do it in float, it can be a lossless transform.

Converting to 10bit YUV to 8bit RGB is not lossless. (You're just using it as a rough "preview")




2. I am not sure I fully understood the luminance range matter.


You did. The flag in the original recording signalled Edius to convert to RGB (for preview) using full range equation. You can simulate the same using "PC" matrix in avs


2a. Which luminance range would you recommend...
- for recording (in the camera)
- during video post processing for color grading with a calibrated sRGB monitor
- for the final version to be watched using a good video projector or modern TV

Pros/cons to acquisition format . Full range is probably better if the tools used can manipulate in float . If not, there is more loss because you have to adjust the ranges to a larger extent to make them "standard range"

But final version should always be "standard" range. So fast turn around situations (direct to TV, broadcast ENG) never use "full range" for acquistion


2b. Does the change of the luminance range have any lossy impact on my video...
- as long as I stay in the YUV world (e.g. avs+ filtering) or is it just a gamma curve that does not spoil anything ?
- when I change it being already in RGB ?


YUV to RGB integer is not lossless. RGB float can be lossless transform if done properly. Not very many tools do the round trip correctly, not even Resolve (it fails on the RGB float to YUV part of the trip).

Vapoursynth can do this correctly. In order to be 100% lossless the chroma up/down sampling has to use nearest neighbor algorithm, and float used on all calculations

But these are relatively minor issues. Don't loose sight of the "big picture" as Kolak said




I heard that the latest "Primary color correction" filter of Edius-X works in RGB. Does this potentially come with IQ loss (there are YUV alternatives on board) ? I mean Edius would internally convert to RGB48 or something and convert back to YUV before it exports YUV e.g. with HQX.

If it works in RGB48, not float, then potentially there is a problem. You'd have to "legalize" YUV beforehand, or risk significant clipping




And if you finally could tell what PSNR means and how you can measure it (as amateur) - would be great.

PSNR is a measure of "quality" compared to a "source".
One way can use ffmpeg to measure
https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio
https://ffmpeg.org/ffmpeg-filters.html#psnr

It's not a good measure for subjective testing, but it's excellent to determine if something is lossless or not (infinity dB)

[kolak]

If Rec.709 and Rec.601 gamuts are slightly different how can you guarantee lossless conversion (I assume you don't clamp to gamut limits)?

PSNR may not be good measure for heavy compression but it's ok for intermediate codecs as in this cases differences are more mathematical than perceptual (overall differences are small and quite often not visible at 1:1 frame).

[poisondeathray]

If Rec.709 and Rec.601 gamuts are slightly different how can you guarantee lossless conversion (I assume you don't clamp to gamut limits)?

You don't "guarantee" anything. You test it.

There are no "limits" to float, nothing is clamped



PSNR may not be good measure for heavy compression but it's ok for intermediate codecs as in this cases differences are more mathematical than perceptual (overall differences are small and quite often not visible at 1:1 frame).

Agreed - it's ok for high quality intermediates and trend testing. Not so much for end deliverable bitrates

[kolak]

ffmpeg -i ref_source -i source -filter_complex psnr -f null -

this how you do it with ffmpeg. At the end you get stats. Anything within +-3dB is not a very meaningful difference. Eg ProRes may show average PSNR of 56dB and DNxHR 54dB. For clean good quality sources (like my BRAW 12K sample downscaled to HD) PSNR can go above 60dB which means loos is very small. For difficult scenes PSNR for intermediate codecs can go <50dB.

You can read about HQX here in its whitepaper:

https://www.syntex.sk/media/document/1107/75640_grass-valley-hqx-whitepaper.pdf

you also have Apple white paper about ProRes which has very similar results: https://www.apple.com/final-cut-pro/docs/Apple_ProRes_White_Paper.pdf )

you have PSNR graphs against other codecs. GV uses same source as Apple (STEM footage), which is specially prepared footage for digital cinema validation. It's never compressed (shot on 35M film) source which is very important. I use to have access to it- very demanding source.
If you take some source from the internet (which you have no idea how it was shot) your results can be very misleading. You may get some nice source which was shot on eg. ALEXA in ProRes natively. Then if you convert that to ProRes then it's 2nd generation fo the same codec and your PSNR will be very high. It won't be true for other codecs as they use different math. You have to really know what you are doing in those tests as you can easily produce something very false :) Cineform behaves quite differently on uncompressed sources with original noise/grain compared to source which are already went through DCT based compression (so ProRes, DNxHR etc). Internet is full of fundamentally flawed tests.

[kolak]

You don't "guarantee" anything. You test it.

There are no "limits" to float, nothing is clamped




Agreed - it's ok for high quality intermediates and trend testing. Not so much for end deliverable bitrates

But Re.709 gamut doesn't 100% align with Rec.601, so how can you represent color from Rec.709 gamut (which is not part of 601) in Rec.601 and then get it back?

[poisondeathray]

But Re.709 gamut doesn't 100% align with Rec.601, so how can you represent color from Rec.709 gamut (which is not part of 620) in Rec.601 and then get it back?

For the same reasons other conversions that normally are clipped

(-) values and values > 1.0 are retained in float

It works, you can compare with PSNR and diff .

You'd have to run some matlab calculation to test every single iteration and combination to verify, but it's lossless in every synthetic and camera acquisition clip I've tested

[kolak]

YUV to RGB integer is not lossless. RGB float can be lossless transform if done properly. Not very many tools do the round trip correctly, not even Resolve (it fails on the RGB float to YUV part of the trip).


Resolve use to terribly bad with it. Few generation of v210 to v210 and your red channel was shifted few pixels :)
They improved it and made v210 to v210 (if we do pure transcode process) not going to RGB at all I think (may be true for other YUV based codecs).

[kolak]

For the same reasons other conversions that normally are clipped

(-) values and values > 1.0 are retained in float

It works, you can compare with PSNR and diff .

You'd have to run some matlab calculation to test every single iteration and combination to verify, but it's lossless in every synthetic and camera acquisition clip I've tested

But it means you actually producing files which are not really properly clamped to your end gamut (which for sake of conversion itself is not an issue).
There is no way that certain color can go from Rec.709->Rec.601->Rec.709 as this color doesn't exist in strict Rec.601 gamut.

It's like you had P3 gamut, converted to strict Rec.709 and then back. No way it can be ever lossless.

[poisondeathray]

Resolve use to terribly bad with it. Few generation of v210 to v210 and your red channel was shifted few pixels :)
They improved it and made v210 to v210 (if we do pure transcode process) not going to RGB at all I think (may be true for other YUV based codecs).

ffmpeg has a v210 "bug" of sorts too, same with prores (for encoding). Values are clamped to 4-1019. vdub's v210 and prores encoder are not. You can argue that it's "intended" behaviour, because certified prores never records those values either

On one hand, yes, code values 0,1023 are reserved for sync, important for equipment - but for synthetic tests could argue a codec should not artifically limit a range

It's a bit inconsistent handling, because ffmpeg dnxhr does not limit the range. Neither do official implementations of v210 or dnxhr

[kolak]

ffmpeg has a v210 "bug" of sorts too, same with prores (for encoding). Values are clamped to 4-1019. vdub's v210 and prores encoder are not. You can argue that it's "intended" behaviour, because certified prores never records those values either

On one hand, yes, code values 0,1023 are reserved for sync, important for equipment - but for synthetic tests could argue a codec should not artifically limit a range


This comes from HDMI spec where those few values are reserved for special usage etc.
Eizo monitors even have setting which either passes 1:1 or stretches 4-1019 to full 0-1023 range.

FFmpeg probably follows Apple official encoder (and Apple follows some SMPTE/EBU etc specs). Not a fan of those "reserved bits" - they only produce confusion :)

[poisondeathray]

But it means you actually producing files which are not really properly clamped to your end gamut (which for sake of conversion itself is not an issue).
There is no way that certain color can go from Rec.709->Rec.601->Rec.709 as this color doesn't exist in strict Rec.601 gamut.

It's like you had P3 gamut, converted to strict Rec.709 and then back. No way it can be ever lossless.

You're talking about slightly different things -

Think about it - you're not producing a file in float. It's for intermediate. You're not clamping anything

Recall , the topic was question was changing 709/601 in YUV. A colormatrix transform, as if the file in YUV had used 709 instead (or 601 instead)

The background info was intermediate RGB preview (recall vdub uses 601) , was different than using 709





1. Does color-conversion Rec.601<>Rec.709 back and fourth in AVS+ / LSmash ... come with any losses or is it just fully reversible ?


Not sure what you mean ?

If you perform a colormatrix transform in YUV integer (8bit, 10bit, 16bit) - it's not lossless, not 100% reversible. If you do it in float, it can be a lossless transform.

Converting to 10bit YUV to 8bit RGB is not lossless. (You're just using it as a rough "preview")


And this is correct - You can apply the colormatrix transorm in YUV444PS . It's reversible. You can get the original file's code values back

[kolak]

You're talking about slightly different things -



Yep, different things.

[poisondeathray]

This is comes from HDMI spec where those few values are reserved for special usage etc.
Eizo monitors even have setting which either passes 1:1 or stretches 4-1019 to full 0-1023 range.

FFmpeg probably follows Apple official encoder (and Apple follows some SMPTE/EBU etc specs). Not a fan of those "reserved bits" - they only produce confusion :)


Yes, but "v210" exported from Adobe, Resolve , etc, retain 0-1023 code values. ffmpeg v210 does not.

Do you see the problem ?

Yeah, it's not important in the big picture; but you better believe that "little detail" becomes important when testing decoders, psnr . Details matter. I reported it before on ffmpeg bug tracker. Nobody seems to care :(

At the very least there is inconsitent behaviour within ffmpeg, and between professional apps

[kolak]

Yep, definitely wasn't aware ffmpeg does it in such a selective manner.


PSNR on v210 out of Resolve encoded to v210 in ffmpeg is still inf.
PSNR on DNxHR encoded to v210 in ffmpeg is still inf.

[poisondeathray]

Yep, definitely wasn't aware ffmpeg does it in such a selective manner.


PSNR on v210 out of Resolve encoded to v210 in ffmpeg is still inf.
PSNR on DNxHR encoded to v210 in ffmpeg is still inf.




Check if your original code values had Y <4, >1019 to begin with. Make sure export had "retain sub-black...superwhite" enabled if using resolve

Verify in other programs that the code values are there with the export. (e.g. vapoursynth, vsedit colorpicker, or avisynth can now with histogram(bits=10))

I use a wrapper function to turn right , left

Thisto(10)



function THisto(clip c, int "bits")
{
bits = Default(bits, 8)
c.TurnRight().Histogram(bits=bits).TurnLeft()
}



ffmpeg v210 encoder is definitely not "infinity" on a ramp, and you can see it on the waveform in multiple programs. You don't need PSNR to know it's not lossless. You can see it. Even ffmpeg's own -vf waveform detects the discrepancy.

To be clear, ffmpeg can still DEcode 0-1023 from other formats e.g. if you test 10bit422 AVC input, it still outputs 0-1023 raw values. But if you encode a v210 version , that will only contain 4-1019 values.

Let me know if me to upload a demo package



(When using ffmpeg psnr, it's good idea to reset timebase - different containers use different timebases and this can cause ffmpeg to compare different frames (not important for 1 frame
obviously, but when testing "video"))

This is a 0-1023 ramp test.

Resolve v210 vs. original v210
ffmpeg -r 24 -i 1_resolve_v210.mov -r 24 -i 0_1023x720_0-1023.mov -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -
[Parsed_psnr_4 @ 00000027ffcb73c0] PSNR y:inf u:inf v:inf average:inf min:inf max:inf

FFmpeg v210 vs. Resolve v210
ffmpeg -r 24 -i 2_ffmpeg_v210.mov -r 24 -i 1_resolve_v210.mov -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]psnr" -f null -

[Parsed_psnr_4 @ 0000001488292ec0] PSNR y:72.519000 u:inf v:inf average:75.529300 min:75.529300 max:75.529300

[kolak]

I think Resolve ramp generator also skips them, so this is why I see no issue.

[poisondeathray]

Here is a 10bit ramp package. 1024x720 perfect ramp. Each column corresponds to pixel value in Y'
https://www.mediafire.com/file/cvulp8cougjm4nb/10bit_ramp_tests.zip/file

0_1023x720_0-1023.mov is "original"

1_resolve_v210.mov is original exported from resolve

2_ffmpeg_v210.mov is generated from
ffmpeg -i 1_resolve_v210.mov -c:v v210 -an 2_ffmpeg_v210.mov



e.g ffmpeg -vf waveform
ffmpeg -i 1_resolve_v210.mov -vf waveform=g=green 1_resolve_v210.png
https://i.postimg.cc/pdnmxW6C/1-resolve-v210.png (https://postimg.cc/WDTNwvjq)


ffmpeg -i 2_ffmpeg_v210.mov -vf waveform=g=green 2_ffmpeg_v210.png
https://i.postimg.cc/mgPDNZB4/2-ffmpeg-v210.png (https://postimg.cc/G41dbC4g)

[kolak]

Got it now- those ramp generators in Resolve/Premiere are crap :)

So this is basically v210 encoder and ProRes encoder (and decoders are passing whatever is there)?

Also- both Resolve and Premiere are quite crap when it comes to trying to see it. Resolve parade shows for both files same result, so it means it's probably made to skip those reserved values!
Interesting. There is always some new crap coming out :)

DNxHR encoder in my ffmpeg also clips them (4.3.2). It doesn't do for Cineform though.

[poisondeathray]

So this is basically v210 encoder and ProRes encoder?

Yes, ffmpeg v210 and ffmpeg prores encoder(s) (native, and prores_ks, and prores_aw)

[kolak]

DNxHR as well (at least in ffmpeg 4.3.2+)
Good to know- thanks.

update: it's Resolve decoder, not ffmpeg encoder.

[poisondeathray]

Also- both Resolve and Premiere are quite crap when it comes to trying to see it. Resolve parade shows for both files same result, so it means it's probably made to skip those reserved values!
Interesting. There is always some new crap coming out :)



I heard apparently there are some bugs in newest resolve waveform...I'm still using older version as I have a few projects in progress and I never upgrade in between

Temporarily assign clip attributes to "full" range, and you will see difference between original v210 (or resolve v210) and ffmpeg v210 . As you already know, resolve is working in RGB float. When "auto" is set and clip is unflagged, or flagged limited, it takes Y 64-940 to "map" to RGB 0-1023 . But internally nothing is clipped at that stage, it's working in float RGB. It's just the display at that point or RGB interpretation, not the actual file

For premiere you can see in lumetri scopes too set to float, you might have to increase the window size, but there is clipping at low and high level with ffmpeg v210 version, but not "original v210", or "resolve v210". But I agree it is a "crap" display compared to ffmpeg waveform, or avisynth histogram(10)



DNxHR encoder in my ffmpeg also clips them (4.3.2). It doesn't do for Cineform though.

It didn't in the past and it doesn't for me (just checked)

Exactly what are you doing for input and for measuring output ? What decoder is being used

[kolak]

Yes, in Premiere I sort of saw it when setting to float.

I used Resolve to check, so problem is probably there.

[poisondeathray]

Yes, resolve's "waveform" is not a true Y' waveform . It's actually RGB converted values - so that depends on how you convert to RGB

ffmpeg -vf waveform or avs's histogram(10) are true Y' waveforms , they are measuring Y' values directly

And it's not a "knock" against resolve. For grading software, that's what the scopes should be doing

[kolak]

I see it now in Resolve as well (with forced full range). It shows it in form of missing data, so your line basically ends at 1019. At the bottom it shows it in some strange way, but it's also visible (Resolve 17.1.1.).

[kolak]

It's Resolve DNxHR decoder is doing it. ffmepg waveform is fine.
Looks like anything "official" (Apple, AVID etc.) is now skipping those reserved bits and rest of the codecs/tools not. More mess to the already messy world of codecs etc. :)

[poisondeathray]

It's Resolve DNxHR decoder is doing it. ffmepg waveform is fine.
Looks like anything "official" (Apple, AVID etc.) is now skipping those reserved bits and rest of the codecs/tools not. More mess to the already messy world of codecs etc. :)

Yes

I see it in resolve import of ffmpeg dnxhr_hqx too, but not with libavcodec decoding (0-1023)

If you use resolve to encode v210 (0-1023) to dnxhr_hqx, it's 0-1023 with libavcodec decoding, but reimporting to resolve also some clipping (again, temporarily set to full range clip attribute)

So that suggests it's resolve's dnxhr_hqx decoder behaviour



Cineform is full scale.


But you said cineform is 0-1023 ? In resolve ? Was that go pro cineform official, or libavcodec sdk implementation, or resolve cineform, or something else
So not all "official" :D

But this isnothing "big" in terms of grand schemes.

It just can "skews" some comparisons sometimes, like PSNR, depending on what exactly is being used

[poisondeathray]

Canopus/GV HQX exported from resolve is also 0-1023 in both libavcodec and re-import back into resolve is 0-1023

[kolak]

ffmpeg Cineform.
I don't think David was that paranoid with all those SMPTE specs :)

[poisondeathray]

Another thing maybe shekh can look at is vdub2's prores encoding show gaps when encoding v210 . Not just the clipping, but almost like 8bit conversion. ffmpeg's equivalent v210 to prores does not (4-1019, but smooth)

vdub2_v210_to_prores
https://i.postimg.cc/1RNb6fgP/vdub2-v210-to-prores.png (https://postimg.cc/vDbPw8LS)

ffmpeg_v210_to_prores
(see earlier post)
https://forum.doom9.org/showthread.php?p=1940546#post1940546

[kolak]

But this isnothing "big" in terms of grand schemes.

It just can "skews" some comparisons sometimes, like PSNR, depending on what exactly is being used

Yes, but they should leave files world.
Those reserved bits are actually coming from SDI I think.
I don't think HDMI actually needs them at all, but I may be wrong.

Yes, there may be bug in Vdub and data may be going over 8bit at some point (or maybe some auto format choice bug). It quite clearly shows 4 levels pattern.
Vdub2 is not the same as old Vdub. New features, but new mess related to whole ffmpeg etc as well.

[poisondeathray]

I just don't like the inconsistencies;

And I'd argue that v210 should never be clipped

Clip when you have some end deliverable, ok that would be fine, but it should be up to user. But not these intermediates.

Ok maybe Prores, because I've never seen 0-1023 official prores software/hardware mac, scratch

[Hotte]

Thanks poisondeathray for all your answers.

Yes, minor color chanage.

-Compare v210 instead of HQX. There are also detail differences. Is that from jpeg, or decoder differences for GV HQX ?

The detail differences in the jpegs are not visible in the original TIFFs => JPEG issue.

You can use z_convert_format (avsresize) instead of ConvertToRGB(blah).

No idea how that works. Did not find anything like "z_convert_format". Could you give a little code example ?


You're just using it for a rough "preview" instead of vdub's 601 preview, so it shouldn't matter too much

Yes if you think that this is acceptable for a preview I am with you.


If you want to include other accurate testing - use 10bit colorbars . 10bit colorbars should reproduce exact 8bit RGB colors as per rp219. Also include some other test material like gradients in your workflow to debug things end to end


Could you give me a good link where I can download test material ?

Just another question to add: What if I record with standard luminance range instead of full ? You said I should go for standard in the end anyway and during the process there is risk of IQ loss if conversion is not being done properly. My subjective feeling is always, that I loose dynamic range if I record with standard range. Is that correct or not ?

[kolak]

I just don't like the inconsistencies;

And I'd argue that v210 should never be clipped

Clip when you have some end deliverable, ok that would be fine, but it should be up to user. But not these intermediates.

Ok maybe Prores, because I've never seen 0-1023 official prores software/hardware mac, scratch

Yep. And after quick search I'm almost sure that those reserved bits are coming from SDI and HDMI doesn't actually need them, but for "compatibility" they preserved them in HDMI spec. Now they force them to files world. Don't like it.
Sometimes 4-1019 is called SDI Full :)

[kolak]

Just another question to add: What if I record with standard luminance Range instead of Full ? You said I should to go for standard in the end anyway and during the process there is risk of IQ loss if it is not done properly. My subjective feeling is, that I loose dynamic range if I record with standard range. Is that correct or not ?

This is somehow important for 8bit recording. You simply have less range: 235-16=219 vs 0-254=255, so actually quite a big difference (more banding etc.). For 10bit (or more) this is less relevant.

[poisondeathray]

Could you give me a good link where I can download test material ?


I posted a 0-1023 10bit greyscale gradient earlier

All NLE's should have 75% colorbars. I'd imagine Edius has them. Free version of resolve can generate them too. If you have problems I can upload bars later

"official" 8bit, 10bit, 12bit YUV values are listed in rp219

A proper 10bit YUV to 8bit RGB conversion should give you the exact numbers for 75% colors. e.g. 75% "red" should be 191,0,0 in 8bit RGB using limited range equations (rec matrix in avisynth). Or 180,0,0 using full range equations (pc matrix in avisynth)

(Subsampling is not an issue with "large" bars, only at the borders between colors)



Did not find anything like "z_convert_format". Could you give a little code example ?

This is avsresize, the avisynth implemenation of the zimg library
http://avisynth.nl/index.php/Avsresize


eg. avsresize. "RGBP" is 8bit RGB planar

z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")

"point" is just nearest neighbor algorithm. It just doubles the chroma samples in the width. e.g. If you had 1920x1080 4:2:2, the U,V samples are 960x1080. It just duplicates them to 1920x1080. So on a pattern, the color borders will look "blocky"

"Bilinear" samples a 2x2 grid (4 pixels)
"Bicubic" samples a 4x4 grid (16 pixels)

There are dozens (or more like 100's) of resampling algorithms.

So you can get different pixel colors, depending on the chroma upscampling algorithm for RGB conversion, but a large "bar" such as in color bar patters would be independent of chroma upsampling. But this is one of the reason why one program might do things differently tha another program

If you get "wrong" colors on large colorbars, then something is ... wrong

Chroma location interpretation can also affect results (one of the reasons for suggesting avsresize, one possible explanation for some of the differences you see), but more for 4:2:0 material because it's 1/2 sampled width and height, 4:2:2 is "only" 1/2 sampled width, the errors are less drastic. When you have different chroma interpretations ("center" or "mpeg1") vs. ("mpeg2" or "left") , the color borders can shift when do subsequent conversions. It's usually not that noticable with 1 or 2 generations. It's more of an academic issue, or if you deal with RGB content and graphics. Avisynth is a bit non standard, because "mpeg2" or "left" is the standard chroma siting in all programs

[Hotte]

All NLE's should have 75% colorbars. I'd imagine Edius has them.
Yes. There is a collection, e.g. SMPTE RP 219. So the principle is to pass them through the chain, get the rgb-colors with a color picker and compare them to the lookup table ?

Is SMPTE RP 219 suitable for both 8-bit and 10-bit material ?


z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:l=>rgb:709:709:f")

I tried
z_ConvertFormat(pixel_type="RGBP", resample_filter="point", colorspace_op="709:709:709:f=>rgb:709:709:f")
(exchanged the first l with an f). This shows only minor changes on a pixel level in comparison to ConvertToRGB24, no matter if I go for bilinear, bicubic or whatever. The overall very slight shift vs. yellow in the grass which the VDub2-TIFF holds against the Edius-TIFF (and this time I was using uncompressed V210 instead of HQX) is still there, but hey, differences are very slight and should be just fine for preview purpose.

[kolak]

I just don't like the inconsistencies;

And I'd argue that v210 should never be clipped

Clip when you have some end deliverable, ok that would be fine, but it should be up to user. But not these intermediates.

Ok maybe Prores, because I've never seen 0-1023 official prores software/hardware mac, scratch

Side note- ProRes444 modes do pass those reserved bits (in official Apple encoder).

[poisondeathray]

Yes. There is a collection, e.g. SMPTE RP 219. So the principle is to pass them through the chain, get the rgb-colors with a color picker and compare them to the lookup table ?

Is SMPTE RP 219 suitable for both 8-bit and 10-bit material ?



Yes. You have known colors. In YUV and RGB. In 8,10,12 bit . And rp219 lists the exact values for Y,U,V in 8,10,12 bit

For 8bit YUV, the colors in 8bit RGB will be slightly off, and it's expected to be +/-3. For example "75% Red" might be 192,1,0 or something (when using standard range equations). But 10bit YUV bars should yield perfect 8bit RGB values.

It's common to get slight color shifting when improper conversion is performed. A common scenario is a "fast" 8bit conversion to YUV before RGB. You're just ruling out if something was done incorrectly somewhere , maybe some setting, some switch - that's what all these low level tests are used for - so when you're doing your actual projects, there aren't any unexpected surprises (like clipping)