How to apply BT.2407 "simple linear matrix transformation" HLG to 709

[emcodem]

Dears,

until attachments are approved, here a link containing all attachments of this post: Attachments

so i desperately try since 3 days now to find some way to do this but i really struggle with it. The thing is that i am ffmpeg/avisynth (ffastrans) guy and developer and was not able to do this but some enthusiastic (but non scientific) user of Blackmagic Davinci was actually able to do it. I really cannot let it stand like this!

So, the goal is for research/scientifical reasons to reproduce more or less exactly what they do in the ITU BT.2111 colorbar documentation, the last 2 pages of it (figure 10 and 12 embeeded below).

Relevant documents:

Bt.2111 Color bar (scroll to pre-last page!)

• https://www.itu.int/dms_pubrec/itu-r...2-I!!PDF-E.pdf

Which links to:

• https://www.itu.int/dms_pub/itu-r/op...2017-PDF-E.pdf

I must work with yuv42210p source so i uploaded one uncompressed frame of this in .nut container (sorry for that, didnt know a better way to keep the source clean).

• Source file: Download

I started by validating to check if i can see the original, unmodified values in AVSPmod. The 2111 document has them in RGB, that is why i use z_ConvertFormat before display them in AVSPmod "on mouse hover"

FFVideoSource("c:\temp\2111.nut")
z_ConvertFormat(pixel_type="rgbp10",colorspace_op="2020ncl:std-b67:2020:limited=>rgb:same:same:limited") #yuv uhd to rgb

This seems to work (with exception that in the colors we have some pixels +-1, which is great already!

Now, my problem is that i don't manage to get even anywhere close to the images or values the last 2-3 pages of the 2111 pdf describes the scene referred and display referred converted values after "simple linear matrix transformation" (which is described in point 2 of Bt.2407.pdf)

Figure 10 in 2111.pdf


Scene Referred should end up in e.g. yellow 75% at RGB values "940 940 64" or "0x3AC 0x3AC 0x40"

Figure 12 in 2111.pdf


Display referred should end up in e.g. yellow 75% (RGB) values "940 939 64" or "0x3ac 0x3ab 0x40" (in other colors, the difference between scene and display conversion is more obvious)

So for start of course i concentrated on scene-referred Method because as far as i understand, we never really work with display referred stuff at all when applying filebased conversion steps to HLG.

The avs plugin that seems to deliver most scientific ways of controlling Primary, Matrix and Transfer conversion and also some control over OOTF, OETF and reverse seems to be fmtc plugin. I tried not only that but also hdrtools and zimg conversion but i was not at all able to get anywhere near to the expected values the 2111.pdf mentions:



From all methods i tried so far, fmtc was the most promising was using fmtconv like this:

loadplugin("E:\FFAStrans1.3.0.2\Processors\avs_plugins\ffms2\x64\ffms2.dll")
loadplugin("E:\Downloads\FFAStrans1.3.0.2\Processors\avs_plugins\ffms2\x64\fmtconv.dll")

FFVideoSource("c:\temp\2111.nut")
ConvertToYuv444()
fmtc_matrix ("2020",bits=16)
#now we are RGB
fmtc_transfer (transs="hlg",transd="709")
fmtc_primaries (prims="2020",primd="709")
fmtc_matrix (mat="709")
#we are back to yuv again
fmtc_bitdepth (10,dmode=6)
#convert to rgb for avspmod mouseover value investigation
z_ConvertFormat(pixel_type="rgbp10",colorspace_op="709:709:709:limited=>rgb:same:same:limited") #yuv uhd to rgb

using this method, i get pretty close to the wanted values for the main bars but especially the bars on the bottom right (original 709 bars) are totally off:




The values for this BLUE at the bottom right (which is 709 in the original colorbar) should be after conversion:
Scene referred: 66/64/940 (0x42,0x40,0x3ac)
Display referred: 93/64/768 (0x60,0x40,0x300)

But i have: 167/76/769 (0xA7,0x4c,0x301)
Any clues?

[DTL]

First AVSpmod may be not samples monitor instrument - it may use some display conversion tools like narrow to full remap or other to make PC-monitoring better. Your screenshot show clipped underblacks and it is not normal for PC-display (sRGB) of industry narrow data format of this colour bars pattern. Also Y-ramp looks like badly clipped at whites (and colours patches are over-brighten in compare with typical 75%) - so something is broken at your transform filter chain. May be somewhere happen range expanding (or even several times).

The industry (continental/regional EBU/ARIB or all-planet ITU) documents typically defined code values in 'narrow' (limited) range.

So better to start from some MS Excel or Open Office design of the conversion that you need and check code values of each plugin input/output in the digital software debugger. I not sure if samples sampler in AVSpmod display output code values of the last filter in the script provided ?

Also as you set lots of filters in the chain - it is better to check each filter output to found where some non-expected started to happen (using your prepared calculations of all chain elements for all colour patches). Part of your processing is in the RGB so not require additional YUV->RGB convert for display.

[emcodem]

Hey DTL and thank you very much for your answer.
Sorry up in front for the many quotes but i fear it is needed for correct clarification.

Originally Posted by DTL

First AVSpmod may be not samples monitor instrument - it may use some display conversion tools like narrow to full remap or other to make PC-monitoring better.

Absolutely agreed, we must be perfectly clear about how our measuring equipment works but i am sure AVSPmod shows the values of the returned clip and nothing else. - at least referring to the yuv/rgb values we see displayed on mouse hover.
What i did was to verify the output of AVSPmod the following way:

• avs Script: only decode source, no filtering. Compare the AVSPmod displayed hex values to ffmpeg signalstat measured YUV values (100% match).
• avs Script: Add matrix conversion to rgb but leave everything else unchanged, compare to Bt.2111 documented "original" values

All tests went 100% as expected, so AVSPmod displays the returned Pixels original values, if you return YUV, you see YUV values, if you return RGB you see RGB values. Any displayed value seems to be unmodified return values of the last clip.
From my perspective this should suffice for this kind of experimenting.

Originally Posted by DTL

Your screenshot show clipped underblacks and it is not normal for PC-display (sRGB) of industry narrow data format of this colour bars pattern. Also Y-ramp looks like badly clipped at whites (and colours patches are over-brighten in compare with typical 75%) - so something is broken at your transform filter chain. May be somewhere happen range expanding (or even several times).

Well it is clearly the goal here to clip 75% HLG to 100% 709, so don't worry about clipped. Looking at my measured values, my current method is still clearly somewhat "off" from the expected values, also it differs a little bit from the sceenshots we see in 2111.pdf.

Originally Posted by DTL

So better to start from some MS Excel or Open Office design of the conversion that you need and check code values of each plugin input/output in the digital software debugger.

Not 100% sure what you mean. I could reproduce what my current filters do in excel but i would only copy the exact thing that i read in the source code of the plugin, thus coming to the exact same result.
Also, the 2111 document does not describe all the values after each step, so no way for me to check every step against standard conformance.
Also, which kind of "digital software debugger" you mean? You mean like building and running a debug build and attach debug breakpoints or such?

Originally Posted by DTL

Also as you set lots of filters in the chain - it is better to check each filter output to found where some non-expected started to happen (using your prepared calculations of all chain elements for all colour patches). Part of your processing is in the RGB so not require additional YUV->RGB convert for display.

This is very interesting actually. As the document 2111 colorbar.pdf is only built on RGB values, i should get rid of any YUV related stuff in my testing at all. This means i need to generate a valid RGB version of the colorbar and work with it as input.
Let me investigate how to do that in a clean way

[DTL]

If you completely sure in the displayed code values in AVSpmod - you not need other debugger.

About HDR to SDR AVS plugins activity you can look into thread https://forum.doom9.org/showthread.php?t=183224 (mostly last posts). Different plugins may require some 'magic numbers' to make simply equal or at least close results in 'standard conversion'.

Last comment there: https://forum.doom9.org/showthread.p...55#post1972755
About the 'magic' amplitude correction:
- fmtconv - after rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear", sceneref=true) the ouput is within range [0,~1]. After applying RGBAdjust values of 0.5, the output becomes within range [0, ~0.5].
- avsresize after rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:l", scene_referred=true) the output is within range [0, ~3.188]. After applying RGBAdjust values of 0.157, the output becomes within range [0, ~0.5].


Also your used plugins have very much of params and 'default' state may be not correct for your task - so it may recommnended to fill all params explicitly in your script.

As some even next level of nighmare for enduser of AVS+ - it support extensive set of 'metadata' exchange (clip and frame properties) between filters/plugins and they also can change action of each plugin (from defaults).

[emcodem]

Again thank you very much DTL, your linked post and your inputs seemed to have brought me on the correct way.
By going deeper into everything, the standard papers as well as the plugins parameter documentations, i was able to get at least Scene referred Conversion working 99.99% as exepcted.
I am not yet totally happy with the Display referred (it's +-10 which makes me doubt that the approach is correct yet or maybe dithering is bad?) result and also i am not at all able to do the transfer function from hlg to linear using z_convertformat (the result is TOTALLY off with z_conv), i would have expected that fmtc and zimg must do exactly the same when going from hlg to linear. But i'll probably open a separate topic for that. For now i am very happy that at least i got the Scene referred approach working.

Both methods share the startup/preparation and also both methods return 10-bit RGB which makes it easy for me to read the values in AVSPmod and compare them to the standard pdf.

Some explainations and further questions:
-) fmtc docs mention that once we are in rgb, we should work in full and only go back to limited for the final yuv conversion
-) in scene referred, it does not influence the final measured values if i do the final matrix conversion (rgb to yuv) which i don't really understand. (fmtc_matrix ("709"...)
* fmtc cannot know that i already applied the 709 conversion with my custom coef function before so why does it not change values when i execute fmtc_matrix ("709",fulls=true,fulld=false) as final step?
-) display referred is actually straight forward but we use 1886 transfer curve to come from linear to CRT display light finally. (as written in BT.2087 documents the functions they used in BT.2111 for reference conversion on last page)

Shared Code:

loadplugin("C:\dev\FFAStrans\Processors\avs_plugins\ffms2\x64\ffms2.dll")
loadplugin("C:\AvsPmod\plugins\fmtconv.dll")
loadplugin("C:\AvsPmod\plugins\avsresize\x64\Release\avsresize.dll")
FFVideoSource("c:\temp\2111.nut")

# Source yuv to rgb
ConvertToYuv444()
fmtc_matrix ("2020",bits=16,fulls=false,fulld=false) #yuv to rgb

Scene referred (nearly perfect result):

##SCENE REFERRED  expected output values, blue bottom right:#42 40 3ac
fmtc_transfer (transs="hlg",transd="linear",sceneref=true,fulls=false,fulld=true)
fmtc_matrix (coef="1.6605 -0.5876 -0.0728 0 -0.1246 1.1329 -0.0083 0 -0.0182 -0.1006 1.1187 0",fulls=true,fulld=true) # matrix from  BT.2407 2.2
#format is now rgbp16
### fmtc_matrix ("709",fulls=true,fulld=false) #this step does not influence the final values, so we can skip it for now
z_ConvertFormat(pixel_type="rgbp10",colorspace_op="rgb:709:709:full=>rgb:same:same:limited") #full-limited and to 10 bit
## returned actual values: # 41 40 3a9 (expected:42 40 3ac)

Display referred (a little bit off):

##DISPLAY REFERRED  expected output values, blue bottom right: 93 64 768  (0x5d, 0x40, 0x300)
fmtc_transfer (transs="hlg",transd="linear",sceneref=false,fulls=false,fulld=true)
fmtc_matrix (coef="1.6605 -0.5876 -0.0728 0 -0.1246 1.1329 -0.0083 0 -0.0182 -0.1006 1.1187 0",fulls=true,fulld=true) # matrix from BT.2407 2.2
fmtc_transfer(transs="linear",transd="1886",sceneref=false,fulls=true,fulld=true)
z_ConvertFormat(pixel_type="rgbp10",colorspace_op="rgb:709:709:full=>rgb:same:same:limited")  #full-limited and to 10 bit
# returned values: 64 49 2FE (expected: 5d 40 300)

[DTL]

"##SCENE REFERRED expected output values, blue bottom right:#42 40 3ac
fmtc_transfer (transs="hlg",transd="linear",sceneref=true,fulls=false,fulld=true)
fmtc_matrix (coef="1.6605 -0.5876 -0.0728 0 -0.1246 1.1329 -0.0083 0 -0.0182 -0.1006 1.1187 0",fulls=true,fulld=true) # matrix from BT.2407 2.2
#format is now rgbp16
### fmtc_matrix ("709",fulls=true,fulld=false) #this step does not influence the final values, so we can skip it for now
z_ConvertFormat(pixel_type="rgbp10",colorspace_op="rgb:709:709:full=>rgb:same:same:limited") #full-limited and to 10 bit"

This not looks completely nice - first you go to linear RGB, next apply RGB matrix to convert rec.2020 to rec.709 colour gamut(?) and next you need to convert RGB linear to rec.709 transfer to got R'G'B' standard data in transfer-domain as FIGURE 1 of BT.2407 also shows at output (also it is better to do in 16bit or float format for better precision).

But with z_ConvertFormat you assume transfer already in 709 - Format is "matS[ :transS[: primS[: rangeS]]]=>matD[: transD[: primD[: rangeD]]]" .
May be better is
z_ConvertFormat(pixel_type="rgbp10",colorspace_op="rgb:linear:709:full=>rgb:709:709:limited") #full-limited, transfer from linear to rec.709 and to 10 bit

Also as Figure 10 in 2111.pdf shows - it may be required additional 'gain' control after HLG to linear RGB and before applying RGB matrix from 2020 to 709. This gain correction may add more distortions in 16bit linear - so may be better to do all scientific stuff first in best precision float32 format and only after getting good results try to check how 16bit immediates are good to hangle all operations chain (the 16bit may be good enough for SDR linear but may be not great for HLG linear and may be completely low dynamic range for PQ linear). Also with float32 samples you may have less jumps between narrow/full domains in 16bit integer and it also accumulate compute errors. Though float32 in AVS plugins additional separate filed of possible errors - it is typically not simply convert integer domain to float but convert integer domain into something about 0.0 to 1.0f range (with zero at 0.0 or shifted and nominal/max white may be around 1.0 and also linear HDR may be completely out of 0..1 range in max values). So when you use different plugins for data domain handling it may require additional research about proper data range mapping between. So users typically happy when it is possible to do _all_ required conversions in single step in single plugin or at least with plugins from single author. Mixing plugins of different authors may be additional nightmare to connect and setup in proper way. There is no any 'strict internal standards' in AVS environment for data range mapping.

So when you design some chain of plugins it is better to check each plugin output for correct (or required by next plugin) data range. Because plugin may do correct computing in conversion but output data in not good range mapping for next plugin so you will quickly accumulate lots of errors in a long chain. So may be you also need to put RGBAdjust() somewhere to proper match range mapping in your HDR to SDR conversion.

[emcodem]

Thanks for the suggestions DTL, of course i am very aware about each and every step's inputs and outputs. Basically i stay in fmtc functions and only use z_Convertformat normally for the final YUV to RGB10 conversion for measuring. OF course i also compared the final conversion to fmtc methods and they are the same. So we don't really need to care about the last line stuff that is only there to convert the final values in a way they can be displayed and read nicely in AVSPmod. I take very good care that this final step does not influence the actual conversion result.

About the "gain" they mention in Bt.2111, this is one of the most confusing parts for me because the didn'd mention more than "the gain was set so that 75% HLG corresponds to 100% 709". I thought that fmtc applies exactly this gain automatically in the transfer step but i just checked the docs and source code and it looks like by default no gain is applied, so i really wonder why it works for me even without taking care about the gain.
About rgbadjust, i tend to use only and exactly what the documents say that is used for now - and i believe i am really really close now with fmtc functions.

You mention doing the calculations in excel, do you mean to write an excel calculation for eotf, ootf and matrix conversions? - unfortunately this would take a long time for me because i'm not good at math. I can barely read and understand the formulas in the documentations and codes. Would you maybe be able to share something you were using last time?


OK so i simplified further and generated a more clean source (the original colorbar i worked with was a recording, some colors were of by default).
My new colorbar source was generated with vapoursynth like that:

import vapoursynth as vs
core= vs.core
# Generate HLG UHD Bars
c = core.colorbars.ColorBars(format=vs.RGB30, resolution=5, hdr=1)
c.set_output()

Which gave me a RGBP10 colorbar, so we don't even need the first conversion step anymore.
I think we are now even closer to the BT2111 document because they don't mention matrix conversion at all.

New script:

fmtc_transfer (transs="hlg",transd="linear",sceneref=true,fulls=false,fulld=true)
fmtc_matrix (coef="1.6605 -0.5876 -0.0728 0 -0.1246 1.1329 -0.0083 0 -0.0182 -0.1006 1.1187 0",fulls=true,fulld=true) # matrix from  BT.2407 2.2
fmtc_matrix ("709",fulls=true,fulld=false) 
z_ConvertFormat(pixel_type="rgbp10",colorspace_op="709:709:709:limited=>rgb:same:same:limited") #full-limited and to 10 bit

Conclusion: we are now even more close (blue is now only 1 value in a single component off, before it was off in multiple components and up to value of 7). This is only due to the "perfectness" of the original colorbar.

New Script also uses fmtc to do the final matrix conversion but again, after applying the first matrix, it does not matter if after that one i do RGB-YUV conversion withg 709 matrix (and then back to rgb10 for result measuring in AVSPmod), the final values are the exact same as when not doing this step. I guess BT.709 compatible values are not changed by applying a BT.709 matrix again?

Unfortunately i was not anyhow able to make fmtc work with more than it automatically chooses to work with: By default it works with 16bit int rgb. As soon as i convert to 32 bit int or even float, everything is totally off. I believe i would need to change the BT.2407 matrix coeffs to match the new number formats maybe?

[DTL]

"You mention doing the calculations in excel, do you mean to write an excel calculation for eotf, ootf and matrix conversions? - unfortunately this would take a long time for me because i'm not good at math."

Yes - it is typically not hard to write all computing in e-table software like MS Excel (or freeware OpenOffice Calc). I typically do it to check my software at colour-bars but I do not have example with matrix-based RGB transform (it is not hard - just lots of mul + add in 1 line).

"My new colorbar source was generated with vapoursynth like that:"

The task for computing in RGB is simple - the 75% HLG ColourBars in 10bit is 721/64 (high/low) RGB triplets so it can be done with AVS ColorBarsHD + RGBAdjust (simply assume it as HDR HLG encoded RGB in its natural transfer domain) . If YUV dataset is required (for checking of code values after ConvertToYUV444()) - the EBU HDR colour bars have YUV (Y'CbCr) table for 10bit HLG - https://tech.ebu.ch/docs/tech/tech3373.pdf Table 3 R'G'B'->Y'CbrCr .

R'G'B' in 10bit 721/64 colour bars may be in such hacky way:

ColorBarsHD(640, 480, pixel_type="YUV444P10")
ConvertToPlanarRGB(matrix="Rec709")
RGBAdjust(r=2,g=2,b=2,rb=-100,gb=-100,bb=-100)
RGBAdjust(r=0.64223, g=0.64223, b=0.64223,rb=64, gb=64, bb=64)
Crop(0,0,640,270)

It looks internal colourbars +RGB decode calc produce some errors so it is better to clip RGB to max/zero and set required RGB triplets with RGBAdjust gain/bias.

For getting YUV equal to EBU Tech 3373 Table 3 calc it looks not very simple - the
ConvertToYUV444(matrix="PC.2020")
Create not equal 10bit values with that table - Yellow is 682,184,538 (EBU is 682, 176, 539). May be need to try fmtc or avsresize.

Looks working good avsresize:
z_ConvertFormat(pixel_type="YUV444P10",colorspace_op="rgb:std-b67:2020:l=>2020:std-b67:2020:l")

So for YUV444P10 HLG 75% colour bars narrow (perfectly matching EBU Tech 3373 Y'CbCr 10bit values Table 3) it is generator (using avsresize_r21 as R'G'B' to Y'CbCr convertion engine):

LoadPlugin("avsresize.dll")
ColorBarsHD(640, 480, pixel_type="YUV444P10")
ConvertToPlanarRGB(matrix="Rec709") // simply some abstract colour bars patch source
RGBAdjust(r=2,g=2,b=2,rb=-100,gb=-100,bb=-100) // clamp badly computed RGBs to 1023 and 0
RGBAdjust(r=0.64223, g=0.64223, b=0.64223,rb=64, gb=64, bb=64) // precisely craft 721 and 64 high/low code values for our lovely Colour Bars pattern
Crop(0,0,640,270) // assume it R'G'B' in HLG transfer and 2020 colour space with 10 bit code values (only levels check - transients are NOT any conditioned)
z_ConvertFormat(pixel_type="YUV444P10",colorspace_op="rgb:std-b67:2020:l=>2020:std-b67:2020:l")

And about AVS internal ConvertToYUV(matrix="PC.2020") it may be need to again open issue about significant errors in 10bit or require additional Levels() hacking to generate close to avsresize values ?

Also I found AVSpmod can display YUV decimal directly in the status bar - add %YUV at the end of status message config in settings. May be ask developer to auto-switch display code values depending on current script output colour space (RGB or YUV) ?

[emcodem]

@DTL thank you so much for your kind and generous help. I spent the whole day again with the desired conversion i am still learning so much that i did not yet set up an excel sheet for manually testing the single conversion steps.

But today at least i managed to get 100% matching values to the described colors after scene-referred conversion from bt.2111.pdf (last page, scene referred)
In my previous conversions above, the gray was totally off, it was nearly black. The missing part was something that @DTL mentioned above already, i missed the linear to 709 OETF conversion which they mention in BT.2111.pdf Figure 12

Here the generated perfect BT.2111 colorbar in RGBP10 raw format:
https://1drv.ms/u/s!AkS-A9Jqq09FhEPc...2dmtz?e=gVjfdc

Here is the current script with 100% color match against the 2111.pdf

FFVideoSource("c:\temp\rgb2111.nut") #rgb2111.nut is generated bgrp10le colorbar hlg,2020,2020

##SCENE REFERRED 

fmtc_transfer (transs="hlg",transd="linear",sceneref=true,fulls=false,fulld=true,debug=1) #also applies limited-full
#we are now RGB16bit full and stay like that until done

fmtc_matrix (coef="1.6605 -0.5876 -0.0728 0 -0.1246 1.1329 -0.0083 0 -0.0182 -0.1006 1.1187 0",fulls=true,fulld=true) # matrix from  BT.2407 2.2
fmtc_transfer (transs="linear",transd="709",sceneref=true,fulls=true,fulld=false,debug=2) #also applies full-limited

#change bits for measuring in avspmod and comparing to bt2111 table but stay in rgb
fmtc_bitdepth(bits=10,dmode=1,fulls=false,fulld=false) #fmtc bitdepth gives same result as z_convertFormat
#z_ConvertFormat(pixel_type="rgbp10",colorspace_op="rgb:709:709:limited=>rgb:same:same:limited") #full-limited and to 10 bit

sidenote, "fmtc_primaries (prims="2020",primd="709")" would come to a very similar but not exactly the same result as the custom matrix above

#####
# Display referred
#####
z_ConvertFormat(pixel_type="RGBPS",colorspace_op="RGB:std-b67:2020:limited=>rgb:same:same:limited") #to get full precision, we convert to float before we start calculations

fmtc_transfer (transs="hlg",transd="linear",sceneref=false,fulls=false,fulld=true)
fmtc_matrix (coef="1.6605 -0.5876 -0.0728 0 -0.1246 1.1329 -0.0083 0 -0.0182 -0.1006 1.1187 0",fulls=true,fulld=true) # matrix from BT.2407 2.2
#Last transform step also returns int instead of float
fmtc_transfer(transs="linear",transd="1886",sceneref=false,fulls=true,fulld=false,flt=false)

z_ConvertFormat(pixel_type="rgbp10",colorspace_op="rgb:709:709:limited=>rgb:same:same:limited")  #full-limited and to 10 bit

####

This is fantastic work from the creators of fmtc, i would not have thought that it is at all possible to get to 100% matching results compared to bt.2111 ! Really unbelieveable that we seem to be able to do exactly what they do in the standard pdf in the exact same way, coming to the exact same values!

Yet i have a problem with the gray bars left and right (40% HLG).
The thing is, the bt.2111.pdf does not describe the target values after simple BT2407 matrix conversion to 709. So i can only "visually" compare them to the bt2111.pdf which i don't like.
Any clues how i could find out the target values for 40% gray HLG mapped to 709 (when 75% HLG is 100% 709)?

[emcodem]

Originally Posted by DTL

And about AVS internal ConvertToYUV(matrix="PC.2020") it may be need to again open issue about significant errors in 10bit or require additional Levels() hacking to generate close to avsresize values ?

I'll try to verify in the next days. I still need to get the display referred stuff right and also i want to try if i can do the same perfect conversion using different plugine as well. Last but not least if i ever want to load yuv sources, i'll need to take this matrix convertion into account too.

Originally Posted by DTL

Also I found AVSpmod can display YUV decimal directly in the status bar - add %YUV at the end of status message config in settings. May be ask developer to auto-switch display code values depending on current script output colour space (RGB or YUV) ?

Thank you, i didn't yet find this (but also not look for it ). It could be nice to have an %INT feature as company for the existing %hex feature. Honestly i would not expect to see an actual conversion feature in the player (the current %YUV actually converts always to 8bit yuv but i like raw values instead of calculated). Anyway, it is a veeery long time ago since i was actually contributing to this forum, i don't know all the people so maybe i am not in a position to raise feature requests

[DTL]

"how i could find out the target values for 40% gray HLG mapped to 709 (when 75% HLG is 100% 709)?"

Make calculations in math-modeling software (Excel/Calc/Mathcad/...) or on paper and windows/OS/standalone calculator scientific. If your colour values are perfect it generally mean the Y-only will also be good.

"(the current %YUV actually converts always to 8bit yuv but i like raw values instead of calculated)"

It may be display in natural bitdepth only YUV colour space values and make calc in 8bit for RGB (current output by script). For raw RGB in decimal may be you need other setting - try to see all possible in status line configure dialog.

[tormento]

Perhaps I am missing the whole point of the discussion but why not to use DGCube or similar plugins?

[emcodem]

Originally Posted by tormento

Perhaps I am missing the whole point of the discussion but why not to use DGCube or similar plugins?

Well, if i had a cube file that leads to the same 100% match result, i would have had an easier time (or you think i can use DGCube somehow to apply above custom matrix?)

I guess the point is mostly the same as with other colorbar engineering: learning, research and building a base that allows validation of different methods and equipment... E.g. now that i have 100% match with documented values in HDR->SDR conversion, i can use it to validate the possibility of mathematically correct, lossless (after initial loss), roundtrip. This of course leads to more and more understanding and the possibility to test/validate more different transform mechanisms.

Example: Simple Roundtrip as desribed in BT.2407, using upconvert from BT.2087 and downconvert with above described §2 method from BT.2407:

loadplugin("C:\dev\FFAStrans\Processors\avs_plugins\ffms2\x64\ffms2.dll")
loadplugin("C:\AvsPmod\plugins\fmtconv.dll")
loadplugin("C:\AvsPmod\plugins\avsresize\x64\Release\avsresize.dll")

FFVideoSource("C:\temp\vistek.mxf") #vistek is a bt.709 colorbar
#ColorBarsHD(pixel_type = "YV24") # simulated 709 yuv 420 colorbar 

#uncomment to check initial values in rgb 10 bit format
#z_ConvertFormat(pixel_type="rgbp8",colorspace_op="709:709:709:limited=>rgb:same:same:limited")  #full-limited and to 10 bit
#return last

# convert matrix from yuv422 to RGB Float 32  and limited-full
z_ConvertFormat(pixel_type="RGBPS",colorspace_op="709:709:709:limited=>rgb:same:same:full")  

# execute the 709-HLG-709 roundtrip (duplicate the lines to simulate mutliple rounds)
c = last
c= BT2087_SDR_TO_HDR_SceneRef(c)
c = BT2407_HDR_TO_SDR_Sceneref(c)

#Finally convert Full-Limited and from RGB Float32 to output format, in this case we use rgb10 for measuring values in avspmod. 
c = z_ConvertFormat(c, pixel_type="rgbp10",colorspace_op="rgb:709:709:full=>rgb:same:same:limited")  #from rgb float32 to rgb10, full-limited
#Display the Result
return c

#rbg to linear rgb, input is assumed to be RGBP32 and full. 
function BT2087_SDR_TO_HDR_SceneRef(clip c){
    # Case #2 on Page3 Bt.2087 ( the goal is to match the colours of a direct Rec. 2020 camera output)
    c = fmtc_transfer (c, transs="709",transd="linear",sceneref=true,fulls=true,fulld=true,flt=true)
    # M2 on Page 4 Bt.2087 
    c = fmtc_matrix (c, coef="0.6274 0.3293 0.0433 0 0.0691 0.9195 0.0114 0 0.0164 0.0880 0.8956 0",fulls=true,fulld=true) 
    # Case #2 on Page4 Bt.2087  
    c = fmtc_transfer(c, transs="linear",transd="2020",sceneref=true,fulls=true,fulld=true,flt=true)
    #output is 2020 full RGB float32
    return c
}

#Linear RGB HLG to 709, "Simple linear matrix conversion" of §2 in Bt.2407
function BT2407_HDR_TO_SDR_Sceneref(clip c){
    #Bt.2407 §2.1 Non-linear to linear conversion (N to L)
    c = fmtc_transfer (c, transs="2020",transd="linear",sceneref=true,fulls=true,fulld=true,flt=true)
    #Bt.2407 §2.2 Matrix (M)
    c = fmtc_matrix (c, coef="1.6605 -0.5876 -0.0728 0 -0.1246 1.1329 -0.0083 0 -0.0182 -0.1006 1.1187 0",fulls=true,fulld=true)
    #Bt.2407 §2.3 Linear to non-linear conversion (L to N)
    c = fmtc_transfer(c, transs="linear",transd="709",sceneref=true,fulls=true,fulld=true,flt=true)
    #output is 709 full RGB float32
    return c
}

[tormento]

Originally Posted by emcodem

Well, if i had a cube file that leads to the same 100% match result

The ones from FranceBB are really good.

[emcodem]

Yeah he constantly tells me so (we are both in the ffastrans team)
Anyway, what i do here does not require anything good or similar but instead it requires something that does exactly what's described in the standard (artistical vs technical conversion). But no worries, from my perspective the mission is finished!

[emcodem]

Originally Posted by DTL

And about AVS internal ConvertToYUV(matrix="PC.2020") it may be need to again open issue about significant errors in 10bit or require additional Levels() hacking to generate close to avsresize values ?

Dear DTL,
i struggle a little when trying to reproduce problems with internal converttoyuv function. Example

colorbarshd(pixel_type="yuv444p10")
z_ConvertFormat( pixel_type="rgbp10",colorspace_op="709:709:709:limited=>rgb:same:same:full")  
z_ConvertFormat( pixel_type="yuv444p10",colorspace_op="rgb:709:709:full=>709:same:same:limited")

Original yellow: 2a0 b0 220, returned yellow: 2a0 b1 220

colorbarshd(pixel_type="yuv444p10")
z_ConvertFormat( pixel_type="rgbp10",colorspace_op="709:709:709:limited=>rgb:same:same:full")  
converttoyuv444(matrix="rec709")

Above gives the same result as the first z_convertformat function, yellow: 2a0 b1 220

But i was not able to get anywhere close to the original value when using limited rgb range.
Example:

colorbarshd(pixel_type="yuv444p10")
z_ConvertFormat( pixel_type="rgbp10",colorspace_op="709:709:709:limited=>rgb:same:same:limited")  
z_ConvertFormat( pixel_type="yuv444p10",colorspace_op="rgb:709:709:limited=>709:same:same:limited")

This works as expected, it returns the exact original value from the colorbarshd(): yellow: 2a0 b0 220

But there is no way to get anywhere close to the expected values when feeding limited RGB to converttoyuvXXX() function. As far as i see we can only use following matrices:
709:l or rec709 -> 27f e0 21b
709:f or pc.709 -> 2a0 b7 21f

I guess that the function always expects RGB full as we cannot specify the input range but only the output range.
It is a little confusing that pc.709 is documented as "keep range unchanged".

[poisondeathray]

Originally Posted by emcodem

I guess that the function always expects RGB full as we cannot specify the input range but only the output range.
It is a little confusing that pc.709 is documented as "keep range unchanged".

This was discussed in the avs thread - one way for studio rgb (limited range rgb) using internal avs functions is with the levels workaround

colorbarshd(pixel_type="yuv444p10")
levels(0,1,1023,64,940,false).converttoplanarrgb(matrix="rec709") #limited range yuv to studio rgb
levels(64,1,940,0,1023,false).converttoyuv444(matrix="rec709") #studio rgb to limited range yuv, roundtrip

The "range unchanged" for pc matrix possibly refers to the is a 1:1 range mapping of Y,R,G,B . It's linear , with no gaps or duplicates. See the vegas studio RGB example, or the avs levels workaround in the graphic in this link to the post - Their slope is 1, but there are discontinunities with gaps or repeated values. The PC matrix has perfect range mapping, but colors are off

http://forum.doom9.org/showthread.ph...92#post1987992

The linear has use cases - such as full range YUV video, masks/compositing . Studio RGB would be less accurate in those cases

[DTL]

"But there is no way to get anywhere close to the expected values when feeding limited RGB to converttoyuvXXX() function. "

Old AVS core (up to 3.7.3 test11) Convert() may be skipped now. Pinterf currently busy with sort of complete redesign of Convert YUV<->RGB (and some other) functions. Need to wait may be several days for next commit and builds. Hope after redesign we will have significantly faster and more precise processing with both narrow/limited and full data levels mapping. Currently it looks only external plugins may be used as 'reference' - avsresize and fmtc.

It looks in old AVS core design at some conversions data paths the equalizing of Y and UV ranges was missed so computing give more errors in compare with quantization noise only. As described in all 601/709/2020 standards the Digital YUV system have a bit different Y and UV channels range mapping (scale) in quantized form. The scale factor of Digital Y is 219 and scale factor of Digital UV is 224 before quantization to integer form. The ratio is not large - only 224/219 ~= 1.02283 but if it is missed from calculations it make additional damage to precision.

It looks like one more 'kiss from the grave' of the poor digital past when the ugly chroma-subsampled compression was designed and required YUV data matrix. In RGB systems typically all 3 channels have equal range and less possible sources of such errors.

So as it looks 224/219 additional UV scale correction is missed in old AVS cores Convert() with PC.x matrices it can be fixed with additional scripting:

colorbarshd(pixel_type="yuv444p10")
z_ConvertFormat( pixel_type="rgbp10",colorspace_op="709:709:709:limited=>rgb:same:same:limited")  
ConvertToYUV444(matrix="PC.709")
ColorYUV(cont_u=6, cont_v=6)

Now Yellow bar YUV is 2A0, AF, 220.

From http://avisynth.nl/index.php/ColorYUV
If cont_y = k, then Y becomes (Y-128) · k / 256 + 128 – for example:
6/256 (+1 ?) ~ 0.023 (or 1.023 ?) and about close to 224/219 ~ 1.0228

Before convert YUV to RGB narrow the
ColorYUV(cont_u=-6, cont_v=-6)
may be used.