Log in

View Full Version : Plugin HDRTools for Avisynth


Pages : [1] 2 3 4 5

jpsdr
30th May 2018, 11:11
This is a plugin with functions made to work on HDR videos, but can also eventualy work to do some color conversions.

Current version: 1.2.3

Sources are here (https://github.com/jpsdr/HDRTools).
Binaries are here (https://github.com/jpsdr/HDRTools/releases/download/1.2.3/HDRTools_v1_2_3.7z).

Change list:
29/11/2025 v1.2.3 : Minor changes (more code refactory).
19/07/2025 v1.2.0 : Add more AVX512 code path.
19/07/2025 v1.1.0 : Add AVX512 code path.
19/11/2023 v1.0.5 : Update to new AVS+ headers.
25/02/2023 v1.0.4 : Update to new AVS+ headers.
21/11/2022 v1.0.3 : Fix the value of a BT.1886 parameter (2.404 -> 2.40).
20/11/2022 v1.0.2 : Update on threadpool, no user limit (except memory).
21/06/2022 v1.0.1 : Some fixes and add Exposure parameter to ACES tonemap.
15/06/2022 v1.0.0 : Add ACES tonemap.
22/02/2022 v0.6.4 : Fix on threadpool, using prefetch parameter created hang. Add negative prefetch for triming, read Multithreading.txt.
20/07/2021 v0.6.3 : Fix on threadpool.
30/04/2021 v0.6.2 : Update to new avisynth headers.
28/04/2020 v0.6.1 : Update to new avisynth headers.
14/12/2019 v0.6.0 : Add BT2446 A & C methods for HDR to SDR convert.
15/09/2019 v0.5.3 : Add Crosstalk parameter, add EOTF path for SDR, fix in 16 bits lookup XYZ.
06/06/2019 v0.5.2 : Minor code change after threadpool update, fix in the number of threads.
30/05/2019 v0.5.0 : Update of the threadpool, add ThreadLevel parameter.
14/08/2018 v0.4.2 : Update ReadMe, minor changes.
10/08/2018 v0.4.1 : Minor changes.
09/08/2018 v0.4.0 : Some fix on HLG, rename and change of HDRtoSDR/SDRtoHDR functions, add Hable, Mobius and Reinhard tonemap (formula from Vapoursynth). Reinhard uses the "exposure" parameter.
03/08/2018 v0.3.1 : Fix some crash issues.
31/07/2018 v0.3.0 : Add HLG support, update PDF document.
08/06/2018 v0.2.0 : Add support of float planar RGB input/ouput on ConvertXYZtoRGB and ConvertRGBtoXYZ. Add document.
30/05/2018 v0.1.0 : Initial release.

Note about HDR HLG:
For speed-up:
8 bits input use 3D lookup (startup is slow).
10 and 12 bits use 2D lookup (startup is slow), but there is a resolution loss noticeable effect on low values.

Note about YV12/YV16/YV24 input/ouput:
For ConvertYUVtoxxx functions, convertion to YV24 if input is not YV24 is "quick".
If you want a true correct precise/perfect convertion, i would avise you to use a resampler and feed directly to YV24.
For ConvertxxxtoYUV functions, if you choose an ouput different from YV24, same remark, convertion is "quick".
If you want a true correct precise/perfect convertion, i would avise you to ouput to YV24, and use a resampler to achieve the real output format you want.

The functions avaible in this plugin are:

ConvertYUVtoLinearRGB(int Color,int OutputMode,int HDRMode,float HLGLb,float HLGLw,int HLGColor,bool OOTF,bool EOTF,bool fullrange,bool mpeg2c,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertLinearRGBtoYUV(int Color,int OutputMode,int HDRMode,float HLGLb,float HLGLw,int HLGColor,bool OOTF,bool EOTF,bool fullrange,bool mpeg2c,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertYUVtoXYZ(int Color,int OutputMode,int HDRMode,float HLGLb,float HLGLw,float Crosstalk,int HLGColor,bool OOTF,bool EOTF,bool fullrange,bool mpeg2c,float Rx,float Ry,float Gx,float Gy,float Bx,float By,float Wx,float Wy,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertXYZtoYUV(int Color,int OutputMode,int HDRMode,float HLGLb, float HLGLw,float Crosstalk,int HLGColor,bool OOTF,bool EOTF,bool fullrange,bool mpeg2c,bool fastmode,float Rx,float Ry,float Gx,float Gy,float Bx,float By,float Wx,float Wy,int pColor,float pRx,float pRy,float pGx,float pGy,float pBx,float pBy,float pWx,float pWy,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertRGBtoXYZ(int Color,int OutputMode,int HDRMode,float HLGLb,float HLGLw,float Crosstalk,int HLGColor,bool OOTF,bool EOTF,bool fastmode,float Rx,float Ry,float Gx,float Gy,float Bx,float By,float Wx,float Wy,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertXYZtoRGB(int Color,int OutputMode,int HDRMode,float HLGLb,float HLGLw,float Crosstalk,int HLGColor,bool OOTF,bool EOTF,bool fastmode,float Rx,float Ry,float Gx,float Gy,float Bx,float By,float Wx,float Wy,int pColor,float pRx,float pRy,float pGx,float pGy,float pBx,float pBy,float pWx,float pWy,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertXYZ_Scale_HDRtoSDR(float Coeff_X,float Coeff_Y,float Coeff_Z,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertXYZ_Scale_SDRtoHDR(float Coeff_X,float Coeff_Y,float Coeff_Z,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertXYZ_Hable_HDRtoSDR(float exposure_X,float whitescale_X,float a_X,float b_X,float c_X,float d_X,float e_X,float f_X,float exposure_Y,float whitescale_Y,float a_Y,float b_Y,float c_Y,float d_Y,float e_Y,float f_Y,float exposure_Z,float whitescale_Z,float a_Z,float b_Z,float c_Z,float d_Z,float e_Z,float f_Z,int pColor,float pRx,float pRy,float pGx,float pGy,float pBx,float pBy,float pWx,float pWy,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertRGB_Hable_HDRtoSDR(float exposure_R,float whitescale_R,float a_R,float b_R,float c_R,float d_R,float e_R,float f_R,float exposure_G,float whitescale_G,float a_G,float b_G,float c_G,float d_G,float e_G,float f_G,float exposure_B,float whitescale_B,float a_B,float b_B,float c_B,float d_B,float e_B,float f_B,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertXYZ_Mobius_HDRtoSDR(float exposure_X,float transition_X,float peak_X,float exposure_Y,float transition_Y,float peak_Y,float exposure_Z,float transition_Z,float peak_Z,int pColor,float pRx,float pRy,float pGx,float pGy,float pBx,float pBy,float pWx,float pWy,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertRGB_Mobius_HDRtoSDR(float exposure_R,float transition_R,float peak_R,float exposure_G,float transition_G,float peak_G,float exposure_B,float transition_B,float peak_B,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertXYZ_Reinhard_HDRtoSDR(float exposure_X,float contrast_X,float peak_X,float exposure_Y,float contrast_Y,float peak_Y,float exposure_Z,float contrast_Z,float peak_Z,int pColor,float pRx,float pRy,float pGx,float pGy,float pBx,float pBy,float pWx,float pWy,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertRGB_Reinhard_HDRtoSDR(float exposure_R,float contrast_R,float peak_R,float exposure_G,float contrast_G,float peak_G,float exposure_B,float contrast_B,float peak_B,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertLinearRGBtoYUV_BT2446_A_HDRtoSDR(float Lhdr,float Lsdr,float CoeffAdj,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConverXYZ_BT2446_C_HDRtoSDR(bool ChromaC,bool PQMode,float Lhdr,float Lsdr,float pct_ref,float pct_ip,float pct_wp,float pct_sdr_skin,float pct_hdr_skin,float WhiteShift,int pColor,float pRx,float pRy,float pGx,float pGy,float pBx,float pBy,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

ConvertXYZ_ACES_HDRtoSDR(float a_X,float b_X,float c_X,float d_X,float e_X,float a_Y,float b_Y,float c_Y,float d_Y,float e_Y,float a_Z,float b_Z,float c_Z,float d_Z,float e_Z,float exposure_X,float exposure_Y,float exposure_Z,int pColor,float pRx,float pRy,float pGx,float pGy,float pBx,float pBy,float pWx,float pWy,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)
ConvertRGB_ACES_HDRtoSDR(float a_R,float b_R,float c_R,float d_R,float e_R,float a_G,float b_G,float c_G,float d_G,float e_G,float a_B,float b_B,float c_B,float d_B,float e_B,float exposure_R,float exposure_G,float exposure_B,bool fastmode,int threads,bool logicalCores,bool MaxPhysCore,bool SetAffinity,bool sleep,int prefetch,int ThreadLevel)

Brief description of most important parameters:
Color: Set the colorspace mode of the data.
OutputMode: Set the output data mode.
HDRMode: Set the HDR mode (PQ or HLG).
OOTF: Allow the output of linear displayed data instead of linear scene data.

See the ReadMe file for more informations and descriptions of the functions and their parameters.

Note about linear values:
For ConvertYUVtoxxx functions, the output is :
For HDR data (Color=0), HDRMode=(0 or 1), the output (R,G,B stage) is normalized for 1.0=10000 cd/mē (or 255 or 65535).
For SDR data (Color<>0),the output (R,G,B stage) is normalized for 1.0=100 cd/mē (or 255 or 65535).
For ConvertxxxtoYUV functions, the input is :
For HDR data (Color=0), HDRMode=(0 or 1),the input (R,G,B stage) is normalized for 1.0=10000 cd/mē (or 255 or 65535).
For SDR data (Color<>0),the input (R,G,B stage) is normalized for 1.0=100 cd/mē (or 255 or 65535).

Note: ConvertXYZ_Scale_HDRtoSDR is a simple scalar function, i've made just to see what result it produces. Don't have expectations on this plugin, but if by luck it works for you...

In the Document directory i've put a pdf file describing how things work, with detailled explainations and formula.

Some exemples:
BT.2020 to BT.709 convertion, do the following:

ConvertYUVtoXYZ(Color=1)
ConvertXYZtoYUV(pColor=1)


BT.709 to BT.2020 convertion, do the following:

ConvertYUVtoXYZ()
ConvertXYZtoYUV(Color=1,pColor=2)


HDR HLG (with mastering Lw at 1500 and HLG mastering display colorspace BT.2020) to HDR PQ convertion:

ConvertYUVtoLinearRGB(Color=0,HDRMode=1,HLGLw=1500,OOTF=false)
ConvertLinearRGBtoYUV(Color=0,OOTF=false)


HDR PQ to HDR HLG (with mastering Lw at 1200 and HLG mastering display colorspace BT.709) convertion:

ConvertYUVtoLinearRGB(Color=0,OOTF=false)
ConvertLinearRGBtoYUV(Color=0,HDRMode=1,HLGLw=1200,HLGColor=2,OOTF=false)

jpsdr
30th May 2018, 11:17
Some pre-release, for people to try and make feedback.
Don't get hope yet, these are tools for me for working on a real HDR to SDR. I have something i want to try. It will take a long time, and i don't even know if it will work, but i'm curious to try.
For now, the HDRtoSDR is not working, it's a simple linear scalling i just made for simple testing. But all the others functions are supposed to work (unless bug found).

I'll probably add a little later the simple standard tonemap commonly used.

Feel free to comment, feedback, and correct me if i have misunderstood how things work.

jpsdr
4th June 2018, 10:07
Note on use : If your chromaticity parameters are the same of the BT.xxxxx defined by the Color parameter, there is no need to specify them.
I'm working on a pdf document, which will explain and detail all of the formulas used, and the process and how i'm doing things, i hope i'll be able to put it with next release, but will not be before a week or two.
You'll be welcome at the time to read it, and most of it, correct me if there was things i've misunderstood.
Stay tuned... ;)

Otherwise, about perfomance, as i'm purely soft, even multi-threaded and vectorized ASM, DGHDRtoSDR will always beats this plugin, a lot !!! :D
Also, i didn't thought the simple linear scaling would realy produce interesting result, as i just made quick it in the meantime, focussing for now on my real needs, the YUV/XYZ/YUV converters.

jpsdr
4th June 2018, 17:57
Some quick benchmark on my system :

Coeff=50.0
Colorbars(width=1920,height=804,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,999)
ConvertYUVtoXYZ(Color=0,OutputMode=1,Rx=0.680,Ry=0.320,Gx=0.265,Gy=0.690,Bx=0.150,By=0.060)
ConvertXYZ_HDRtoSDR(Coeff_X=Coeff,Coeff_Y=Coeff,Coeff_Z=Coeff)
ConvertXYZtoYUV()

288.9fps


Coeff=50.0
Colorbars(width=3840,height=1608,pixel_type="yv12").killaudio().assumefps(25,1).trim(0,999)
ConvertYUVtoXYZ(Color=0,OutputMode=1,Rx=0.680,Ry=0.320,Gx=0.265,Gy=0.690,Bx=0.150,By=0.060)
ConvertXYZ_HDRtoSDR(Coeff_X=Coeff,Coeff_Y=Coeff,Coeff_Z=Coeff)
ConvertXYZtoYUV()

47.66 fps

And real life 4k file bench :

Coeff=50.0
DGSource("DCrystal.dgi",i420=false,deinterlace=0,fieldop=0,fulldepth=true).crop(0,276,0,-276).trim(12632,12731)
ConvertYUVtoXYZ(Color=0,OutputMode=1,Rx=0.680,Ry=0.320,Gx=0.265,Gy=0.690,Bx=0.150,By=0.060)
ConvertXYZ_HDRtoSDR(Coeff_X=Coeff,Coeff_Y=Coeff,Coeff_Z=Coeff)
ConvertXYZtoYUV()

29.0fps

videoh
4th June 2018, 18:56
And what is your system please?

jpsdr
4th June 2018, 19:36
A Broadwell 10 cores (i7-6950X) overclocked @4.1GHz.

videoh
4th June 2018, 21:19
Thanks.

FranceBB
4th June 2018, 22:12
You have no idea how long I've been waiting for this.
Very appreciated! Thank you very much indeed! :D

jpsdr
5th June 2018, 09:06
Happy to see it's usefull to some people. But i warn again, unless bug found, everything is expected to work properly, except the HDR to SDR wich is just a multiply scaling, i've just made it out of curiosity to see what result a very simple thing would produce, without expectation of course, because if it would have been so simple, it would have been done a long time ago.
I'll had for 0.2 input RGBPS on ConvertRGBtoXYZ and ConvertLinearRGBtoYUV.
With also a document explaining things more precisely.
After, for 0.3, i'll try to add the standard tonemap. They are "simple" functions (multiply, div, add, sub), so, they can be vectorized.
And then, i'll "slowly" or "quietly" work on the real HDR to SDR process i want to try, but don't know if it will produce results...

jpsdr
8th June 2018, 15:03
New version, and i've added a document with the release.
So, feel free to read it, and provide feedback in case i was doing things wrong, and/or misunderstood things.

jpsdr
10th June 2018, 09:06
I've updated the 0.2 file (no changes in the binaries) with a new version of the document, a few very minor corrections on it.

Yanak
14th June 2018, 13:57
Hello jpsdr,

It's the first time i have a HDR file and try to do something with it so please forgive me if I'm asking silly things but after spending a couple of hours on this i didn't manage to get things working correctly.

This if the file mediaInfo from the mkv container :
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Commercial name : HDR10
Format profile : Main 10@L5.1@High
Codec ID : V_MPEGH/ISO/HEVC
Duration : 2 h 1 min
Bit rate : 45.2 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.227
Stream size : 40.6 GiB (85%)
Default : Yes
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0000 cd/m2, max: 1100 cd/m2

I first open the file with DSS2 or FFVideoSource ( of course the colors are off here), i don't have and not yet tried with DGSource like in your real life 4k file bench script, I'm not sure if i need this or if DSS2 or FFms2/ffvideosource are enough.

I then tried to add like in your first post
ConvertYUVtoXYZ(Color=1)
ConvertXYZtoYUV(pColor=1)
plus tried a few other variations, i get the colors shifting but not getting the right colors out in avisynth, It's my first time messing with HDR stuff and i'm not sure if i am doing something wrong with your plugin or if the HEVC video is the problem, so if you have any hints to share please.

Thanks a lot ( and thanks for your very useful Avisynth plugins pack while i'm at it :p )

jpsdr
14th June 2018, 16:36
First, again, as i've explained several times, the plugin do only XYZ/RGBLinear conversion, but the HDR to SDR part is not realy properly implemented, there is only a simple quick linear scaling which may produce results on some case, but that's all.
If you want try something like this :

Coeff=20.0
ConvertYUVtoXYZ(Color=0,OutputMode=1)
ConvertXYZ_HDRtoSDR(Coeff_X=Coeff,Coeff_Y=Coeff,Coeff_Z=Coeff)
ConvertXYZtoYUV()

Your mediainfo information doesn't provide the chromaticity parameters metadata, so the script assume default for BT.2100.
Try different values of Coeff.
But again, for now, the HDR to SDR part is not realy implemented.

Yanak
14th June 2018, 16:57
Thanks a lot for the reply,

Will give more tries later at night and dig a bit more into this, never had to deal with HDR content before and i am still discovering all this.

Thank you very much for the hint.

Edit :
Got it working and displaying the right colors now, might need a little tweak with levels or hue/sat but it's good already, thanks again for the help.
I looked to see if i could get more infos about the chromacity on it but the only thing mediainfo is giving me that i forgot to post is "Writing library: ATEME Titan KFE 3.7.3 (4.7.3.1001)" and i don't think this will help anyways.

It's the first time i have to deal with such material and probably won't have to do this a lot in the near future so thank your very much for the help and patience with a nooby in this domain.

jpsdr
19th June 2018, 08:56
Message to Narkyy (from another realm). Can you tell me what is your issue between 0.1 and 0.2 version ?

Edit :
Ok, reproduce and found. Will be fixed in next release.

videoh
19th June 2018, 15:13
With the AVX2 build disabled, this test is being hit with b2[i] equal to zero.

if (fabs(b2[i])<=_zero) return(-2);

I guess you already found it.

jpsdr
19th June 2018, 16:44
I'll push soon (time for me to get back home) the fix on github, you'll be able to make a build if you want.

videoh
19th June 2018, 17:34
Great, thank you.

videoh
20th June 2018, 22:27
I wonder if you should consider renaming your package to avoid confusion with this:

https://gitlab.com/standards/HDRTools/

StainlessS
20th June 2018, 23:34
I wonder if you should consider renaming your package to avoid confusion with this:

Good idea, there have been such confilcts before on D9 [even if linked one aint on D9], HdrToolKit, or something, perhaps.
I once was attending a computer show where one of my lumps of s/w was only in demo form (could not manage to complete in time),
and a game with identical name was released at same show, I had to make a quick name change [which turned out to be better I think],
and avoided what could have been a few legal wrangles, and lots of user confusion.

jpsdr
21st June 2018, 08:45
... I'll see... in the future...

kolak
21st June 2018, 09:55
What bout HDR PQ to HLG conversion based on this paper:

http://downloads.bbc.co.uk/rd/pubs/papers/HDR/BBC_HDRTV_PQ_HLG_Transcode_v2.pdf

which suggests it should be pure math (no tone mapping, etc at leats for 1000 nits masters).

jpsdr
21st June 2018, 13:50
I'll take a look later, but will not this just shift the issue from converting [PQ-HDR -> SDR] to converting [HLG-HDR -> SDR] ?

kolak
21st June 2018, 15:46
Well, HDR to SDR is different story as it's difficult conversion and ideally should be done per scene, not global.
HDR PQ to HLG should be way easier (more like Rec.709 to Rec.601).

jpsdr
21st June 2018, 16:15
My final purpose (the reason i've made these tools) is to work on HDR to SDR.

kolak
21st June 2018, 23:02
Starting with something easier (which is still useful) is no harm :)

jpsdr
22nd June 2018, 09:03
I have no needs for PQ to HLG for now. But technicaly, once i'll implement the HLG in the plugins (because for now there is only PQ, my plugins have a HLGMode parameter, not implemented yet), doing something like :

ConvertYUVtoLinearRGB(Color=0,HLGMode=true)
ConvertLinearRGBtoYUV(Color=0)

will do the convertion.
I didn't implement HLG yet, because i've trouble to understand properly how it works. There is several range mode (so i'm a little loss).
But...! Maybe the doc you provided will help me to understand things. But, HLG is not my prority for now.
Ah... Just take a look. The french version and english version of the same release version document don't say the same thing.
French version (R-REC-BT.2100-1-201706-I!!PDF-F) talk of [0:12] range and possibility of [0:1], when english (R-REC-BT.2100-1-201706-I!!PDF-E) talk only of [0:1]...
Nice !

Edit :
Quick look, it seems that PQ has fixed parameters functions, for luminance range up to 10000 cd/mē. It seems that for HLG, the parameters changes according (Lw/Lb) values, when they are fixed for PQ. Meaning i'll have to add these parameters on my plugins, for when i'll implement HLG. I'll also have to add some "WhitePeak" value (at least on plugins using linear data on input), if i want to properly handle both HLG and PQ, because it seems that i can't assume (like i thought), that in linear value, 1.0 = 10000 cd/mē.
Well... I'll see this for later... when i'll do the 0.3 release ;)

kolak
22nd June 2018, 10:40
There are many references to [0,1] or [0,12] in English version.
Is there anything useful in link to HDRTools project provided in the document?

jpsdr
22nd June 2018, 11:38
Didn't read properly yet (but retrieved), i'm on something else for now, but there is a big chance it may help me to understand better HLG and this eventualy "whitepeak" information.

Edit :
Read a little more, it will help me. From what i understood (if properly), PQ is "fixed" -> 1 = 10000cd/mē, but HLG is adaptative.
Stop investigating for now, will continue in second time.

jpsdr
26th June 2018, 14:50
I've read doc more carefully, kolak doc helped... HLG is a pain in the ass !!!
I'll implement what i've understood, i just hope i have undersood properly... :D

FranceBB
26th June 2018, 17:42
I've read doc more carefully, kolak doc helped... HLG is a pain in the ass !!!
I'll implement what i've understood, i just hope i have undersood properly... :D

That would be amazing, 'cause last time I used Davinci Resolve to convert HDR10 to HLG, but it would indeed be very useful to have HLG working in your plugin.

jpsdr
26th June 2018, 19:28
And having someone to test/verify also ;)

Otherwise, does anyone know if it's possible to make a script which display, in the frame, the max value of Y for the curent frame (for an YUV video) ?
If yes, what would it be ?

StainlessS
26th June 2018, 20:44
Something like this maybe [EDIT: Below intended for 8 bit only]


Function TestFunc(clip c,Float Thresh_Min, Float Thresh_Max,Bool Debug,String LogFile) {
c
Yave=AverageLuma YMin=YPLaneMin(Thresh_Min) YMax=YPLaneMax(Thresh_Max)
Subtitle(String(YAve,"YAve=%7.3f : ")+String(YMin,"YMin=%3.0f : ")+String(YMax,"YMax=%3.0f"),Align=8) # Added
RT_Subtitle("%d] YAve=%7.3f YMin=%3d YMax=%3d",current_frame,YAve,YMin,YMax,Align=5)
(Debug) ? RT_DebugF("%d] YAve=%7.3f YMin=%3d YMax=%3d",current_frame,YAve,YMin,YMax,name="TestFunc: ") : NOP
(LogFile!="") ? RT_Writefile(LogFile,"%d] YAve=%7.3f YMin=%3d YMax=%3d",current_frame,YAve,YMin,YMax,Append=True) : NOP
Return last
}


FN = "D:\TestClip.avi"
LOGFILE = "D:\TestClip.Log"
DEBUG = True
Thresh_Min = 0.0
Thresh_Max = 0.0

(LOGFILE!="") ? RT_FileDelete(LOGFILE) : NOP # We Will Append

SSS = "TestFunc(" +
\ String(Thresh_Min,"%f") +
\ String(Thresh_Max,",%f,") +
\ String(DEBUG) +
\ "," + Chr(34) + LogFile + Chr(34) +
\ ")"

#SSS=RT_String("""TestFunc(%f,%f,%s,"%s")""",Thresh_Min,Thresh_Max,DEBUG,LogFile) # Alternative
#RT_DebugF("SSS=%s",SSS)

AviSource(FN)
Return Last.Scriptclip(SSS)


EDITED:

EDIT: Alternative to RT_string, StrFmt(Small dll):- https://forum.doom9.org/showthread.php?t=174649&highlight=StrFmt

EDIT: Or RT_Stats for 'RT_xxx' funcs:- https://forum.doom9.org/showthread.php?t=165479&highlight=RT_Stats

jpsdr
27th June 2018, 09:08
Tested, and seems there is a problem. I've used your 2.00 beta version.
The message is telling me that Average Plane can only be used within run-time filters.
No log file is created.

StainlessS
27th June 2018, 09:40
That is why its called from ScriptClip() in given code.
AverageLuma can only be called from within runtime environment,
or alternatively, cheat a bit, simulate within runtime filter, must eg current_frame=n, before the call to AverageLuma. (Scriptclip et al, set current_frame at every frame so it is known what frame is current).
For RT_AverageLuma(n=n).will also suffice, where n is accepted as current_frame, but MUST be done for every frame, thats is why Scriptclip is a good idea, unless using eg GScript for next to access frames.

"No log file is created", thats because its already caused an error and exited.
Try the script as given.

MOBILE:

EDIT: Post your failing script.

EDIT: do frame 42 only (untested)

Function TestFunc(clip c,Float Thresh_Min, Float Thresh_Max,Bool Debug,String LogFile) {
c

n=current_frame
trim(n,-1)
current_frame=0
Yave=AverageLuma YMin=YPLaneMin(Thresh_Min) YMax=YPLaneMax(Thresh_Max)
Subtitle(String(YAve,"YAve=%7.3f : ")+String(YMin,"YMin=%3.0f : ")+String(YMax,"YMax=%3.0f"),Align=8) # Added
RT_Subtitle("%d] YAve=%7.3f YMin=%3d YMax=%3d",n,YAve,YMin,YMax,Align=5)
(Debug) ? RT_DebugF("%d] YAve=%7.3f YMin=%3d YMax=%3d",n,YAve,YMin,YMax,name="TestFunc: ") : NOP
(LogFile!="") ? RT_Writefile(LogFile,"%d] YAve=%7.3f YMin=%3d YMax=%3d",n,YAve,YMin,YMax,Append=True) : NOP
Return last
}


FN = "D:\TestClip.avi"
LOGFILE = "D:\TestClip.Log"
DEBUG = True
Thresh_Min = 0.0
Thresh_Max = 0.0

(LOGFILE!="") ? RT_FileDelete(LOGFILE) : NOP # We Will Append

AviSource(FN)
current_frame=42 /* Simulate runtime */
TestClip(Thresh_Min,Thresh_Max,DEBUG,LOGFILE)
Return Last


EDIT: Oops, would print stats from frame 42 on frame zero, i think(not sure), edited in blue, should work.
EDIT: Oops again, will print frame 0] in stats, well you get the idea anyways I hope.

Alternative, MUST call with n=frame number, or n=current_frame from within ScriptClip.
Function TestFunc(clip c,int n,Float Thresh_Min, Float Thresh_Max,Bool Debug,String LogFile) {
c
trim(n,-1)
current_frame=0
Yave=AverageLuma YMin=YPLaneMin(Thresh_Min) YMax=YPLaneMax(Thresh_Max)
Subtitle(String(YAve,"YAve=%7.3f : ")+String(YMin,"YMin=%3.0f : ")+String(YMax,"YMax=%3.0f"),Align=8) # Added
RT_Subtitle("%d] YAve=%7.3f YMin=%3d YMax=%3d",n,YAve,YMin,YMax,Align=5)
(Debug) ? RT_DebugF("%d] YAve=%7.3f YMin=%3d YMax=%3d",n,YAve,YMin,YMax,name="TestFunc: ") : NOP
(LogFile!="") ? RT_Writefile(LogFile,"%d] YAve=%7.3f YMin=%3d YMax=%3d",n,YAve,YMin,YMax,Append=True) : NOP
Return last
}

jpsdr
27th June 2018, 12:26
Thanks, i finaly been able to get what i want.

jpsdr
28th June 2018, 15:33
That would be amazing, 'cause last time I used Davinci Resolve to convert HDR10 to HLG, but it would indeed be very useful to have HLG working in your plugin.

Do you have, by any chance, an HLG encoded stream you could share ?

videoh
28th June 2018, 17:57
http://4kmedia.org/travelxp-4k-hdr-hlg-sample/

Or google for "hdr hlg sample files". You'll find a lot of them.

Not all have the SEI denoting HLG, but some do.

jpsdr
28th June 2018, 18:18
It didn't even cross my mind to google search... :(
Thanks.

What are exactly HLG SEI information ? Is there some other mastering informations in the H265 spec ? I've tried a search for "HLG" in the T-REC-H.265-201612-I!!PDF-E document, but found no information about such a thing.

videoh
28th June 2018, 18:49
You can google "HLG SEI message". BTW, DGIndexNV notifies this SEI in the DGI file.

jpsdr
28th June 2018, 20:12
Ok, i see (at least, i hope...).
... Hum... If i understand properly, it seems i have to add again another parameter for HLG, which specifies what transfer color matrix use within the OOTF function, because it's not necessary BT.2020... Argh....

jpsdr
30th July 2018, 16:30
After a long fight (HLG is realy a pain in the ass if you want to optimize things), i'm near the end. But i have two choices, for a specific point. For speed-up i've implemented lookup. 8 bit is 3D lookup, so it's fine, but for 10 and 12 bit i've tried 2D lookup. It's faster than 16 bits, but... there is a noticeable resolution loss on low values (visible on some colors of ColorBars when testing YUVtoRGB followed by RGBtoYUV).
Choice 1 : Keep actual status, with 2D lookup for 10/12 bits.
As most of source filters produce 16bits ouput, if you don't want speed, you keep 16 bits, it you want speed-up, you add ConvertBits to reduce the bitdepth to the actual true bitdepth of the source. With this option, you keep the possibility to choose.
Choice 2 : No 2D lookup, 10 to 16 bit, same speed.
For now, i'm more on "Choice 1", allowing users to keep the possibility to choose, but i want to know what others users thinks what would be better.

BTW :I personnaly have nothing to display HDR video, no reader, no screen, nothing. The only tests i can do is checking ouput is not obvious garbage, no crash or things like this, that an operation followed by its reverse produce output indentical to input, but i have no way to check if output is realy proper.

jpsdr
31st July 2018, 17:52
Finaly, the Hell Log Gamma is done. New version, add HLG support, see first post.
I've also updated the PDF document, there was some still some mistake (things i've not totaly properly understood) on the last one.
I've made a lot of tests, but there's still the possibilty that i've missed something, but i hope not... ;)

FranceBB
1st August 2018, 03:58
Thank you very much indeed.

By the way there's a typo in the Readme "Outpout RGB64." instead of "Output" in "ConvertXYZtoRGB".

Anyway, I tried this:

FFMpegSource2("test.mov", atrack=-1)

ConvertYUVtoXYZ(Color=2, OutputMode=2, fullrange=false)

ConvertXYZtoRGB(OutputMode=0)

ConvertXYZ_SDRtoHDR()

But ConvertXYZ_SDRtoHDR keeps complaining that input format must be RGB64 or RGBPS.

So I tried with:

ConvertYUVtoLinearRGB(Color=2, OutputMode=2, fullrange=false)

ConvertXYZ_SDRtoHDR()

But again ConvertXYZ_SDRtoHDR keeps complaining that input format must be RGB64 or RGBPS.

I'm trying to convert a normal 8bit yv12() footage to XYZ Planar float RGB 8bit and then use ConvertXYZ_SDRtoHDR() to get an HDR BT2020 PQ 8bit output.


Windows XP x86 Build, Avisynth 2.6.1.

Am I doing something wrong?

Thank you in advance.

Oh, by the way, are you considering to integrate this into your jpsdr_plugin.dll?

jpsdr
1st August 2018, 08:34
Yes, there is "something wrong" : standard avisynth doesn't support RGB64 neither RGBPS, so, you need avs+.

8 bits creates too much precision loss on linear data on HDR video. You can easely see it just by doing, on 8 bit video (even if it's not an HDR video) the following (it will use for this test the PQ HDR path) :
ConvertYUVtoLinearRGB(Color=0)
ConvertLinearRGBtoYUV(Color=0)
Theoricaly, the ouput should be identical to the input, but you'll see effect of precision loss. So, doing HDRtoSDR functions on 8 bits data is a total nonsense (when your data are the linear sensor data).
This could be possible if you're doing a function which will do the 3 functions in one (YUV -> RGB -> HDRtoSDR -> RGB -> YUV) and working internaly with 16 bits data, but it's not the path i've choosen. I choose modular functions option.
So if you stay with standard avisynth, you'll not be able to use the specific HDR/SDR functions.

I've re-read again, and there is nothing wrong in the description of "ConvertXYZtoRGB" in the readme.

Yes, i'll integrate it later, after the 0.4.0, where i'll add Hable, Mobius and Reinhard (but again, they'll be usable only with avs+).
Expect 6 new functions like ConvertXYZ_Mobius_HDRtoSDR, ConvertRGB_Mobius_HDRtoSDR, etc...

FranceBB
1st August 2018, 10:07
avisynth doesn't support RGB64 neither RGBPS


I see...


8 bits creates too much precision loss on linear data on HDR video.


Well... yes...
But 16bit stacked conversion to RGB48y would probably provide enough precision, I think and Avisynth supports that thanks to Dither Tool, but I don't think I'll ever see it supported as you said that you didn't like the stacked 16bit format 'cause you considered it an "hack" back when I asked you about NNEDI... :'(

jpsdr
1st August 2018, 10:14
Yes, as it exists avs+ which supports 16 bits, i don't intend to spend/waste time to add stacked version on my plugins.

Is it XP which prevents you from using avs+ ? avs+ can't be installed or don't work on XP ?

FranceBB
1st August 2018, 10:52
Is it XP which prevents you from using avs+ ? avs+ can't be installed or don't work on XP ?

I don't know whether it's officially supported or not, but it installs on XP. It automatically detected my Avisynth installation and offered to "migrate my plugins to AVS+". I clicked "yes" and it only copied some of the plugins I had in my plugin folder. Then, when I tried to run it with a simple command, it loaded them all together automatically ('cause they are .avsi) and it crashed.
I've been using Avisynth since 2006 and I have many plugins in my plugins directory. I gotta find out which one/ones are causing the problem, why they are causing the problem and a workaround to solve it. I'm gonna open up a new thread about this in "Avisynth usage" though.

StainlessS
1st August 2018, 13:09
FranceBB,
Just get Groucho2004 AvsMeter

AvsMeter Avsinfo

Avs+ officially supports XP and is much better than avs standard.

jpsdr
3rd August 2018, 21:06
New version, see first post.