Plugin HDRTools for Avisynth

[joearmstrong]

I found the best overview of all the tonemappers with further links here:
https://github.com/tizian/tonemapper

[Dogway]

Originally Posted by joearmstrong

Is it possible to implement ACES tonemapping in HDRTools? I see no way to use this curve in Avisynth even though it's now the current standard in tonemapping.

I plan to add it to TransformsPack, but not only the TM but the whole implementation. It's on the roadmap since the beginning since I already made a few tonemappers for Substance Painter a few years ago. The ACES I did then was the Stephen Hill version which is far from the real deal.

[jpsdr]

The ACES function transfert is "simple" (using only one parameters, not alls), also it seems to be XYZ specific, so i'll do only an XYZ version, and i can use allready existing code function as base template, so a "maybe" with a good chance...

[jpsdr]

Begin slowly to add ACES...

[jpsdr]

New version, see first post.
WARNING : Not tested ! (But chances of working properly are good)
Will not be able to test before a week at least, but make a release anyway in case others want to test.

[StainlessS]

Confidence is hi,
Love that !

Rock On Tommy,

[-QfG-]

Hi there,

using for testing this values:

Loadplugin("%startup_dir%\Apps\Plugins\AVS\HDRTools\hdrtools.dll")
ConvertYUVtoXYZ(Color=1, HDRMode=0)
ConvertXYZ_ACES_HDRtoSDR()
ConvertXYZtoYUV(pColor=1)
ConvertBits(8)

Colors are awesome, but the Brightness of the picture is to low. Any Ideas which Parameters i must set by ConvertXYZ_ACES_HDRtoSDR() ?

[jpsdr]

Everyone will say "Why the developper is asking this question....????"

But in fact, i've programmed the standard tonemap, but never used them (don't remember how i tested them).

Then, when i wanted to see, out of curiosity the results of ACES (and no, no testing yet, don't have access to PC with AVX & AVX2, which trigged me an issue never seen, but it's another story) result was black...
Tested Mobius, Hable, black... (well not totaly black, but very black).

Maybe the HDR -> SDR tonemap is not working as i thought...?

I thought, with my functions, if your input is standard PQ Blu-Ray, you do :

ConvertYUVtoXYZ(Color=0)
tonemapXYZ(pColor=0)
ConvertXYZtoYUV(pColor=0,Color=2)

For me, the XYZ input of a tonemap is HDR (so basicaly BT2100), and output is SDR (so basicaly BT709).

Except, as i said, when i tested, is was, almost, black.
So, what these tonemaps finaly realy do ?

Still don't give up on my first idea (maybe use Neural Network different way) of trying something with the 3 databases i've build.

[FranceBB]

Originally Posted by jpsdr

But in fact, i've programmed the standard tonemap, but never used them (don't remember how i tested them).

I do, we tested them on TV Series, Sports and other movies as I was desperately eager to get something to work with HDR stuff back in 2018. We didn't have LUT support (no cube) and we had literally nothing to handle PQ contents in Avisynth which made me worry a lot, so I was like really really pushing to get this. I even remember some of the assets: one was Riviera Season 1 coming from a DNxHQX 12bit in HLG, one was Avengers Infinity War coming from an H.265 in PQ at 1000 nits 10bit, one was Naples vs Juventus, a Serie A game in XAVC Intra Class 300 in HLG 10bit and then there were some other assets I tested 'em on which I forgot.
There were some I had to make the tests on myself and we couldn't share, while we worked together on some others with the FTP (I saturated it that time and you sent me a reply shortly after hahahahha) and also some other of them were disclosed publicly here on Doom9 as far as screenshots are concerned: https://forum.doom9.org/showthread.php?t=175488&page=4

Originally Posted by jpsdr

Tested Mobius, Hable, black... (well not totaly black, but very black).

Maybe the HDR -> SDR tonemap is not working as i thought...?

It is, but as per the tests I made in May 2018, I always suggest:

#BT2020 HLG HDR to BT709 SDR
ConvertBits(16)
ConvertYUVtoXYZ(Color=0, HDRmode=2)
ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.1)
ConvertXYZtoYUV(pColor=0)

and

#BT2100 HDR PQ to BT709 SDR
ConvertBits(16)
ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)
ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.5, contrast_X=0.9)
ConvertXYZtoYUV(pColor=0)

Originally Posted by jpsdr

Except, as i said, when i tested, is was, almost, black.

Yeah, that's "normal".

Originally Posted by jpsdr

So, what these tonemaps finaly realy do ?

They do what they are supposed to do, so what you wrote in the PDF, but only when properly tweaked.

Originally Posted by jpsdr

Still don't give up on my first idea (maybe use Neural Network different way) of trying something with the 3 databases i've build.

More than a neural network, please please please make them scene-aware, that would be amazing.
My idea from here is still valid: https://forum.doom9.org/showthread.php?t=182823

[jpsdr]

Originally Posted by FranceBB

one was Avengers Infinity War coming from an H.265 in PQ at 1000 nits 10bit

Oh... That's one of the 3 movies i used to build my HDR -> SDR database for my NeuralNetwork first idea.

If i do something for now, it will unfortunately not be as complex as scene aware. Also, the database i've build don't have this kind of information... And even, maybe, database is a big word for it.

If i do something for now, it will be some kind of tonemap function in 1-exp(x) form, maybe keep the NN part to compute the parameters. If you're curious of the curves i have, i can send you excel sheets.

[joearmstrong]

Originally Posted by -QfG-

Hi there,

using for testing this values:



Colors are awesome, but the Brightness of the picture is to low. Any Ideas which Parameters i must set by ConvertXYZ_ACES_HDRtoSDR() ?

I have the same question. With more brightness the converted video would look very good. I'm missing the exposure parameter from the other tonemapping curves.
Thank you for implementing ACES tonemap!

[jpsdr]

Didn't see any kind of exposure parameter in ACES description.
If it's just a multiply coeff factor, for now, you can use in that case ConvertXYZ_Scale_HDRtoSDR just before to multiply the input, or after to multiply the output.
It's true indeed that the others tonemap have exposure parameter.
Maybe i'll think to add it for a next release...

[jpsdr]

New version, see first post, still not fully tested...

Edit :
More tests done now, seems Ok, i'll update the complete plugins package soon.

[jpsdr]

Here some result HDR (X axis) -> SDR (Y axis) curves extracted from my database.

It's clearly an a.(1-exp(-b.x)) formula.

[joearmstrong]

I'm trying the new exposure parameters in ACES. Anybody found good settings to recommend?

[FranceBB]

Sorry for not replying earlier, but I've been extremely busy at work.
Anyway, yeah, I'd love to see the results you get and the values of the training you're doing on the experimental neural network.

[jpsdr]

I may have not been clear enough... For now, i only built a database, and have an excel of curves like in post #194.
Throw data to a NN just "like that" didn't work. Also, i'm far far far from being a expert in NN, i'm more a noob in the field.
The NN idea may work, but with a little thinking... Like, as said in post #194, curves seems to follow a specific formula pattern, maybe the NN can find the a and b values.
Anyway, i'll PM you.

For now, i'm wondering what to do with this. As it can be seen in post #194, for the same HDR value, there is several SDR results.
For now, i have to think how to use this database, and for exemple, figure out if there is some kind of reproduce things. The Xsdr is strongly from Xhdr, but as for same Xhdr there is several Xsdr, others things may have, small effect, like Y and Z. Maybe something like, according formula of #194 : Xsdr=a(Yhdr,Zhdr,...?).(1-exp(-b(Yhdr,Zhdr,...?).Xhdr)
All curves show something interesting : "big belly" with 2 convergence points. 0, obvious, and at "the end". This is interesting property, help and reduce parameters exploration.

[DTL]

If you have a time can you look in this difference:

LoadPlugin("plugins_JPSDR.dll")
LoadPlugin("avsresize.dll")
LoadPlugin("fmtcavs.dll")

Function Convert422ToRGB24mon_lin_x4_HLG(clip c)
{
  uc=UToY(c)
  vc=VToY(c)
  uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=105, c=0)
  vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=105, c=0)
  uc=SincLin2ResizeMT(uc, src_left=0, uc.width*2, uc.height, taps=8)
  vc=SincLin2ResizeMT(vc, src_left=0, vc.width*2, vc.height, taps=8)
  yuv444=CombinePlanes(c, uc, vc, planes="YUV", source_planes="YYY", pixel_type="YUV444P10")

# jpsdr_plugin
 rgb_lin=ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2)
 rgb_lin=RGBAdjust(rgb_lin,r=3,g=3,b=3)

# avsresize
#rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:l", nominal_luminance=800)
#rgb_lin=RGBAdjust(rgb_lin,r=3,g=3,b=3)

#fmtconv
#  rgb_lin=fmtc_matrix(yuv444,mat="RGB", mats="2020",fulls=false, fulld=false)
#  rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear",bits=32,fulls=false, fulld=false)
#  rgb_lin=RGBAdjust(rgb_lin,r=3,g=3,b=3) 

  rgb_lin=SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
#return z_ConvertFormat(rgb_lin, pixel_type="rgbp8", colorspace_op="rgb:linear:709:l=>rgb:709:709:l").ConvertToRGB()
#return fmtc_transfer(rgb_lin,transs="linear",transd="709",fulls=false, fulld=false).ConvertBits(8,fulls=true,fulld=false).ConvertToRGB()
return ConvertLinearRGBToYUV(rgb_lin,Color=2).ConvertToRGB24(matrix="PC.709")  
}
plY=Blankclip(width = 10, height = 1, pixel_type="Y10").Expr("sx 0 == 509 sx 1 == 509 sx 2 == 509 sx 3 == 509 sx 4 == 509 sx 5 = 498 sx 6 = 446 sx 7 == 361 sx 8 == 292 276 ? ? ? ? ? ? ? ? ?")
plU=Blankclip(width = 5, height = 1, pixel_type="Y10").Expr("sx 0 == 270 sx 1 == 263 sx 2 == 330 sx 3 == 761 754 ? ? ? ?")
plV=Blankclip(width = 5, height = 1, pixel_type="Y10").Expr("sx 0 == 203 sx 1 == 224 sx 2 == 271 sx 3 == 832 821 ? ? ? ?")
tr=CombinePlanes(plY, plU, plV, "YUV", "YYY", pixel_type="YUV422P10")

sust_l=Blankclip(width = 4, height = 1, pixel_type="YUV422P10").Expr("sx 0 == 509 sx 1 == 509 sx 2 == 509 509 ? ? ?","sx 0 == 270 270 ?","sx 0 == 203 203 ?").PointResize(50,1)
sust_r=Blankclip(width = 4, height = 1, pixel_type="YUV422P10").Expr("sx 0 == 276 sx 1 == 276 sx 2 == 276 276 ? ? ?","sx 0 == 754 754 ?","sx 0 == 821 821 ?").PointResize(52,1)

StackHorizontal(sust_l,tr)
StackHorizontal(last,sust_r)
PointResize(width, 100)
# source finishes here, 75% green-magenta transient HLG 10bit 4:2:2

Convert422ToRGB24mon_lin_x4_HLG()

Results are:


So the functions from plugins_JPSDR_v3_3_0 return a bit lower RGB levels and the shape of transient also a bit different from 2 others HDR converting plugins. What may cause this difference ?
I tried to calculate levels values at https://forum.doom9.org/showthread.p...56#post1952556

"my advice would bet to test these 2 settings : Color=0,HDRMode=2,HLGLw=800 and Color=0,HDRMode=1,HLGLw=800"

With HDRMode=2 the HLGLw adustment change nothig.
With HDRMode=1 the mostly closely looking green is with HLGLw=11000 (or may be about 11500) but RB channels at magenta become unbalanced.

"try with OOTF=false, it will ouput the displayed linear data.
But in that case you have to put also OOTF=false in ConvertLinearRGBToYUV."

This returns very dark image with HDRMode=1.
With

rgb_lin=ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2, OOTF=false, HLGLw=350)
return ConvertLinearRGBToYUV(rgb_lin,Color=2, OOTF=false).ConvertToRGB24(matrix="PC.709")

The sustained RGB looks mostly close to other plugins and green-magenta transient may be looks the mostly nicely (with much less dark stripe and ringing). So this HDR magic is so magic. How to connect this magic value HLGLw=350 with 'typical Lw' I still do not know.


As I test in MS Paint with cut-paste region - in RGB24 the fmtconv and avsresize output looks perfectly matches between each other at both green and magenta sustained levels and transient between colours.

[jpsdr]

Don't have time unfortunately right now.
But, if you want, you can do :
If both others plugins produce same result, mix the YUV -> RGB and RGB -> YUV (for exemple z_ConvertFormat doing YUV -> RVB followed by fmtc_transfer doing RVB->YUV), check if output is still the same. If yes, revert the mix.
If both mix are Ok, now do the same mix with my plugin. This will eventualy allow to see (if it's the case) if difference is only on one part.
If, for exemple, ConvertYUVtoLinearRGB followed by z_ConvertFormat or fmtc_transfer produces the same result than a complete conversion with another plugin, it meens it's ConvertLinearRGBToYUV which behaves differently.

[jpsdr]

New version, see first post.