Plugin HDRTools for Avisynth

[jpsdr]

Yes, i'm a dude.

Almost everything was allready done, but i wanted to finish fixing the aWarp4 issue before releasing new builds.
I was waiting to have time to investigage it, and this is my first day of my holydays, so i finaly got the time, and also i wanted to "get rid of it", to be quiet for the rest of my holydays... :D

[StainlessS]

OK dude, have a nice holiday.

[FranceBB]

i wanted to "get rid of it", to be quiet for the rest of my holydays... :D

Smart move. Enjoy your vacation ;)

[Hotte]

I have some more general, simple questions concerning HDR.

My camera allows me to record in Rec.2020 (HLG) or Rec.709 while shooting in 10-bit modes. As far as I understood the idea of colorspaces, BT.2020 also covers colors outside 709 hence a larger visible colorspace if I have the right monitor. Although I have an sRGB-display I might be going for a HDR-Monitor in the near future to participate.

So I thought it is a good idea to start recording everything new in HLG rather than keep on shooting in 709.

But:

Isn't HDR only a mapping of colors rather than a quality push ? Could I e.g. remap rich 10-bit footage captured in 709 to become satifying HDR if needed (maybe using HDR-Tools) ?

Or is the chance for HDR gone if it was not recorded in BT.2020 right from the start ?

Generally I found it hard to get satisfying 709 results from HLG source: The colors, especially red spectrum ist somewhat out of balance and I have not found any approach yet to cure this.

What would you recommend ?

[videoh]

Isn't HDR only a mapping of colors rather than a quality push ? Could I e.g. remap rich 10-bit footage captured in 709 to become satifying HDR if needed (maybe using HDR-Tools)? "quality push"? "rich"? "satisfying"?

Bit depth is independent from color space, but BT2020 is usually chosen for HBD.

Generally I found it hard to get satisfying 709 results from HLG source: The colors, especially red spectrum ist somewhat out of balance and I have not found any approach yet to cure this.

What would you recommend ? Post a link to the problematic HLG source video.

[Hotte]

"quality push"? "rich"? "satisfying"?
Bit depth is independent from color space, but BT2020 is usually chosen for HBD.

I am clear about bit depth being independent from colour space.

The essence of my question was if I should always record my 10-bit modes in BT2020 to be future-proof.

Currently I am on Rec709 (monitor, projector...). Recording everything in BT2020 today would mean conversion down to 709 which probably comes with some compromise.

Also the HLG-setting in my cam does not allow me to use a flat log-profile. Or is a flat profile not needed anymore with HDR ?

Or is there a way up from recorded Rec709 to BT2020 ?

Sorry, maybe silly questions. I just want to make sure I am going into the right direction.

[FranceBB]

The essence of my question was if I should always record my 10-bit modes in BT2020 to be future-proof.


Yes



Currently I am on Rec709 (monitor, projector...). Recording everything in BT2020 today would mean conversion down to 709 which probably comes with some compromise.


Yes, you would have to convert, but I would strongly suggest not to shoot in Linear BT709 SDR 100 nits in 2021 especially if the camera has far more stops than 6 stops which is 100 nits 'cause you would be wasting the sensor capacity otherwise... Also BT2020 alone won't achieve anything, it's just a color matrix, what's important is the color curve, in this case HLG which will allow you to record as many nits as the stops of the sensor.



Also the HLG-setting in my cam does not allow me to use a flat log-profile. Or is a flat profile not needed anymore with HDR ?


Ok, a bit of confusion.
BT709 and BT2020 are just matrices, those are color matrices and both are SDR 100 nits. Sure, BT2020 would give you a better result, but you're still gonna be limited to 6 stops, 100 nits SDR, so, if you shoot a subject with the sky behind and sunlight, either the subject will be lit and the sky clipped out and white averaged out at 940 or the sky is gonna be there and the subject is gonna be totally dark 'cause you still have only 6 stops, 100 nits.
Now, when it comes to color curve, those give you "HDR" aka "High Dynamic Range". Now, I don't know which camera you have and you didn't specify which log you're shooting in, so I'm gonna assume Slog3 for the sake of the argument.
If you compare Slog3 vs HLG, both have their pro and cons. You gotta remember that HLG is not a purely logarithmic color curve, therefore it starts just like a normal linear SDR curve and then it slowly curves and becomes logarithmic, which means that the blacks won't have many details and they're gonna be just like the SDR ones, but the more you "go up" the more logarithmic you get, so the highlights are gonna be preserved and you're gonna get more nits (up to 1000 nits, although I strongly doubt that your camera will reach that and it's gonna sit probably around 600-700). The reason why HLG is an "Hybrid Log Gamma" (emphasis on "hybrid") is that it allows backwards compatibility with SDR monitors and decoders who ignore the color curve and just interpret the color matrix. Of course, this "sacrifices" the details in the black 'cause you won't get anything more than shooting in SDR in the blacks. This is NOT true for Slog3, though. Just like other purely logarithmic curves like Clog3, Log-C, PQ etc, it starts logarithmic and continues as logarithmic, therefore you're gonna have many more nits available and you'll have many more details in the blacks as well. Cameras nowadays (assuming you're not using a Red Monstro or something insanely pricey and bleeding edge) sit around 700 nits anyway, so you won't have more details in the highlights using a purely logarithmic color curve anyway (even if they can go up to 10'000 nits) but you will get more details in the blacks as they start logarithmic and not linear. The "problem" with this is that this way you're literally getting as many frequencies as the sensor can get and many sensors do a fairly BAD JOB with dark/black areas, so the details you're gonna get are very likely going to be full of combing in the chroma and noise in the luma anyway. Nonetheless, if you ask me, from a purely mathematical point of view, it's better to shoot in a purely logarithmic color curve. That being said, I would suggest you to shoot in HLG for your use case and perform a linear transformation with a matrix like a custom made LUT in Davinci or any NLE to bring it to Linear BT709 SDR so that you can pick which frequencies are gonna be kept and which other are gonna be cut or kneed down to SDR. I also suggest you to archive the HDR HLG BT2020 master for future use. The reason why I'm suggesting you this is that *some* cameras that were produced in the middle of a "transition" only allow purely logarithmic curves with the BT709 matrix but not BT2020. This historically has a reason: when PQ and HLG were not yet invented camera manufacturers were already able to shoot with more than 6 stops 100 nits, so they invented their own color curves: Sony invented Slog, Canon invented Clog, Arri invented LogC, Fuji invented FLog etc and they did that with the only color matrix that was available at the time: BT709. Some manufacturers were quick to catch up and allow to record in Log with BT2020 (BT2100) as well, but some others were not, therefore:

- If your camera allows you to shoot log in BT2020 (BT2100), then do that and then apply a matrix to go to BT709 SDR and archive the original master. The SDR you're gonna get is gonna be far better than anything you can get by shooting in SDR directly from the camera.

- If your camera allows you ONLY to shoot log in BT709 but allows you to use BT2020 for HLG, then shoot in HLG BT2020 HDR, archive the original master and then use a matrix to bring it to Linear BT709 SDR. The SDR you're gonna get is STILL gonna be far better than anything you can get by shooting in SDR directly from the camera, at least in the highlights.


p.s also remember that purely logarithmic curves are NOT backwards compatible with SDR.


I hope this cleared the confusion a bit and helped you pick the correct profile.

Cheers,
Frank



Side note: Oh, I almost forgot: HDRTools from Jean Philippe currently only supports PQ and HLG but not other curves, so using it for Slog1, Slog2, Slog3, Clog1, Clog2, Clog3, LogC, FLog, DLog etc is out of question.
Side note 2: A personal appeal to Jean Philippe -> if you wanna introduce support for them as well and you need to work out how they work along with samples I can provide them to you via FTP, just like we've done back when we were testing BT2446 in 2019. You know, I'm always here, buddy ;)

[jpsdr]

So much................?????????????? Fleeing (with Benny Hill music)...:p

[videoh]

Fleeing (with Benny Hill music) Me too.

[Hotte]

:thanks: very much, Frank. This makes me learn a lot.

(To be honest: I had seen you being active in this thread and I hoped, that you would contribute with your excellent knowledge and the ability to explain complicated things from scratch in a very understandable manner).

I would like to discuss this a bit further, but I saw too late this is the developer's chatroom (it is where the search engine sent me to), and maybe continuation is inappropriate here. If so, just say and I am happy to move elsewhere. However I think that a broader community benefits a lot more from sophisticated discussions here if some basics are being explained from time to time.

I`ve got a Panasonic G9 which can (latest firmware) record 10-bit YUV422 BT.2020 with HLG-profile available. My Edius-X shows the footage as BT.2020 explicitly with "BT.2100 HLG"-gamma curve. It is HEVC-coded. There is an ultraflat VLOG-L profile available for 100$ extra. They say it increases DR by 2 f-stops to 12 f-stops.

Question 1: As far as I understood you, there is no need to go for a VLOG-L profile: If I am doing it right, I will be able to squeeze all available shadow-DR out of the HLG master too if needed. Correct ?

Question 2: The "Primary Color Correction"-Filter in Edius allows me to apply LUTs and conversions. Leaving Input and Output in HLG results in this well described flat, colorless look on my calibrated Rec709-Monitor. The question is, why then HLG in some way is told to be "Rec709-compatible" if it looks so unnaturally washed out ?

Question 3: If I apply BT2020 / BT.2100 Input and Rec709 output in the PCC-Filter it looks much better. But still the greens and reds are not fully convincing and it is difficult to get the midtones into some harmonic balance. I`d say you achieve acceptable but not really good results. I thought BT2020/BT.2100 as well as rec709 are standard profiles so why are colors and tones getting messed up like this ?

Question 4: Is HDRTools something to improve this or do I have to take the hard track of trying out numerous LUTs until I find sth convincing ?

[FranceBB]

:thanks: very much, Frank.


No worries, I like to reply on Doom9 (when I can contribute), it's like my favorite hobby (which is far more useful than posting memes on Facebook as some other people do instead xD).




I would like to discuss this a bit further, but I saw too late this is the developer's chatroom (it is where the search engine sent me to), and maybe continuation is inappropriate here.


I don't think Jean Philippe minds if we stick here. After all it is HDR Tools related, so...
But the final decision is his since it's technically his topic and his plugin xD




Question 1: As far as I understood you, there is no need to go for a VLOG-L profile: If I am doing it right, I will be able to squeeze all available shadow-DR out of the HLG master too if needed. Correct ?


V-Log would probably be better in preserving blacks. Simply put, if you take a look at the waveform in a V-Log file, you're gonna see it exactly in the middle.
This is V-Log plotted in the graph against Stop/IRE:

https://i.imgur.com/6JbaNpK.png

As you can see, it starts "high" and it slowly goes all the way up.
It is a truly logarithmic curve which means that blacks are "high" as they start "high" and hence are preserved:

https://i.imgur.com/0oAiKp0.png

HLG is not a truly logarithmic curve, it starts like a normal curve and then slowly becomes logarithmic.
What this means is that the blacks are just like you would see them in a normal SDR recording, but the more it goes up, the closer it gets to be a Logarithmic curve.
Take a look at the graph:

https://i.imgur.com/uBdkzIF.png

There are two things you should notice:

1) the blue curve (HLG) starts much lower compared to V-Log
2) it goes up much faster than V-Log

The reason for the first point (1) is that HLG starts lower 'cause it's an Hybrid Log Gamma, so it's not logarithmic, hence blacks are not "pushed up" and hence not preserved. The reason for the second point (2) is that by doing this "trick" to be hybrid and not truly logarithmic it doesn't have as many nits as a truly logarithmic curve like V-Log, Slog, Clog, PQ, have and it peaks at 1000 nits max.

They say it increases DR by 2 f-stops to 12 f-stops.


That would be correct, a truly logarithmic curve is better than an hybrid curve like HLG, so the additional stops you get are almost definitely details in the blacks. Technically you would get more in the whites too, but the camera sensor can't record more than 1000 nits anyway so you won't get any benefit in the whites from using V-Log compared to HLG.



The question is, why then HLG in some way is told to be "Rec709-compatible" if it looks so boringly washed out ?


It's not.
This is a very common mistake: when people say "HLG is backwards compatible with SDR monitors" they clearly mean BT2020 SDR monitors, but most people associate SDR with BT709. You see, SDR has nothing to do with color matrices: BT601, BT709, BT2020 can all be SDR 100 nits. Standard Dynamic Range basically means that what you get only has a color matrix with the same gamma and primaries, so no fancy color curve. An SDR file can be an SD BT601 100 nits, an SD BT709 100 nits or a BT2020 100 nits. So, when you get an HLG BT2020 stream, it is backwards compatible with TVs that understand just the color matrix but not the curve and they're gonna ignore the curve. The result will look ok-ish 'cause blacks are just fine 'cause it starts linear just like you would have with a normal BT2020 SDR stream, but as it goes up it becomes logarithmic. This is a compromise that broadcaster decided to adopt 'cause this way with just one channel you can cover 4K BT2020 SDR TVs and 4K BT2020 HDR TVs. You couldn't do that with truly logarithmic curves like PQ 'cause the result would be washed out as levels wouldn't match in a BT2020 SDR TV as blacks would start high. By being "hybrid", though, HLG peaks at 1000 nits and can't go beyond. On top of that, you don't get any additional details in the blacks 'cause they're effectively just like an SDR stream.


So, the HLG compromise is simple: the closer you get to HDR, the worse it looks on SDR BT2020 compatible monitors, the closer you get to SDR BT2020, the less dynamic range you have on those with an HDR compatible TV.
Since HLG is not a purely logarithmic curve, its lower part resembles the one of an SDR curve like the Linear BT2020nc SDR 100 nits, but as it gets to the higher values (whites) it resembles more and more a logarithmic curve and therefore introduces dynamic range, hence it doesn't offer many details in the blacks, but it does offer them in the mid-white and therefore it avoids the sky to be clipped out etc. As I said, the problem is that the TVs that don't understand any color curve, but do understand BT2020 are not gonna interpret those things, hence displaying the lower values correctly but not the highlights. Of course, this is intended and it's what the BBC made it for, in fact we've been airing like this for years now.

Here's a comparison of an HDR HLG BT2020nc 600 nits shot with a Sony A7 III:

On a TV which interprets both the color matrix (BT2020nc) and the color curve (HLG) and displays it as intended:
https://i.imgur.com/T7I2bRK.png

On an SDR TV 100 nits which does interpret the color matrix (BT2020nc) but totally ignores the color curve (the "not so bad" fallback as intended by the BBC specs):
https://i.imgur.com/W9TViA6.png

On an SDR TV 100 nits which has no clue about both the color matrix and the color curve and ignores them both like your BT709 display (totally wrong display, not acceptable, never should happen):
https://i.imgur.com/sXNb8CI.png


The first image is what we see in our monitors and it's how the whole content is graded and how it should be visualized by people at home. It's HDR, it has a dynamic range of 600 nits.
The second image is what people at home who have an old 4K TV which doesn't support HDR are gonna see. Ignoring the color curve and just interpreting the matrix isn't so bad and it's what the BBC guys were thinking about. The fact that the color curve is ignored makes the image look ok in the lower values/blacks ('cause the HLG is essentially like the Linear BT2020 SDR on the lower part) and "dull"/greyish in the higher values/whites ('cause as it goes up it embraces its logarithmic part).
Lastly, we have something that should never happen and that it's not part of the specs as not only it ignores the color curve, but it wrongly translates the values of the BT2020 so it's totally wrong. This is what happens if you try to watch a BT2020nc HLG stream on a BT709 aware only monitor (so don't do it).


Question 3: If I apply BT2020 / BT.2100 Input and Rec709 output in the PCC-Filter it looks much better. But still the greens and reds are not fully convincing and it is difficult to get the midtones into some harmonic balance. I`d say you achieve acceptable but not really good results. I thought BT2020/BT.2100 as well as rec709 are standard profiles so why are colors and tones getting messed up like this ?


I don't know what that filter is doing, so I can't say, but yes, the conversion is the reason why you see "funny" results.


Question 4: Is HDRTools something to improve this or do I have to go the hard track of trying out numerous LUTs until I find sth convincing ?

Definitely. You can use HDRTools to go from HLG BT2020 HDR to Linear BT709 SDR 100 nits in several different ways and it has several different parameters.

There are three different tonemappers: Hable, Mobius and Reinhard + the official BT2446_C implementation to go from HLG to BT709 SDR. I'm not gonna get into which one you should pick and why and what are the pros and cons of each tonemapping algorithm as it has been discussed several times here and also 'cause my reply is already long enough I think xD

Still, this is a start:


ConvertYUVtoXYZ(Color=0, HDRmode=2)
ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.1)

ConvertXYZtoYUV(pColor=0)



That conversion takes care of the color matrix + color curve conversion, while this one is just the color matrix without the color curve:

ConvertYUVtoXYZ(Color=1)
ConvertXYZtoYUV(pColor=1)

it's useful to see how people at home who have an SDR monitor would see the BT2020 HLG stream ignoring the color curve. Anyway, for what you have to do, try with the first script I posted and read the documentation and go on from there. Jean Philippe beautifully described what he has done in the filter and you can see it explained in detail in a PDF file that you can find in his repository. ;)

[Hotte]

Frank, for a first reaction...

This is an excellent description of the matter and a most precious contribution for alle HDR-Newbies.

It will take me some time to study this in detail and experiment And I might be coming back at a later stage with some need for advice.

Just one quick thought upfront: After what you've said I have a first impression that HLG is not the ideal profile for its compromises and limited DR in the low lights and VLOG-L could be worth while having a closer look into. This is an important question, because workflow optimization could be much different. My basic understanding is that VLOG-L conserves all the DR better and could be graded and matrixed into whatever target is needed. But then I'd need a VLOG-L => SDR 709 and a VLOG-L => HLG 2020/2100 conversion dependant of where I want to watch the content. Is this also sth HDRTools could be helpful for (if it's in the Docs "Yes" is the short answer needed and I am going to check this out for myself).

But some people say: Don't shoot VLOG-L all the time (sunny days...). It demands a minimum of ISO400 (more noise as standard ISO200), DR is not needed... But permanent change of profiles would make the workflow extremely difficult. Never heard this about HLG. So how is that to be taken ?

And what I have not yet quite unterstood is: Will I use a wide color gamut display oder projector to its full capabilities if I grade say a VLOG-L master to HLG2020/2100 later ? The emphasis here is on larger colorspace, because I do not know which matrix I end up with VLOG-L (haven't bought it yet) and if it is, say, 709 the matrix may not be extendable without significant loss in post ?

For my workflow it is a bit difficult to use HDRTools as converter because it would have to be done before the source file arrives in the NLE. But that could may be done someway with mounted avisynth... I will be trying it out.

[FranceBB]

This is an excellent description of the matter and a most precious contribution for alle HDR-Newbies.


Thanks. :) I hope it's gonna help other people too. :D



VLOG-L conserves all the DR better and could be graded and matrixed into whatever target is needed. But then I'd need a VLOG-L => SDR 709 and a VLOG-L => HLG 2020/2100 conversion dependant of where I want to watch the content.


That's exactly right. From V-Log you can go both to BT2020 HLG HDR and to Linear BT709 SDR. Here at Sky but also in other professional productions we always shoot log and then grade to other version like PQ for UHD-BD, HLG for linear UHD channels and BT709 SDR for FULL HD channels.
This movie I worked on: https://www.imdb.com/title/tt11462526/ was shot in Clog3 and then graded to HDR, although limited to 400 nits due to the camera limitations I faced with Canon at the time.



Is this also sth HDRTools could be helpful for?


Sadly no, HDRTools only supports HLG and PQ but no other curve... :(



But some people say: Don't shoot VLOG-L all the time (sunny days...). It demands a minimum of ISO400 (more noise as standard ISO200), DR is not needed...


Pff! You can shoot log all the time, totally. Also, if it's a sunny day, you're gonna have the clouds in the sky if you shoot log, however you won't if you shoot like in BT709.
There's NOTHING stopping you from shooting log in this day and age.
About the noise part, that's sadly true, though. Log will introduce more noise all the time, but that's normal and for the reason I explained before. ;)




Never heard this about HLG. So how is that to be taken ?


There will be noise in HLG as well, although much less 'cause the added noise is generally in the blacks and HLG is like an SDR at the beginning (in the blacks) so you won't get noise there which is the most visible and noticeable one. You'll still get more noise in the mid-highlights but the sensors generally do a very good job there, so you won't notice.



Will I use a wide color gamut display oder projector to its full capabilities if I grade say a VLOG-L master to HLG2020/2100 later ? The emphasis here is on larger colorspace, because I do not know which matrix I end up with VLOG-L (haven't bought it yet) and if it is, say, 709 the matrix may not be extendable without significant loss in post ?


Who knows...
As I said, some manufacturers allow people to shoot log with BT2020, so if that's the case you're gonna be fine grading to BT2020 HLG later on and/or to BT709 SDR now. If, for historical reason, it's still with BT709, then you're fuc... ehm no I can't write it in a public forum, let's say you're "screwed" eheheheh
Jokes aside, you could still go to BT2020 HLG, but you would just make use of the color curve nits but not of the wider color gamut. In that case I would suggest you to use HLG BT2020 instead as using V-Log with BT709 wouldn't be worth it. Ask the manufacturer, they'll tell you, hopefully.

[Hotte]

...I would suggest you to use HLG BT2020 instead as using V-Log with BT709 wouldn't be worth it.

One of those precious suggestions that really help to take off from sbdy who really knows what he's talking about.

For now I've got enough information. Will develop my workflow further and might be coming back at a later stage. Thanks!

[DTL]

"There's NOTHING stopping you from shooting log in this day and age."

Actually the whole playing around HDR and more badly HDR shooting for SDR output is not any everyday shooting idea. HDR more looks like 'poor people' exposition fool-proof shooting but getting every time the worst result in color and tone that the only matter for true art shooting. If we look at the curves of long used by pro photographers reversable film curves we will find the standard transfer curve for decates of old pro landscape and other pro shooting. And it really not High-DR process but Low-DR process with not compressing but expanding the nice low DR to standard DR. Typical pro-standard for landscapes Fuji Velvia film have about 6 f-stops of full DR and specially bended transfer curve with expanding DR at some points to 2..3 times of linear input.

If someone will shoot log (v-log) all the time with low bits it will cause the position of the nice to pick range in several bit encoding and result will be degraded by quantization noise and camera noise. To have headroom for HDR highlights camera put really valueable levels to low and noisy levels. The color degraded.

So HDR capture mostly toy for cheap broadcast live TV where the real scene lighting can not be controlled or selected for art purposes and hardware non-clipped highlights saves some work of live broadcast aperture-control engineers and camera knee-control engine (if auto-knee enabled). For serious art work with pro-controlled lighting and very limited DR for really important subjects of the scene using HDR/V-log capture will wastes most of bits in recording and produce low signal to quantization noise result. Because pro-controlled lighted scene have very low DR like 2..3 f-stops for most important subjects.

Keeping the natural DR or more badly compressing natural DR may be boring to viewer - you see this even without paying for process. In compressed DR the viewer lost even natural textures because the contrast sensitivity of viewer is limited and log-based. So to show to the viewer more details of natural subjects we need to expand DR. And expanding DR with pro processing may be new and interesting to viewer because it have natural logariphmic eyesense and can not see subtle levels of lighting changes well. Using log/v-log encoding of pro-lit scene with very low DR and intended for expanding DR at processing and displaying will leave only several bits for encoding the required levels of brigtness and may cause to lost subtle brightness details (due to more high quantization noise).

"But some people say: Don't shoot VLOG-L all the time "

If your scene is pro-lit and the lighting artist of good educated and understand what he doing and shooting operator is artist (not just camera pointing guy to there director says to point) you need to measure the actual scene linear DR and use the camera shooting mode to put this DR to as many recored useful bits as possible. For camera histogram it may mean not only 'Exposure To The Right' but more important 'Exposure to Fill Full Histogram'.

Also the release footage _displaying_ format may be HDR because of DR-expanding technique at post processing described above and _shooting_ format may be SDR to capture as many light levels of low DR scene as possible without degrading by quantization noise and later with comparession (if used).

HDR/log (compressing-DR) shooting is mainly when your lighting director do not understand what he is doing, scene illumination is bad and non-controllable and the content and lighting of the scene contrast is awfully high, you can not re-shoot the scene in better lighting and better arrangement of contrast items like live one-time event, etc. HDR/log shooting tool will let to show badly lit and bad contrast scene not as bad as with SDR shooting but will damage high-art low-DR and low-contrast scene with quantization noise and lost of fine texture details, and so on.

May be in short form: Camera's DR must follow scene's DR and typical high-art scenes are of low DR and designed for future expansion of DR for displaying.

[FranceBB]

HDR/log (compressing-DR) shooting is mainly when [...] scene illumination is [...] non-controllable and the content and lighting of the scene contrast is awfully high, you can not re-shoot the scene in better lighting and better arrangement of contrast items like live one-time event, etc.


Which covers 99.9% of the events I deal with, as I work mainly on sports. You know, you can't tell a player to take a free kick twice 'cause I didn't like the lighting xD
Besides, sporting events are played outside and you have the direct light coming from the sun to the pitch and it might change out of the blue due to a cloud or whatever. It would be a nightmare...



HDR/log shooting tool will let to show badly lit and bad contrast scene not as bad as with SDR shooting but will damage high-art low-DR and low-contrast scene with quantization noise and lost of fine texture details, and so on.


Of course compressing a waveform in the middle like it happens in this case and pushing the low values high means that we're gonna use the very same bits to code "more" values which is gonna lead to quantization noise, indeed, granted, but log footages are generally easier to encode so that the difference is gonna be attenuated.




May be in short form: Camera's DR must follow scene's DR and typical high-art scenes are of low DR and designed for future expansion of DR for displaying.

I know what you mean here and indeed it's a tough case in the sense that his final result is always gonna be SDR, so, in a controlled environment with controlled lights and if he can take several takes at a scene, he might (just might) be better off with SDR, but honestly, it's summer 2021, if it was me I would love to have all my shots future-proof and I wouldn't really feel comfortable into expanding DR from SDR even in a controlled environment 'cause if one day I really want to display those, I won't be able to and I'm gonna regret it forever 'cause the camera was able to and I choose not to, you know what I mean...?

[jpsdr]

New version, see first post.

[Atlantis]

First thank you for your tools. I use it for HDR to SDR and it gives the best results that I have seen. I try to go simple. I use this

ConvertYUVtoXYZ(Color=0, HDRMode=0, OOTF=false, OutputMode=2)
ConverXYZ_BT2446_C_HDRtoSDR(PQMode=true, Lhdr=10000, Lsdr=100.0, pColor=0)
ConvertXYZtoYUV(Color=2, pColor=0, OOTF=false, OutputMode=2)
ConvertBits(10)

It looks great. However I need a little extra tweak. I want to add a little higher contrast. A little deeper blacks. How can I achieve this?

[jpsdr]

Sorry, i don't know... You have to eventualy check the BT2446 spec (R-REP-BT.2446-2019-PDF-E p17-20, figure 10 p20), check the curves/parameters (the pct_xxx parameters) and eventualy whiteshift and "play" with them.
I know i've made the filter, but i don't use them.
I've done these as basic tool set for trying to do a neural network HDRtoSDR converter, but unnfortunately without sucess.
I've implemented the common know HDRtoSDR method, but i'm not using them...

[FranceBB]

I've done these as basic tool set for trying to do a neural network HDRtoSDR converter, but unnfortunately without sucess.

but there's room for improvement and I'll take the chance to write my request.

Since all the tonemapping algorithms are already there, what if:

1) A user-defined lookahead is introduced

2) A threshold parameter is introduced

2) A bool is introduced so that the user can choose "static" true or false. If true, then it behaves as it has been behaving so far, if false, then the user can specify the two parameters above.

What those parameters do is simple: take a bunch of frames into consideration, analyze the peak luminance of these frames and pick the best possible values for the tonemapping for that scene. The threshold parameter would define how different luma should be for the filter to consider that being a "scene change".


Quoting myself from some time ago:


Hi there,
in MPV tonemapping is applied using Hable, Mobius or Reinhard on a scene by scene basis by using the HDR Compute Peak, the HDR Peak Decay Rate and the HDR Scene Threshold.

Those three parameters are what are really doing the trick to the very good SDR representation of MPV:

--hdr-compute-peak=<auto|yes|no>
Compute the HDR peak and frame average brightness per-frame instead of relying on tagged metadata. These values are averaged over local regions as well as over several frames to prevent the value from jittering around too much. This option basically gives you dynamic, per-scene tone mapping.


--hdr-peak-decay-rate=<1.0..1000.0>
The decay rate used for the HDR peak detection algorithm (default: 100.0). This is only relevant when --hdr-compute-peak is enabled. Higher values make the peak decay more slowly, leading to more stable values at the cost of more "eye adaptation"-like effects (although this is mitigated somewhat by --hdr-scene-threshold). A value of 1.0 (the lowest possible) disables all averaging, meaning each frame's value is used directly as measured, but doing this is not recommended for "noisy" sources since it may lead to excessive flicker. (In signal theory terms, this controls the time constant "tau" of an IIR low pass filter)


--hdr-scene-threshold-low=<0.0..100.0>, --hdr-scene-threshold-high=<0.0..100.0>
The lower and upper thresholds (in dB) for a brightness difference to be considered a scene change (default: 5.5 low, 10.0 high). This is only relevant when --hdr-compute-peak is enabled. Normally, small fluctuations in the frame brightness are compensated for by the peak averaging mechanism, but for large jumps in the brightness this can result in the frame remaining too bright or too dark for up to several seconds, depending on the value of --hdr-peak-decay-rate.



I know that tonemapping is something we've been talking about a LOT in the last... well... 5 years (or more) and that there are different open source tonemapping implementation both in FFMpeg and in Avisynth like HDRTools and DGTonemap and I've been using HDRTools inside Avisynth for quite some time but what I wonder is: is there a way to tweak it to make it act on a scene by scene basis like in MPV? And is there a way to set how many frames have to be used for the scene-by-scene detection and a threshold to tweak how much change there has to be for a "scene change" to be considered as such?
In other words, can a dynamic scene by scene tonemapping be done in Avisynth?

[jpsdr]

I'll try to remember this for the days i'll be going back to work on new filters.
I've made, using 3 movies where i know SDR/HDR masters where done on the same time (this way i'm almost sure both use the same source film material and also maybe the same... i don't know how to translate... scan/digitalise film source material), 3 "database" with HDR <-> SDR information in XYZ format (get HDR pixel from HDR 4k BluRay, and get the same pixel on the SDR Bluray), try to "brutaly" feed them on neural network, but... need to be less "brutal" to be able to do somethig with them...
This was around 2 years ago, didn't work on it since, but, probably one day i'll try to work again on it.

[joearmstrong]

For optimal results I convert my videos to YV24 16-bit first using Avsresize ("YUV444P16").
In the description of HDRTools it says that converting to YV24 is done "quick" and for perfect results a resampler is recommended.
At the end of my script I convert to YV12 with dithered output. Are my settings correct or can you do the YUV conversion even better?

My example script:
Loadplugin("%startup_dir%\Apps\Plugins\AVS\avsresize.dll")
z_ConvertFormat(pixel_type="YUV444P16",dither_type="none")
ConvertYUVtoXYZ(Color=0,HDRMode=0,OOTF=false,OutputMode=2,threads=1)
ConverXYZ_BT2446_C_HDRtoSDR(PQMode=true,Lhdr=50000.0,Lsdr=100.0,pColor=0,pct_ref=0.6,pct_ip=0.6,pct_wp=1.0,pct_sdr_skin=1.0,pct_hdr_skin=0.44,threads=1)
ConvertXYZtoYUV(Color=2,pColor=0,OOTF=false,threads=1)
z_ConvertFormat(pixel_type="YV12",dither_type="ordered")

[jpsdr]

No, i can't do better YV24<->YV12 convertion, for this, your settings are correct. But... Is there any reason you limits threads to 1 ? It will be the slowest.

[joearmstrong]

There is no specific reason - I just used the HDRTools-Settings from Ripbot264.

[FranceBB]

There is no specific reason - I just used the HDRTools-Settings from Ripbot264.

If you don't use Prefetch(), you can easily use the Jean Philippe threadpool which will be faster, so you can avoid to specify threads=1 and get a speed performance increase with no downside (other than an increase in RAM probably).

[joearmstrong]

Thanks for your tips! I wish you all a happy new year.

[MeteorRain]

https://github.com/msg7086-forks/HDRTools

Ported to Linux64. Not sure if anyone did anything similar, but I recently need to work with Rec2020->Rec709 and wanted this to work under Linux, so I patched the code to make it work.

Deleted all thread pool stuff, so it's single threaded now.

Deleted 32-bit ASM.

Didn't touch much on the ASM code. I compiled it with UASM but you can find your favorite MASM tool if there's any other.

I had no idea what I was doing on the MS_ABI, but examining the stack space, all arguments are 1 QW deeper than they should be, so I let everything +8 and it magically works.

Most code paths are untested, so use at your own risk. PR welcome but I'm not going to spend time and efforts on it. Feel free to take the patch and polish on your own.

EDIT: Had issues on AVX2 CPUs, PR sent to fix a CPU flag condition check.

[jpsdr]

Finaly sooner than expected, new version, see first post.

[joearmstrong]

Is it possible to implement ACES tonemapping in HDRTools? I see no way to use this curve in Avisynth even though it's now the current standard in tonemapping.

[jpsdr]

Maybe, but don't expect things soon. I'll have to search for informations about it.

[joearmstrong]

I found the best overview of all the tonemappers with further links here:
https://github.com/tizian/tonemapper

[Dogway]

Is it possible to implement ACES tonemapping in HDRTools? I see no way to use this curve in Avisynth even though it's now the current standard in tonemapping.

I plan to add it to TransformsPack, but not only the TM but the whole implementation. It's on the roadmap since the beginning since I already made a few tonemappers for Substance Painter a few years ago. The ACES I did then was the Stephen Hill version which is far from the real deal.

[jpsdr]

The ACES function transfert is "simple" (using only one parameters, not alls), also it seems to be XYZ specific, so i'll do only an XYZ version, and i can use allready existing code function as base template, so a "maybe" with a good chance... ;)

[jpsdr]

Begin slowly to add ACES...

[jpsdr]

New version, see first post.
WARNING : Not tested ! (But chances of working properly are good)
Will not be able to test before a week at least, but make a release anyway in case others want to test.

[StainlessS]

Confidence is hi,
Love that !

Rock On Tommy, :)

[-QfG-]

Hi there,

using for testing this values:

Loadplugin("%startup_dir%\Apps\Plugins\AVS\HDRTools\hdrtools.dll")
ConvertYUVtoXYZ(Color=1, HDRMode=0)
ConvertXYZ_ACES_HDRtoSDR()
ConvertXYZtoYUV(pColor=1)
ConvertBits(8)

Colors are awesome, but the Brightness of the picture is to low. Any Ideas which Parameters i must set by ConvertXYZ_ACES_HDRtoSDR() ?

[jpsdr]

Everyone will say "Why the developper is asking this question....????" :scared:

But in fact, i've programmed the standard tonemap, but never used them (don't remember how i tested them).

Then, when i wanted to see, out of curiosity the results of ACES (and no, no testing yet, don't have access to PC with AVX & AVX2, which trigged me an issue never seen, but it's another story) result was black...
Tested Mobius, Hable, black... (well not totaly black, but very black).

Maybe the HDR -> SDR tonemap is not working as i thought...? :confused:

I thought, with my functions, if your input is standard PQ Blu-Ray, you do :

ConvertYUVtoXYZ(Color=0)
tonemapXYZ(pColor=0)
ConvertXYZtoYUV(pColor=0,Color=2)

For me, the XYZ input of a tonemap is HDR (so basicaly BT2100), and output is SDR (so basicaly BT709).

Except, as i said, when i tested, is was, almost, black.
So, what these tonemaps finaly realy do ?

Still don't give up on my first idea (maybe use Neural Network different way) of trying something with the 3 databases i've build.

[FranceBB]

But in fact, i've programmed the standard tonemap, but never used them (don't remember how i tested them).


I do, we tested them on TV Series, Sports and other movies as I was desperately eager to get something to work with HDR stuff back in 2018. We didn't have LUT support (no cube) and we had literally nothing to handle PQ contents in Avisynth which made me worry a lot, so I was like really really pushing to get this. I even remember some of the assets: one was Riviera Season 1 coming from a DNxHQX 12bit in HLG, one was Avengers Infinity War coming from an H.265 in PQ at 1000 nits 10bit, one was Naples vs Juventus, a Serie A game in XAVC Intra Class 300 in HLG 10bit and then there were some other assets I tested 'em on which I forgot.
There were some I had to make the tests on myself and we couldn't share, while we worked together on some others with the FTP (I saturated it that time and you sent me a reply shortly after hahahahha) and also some other of them were disclosed publicly here on Doom9 as far as screenshots are concerned: https://forum.doom9.org/showthread.php?t=175488&page=4






Tested Mobius, Hable, black... (well not totaly black, but very black).

Maybe the HDR -> SDR tonemap is not working as i thought...? :confused:


It is, but as per the tests I made in May 2018, I always suggest:


#BT2020 HLG HDR to BT709 SDR
ConvertBits(16)
ConvertYUVtoXYZ(Color=0, HDRmode=2)
ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.1)
ConvertXYZtoYUV(pColor=0)

and

#BT2100 HDR PQ to BT709 SDR
ConvertBits(16)
ConvertYUVtoXYZ(Color=0, OutputMode=1, HDRMode=0, fullrange=false)
ConvertXYZ_Reinhard_HDRtoSDR(exposure_X=2.5, contrast_X=0.9)
ConvertXYZtoYUV(pColor=0)




Except, as i said, when i tested, is was, almost, black.


Yeah, that's "normal".



So, what these tonemaps finaly realy do ?


They do what they are supposed to do, so what you wrote in the PDF, but only when properly tweaked. ;)


Still don't give up on my first idea (maybe use Neural Network different way) of trying something with the 3 databases i've build.

More than a neural network, please please please make them scene-aware, that would be amazing.
My idea from here is still valid: https://forum.doom9.org/showthread.php?t=182823

[jpsdr]

one was Avengers Infinity War coming from an H.265 in PQ at 1000 nits 10bit
Oh... That's one of the 3 movies i used to build my HDR -> SDR database for my NeuralNetwork first idea.

If i do something for now, it will unfortunately not be as complex as scene aware. Also, the database i've build don't have this kind of information... And even, maybe, database is a big word for it.

If i do something for now, it will be some kind of tonemap function in 1-exp(x) form, maybe keep the NN part to compute the parameters. If you're curious of the curves i have, i can send you excel sheets.

[joearmstrong]

Hi there,

using for testing this values:



Colors are awesome, but the Brightness of the picture is to low. Any Ideas which Parameters i must set by ConvertXYZ_ACES_HDRtoSDR() ?


I have the same question. With more brightness the converted video would look very good. I'm missing the exposure parameter from the other tonemapping curves.
Thank you for implementing ACES tonemap!

[jpsdr]

Didn't see any kind of exposure parameter in ACES description.
If it's just a multiply coeff factor, for now, you can use in that case ConvertXYZ_Scale_HDRtoSDR just before to multiply the input, or after to multiply the output.
It's true indeed that the others tonemap have exposure parameter.
Maybe i'll think to add it for a next release...

[jpsdr]

New version, see first post, still not fully tested... :D

Edit :
More tests done now, seems Ok, i'll update the complete plugins package soon.

[jpsdr]

Here some result HDR (X axis) -> SDR (Y axis) curves extracted from my database.

It's clearly an a.(1-exp(-b.x)) formula.
http://jpsdr.free.fr/XBMC/HDR_SDR1.jpg
http://jpsdr.free.fr/XBMC/HDR_SDR2.jpg

[joearmstrong]

I'm trying the new exposure parameters in ACES. Anybody found good settings to recommend?

[FranceBB]

Sorry for not replying earlier, but I've been extremely busy at work.
Anyway, yeah, I'd love to see the results you get and the values of the training you're doing on the experimental neural network.

[jpsdr]

I may have not been clear enough... For now, i only built a database, and have an excel of curves like in post #194.
Throw data to a NN just "like that" didn't work. Also, i'm far far far from being a expert in NN, i'm more a noob in the field.
The NN idea may work, but with a little thinking... Like, as said in post #194, curves seems to follow a specific formula pattern, maybe the NN can find the a and b values.
Anyway, i'll PM you.

For now, i'm wondering what to do with this. As it can be seen in post #194, for the same HDR value, there is several SDR results.
For now, i have to think how to use this database, and for exemple, figure out if there is some kind of reproduce things. The Xsdr is strongly from Xhdr, but as for same Xhdr there is several Xsdr, others things may have, small effect, like Y and Z. Maybe something like, according formula of #194 : Xsdr=a(Yhdr,Zhdr,...?).(1-exp(-b(Yhdr,Zhdr,...?).Xhdr)
All curves show something interesting : "big belly" with 2 convergence points. 0, obvious, and at "the end". This is interesting property, help and reduce parameters exploration.

[DTL]

If you have a time can you look in this difference:

LoadPlugin("plugins_JPSDR.dll")
LoadPlugin("avsresize.dll")
LoadPlugin("fmtcavs.dll")

Function Convert422ToRGB24mon_lin_x4_HLG(clip c)
{
uc=UToY(c)
vc=VToY(c)
uc=UserDefined2ResizeMT(uc,src_left=0.001,uc.width, uc.height, b=105, c=0)
vc=UserDefined2ResizeMT(vc,src_left=0.001,vc.width, vc.height, b=105, c=0)
uc=SincLin2ResizeMT(uc, src_left=0, uc.width*2, uc.height, taps=8)
vc=SincLin2ResizeMT(vc, src_left=0, vc.width*2, vc.height, taps=8)
yuv444=CombinePlanes(c, uc, vc, planes="YUV", source_planes="YYY", pixel_type="YUV444P10")

# jpsdr_plugin
rgb_lin=ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2)
rgb_lin=RGBAdjust(rgb_lin,r=3,g=3,b=3)

# avsresize
#rgb_lin=z_convertformat(yuv444,pixel_type="rgbps", colorspace_op="2020ncl:std-b67:2020:l=>rgb:linear:2020:l", nominal_luminance=800)
#rgb_lin=RGBAdjust(rgb_lin,r=3,g=3,b=3)

#fmtconv
# rgb_lin=fmtc_matrix(yuv444,mat="RGB", mats="2020",fulls=false, fulld=false)
# rgb_lin=fmtc_transfer(rgb_lin,transs="hlg",transd="linear",bits=32,fulls=false, fulld=false)
# rgb_lin=RGBAdjust(rgb_lin,r=3,g=3,b=3)

rgb_lin=SincLin2ResizeMT(rgb_lin,rgb_lin.width*4, rgb_lin.height*4, taps=16)
#return z_ConvertFormat(rgb_lin, pixel_type="rgbp8", colorspace_op="rgb:linear:709:l=>rgb:709:709:l").ConvertToRGB()
#return fmtc_transfer(rgb_lin,transs="linear",transd="709",fulls=false, fulld=false).ConvertBits(8,fulls=true,fulld=false).ConvertToRGB()
return ConvertLinearRGBToYUV(rgb_lin,Color=2).ConvertToRGB24(matrix="PC.709")
}
plY=Blankclip(width = 10, height = 1, pixel_type="Y10").Expr("sx 0 == 509 sx 1 == 509 sx 2 == 509 sx 3 == 509 sx 4 == 509 sx 5 = 498 sx 6 = 446 sx 7 == 361 sx 8 == 292 276 ? ? ? ? ? ? ? ? ?")
plU=Blankclip(width = 5, height = 1, pixel_type="Y10").Expr("sx 0 == 270 sx 1 == 263 sx 2 == 330 sx 3 == 761 754 ? ? ? ?")
plV=Blankclip(width = 5, height = 1, pixel_type="Y10").Expr("sx 0 == 203 sx 1 == 224 sx 2 == 271 sx 3 == 832 821 ? ? ? ?")
tr=CombinePlanes(plY, plU, plV, "YUV", "YYY", pixel_type="YUV422P10")

sust_l=Blankclip(width = 4, height = 1, pixel_type="YUV422P10").Expr("sx 0 == 509 sx 1 == 509 sx 2 == 509 509 ? ? ?","sx 0 == 270 270 ?","sx 0 == 203 203 ?").PointResize(50,1)
sust_r=Blankclip(width = 4, height = 1, pixel_type="YUV422P10").Expr("sx 0 == 276 sx 1 == 276 sx 2 == 276 276 ? ? ?","sx 0 == 754 754 ?","sx 0 == 821 821 ?").PointResize(52,1)

StackHorizontal(sust_l,tr)
StackHorizontal(last,sust_r)
PointResize(width, 100)
# source finishes here, 75% green-magenta transient HLG 10bit 4:2:2

Convert422ToRGB24mon_lin_x4_HLG()


Results are:
https://i3.imageban.ru/out/2022/06/28/0ab51aeedca3f93c64ac6f487b474a3d.png

So the functions from plugins_JPSDR_v3_3_0 return a bit lower RGB levels and the shape of transient also a bit different from 2 others HDR converting plugins. What may cause this difference ?
I tried to calculate levels values at https://forum.doom9.org/showthread.php?p=1952556#post1952556

"my advice would bet to test these 2 settings : Color=0,HDRMode=2,HLGLw=800 and Color=0,HDRMode=1,HLGLw=800"

With HDRMode=2 the HLGLw adustment change nothig.
With HDRMode=1 the mostly closely looking green is with HLGLw=11000 (or may be about 11500) but RB channels at magenta become unbalanced.

"try with OOTF=false, it will ouput the displayed linear data.
But in that case you have to put also OOTF=false in ConvertLinearRGBToYUV."

This returns very dark image with HDRMode=1.
With

rgb_lin=ConvertYUVtoLinearRGB(yuv444,Color=0,HDRMode=2, OOTF=false, HLGLw=350)
return ConvertLinearRGBToYUV(rgb_lin,Color=2, OOTF=false).ConvertToRGB24(matrix="PC.709")

The sustained RGB looks mostly close to other plugins and green-magenta transient may be looks the mostly nicely (with much less dark stripe and ringing). So this HDR magic is so magic. How to connect this magic value HLGLw=350 with 'typical Lw' I still do not know.
https://i6.imageban.ru/out/2022/06/28/760f514040c5803cb74efedf501230ce.png

As I test in MS Paint with cut-paste region - in RGB24 the fmtconv and avsresize output looks perfectly matches between each other at both green and magenta sustained levels and transient between colours.

[jpsdr]

Don't have time unfortunately right now.
But, if you want, you can do :
If both others plugins produce same result, mix the YUV -> RGB and RGB -> YUV (for exemple z_ConvertFormat doing YUV -> RVB followed by fmtc_transfer doing RVB->YUV), check if output is still the same. If yes, revert the mix.
If both mix are Ok, now do the same mix with my plugin. This will eventualy allow to see (if it's the case) if difference is only on one part.
If, for exemple, ConvertYUVtoLinearRGB followed by z_ConvertFormat or fmtc_transfer produces the same result than a complete conversion with another plugin, it meens it's ConvertLinearRGBToYUV which behaves differently.

[jpsdr]

New version, see first post.