Sdr -> hdr?

[asarian]

Topaz Video AI is currently doing (or attempting to do) SDR -> HDR. I know Donald A. Graft (from DGDecNV), made an HDR2SDR tool once. Has anyone tried the reverse yet, from SDR -> HDR? Would make for a very interesting tool. :)

[FranceBB]

I don't really understand what all the hype around AI is for.
Instead of having a non deterministic output generated by some kind of artificial stupidity (got the joke?) I would rather have the SDR to HDR conversion done in a precise, deterministic, reliable way, which is how everyone has been doing it for years now.
Not to quote myself, but here https://forum.doom9.org/showthread.php?t=176091 you can find the LUTs I made free and open source for everyone and among those you're gonna find the options to go from BT709 to both BT2020 HLG HDR and BT2020 PQ HDR. You can do that either by using the LinearTransformation() function I wrote like this:


#BT709 SDR to BT2020 HLG HDR 400 nits
LinearTransformation(Input="Linear_BT709", Output="BT2020_HLG")



#BT709 SDR to BT2020 PQ HDR 400 nits
LinearTransformation(Input="Linear_BT709", Output="BT2100_PQ")


or by using Cube() and pointing to the .cube yourself like this:


#BT709 SDR to BT2020 HLG HDR 400 nits
ConvertBits(16)
ConvertToPlanarRGB()
Cube("C:\Programmi\AviSynth+\LUTs\BT709_to_HLG.cube")
Converttoyuv420()



#BT709 SDR to BT2020 PQ HDR 400 nits
ConvertBits(16)
ConvertToPlanarRGB()
Cube("C:\Programmi\AviSynth+\LUTs\BT709_to_PQ.cube")
Converttoyuv420()


This is a BT709 SDR FULL HD source upscaled to UHD and converted to BT2020 PQ using the method mentioned above (the graph is produced by VideoTek in Avisynth https://forum.doom9.org/showthread.php?p=1832846):

https://i.imgur.com/ibkcItW.png

The advantage of having the LUTs available is that you can use them directly in other software as well, like another frameserver instead of Avisynth (i.e VapourSynth if you want), or FFmpeg or even inside your own preferred NLE like AVID Media Composer, Adobe Premiere, Davinci Resolve etc.

If you wanna use something different and stay within Avisynth, there's also avsresize that can perform the same kind of conversion, albeit in a different way.


#BT709 SDR to HDR HLG
z_ConvertFormat(pixel_type="YUV422P16", colorspace_op="709:709:709:limited=>2020ncl:std-b67:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")


#BT709 SDR to HDR PQ
z_ConvertFormat(pixel_type="YUV422P16", colorspace_op="709:709:709:limited=>2020:st2084:2020:limited", resample_filter_uv="spline64", dither_type="error_diffusion")

however I tend to prefer my own implementation (but hey, I'm biased, obviously eheheheheh).

The reason behind this is simple: use legacy SDR footage when you're editing new HDR one. For instance, suppose you're shooting a documentary and you have a bunch of interviews that were shot in UHD HLG with a Sony A7 IV in HLG and then you have a bunch of legacy FULL HD BT709 footage that you need to add about the person you're interviewing, then you would upscale them and convert them to BT2020 HLG so that you can use them within the same timeline to create the final output file. Sure, those legacy footage will be "fake" HDR, but at least you're gonna be able to use them. :)

[asarian]

I don't really understand what all the hype around AI is for.
Instead of having a non deterministic output generated by some kind of artificial stupidity (got the joke?) I would rather have the SDR to HDR conversion done in a precise, deterministic, reliable way, which is how everyone has been doing it for years now.
Not to quote myself, but here https://forum.doom9.org/showthread.php?t=176091 you can find the LUTs I made free and open source for everyone and among those you're gonna find the options to go from BT709 to both BT2020 HLG HDR and BT2020 PQ HDR. You can do that either by using the LinearTransformation() function I wrote like this:.

Brother, this is pure gold you provided here! :thanks: I will start using this.

[Z2697]

One advantage of using zimg: you can specify the "nomial luminance" of you sdr clip to any valid value you want ;)

[asarian]

Instead of having a non deterministic output generated by some kind of artificial stupidity (got the joke?) I would rather have the SDR to HDR conversion done in a precise, deterministic, reliable way, which is how everyone has been doing it for years now

For the record, Topaz Video Ai is not using A.i. for SDR2HDR, but for upscaling rather. Their attempts at SDR2HDR is just the latest bandwagon they jumped on, as an added bonus.

The advantage of having the LUTs available is that you can use them directly in other software as well, like another frameserver instead of Avisynth (i.e VapourSynth if you want), or FFmpeg or even inside your own preferred

Yeah, I think I'm going to try this is VapourSynth first, as my AviSynth is rather old (but 64-bit nonetheless).

[Z2697]

Then that's typically more stupid than artificial stupidity
The common situation of these "AI" video software companies is they know basically nothing about "real" video technologies, they just use neuro net as like cyber fortune teller and the rest is let's f*ckin YOLO it.
Remember the time topaz used mpeg4 (not part 10!) to output video? (for quite a few major versions) I remember.

[Sagittaire]

Then that's typically more stupid than artificial stupidity
The common situation of these "AI" video software companies is they know basically nothing about "real" video technonoly, they just use neuro net as like cyber fortune teller and the rest is let's f*ckin YOLO it.
Remember the time topaz used mpeg4 (not part 10!) to output video? (for quite a few major versions) I remember.

Well I don't think that. DLSS for Nvidia or FSR for AMD produce really good results.

Never forget that these technology don't upresize 2D matrice to 2D matrice but 3D matrice to 3D matrice with vector reconstruction and prediction (AI is for that).

[Z2697]

Well I don't think that. DLSS for Nvidia or FSR for AMD produce really good results.

Never forget that these technology don't upresize 2D matrice to 2D matrice but 3D matrice to 3D matrice with vector reconstruction and prediction (AI is for that).

Well first of all I think they are chip company not "AI video software company".
Secondly I don't mean that the result of AI is bad, I mean the rest part is bad.
Finally DLSS and FSR are not for video (the early pure spatial version may be "useable" but far from good), I think them as "advanced TAA" with smart temporal reference technique powered by AI or hand tuned algorithm.

[asarian]

Then that's typically more stupid than artificial stupidity
The common situation of these "AI" video software companies is they know basically nothing about "real" video technonoly, they just use neuro net as like cyber fortune teller and the rest is let's f*ckin YOLO it.
Remember the time topaz used mpeg4 (not part 10!) to output video? (for quite a few major versions) I remember.

Sadly, I have to agree. These companies should be forbidden to use the term "A.i." Nothing about it uses intelligence (real or otherwise). A violin player with a bow glued to her face, that's not 'I' in any universe. It's all neural net learning, sans true understanding of the real world. It's, at the end of the day, just statistics. Now, all A.i uses statistics, but not all statistics are A.i.

But we digress. :)

[Sagittaire]

Sadly, I have to agree. These companies should be forbidden to use the term "A.i." Nothing about it uses intelligence (real or otherwise). A violin player with a bow glued to her face, that's not 'I' in any universe. It's all neural net learning, sans true understanding of the real world. It's, at the end of the day, just statistics. Now, all A.i uses statistics, but not all statistics are A.i.

But we digress. :)

Well ChatGPT is just prediction with databases to use good words near initial words. And that work very well.

AI for video is just a prediction to choose the good pixel near other pixel with databases. And that work very well for video too.

[asarian]

Well ChatGPT is just prediction with databases to use good words near initial words. And that work very well.

AI for video is just a prediction to choose the good pixel near other pixel with databases. And that work very well for video too.

It works well for grass and such; for (far-away) faces with things in front of them, not so much. :)

[asarian]

LinearTransformation(Input="Linear_BT709", Output="BT2020_HLG")


So, dumb question no doubt, but how can I use in VapourSynth?

I tried

core.avs.LoadPlugin ("C:/VS/plugins/LinearTransformation/LinearTransformation.avsi")

That gives me all kinds of errors.

[FranceBB]

For VapourSynth you have to check the instructions here: https://forum.doom9.org/showthread.php?t=173361
The plugin you need is vscube which is here: https://github.com/sekrit-twc/timecube

I would write an example script, but I don't personally use VapourSynth, so I wouldn't be able to test it or verify if it works or not. In that thread, though, plenty of people have been able to use the .cube files via vscube, so hopefully you're gonna find your answer there. I'm sure there are plenty of people who know how to use VapourSynth who are better placed to help you than me.

All you need to do is to go to RGBPS and then use the cube function to point to either BT709_to_HLG.cube or BT709_to_PQ.cube according to whether you wanna go to HDR HLG or HDR PQ.

[asarian]

For VapourSynth you have to check the instructions here: https://forum.doom9.org/showthread.php?t=173361
The plugin you need is vscube which is here: https://github.com/sekrit-twc/timecube

I would write an example script, but I don't personally use VapourSynth, so I wouldn't be able to test it or verify if it works or not. In that thread, though, plenty of people have been able to use the .cube files via vscube, so hopefully you're gonna find your answer there. I'm sure there are plenty of people who know how to use VapourSynth who are better placed to help you than me.

Again, thank you kindly!

I reinstalled a new AviSynth+ (as mine has really old), tried

"LinearTransformation(Input="Linear_BT709", Output="BT2100_PQ")

But it gave me a script error:

Cube does not have a named argument 'interp'

But let me use VapourSynth again first. :)


EDIT: Just want to let you know, I got the vapoursyth version to work! Again, thank you!

[FranceBB]

Cube does not have a named argument 'interp'


by default I use interp=1 which is tetrahedral interpolation.
I wrote a paragraph in the AVS Cube section about Trilinear vs Tetrahedral interpolation and why the latter is better: http://avisynth.nl/index.php/AVSCube
Anyway, the "interp" parameter was added later in version 1.4 of Cube so if you update your version with the one you find in the wiki download link you're gonna be able to run it in Avisynth as well. :)


Just want to let you know, I got the vapoursyth version to work! Again, thank you!

Great. Happy to hear that :)

[asarian]

by default I use interp=1 which is tetrahedral interpolation.
I wrote a paragraph in the AVS Cube section about Trilinear vs Tetrahedral interpolation and why the latter is better: http://avisynth.nl/index.php/AVSCube
Anyway, the "interp" parameter was added later in version 1.4 of Cube so if you update your version with the one you find in the wiki download link you're gonna be able to run it in Avisynth as well. :)

Great. Happy to hear that :)

Got it. Updated to v1.4 (on the wiki), which conveniently is a backported vscube from VapourSynth, and now it even works in AviSynth. :)

[Leo 69]

In the latest Davinci Resolve Studio 19 you can do SDR to HDR conversion using RTX HDR.

[asarian]

In the latest Davinci Resolve Studio 19 you can do SDR to HDR conversion using RTX HDR.

I don't have thousand of dollars available for professional software like that. :) Instead I rely in the generosity of people like FranceBB, to make these LUTs available for free!

[Z2697]

If I'm not somehow creating false memories myself, there's method(s) to use "RTX HDR" in open source tools, don't need professional software.
However you do need a RTX card (maybe even a newer one 30- or 40-, I'm not sure), and "RTX HDR" uses "AI" and the result is, well let's say I don't appreciate that.
I'd even consider its addition to a professional software a questionable choice. It's not professional at all IMO. It may impress to the eyes at first glance but something's off if you look more closely.

[asarian]

If I'm not somehow creating false memories myself, there's method(s) to use "RTX HDR" in open source tools, don't need professional software.
However you do need a RTX card (maybe even a newer one 30- or 40-, I'm not sure), and "RTX HDR" uses "AI" and the result is, well let's say I don't appreciate that.
I'd even consider its addition to a professional software a questionable choice. It's not professional at all IMO. It may impress to the eyes at first glance but something's off if you look more closely.

I have an RTX 4090, so I'm good on that front.

As for A.i, yeah, I already made my position on that clear in this thread. :) In fact, I truly rather rely in people like FranceBB, who possess the mathematical wherewithal to actually know what they're doing. The best stuff (like x265 et al.) still comes from this forum (and I've been a member since 2005),

[cubicibo]

At the end of the day, what you're trying to do is generally not useful. There's just no "data" in your source to justify expanding it to HDR without creating ugly artifacts. In general, SDR content presented on UHD BD is delivered as SDR bt709 or SDR bt2020. Most SDR content presented in HDR bt2020 generally remain in SDR levels, with a very occasional leak in some direction of the gamut. This "leak" in HDR domain is can be the result of manual tuning by the video engineer/colourist, but often it comes solely from compression noise!

[Blue_MiSfit]

> SDR content presented on UHD BD is delivered as SDR bt709 or SDR bt2020

Is there any SDR BT. 2020 content distributed anywhere? On UHD BD or otherwise? I've never seen any.

[cubicibo]

JP BD of Evangelion 3.333 (https://www.evangelion.jp/3_0/bd_dvd.html) advertises itself as SDR bt2020.
映像の規格は、通常のテレビ放送用などで使われるSDR(スタンダードダイナミックレンジ（※１）)としていますが、その色域は、通常のHD放送規格のBT.709（※２）より広い、UHD規格であるBT.2020（※３）を使用致しました。
== (GTranslate) ==>
The video standard is SDR (Standard Dynamic Range (※1)), which is used for regular TV broadcasting, but the color gamut is BT.2020 (※3), a UHD standard that is wider than the regular HD broadcast standard BT.709 (※2).
The british Millenium Actress UHD BD is bt709 SDR.

[FranceBB]

Is there any SDR BT. 2020 content distributed anywhere?

Anything that ever aired on linear TV from 2015 to 2017. Then in 2018 HDR HLG arrived to save the day without screwing users over. Almost every sport event broadcasted in that timeframe was BT2020 SDR 100 nits.

[Blue_MiSfit]

...Really? Sports in this timeframe were not delivered in regular SDR BT. 709? Where? I've literally never heard this, even from a close friend who was working for a major regional sports network here in Los Angeles.

When you mean it was BT. 2020 SDR - do you mean perhaps some colors were allowed outside of BT. 709 gamut, or are you suggesting that the camera shading was actually using BT. 2020 primaries and pushed well outside of the 709 gamut for creative intent?? If so, was that actually signaled on the compressed live feed or was it masquerading as BT. 709 primaries?

[FranceBB]

Back then it was all really new and really entertaining. It wasn't just live sports but also things like documentaries etc. Cameras were all set to shoot with their proprietary gamut. For instance, Sony cameras were set to record in S-Gamut3 and were then converted to BT2020. For reference, BT709 covers 35% of CIE 1931, S-Gamut3 covers 53% and BT2020 covers 75%, so back then you had something better than BT709 (i.e S-Gamut3) converted to BT2020 which yield better results. When HDR HLG arrived, the same cameras were then used to produce HDR despite not being able to shoot directly in HLG by combining S-Gamut3 with Slog3 which is a totally logarithmic transfer and allowed people to output BT2020 HLG after going through an RGBPS 16bit roundtrip. It was all a bit hacky with many things that could go wrong, but it worked, at least in Europe.

[benwaggoner]

There's no way to do HDR from SDR that would approach the creative intent of a colorist doing the HDR. as there are many, many ways to do that conversion. And the sources used for the remastering are often ACES, DCP, or other > Rec.709 color volume, so there are more highlights to preserve.

Good ML software that can be given patience and lots of lookahead should be able to outperform what a TV can do in realtime, though.

[asarian]

There's no way to do HDR from SDR that would approach the creative intent of a colorist doing the HDR. as there are many, many ways to do that conversion. And the sources used for the remastering are often ACES, DCP, or other > Rec.709 color volume, so there are more highlights to preserve

Nevertheless, I am very happy with what FranceBB did with his LUTs. :)

[FranceBB]

Well, both the BT709 to BT2020 HLG and BT2020 PQ LUTs were made to target 400 nits which is a very good compromise when it comes to SDR converted to HDR.
Those were tested on a Sony BVM-HX310 reference monitor which peaks at 1000 nits and is connected to a Tektronix Waveform Monitor.
The original reason behind those LUTs was to make life easier for those who wanted to include legacy SDR contents into native HDR ones without having any issues.

First use case:

The first use case was documentaries in which you had interviews shot in Slog3 that could be graded to HLG of the person interviewed, of the surroundings etc and at the same time you had legacy footage of that person that you had to include. One such a documentary was "A proposito di Max" (https://www.imdb.com/title/tt11462526/?ref_=nm_flmg_knf_t_1) which even has scenes in BT601 SDR which were first converted to BT709 SDR and then to BT2020 HLG HDR. Another one was "Leo sulla cresta dell'onda" (https://www.imdb.com/title/tt27728055/?ref_=nm_flmg_knf_t_2) as it also included some clips recorded by Leonardo Fioravanti himself using his phone as he was traveling around the world and were clearly in BT709 SDR. Another one is "Ljubo" (https://www.imdb.com/title/tt31186288/?ref_=nm_flmg_knf_t_3) which is a documentary about the very famous tennis player Ljubicic which again included legacy footage.

Second use case:

Obviously documentaries are just one of the use cases of converting BT709 SDR to BT2020 HLG HDR or BT2020 PQ HDR, another one can be live feeds. For instance, take F1. The feed is delivered by Tata (the official producer) in BT2020 HLG HDR to all the licensors around the world and while some TVs just limit themselves to put the commentary on via a speaker boot, plenty of others go above and beyond to send their own journalists to follow the races across the world, interview pilots etc. Most often than not, those interviews are live and sent back in H.265 with a short delay via IP (i.e internet) using applications like LiveU or using satellites like Elpitel, but due to constraints those cameras are almost always BT709 SDR. So... what do you do? Well, you have the whole race in BT2020 HLG HDR at 1000 nits and then you have the interviews in BT709 SDR converted to BT2020 HLG HDR on the fly. Given the very limited nature of BT709 SDR and its 100 nits corresponding to around 6 stops in a camera (a modern HDR camera can record 15 stops) and the very challenging environment, you may end up with clipping in SDR, so the last thing you wanna do is bring that clipped out white sky from 100 nits to 1000 nits as it would burn your eyes (well, not literally, but it will give you a very unpleasant feeling). Hence, the 400 nits target in my LUTs which should accommodate for any challenging shot as a pure white at 400 nits is still very much tolerable.

Third use case:

The last scenario I wanna depict here is commercials. You see, there are plenty of linear channels and all of those broadcast at a certain native resolution. Nowadays, it's not uncommon to have the very same channel in SD (BT601 SDR), FULL HD (BT709 SDR) and UHD (which used to be BT2020 SDR and evolved into BT2020 HLG HDR for backwards compatibility). In those channels, everything must be consistent, not just the shows (i.e tv series, movies etc), but also all the commercials, so, just like FULL HD BT709 SDR commercials are downscaled to SD and converted to BT601 SDR, they're also upscaled to UHD and converted to BT2020 HLG HDR.

[benwaggoner]

Nevertheless, I am very happy with what FranceBB did with his LUTs. :)
Yeah. And starting with HLG gives a lot more information that good old 709.

[asarian]

Yeah. And starting with HLG gives a lot more information that good old 709.

Sarcsasm? Anyway, I see SDR2HDR like upscaling: you can't really squeeze more definition (pixels) out of the original than it came with; doesn't mean you cannot repair/interpolate things, and thus (sometimes drastically) improve picture quality. Same with SDR2HDR: FranceBB cannot apply highlights/accent the way a creative director would (we'd need some real A.i for that, like twenty years down the line). Still, the result is pretty impressive, and I prefer the result over plain SDR.

[RanmaCanada]

I don't have thousand of dollars available for professional software like that. :) Instead I rely in the generosity of people like FranceBB, to make these LUTs available for free!

I know this is really late, but Davinci Resolve Studio is $300 USD.

[benwaggoner]

JP BD of Evangelion 3.333 (https://www.evangelion.jp/3_0/bd_dvd.html) advertises itself as SDR bt2020.
This would be for just the 4K HDR Blu-ray Disc, not the OG BD disc. There's no support for anything beyond 8-bit Rec. 709 in the original BD.

SDR WCG kinda makes sense for remastered anime, which generally doesn't have a lot of contrast, but can be very colorful.

[benwaggoner]

Sarcsasm? Anyway, I see SDR2HDR like upscaling: you can't really squeeze more definition (pixels) out of the original than it came with; doesn't mean you cannot repair/interpolate things, and thus (sometimes drastically) improve picture quality. Same with SDR2HDR: FranceBB cannot apply highlights/accent the way a creative director would (we'd need some real A.i for that, like twenty years down the line). Still, the result is pretty impressive, and I prefer the result over plain SDR.
Technicolor has demoed some pretty good AI SDR to HDR conversion that's trained on how a bunch of great colorists did SDR to HDR remasters. Still not how a specific colorist might decide to regrade a specific title, lacking any sense of creative intent. But definitely better than a fixed LUT, and often better than algorithmic adaptive LUTs like ColorFront's. But of course, with a risk of doing something hallucinatory.

The best quality solution I've worked with that is extremely reliable is ColorFront's. Regularly superior to any fixed LUT.

[asarian]

I know this is really late, but Davinci Resolve Studio is $300 USD.

Yeah, I saw. Waiting for a possible Black Friday deal. :thanks:

[Leo 69]

Here's a tutorial (https://forum.videohelp.com/threads/416613-Convert-SDR-to-HDR-with-NVIDIA-AI-%28RTX-20xx-or-Above%29-at-No-Additional-Cost) on how to to SDR=>HDR conversions on NVIDIA cards (using only free software).

[benwaggoner]

Here's a tutorial (https://forum.videohelp.com/threads/416613-Convert-SDR-to-HDR-with-NVIDIA-AI-%28RTX-20xx-or-Above%29-at-No-Additional-Cost) on how to to SDR=>HDR conversions on NVIDIA cards (using only free software).
So it just uses the Nvidia TrueHDR algorithms? Are those good for natural images? There's not much info on how they work, and they sound to be game-focused, not TV/movie content.

[Z2697]

So it just uses the Nvidia TrueHDR algorithms? Are those good for natural images? There's not much info on how they work, and they sound to be game-focused, not TV/movie content.

They are both game and TV focused, but the result is not really great.