[DoVi_Scripts] Multi-Function Scripts for Dolby Vision processing and a lot more...

[sk2316]

Originally Posted by Kuler087

no, it cant

thank you

[smaiderman]

Hello.
You updated dovitools since the last time I used it, and I cant find "Convert RPU to XML" option.
Im trying to convert alien HDR to SDR

Summary:
Frames: 624
Profile: 8
DM version: 2 (CM v4.0)
Scene/shot count: 3
RPU mastering display: 0.0001/1000 nits
RPU content light level (L1): MaxCLL: 149.86 nits, MaxFALL: 10.05 nits
L6 metadata: Mastering display: 0.0001/1000 nits. MaxCLL: 335 nits, MaxFALL: 122 nits
L2 trims: 100 nits, 600 nits, 1000 nits
L8 trims: 100 nits
L9 MDP: BT2020
L5 offset: TOP=N.A., BOTTOM=N.A., LEFT=N.A., RIGHT=N.A.
1st Frame is a Scene Cut: YES (good)
Consecutive Scene Cuts: NO (good)
Base Layer (HDR10+P8): "min: 0.0050 cd/m2, max: 1000 cd/m2"(BT.2020), MaxCLL: 714 nits, MaxFALL: 123 nits
Resolution/FPS: 3840 x 1606 @ 24.000

Thnaks in advance

[wyup]

Originally Posted by Kuler087

I guess you meant 8-2-4 because 8-2-2 use libplacebo for the SDR tonemapping.

What algorithm does libplacebo library use for SDR tonemapping?
You can check some here:
https://libplacebo.org/options/
Interesting ones are bt2446a and st2094-40. These seem to accept actual and target display luminances for the OOTF adjustement. Ie: actual=MaxCLL or MaxMDL, target=100. I'm not sure if it stays in PQ or translates to BT.1886 (2.4) for either one.

It looks like st2094-10 takes HDR10+ metadata if available.

I think these algorithms, if tuned correctly might be interesting to compare to Dolby's own.

[Kuler087]

You updated dovitools since the last time I used it, and I cant find "Convert RPU to XML" option.

Workflow 1, then enter ''m''

[Kuler087]

Originally Posted by wyup

What algorithm does libplacebo library use for SDR tonemapping?
You can check some here:
https://libplacebo.org/options/
Interesting ones are bt2446a and st2094-40. These seem to accept actual and target display luminances for the OOTF adjustement. Ie: actual=MaxCLL or MaxMDL, target=100. I'm not sure if it stays in PQ or translates to BT.1886 (2.4) for either one.

It looks like st2094-10 takes HDR10+ metadata if available.

I think these algorithms, if tuned correctly might be interesting to compare to Dolby's own.

Right now it uses the default libplacebo settings but the current dovi_scripts version is using an older version because they changed things a bit in the latest libplacebo.

If you download the latest DS beta + tools, I've updated libplacebo and added more controls on the tone mapping. I've compared the other mode and i much prefer bt2390 + target 125nits honestly but the Resolve Dolby cmv4.0 with balanced tuning is still the best HDR to SDR tone mapping IMO.

:: choose if you want to use dynamic peak detection in 7-2 / 7-1 / 8-2-2 true or false (default= true) true or false : 'true' can be misleading if you compare two HDR sources in SDR. see info here: https://github.com/Asd-g/avslibplace...e#tone-mapping
set peak_detect=true
:: choose the SDR tone mapping function for 7-2 / 7-1 / 8-2-2 (default = bt2390) choices are: "clip", "st2094-40", "st2094-10", "bt2390", "bt2446a", "spline", "reinhard", "mobius", "hable", "gamma", "linear", "linearlight" see info here: https://github.com/Asd-g/avslibplace...e#tone-mapping
set tone_mapping_function=bt2390
:: choose the SDR tone mapping mode in 7-2 / 7-1 / 8-2-2 choices are: (default = perceptual") "perceptual", "softclip", "relative", "saturation", "absolute", "desaturate", "darken", "highlight", "linear" see info here: https://github.com/Asd-g/avslibplace...e#tone-mapping
set gamut_mapping_mode=perceptual
:: This helps block out annoying "sparkling" or "flickering" due to small variations in frame-to-frame brightness for the SDR tone mapping mode in 7-2 / 7-1 / 8-2-2 (default=20.0) see info here: https://github.com/Asd-g/avslibplace...e#tone-mapping
set smoothing_period=20.0
::Which percentile of the input image brightness histogram to consider as the true peak of the scene for the SDR tone mapping mode in 7-2 / 7-1 / 8-2-2 (default=99.995) see info here: https://github.com/Asd-g/avslibplace...e#tone-mapping
set percentile=99.995
:: choose if you want to use dolby vision metadata in 7-2 / 7-1 / 8-2-2 (default = false) true or false. Profile 5 input default to true see info here: https://github.com/Asd-g/avslibplace...e#tone-mapping
set use_dovi=false

@all I'm almost done debugging the new GUI and a new scripts version should be released soon

- (2-7) Added a new workflow that can batch export the DV RPU summary info to a text file (very fast, about 5-10sec per file)
- (9-8) Added a new workflow that simply convert 12-bit PQ values to Nits or Nits values to 12-bit PQ
- (8-6) Audio waveform plot can now downmix 7.1 to 5.1 (line 187, default = NO)
- (6-2) When you opt to CROP, added an option to convert to prores instead of reading a VERY slow avisynth script. Depending on your CPU, it may be faster but require more HDD space. ( line 165 default = NO)
- (6-2) Can now measure HDR PNG images.
- (7-1) (7-2) Can now display the frame maxcll/fall values but it makes the process two times slower and require python and opencv library: ''pip install opencv-python'' (line 143 , default =NO)
- (7-1) (7-2) (8-2-2) added more controls to the SDR tone mapping and changed the default 200nits target to 125nits. avs_libplacebo must be updated.
- (7-1) (7-2) Added an option to use custom filename(s) different than the input(s) (lines 98-99)
- (3-2) Improved HDR10plus and madVR to Dolby Vision metadata conversion. New tool needed: Thank's to doppingkoala !
- Added a timer to all the workflows.
- Added HDD output path free space check
- Introducing a GUI for DoVi_Scripts (not every workflow is supported and require python)
- Bugfix: (7-2) FEL vs FEL screenshots did not work
- Bugfix: (3-1) M2TS/TS/MP4 container input did not work

[smaiderman]

Originally Posted by Kuler087

Workflow 1, then enter ''m''

1) MODE.I= INJECT / EDIT / EXTRACT / INFO / VALIDATE
2) MODE.F= VERIFY SYNC / REMOVER / TRANSFER LEVELS
3) MODE.H= DoVi MAKER from HDR10 (Dolby Algo or MadVR or HDR10+)
4) MODE.7= DoVi Profile 7 Input (MKV/BDMV)
5) MODE.B= DoVi MKV Batch Muxer
6) MODE.P= Plotter (DoVi/HDR10/HLG/SDR)
7) MODE.S= Screenshots & Player
8) MODE.E= Encoders (video and audio)
9) MODE.M= MORE

Choice? [1,2,3,4,5,6,7,8,9]?1


==================================
- INJECTOR / EDITOR / EXTRACTOR -
==================================

-------------------------------------------------------------------------------------
-- This workflow inject / extract and can edit two or one input (press enter to skip the 2nd input)
-- Can batch process files: input(s) can be folder(s) with files
-- Input can be MKV/TS/M2TS/MP4/HEVC/RPU/XML/HDR10plus.json/P5/P8/P7/folder(s)
-- Can auto-Crop, Edit L5/L6/L9 and Resync DV or HDR10plus, validate metadata with metafier(XML)
-- Can use external json config for more rpu edition. json must be same filename/path. do not input as metadata
-- json input for the metadata input must be hdr10plus.
-- You can inject both DV and HDR10plus if the metadata inputs have the same path/filename as the BL.
-- For the batch injector, both folders must contain only the files to process and they must be in the same order
-------------------------------------------------------------------------------------

Drag and drop a file or a folder with files (MKV/TS/M2TS/HEVC/H265/RPU/XML/FOLDER) and press enter...
m

(OPTIONAL) Drag and drop a folder with Dynamic Metadata (RPU/XML/JSON/MKV/MP4/TS/M2TS/HEVC) and/or press enter...

[Kuler087]

[Gatorman3385]

Originally Posted by Kuler087

@all I'm almost done debugging the new GUI and a new scripts version should be released soon

Woohoo!

[smaiderman]

Originally Posted by Kuler087

Thank you.

I'm trying to follow this tutorial, but I'm lost.

https://www.youtube.com/watch?v=lM56zLpKDQ8&t=5s

In the minute 2:29, when you need to import the DV xml, I get this message

The edit rate of Dolby Vision metadata 
is not matched to timeline.

To get this XML, I used DoviTools "1-M"

This is the XML first lines, it says 24000 101 in the edit rate section, and my project and the file are in 24fps

<?xml version="1.0" encoding="UTF-8"?>
<DolbyLabsMDF>
  <xmlns>http://www.dolby.com/schemas/dvmd/4_0_2</xmlns>
  <Version>4.0.2</Version>
  <RevisionHistory>
    <Revision>
      <DateTime>2024-10-20T00:44:34Z</DateTime>
      <Author>Rainbaby</Author>
      <Software>dovi_meta</Software>
      <SoftwareVersion>1d32ca0</SoftwareVersion>
    </Revision>
  </RevisionHistory>
  <Outputs>
    <Output>
      <CompositionName>Timeline</CompositionName>
      <UniqueID>ac5018b5-c906-4937-8026-e25b9f7201a3</UniqueID>
      <NumberVideoTracks>1</NumberVideoTracks>
      <CanvasAspectRatio>1.7777778</CanvasAspectRatio>
      <ImageAspectRatio>1.7777778</ImageAspectRatio>
      <Video>
        <Track>
          <TrackName>V1</TrackName>
          <UniqueID>db77339f-2e48-41b7-bc6d-063c492bc1e3</UniqueID>
          <EditRate>24000 1001</EditRate>
          <ColorEncoding>
            <Primaries>
              <Red>0.708 0.292</Red>
              <Green>0.17 0.797</Green>
              <Blue>0.131 0.046</Blue>
            </Primaries>
            <WhitePoint>0.3127 0.329</WhitePoint>
            <PeakBrightness>10000</PeakBrightness>
            <MinimumBrightness>0</MinimumBrightness>
            <Encoding>pq</Encoding>
            <ColorSpace>rgb</ColorSpace>
            <SignalRange>computer</SignalRange>
          </ColorEncoding>
          <Level6 level="6">
            <MaxCLL>0</MaxCLL>
            <MaxFALL>0</MaxFALL>
          </Level6>
          <PluginNode>
            <DVGlobalData level="0">
              <MasteringDisplay>
                <ID>21</ID>
                <Name>1000-nits, BT.2020, ST.2084, Full</Name>
                <Primaries>
                  <Red>0.708 0.292</Red>
                  <Green>0.17 0.797</Green>
                  <Blue>0.131 0.046</Blue>
                </Primaries>
                <WhitePoint>0.3127 0.329</WhitePoint>
                <PeakBrightness>1000</PeakBrightness>
                <MinimumBrightness>0.0001</MinimumBrightness>
                <EOTF>pq</EOTF>
                <DiagonalSize>42</DiagonalSize>
              </MasteringDisplay>
              <TargetDisplay>
                <ID>1</ID>
                <Name>100-nits, BT.709, BT.1886, Full</Name>
                <Primaries>
                  <Red>0.64 0.33</Red>
                  <Green>0.3 0.6</Green>
                  <Blue>0.15 0.06</Blue>
                </Primaries>
                <WhitePoint>0.3127 0.329</WhitePoint>
                <PeakBrightness>100</PeakBrightness>
                <MinimumBrightness>0.005</MinimumBrightness>
                <EOTF>gamma_bt1886</EOTF>
                <DiagonalSize>42</DiagonalSize>
              </TargetDisplay>
              <TargetDisplay>
                <ID>28</ID>
                <Name>600-nits, BT.2020, ST.2084, Full</Name>
                <Primaries>
                  <Red>0.708 0.292</Red>
                  <Green>0.17 0.797</Green>
                  <Blue>0.131 0.046</Blue>
                </Primaries>
                <WhitePoint>0.3127 0.329</WhitePoint>
                <PeakBrightness>600</PeakBrightness>
                <MinimumBrightness>0</MinimumBrightness>
                <EOTF>pq</EOTF>
                <DiagonalSize>42</DiagonalSize>
              </TargetDisplay>
              <TargetDisplay>
                <ID>49</ID>
                <Name>1000-nits, BT.2020, ST.2084, Full</Name>
                <Primaries>
                  <Red>0.708 0.292</Red>
                  <Green>0.17 0.797</Green>
                  <Blue>0.131 0.046</Blue>
                </Primaries>
                <WhitePoint>0.3127 0.329</WhitePoint>
                <PeakBrightness>1000</PeakBrightness>
                <MinimumBrightness>0</MinimumBrightness>
                <EOTF>pq</EOTF>
                <DiagonalSize>42</DiagonalSize>
              </TargetDisplay>
            </DVGlobalData>
            <Level254 level="254">
              <DMMode>0</DMMode>
              <DMVersion>2</DMVersion>
              <CMVersion>4 1</CMVersion>
            </Level254>
          </PluginNode>
          <Shot>
            <UniqueID>04566c40-df2b-4758-acef-3715f66ecb83</UniqueID>
            <Record>
              <In>0</In>
              <Duration>24</Duration>
            </Record>
            <PluginNode>
              <DVDynamicData>
                <Level1 level="1">
                  <ImageCharacter>0 0.3001221 0.5492064</ImageCharacter>
                </Level1>
                <Level2 level="2">
                  <TID>1</TID>
                  <Trim>0 0 0 -0.00784724 -0.13330078 0.21885157 0 0.0048828125 0</Trim>
                </Level2>
                <Level2 level="2">
                  <TID>28</TID>
                  <Trim>0 0 0 0 0.0068359375 0.04646516 0 0.00048828125 0</Trim>
                </Level2>

[Kuler087]

yes because an rpu doesnt have a framerate and dovi_meta always assumes 23.976 so you have to edit the XML manually. Replace ''24000 1001'' with ''24 1'' or just ''24'' (I don't remember).
I'll script something to automate this in the next version.

[smaiderman]

Originally Posted by Kuler087

yes because an rpu doesnt have a framerate and dovi_meta always assumes 23.976 so you have to edit the XML manually. Replace ''24000 1001'' with ''24 1'' or just ''24'' (I don't remember).
I'll script something to automate this in the next version.

Yes, I was experimenting, and when creating a timeline in Davinci, if you force it to 23.976 it works.

Thanks again

I confirm it is "24"

[wyup]

Originally Posted by Kuler087

Right now it uses the default libplacebo settings but the current dovi_scripts version is using an older version because they changed things a bit in the latest libplacebo.

If you download the latest DS beta + tools, I've updated libplacebo and added more controls on the tone mapping. I've compared the other mode and i much prefer bt2390 + target 125nits honestly but the Resolve Dolby cmv4.0 with balanced tuning is still the best HDR to SDR tone mapping IMO.
[/IMG]

I've tried 8-2-2 in latest 3.0.3 beta & tools file but it doesn't offer algorithm and target nits. Wouldn't the more modern bt2446a and 100 nits as I think mpv player uses by default? Can't find where to edit in script. 125 nits produces darker image. Do you think it preserves highlights better? Bt2446a uses input target in nits which is very appropiate to shrink dynamic range effectively. I.e: input target=1,000 nits, output target=100 nits. I see it is 1,000 nits by default.

My GPU is Intel Graphics UHD 770, which is vulkan compatible 1.3.295.
Script outputs this error constantly:

vpy  [WARN]: Avisynth Compat: requested frame 0 not prefetched, using slow method
Validation failed: !params->renderable || fmt_caps & PL_FMT_CAP_RENDERABLE (../../../../../src_packages/libplacebo/src/gpu.c:234)
  for texture: ../../../../../src_packages/vs-placebo/src/tonemap.c:114

Output file produces a green tint image.

Dolby Balanced may produce better results, but libplacebo bt.2446 with input and output nits would be a contender, since it's a ITU standard.

POST_EDIT: I found by chance SETTINGS rename me to use external settings.bat file which has options for bt.2446 and input/output target nits. I see it has dynamic peak detection on by default, which uses dynamic tonamapping and brightness shifts in mpv player. Best result would be reading MaxCLL or MaxDML from metadata as input and disable peak detection. Even better, it could be manually offered as options on live script run.

[Kuler087]

You sure bt2390 and 125 nits is better than more modern bt2446a and 100 nits

well, tone mapping is subjective but yes, in the test I did, I preferred bt2390 in terms of overall brightness, contrast, shadow and HL details.

input target=1,000 nits, output target=100 nits. Otherwise, which input target does it take: default PQ max 10,000 nits?

No, libplacebo default is 1000 and I did not change that.

src_max, src_min, dst_max, dst_min
Source max/min and output max/min in nits (cd/m^2).
The source values can be derived from props if available.
Default: max = 1000 (HDR)/203 (SDR); min = 0.005 (HDR)/0.2023 (SDR)

Output file produces a green tint image.

no such issue on my end

Script outputs this error constantly:

yes its because I use an avisynth plugin in vapoursynth. It,s just a warning and you can ignore. The msg disappear after about 200 frame I think.

it doesn't let me choose algorithm and target nits

you have to edit the bat file with notepad++, the tone mapping section is around line 102
But I think in 8-2,I didn't do the change yet and the tone mapping mode is hardcoded to bt2390 in the latest beta but the target nits value is working for sure. In the final stable version, you will have the same control as in 7-2.
For testing, you should use 7-2 and avspmod. It's easier to switch from one setting to another.

[halls]

Originally Posted by Kuler087

yes because an rpu doesnt have a framerate and dovi_meta always assumes 23.976 so you have to edit the XML manually. Replace ''24000 1001'' with ''24 1'' or just ''24'' (I don't remember).
I'll script something to automate this in the next version.

Hi!

And how can i edit/modify the fps of the hdr10+ in the metadata file?

[Kuler087]

Originally Posted by halls

Hi!

And how can i edit/modify the fps of the hdr10+ in the metadata file?

You don't have to, AFAIK. HDR10plus and DV metadata don't have a framerate; just a frame order, and the video stream dictates the framerate.

So as long as you sync the metadata frames correctly with the video frames, you can inject the metadata from a 24fps source to a 23.976fps video for example.

[halls]

Thank You!

[wyup]

Thanks Kuler, you are the man :-). Great professional job with your scripts.

Tweaking HDR to SDR tonemapping algos is interesting to compare with previous SDR blu-ray editions in SDR environment such as average computer monitors. I believe the better brightness gradation of HDR should come across in SDR aswell as show your libplacebo stills (in some cases better than DV clorist trims as you say).

What do you think of the regular 100-nit trims extrapolation job to tv brightness? I think colorists should do a 400-nit or 600-nit grade by default since DV tvs are already HDR400, capable at least of 400 nits. No DV tv is 100 nits, which I think already is too much tonemapping to begin with, even with extrapolation.

Aside from custom-nit DV trim outs with your script, which is awesome, it would be nice to do custom HDR to HDR tonemapping aswell, i.e: bt2446a reading metadata. Resolve has CST with PQ to PQ tonemapping I/O nit settings, but plot measurements from Resolve HDR deliver metadata and your script plot don't seem to correlate after rendering. I think for example a 400-nit to 1,000-nit tonemapping would be useful to scale up tv brightness. Usually low HDR nit content does not scale to tv capacity. Although tonemapping always change the picture to some extent as it modifies the original.

On the other hand, I don't know how HDR10+ tools in Resolve can help to deliver a tonemapping output render instead of only the .json metadata.

[Kuler087]

Great professional job with your scripts.

thank's man

What do you think of the regular 100-nit trims extrapolation job to tv brightness? I

I never tested that but assuming all the trim metadata works on the player, I think it should be the same as delivering the 100nits trim from the master. The problem is that almost all the players handle the metadata differently and some trims metadata are ignored depending on the player:
EG:
- Chroma Weight is ignored by all the cmv2.9 devices.
- Some devices ignore any positive lift (offset metadata)
- All the devices I tried ignore ms_weight
- All the devices I tried ignore the vector fields (hue and saturation secondary trims metadata)

So in the end, it is much more accurate to deliver the 100nits from the HDR source since all the metadata works this way and the tone mapping is identical to the official bluray SDR release in 95% of the case.

One other strange behavior I noticed recently with the Dolby trims is in resolve when you adjust 100nits trim of the Chroma Weight and/or the trim saturation controls, the values (not even extreme values) are not extrapolated to the 600-1000nits trims. For all the other controls, any adjustment in 100nits get extrapolated to the 600-1000trim and you can see the difference on your mastering monitor.
What is strange is when you deliver that 100nits trim with CW and Saturation metadata, it has an obvious effect on my 800nits LG C2 TV. So this doesn't make much sense to me.

It's a similar story for the ms_weight control, if I make an adjustment, I see the difference on the mastering display but I don't see it in DV playback.

I think colorists should do a 400-nit or 600-nit grade by default since DV tvs are already HDR400, capable at least of 400 nits.

There's no 400 control in DV but there is a 300nits trim but I have never seen any colorist using it. I do use it sometimes in my personal projects.

No DV tv is 100 nits, which I think already is too much tonemapping to begin with, even with extrapolation.

They see the extrapolation to 600-1000 nits when they make the 100nits trim , so as long as they don't go stupid like in the movie ''the conversation'' which brighten the 600nits trim more than the master, the 100nits trim extrapolation can be very good. They can also blank the 600 and 1000nits trim which will make the 100 nits to
be ignored on any display with a target lower than 600 or 1000. This way the colorist can be very aggressive with the 100nits trim without affecting the other target.

it would be nice to do custom HDR to HDR tonemapping aswell,

the scripts or resolve can already deliver the 300-600-1000 or 2000nits trims but it is not for increasing the brightness. It always reduces the brightness from the master so if your master is 1000 nits and you deliver the 2000nits , there wont be any tone mapping and the output will be the same.
Increasing the master brightness requires grading and Dolby can only map down the brightness, not the other way around.

On the other hand, I don't know how HDR10+ tools in Resolve can help to deliver a tonemapping output render instead of only the .json metadata.

I never tried the HDR10+ tone mapping option, and there's no target so I have no idea what it does.

[Kuler087]

3.0.3 Released: https://github.com/R3S3t9999/DoVi_Sc...ases/tag/3.0.3

[wyup]

No standalone more MKV MP4 TS .bat downloads? It's more "transparent" to see their source code, even when you need additional and updated libraries.
While downloading "tools pack" or the whole DoVi.Scripts.7z is convenient, does it override user installed python scripts and plugins, as well as MadVR, LavFilters, Avisynth, Vapoursynth? I installed all these requirements but I don't know if those in DoVi.Scripts.7z do override my system user installed ones. In such case, is it still required to manually install these tools?