[DoVi_Scripts] Multi-Function Scripts for Dolby Vision processing and a lot more...

[Kuler087]

I wish, but no, sadly I have a 8.6 file and would like to convert to 8.1.

Any ideas on if plausible or not?

Are you sure it doesn't say: Profile 8.1, dvhe.08.06


how have you implemented the FEL brightness expansion detection?
The scripts detect brightness differences in the screenshots. It looks for any average brightness higher than 2nits or any peak brightness higher than 50nits.


I see that you usually show the DV L1 plot in your comparisons to put in evidence that FEL restores the original brightness but what about titles where the DV L1 is a bit "weird". For example, Jack Reacher (2012). For that title, it looks like the DV L1 MaxCLL is capped to 900 nits which is less than the maximum MaxCLL (1454 nits) of the HDR10 BL measured with madVR. In that case, comparing both values would lead to a verdict of no brightness expansion, right? Yet, the screenshot comparison clearly shows a brightness difference which I take this to mean that the actual peak luminance of BL+FEL is higher than maximum MaxCLL of BL and therefore of the maximum DV L1 MaxCLL as well.

Also, for Nosferatu (2024), you mention that FEL expands brightness. So, is this the case here of the HDR10 BL being tonemapped to 600 or 1000 nits? If so, which one? And why is there even a difference between BL and BL+FEL since the movie is so dark?
And on a related note, why does FEL sometimes darken the image, by that I mean, if there is no mapping, what would lead to the residual layer causing darkening? Is it due to a manual adjustment of the BL?
rollmayonnaise is offline Report Post Reply With Quote

If a title is graded to, for example, 2000 nits and you decide to map the base layer to 600 nits or 1000 nits, the algorithm will automatically and uniformly dim the image to match that target.
When the colorist does trim pass(es), those adjustments get baked into the base layer as well when encoding to P7. In some cases, the colorist raise the 100-nit SDR trim above the HDR master’s mid-range, which can result in the FEL appearing darker in comparison.

[prudentavocado]

Are you sure it doesn't say: Profile 8.1, dvhe.08.06



Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@Main
HDR format : Dolby Vision, Version 1.0, Profile 8.6, dvhe.08.06, BL+RPU, no metadata compression, Blu-ray compatible / SMPTE ST 2086, Version HDR10, HDR10 compatible
Codec ID : V_MPEGH/ISO/HEVC
Duration : 2 h 0 min
Bit rate : 17.9 Mb/s
Width : 3 840 pixels
Height : 1 600 pixels
Display aspect ratio : 2.40:1
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.122
Stream size : 15.1 GiB (99%)
Writing library : x265 unknown:[Linux][GCC 7.5.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=3 / numa-pools=8 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x1600 / interlace=0 / total-frames=0 / level-idc=51 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / repeat-headers / annexb / aud / hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=1 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / no-limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=-3:-3 / no-sao / no-sao-non-deblock / rd=4 / selective-sao=0 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=1.00 / psy-rdoq=3.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=cbr / bitrate=18000 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=2 / cplxblur=20.0 / qblur=0.5 / vbv-maxrate=18000 / vbv-bufsize=36000 / vbv-init=0.9 / min-vbv-fullness=50.0 / max-vbv-fullness=80.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / chromaloc-top=2 / chromaloc-bottom=2 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,001) / cll=1033,311 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.03 / no-opt-cu-delta-qp / no-aq-motion / hdr10 / hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / scenecut-aware-qp=0conformance-window-offsets / right=0 / bottom=0 / decoder-max-rate=0 / no-vbv-live-multi-pass
Language : English
Default : No
Forced : No
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primar : Display P3
Mastering display luminance : min: 0.0001 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 1033 cd/m2
Maximum Frame-Average Light Le : 311 cd/m2

Yes, and in searching my library actually have another movie like this too sadly.

[dkangel]

hello
just a little correction to do, in your bat file

::choose DEE 8-1-3 DDP Atmos mode: streaming or bluray (default = bluray) streaming max bitrate = 1024 @ 5.1ch /// Bluray max bitrate = 1536 @ 7.1ch + ac3 core
set DDP_DEE_mode=streaming

streaming is by default

not a big deal but a friend drive nuts because he was losing 7.1 :)

[en6ads]

Why do this for MEL instead of using handbrake?

Because Handbrake is a lossy transcoding tool that will encode video using encoders and settings specified by the user.

This script will losslessly inject a new RPU without touching the underlying video.

[Kuler087]

Yes, I know, and it was fixed in the beta a while ago. Thanks anyway.

hello
just a little correction to do, in your bat file

::choose DEE 8-1-3 DDP Atmos mode: streaming or bluray (default = bluray) streaming max bitrate = 1024 @ 5.1ch /// Bluray max bitrate = 1536 @ 7.1ch + ac3 core
set DDP_DEE_mode=streaming

streaming is by default

not a big deal but a friend drive nuts because he was losing 7.1 :)

[DMD]

Can anyone point me to instructions on how to convert a 8.6 video to 8.1? Or is this not possible? Tried searching this thread but didn't find anything promising.

8.6 giving me fits on Jellyfin via the LG app.
Did you accidentally use a base file such as HDR10+ to create the 8.6 profile?

[rollmayonnaise]

When the colorist does trim pass(es), those adjustments get baked into the base layer as well when encoding to P7. In some cases, the colorist raise the 100-nit SDR trim above the HDR master’s mid-range, which can result in the FEL appearing darker in comparison.

Thanks, I understand much better the reason behind the darkening now.

On a related note, is there a way to produce a madVR plot for the BL+FEL? Because for Nosferatu, I am trying to understand what was the original grade and what the BL was tonemapped to. It is not clear to me from the disc BL madVR and DV L1 plots what is happening that would explain FEL brightness expansion.

[Kuler087]

Thanks, I understand much better the reason behind the darkening now.

On a related note, is there a way to produce a madVR plot for the BL+FEL? Because for Nosferatu, I am trying to understand what was the original grade and what the BL was tonemapped to. It is not clear to me from the disc BL madVR and DV L1 plots what is happening that would explain FEL brightness expansion.

bake fel to prores in 8-2-3 and then measure it in 6-2

[prudentavocado]

Did you accidentally use a base file such as HDR10+ to create the 8.6 profile?

Nope :(

[MwenDavo]

Hi! I'm trying to understand the profile 7 to 8.1 converting process (8-2-1 workflow) so to replicate it on Linux. For now, this is what I have:

1- Obtain MDL, MaxCLL and MaxFAL (What are they used for)
2- Demux MKV to obtain HEVC stream
3- Demux stream with dovi_tool to get BL and EL
4- Index BL and EL
5- Extract RPU P7 from EL
6- Convert RPU P7 to P8 with dovi_tool
7- From RPU P8 get MaxCLL and MAxFALL (What's the difference between these and the ones from the video stream?)
8- From RPU P8 get MinPQ and MaxPQ and map them based on their values
9- Load the AVS script with:
DGDecodeNV
DoviBaker
Libplacebo (Is this being used? No command from the script seems to be using it)
AVSresize
BL with its index
EL with its index
10- Run the script through X265 to bake FEL (Could this be done directly with FFMPEG? If so, could there be support for hardware encoding on other platforms than NVenc?)

If I got any part wrong, please correct me.

[Kuler087]

7- From RPU P8 get MaxCLL and MAxFALL (What's the difference between these and the ones from the video stream?)
8- From RPU P8 get MinPQ and MaxPQ and map them based on their values

The script uses those values from the RPU because the FEL can alter scene brightness, which makes the original HDR10 metadata inaccurate. I recommend measuring your encode using workflow 6-2 to generate more accurate HDR10 metadata, then updating the encode’s metadata with workflow 9-9.

Libplacebo (Is this being used? No command from the script seems to be using it)
Used only when you tone map to SDR

(Could this be done directly with FFMPEG? If so, could there be support for hardware encoding on other platforms than NVenc?)
The script can use ffms2, but it will default to dgdecode when an NVIDIA GPU is detected. (line 315 set force_ffms2=NO)
Yes, ffmpeg can be used to encode the avs file, but dovi_scripts supports only x265.exe and nvenc.exe, and I have no plans to add support for any other indexing, encoding, or decoding options.

[MwenDavo]

The script uses those values from the RPU because the FEL can alter scene brightness, which makes the original HDR10 metadata inaccurate. I recommend measuring your encode using workflow 6-2 to generate more accurate HDR10 metadata, then updating the encode’s metadata with workflow 9-9.


Do you need to convert the RPU from P7 to P8 to get MaxCLL and MaxFALL? Or could you get them from the P7 RPU?

Also, in the case of MEL, will these values be equal to the original HDR10 metadata? If so, could you just use those?


The script can use ffms2, but it will default to dgdecode when an NVIDIA GPU is detected. (line 315 set force_ffms2=NO)
Yes, ffmpeg can be used to encode the avs file, but dovi_scripts supports only x265.exe and nvenc.exe, and I have no plans to add support for any other indexing, encoding, or decoding options.


Is NVIDIA needed for DGdecode? If not, is there any performance or quality difference between that and FFMS2?

[Kuler087]

Do you need to convert the RPU from P7 to P8 to get MaxCLL and MaxFALL? Or could you get them from the P7 RPU?
no

Also, in the case of MEL, will these values be equal to the original HDR10 metadata? If so, could you just use those?
the script use the original HDR10 metadata for MEL

Is NVIDIA needed for DGdecode?
yes

is there any performance or quality difference between that and FFMS2
no

[prudentavocado]

Is it possible to strip the dolby vision out of a p7 file completely and just end up technically with an HDR remux?

[Kuler087]

workflow 2-2

[MwenDavo]

Is cropping something that must be done to certain DV files? or is it optional?

[Kuler087]

Are you talking about the RPU or the video cropping or both?
Check the last part of the first post to understand how L5 and the cropping/letterbox should be handled.

[MwenDavo]

Are you talking about the RPU or the video cropping or both?
Check the last part of the first post to understand how L5 and the cropping/letterbox should be handled.

I'm talking about the RPU cropping, from the JSON you pass into dovi_tool to convert the RPU from P7 to P8. The script gets those through mediainfo and then uses them on the AVS script. Is that something necessary for the conversion? or is done only if the user says yes to the cropping question?

[Kuler087]

theres nothing special about the rpu crop function in dovi_tool, it just resets L5 to 0, and as I mentioned, everything you need to know about cropping and L5 is written in the first post.

[Kuler087]

3.1.1 released

https://github.com/R3S3t9999/DoVi_Scripts/releases

[DMD]

3.1.1 released

https://github.com/R3S3t9999/DoVi_Scripts/releases

Thank you for your great dedication to this new release, as always.

[coopzr]

Can you please let me know how the upgrade to cmv4.0 works on your latest beta (https://drive.google.com/file/d/128gq8aDUTKA_aT7SQsM9dkjA1EP1sosR/view)?

I thought that you couldn't replace the L1 from cmv2.9 to cmv4.0. I don't have a clue how this works.

When should this be used? (Only when the content doesn't exceed display capabilities, or always?)

Are there any downsides or weird behaviour?

[Kuler087]

It doesn’t replace the metadata; it simply generates a CMV 4.0 block with level 9.
L2 will then be ignored at playback, but if the content stays within your target TV brightness, it won’t matter. In that case, you avoid CMV 2.9 black crush and the 4000-nits MDL overdimming issues.

So this is essentially a quick workaround to fix CMV 2.9 playback, rather than generating new metadata from scratch. As you know, CoreELEC will most likely get this feature soon, which is even bette, nothing to process manually before watching a movie.

Workflow 1, skip 2nd input and select ''u''

[coopzr]

So essentially, it changes the RPU version to 4.0. It keeps the same old levels and adds the L9 level.

So it's essentially the old levels in a newer container.

L2 is ignored by the player because L2 is always ignored in cmv4.0.

Is that right?

[Kuler087]

Yes

L2 will be ignored if the TV supports cmv4.0

[coopzr]

In a normal cmv4.0 metadata with no mid tone offset trim, do video players use L3?

If they use it: what happens if it is removed?

If they don't use it: what is the point of the automatic data?

These questions are assuming no mid tone offset trim and only the automatic data generated by the algorithm.

[Kuler087]

Of course, the player uses L3 and L1 together.

I did many L3 test files here:
https://drive.google.com/drive/folders/1RItcDAp3vWG-MkJMMnWeKx45bQirTbcs?usp=drive_link

and many real scene comparisons here:
https://drive.google.com/drive/folders/1g5I-z_sJmVu-SAIPNiiSlcdMiy2ka0mf

[coopzr]

So that would imply that upgrading to cmv4 (which does not generate L3) would be inferior to generating it, right?

[Kuler087]

yes but not for content that is under your display peak brightness

[en6ads]

Workflow 1, skip 2nd input and select ''u''

Hmm. I don't see "u" as an option listed in workflow 1 (version 3.1.1). Is there a newer beta than 3.1.1?

===============
- INPUT INFO -
===============
Parsing RPU file...

Summary:
Frames: 552
Profile: 7 (FEL)
DM version: 1 (CM v2.9)
Scene/shot count: 2
RPU mastering display: 0.0001/1000 nits
RPU content light level (L1): MaxCLL: 147.74 nits, MaxFALL: 10.05 nits
L6 metadata: Mastering display: 0.0001/1000 nits. MaxCLL: 546 nits, MaxFALL: 129 nits
L5 offsets: top=0, bottom=0, left=440, right=440
L2 trims: 100 nits, 600 nits, 1000 nits
Base Layer (HDR10+P7): min: 0.0001 cd/m2, max: 1000 cd/m2(Display P3), MaxCLL: 1000nits, MaxFALL: 129nits
Resolution\FPS: 3840 x 2160 @ 23.976
Input: "U:\movies\Sunset Boulevard (1950)\Sunset Boulevard (1950) - 2160p Paramount 2025 USA-DV P7 FEL CM2.9-1000nit MDL.mkv"
Output Path Free Space: 2321gb

--------------------------------------------------------------------------------------------------
*You can select more than one value but no space between values (except: e/m/p/s/7/v/r/t).
*EG: You can select 569fp4 or 569dp4 but not 569p4fd, 569p4fr, 569p4ms, emsv, etc...

CHOICES: (default=X - check another file)
Check_New_File=X L5=5 L6=6 L9=9 Resync=R or D SceneCut_Edit=F Source_PQ=Q
PLOT=P Extract=E Check_Sync=S Metafier=M Transfer_Levels=T P7_to_P8=7 Remover=V No.DV.TM=N
--------------------------------------------------------------------------------------------------

I tried "u" anyway can it generates this, still a CM2.9 FEL. What am I missing?:

===============
- INPUT INFO -
===============
Parsing RPU file...

Summary:
Frames: 552
Profile: 7 (FEL)
DM version: 1 (CM v2.9)
Scene/shot count: 2
RPU mastering display: 0.0001/1000 nits
RPU content light level (L1): MaxCLL: 147.74 nits, MaxFALL: 10.05 nits
L6 metadata: Mastering display: 0.0001/1000 nits. MaxCLL: 546 nits, MaxFALL: 129 nits
L5 offsets: top=0, bottom=0, left=440, right=440
L2 trims: 100 nits, 600 nits, 1000 nits
Base Layer (HDR10+P7): min: 0.0001 cd/m2, max: 1000 cd/m2(Display P3), MaxCLL: 1000nits, MaxFALL: 129nits
Resolution\FPS: 3840 x 2160 @ 23.976
Input: "C:\video\DoVi.Scripts-3.1.1\Sunset Boulevard (1950) - 2160p Paramount 2025 USA-DV P7 FEL CM2.9-1000nit MDL_DV.mkv"
Output Path Free Space: 2265gb

--------------------------------------------------------------------------------------------------
*You can select more than one value but no space between values (except: e/m/p/s/7/v/r/t).
*EG: You can select 569fp4 or 569dp4 but not 569p4fd, 569p4fr, 569p4ms, emsv, etc...

CHOICES: (default=X - check another file)
Check_New_File=X L5=5 L6=6 L9=9 Resync=R or D SceneCut_Edit=F Source_PQ=Q
PLOT=P Extract=E Check_Sync=S Metafier=M Transfer_Levels=T P7_to_P8=7 Remover=V No.DV.TM=N
--------------------------------------------------------------------------------------------------

[Kuler087]

This new option was added to the beta yesterday. It seems every time I release a new stable, I get a new idea the next day...

[en6ads]

This new option was added to the beta yesterday. It seems every time I release a new stable, I get a new idea the next day...

Awesome I'll try the new beta. Thank you!

[dkangel]

Il y a du plagiat dans l'air !!!

https://github.com/MaitreAnonyme/Maitre-Dolby/releases/tag/2.0

[Kuler087]

héhéhé

[Enet47]

I've done a thread search on heere and checked the Git but I still don't understand where DEE.exe(?) or the DEE folder is or how to install it. Is it part of the much larger Tools package? Also, does the new FEL functionality mean that I no longer need to convert to 8.1?

[Kuler087]

I've done a thread search on heere and checked the Git but I still don't understand where DEE.exe(?) or the DEE folder is or how to install it.

DEE (https://professional.dolby.com/product/media-processing-and-delivery/dee--dolby-encoding-engine/) is a paidware, and you just have to put the whole DEE folder into the dovi_scripts Tools folder.


Also, does the new FEL functionality mean that I no longer need to convert to 8.1?

Not really because:
DEE does hevc re-encoding (lossy) and the P7 to P8 doesnt re-encode (lossless).

The FEL encoding workflow can encode any DV video, but it is mostly intended for 12bit mezzanine file (prores or jpeg 2000).

[Enet47]

DEE (https://professional.dolby.com/product/media-processing-and-delivery/dee--dolby-encoding-engine/) is a paidware, and you just have to put the whole DEE folder into the dovi_scripts Tools folder.




Not really because:
DEE does hevc re-encoding (lossy) and the P7 to P8 doesnt re-encode (lossless).

The FEL encoding workflow can encode any DV video, but it is mostly intended for 12bit mezzanine file (prores or jpeg 2000).
Thanks :)

[Kontrarian]

I can't get 3-1 to work for some reason. I tried using the latest beta .bat, I've tried updating the tools, changing the DIR DoVi_Scripts to DoVi.Scripts, etc. It always gets stuck at this line:

Frame 588 complete. Last frame: 0.534 seconds. ETA: 33:47:46.130

But the completed frame will be different.

Here's the full code from my last attempt: https://pastebin.com/LJdD6Pwe

[Kuler087]

I had those issues before on my Windows 11 PC, and I have no idea how it corrected itself. It seems it is related to the gpu drivers version, Windows update, and cm_analyze...

[ac777103]

just checking my understanding of the new CMV4 upgrade option via workflow 1. Are there any negative effects vs the vanilla 2.9 playback, or only potential benefits? I guess this will work equally for both MEL and FEL? Trying to think if there is a reason not to do this for all my 2.9 files.

[Kuler087]

It depends on how much the trim pass actually matters in the grade.

1- On a display with a target of 1000 nits or higher, trim passes are ignored when the RPU MDL is 1000 nits. (G3 G4 G5 G6 / C5 C6 etc...)

2- The content is graded (https://drive.google.com/drive/u/1/folders/154fBNllwOHL4Lckc7wDV8QKFJwFxnDt-)within your TV’s capabilities, so the trims won’t matter, even though some colorists stupidly make the trims brighter than the master, likely because they don’t properly monitor the 100-nit extrapolation to higher-brightness targets.

[ac777103]

workflow 1 now failing on FEL input, both 3.1.1 stable and latest beta, error "no module named cv2". It crashes on checking expanded brightness. I just updated tools folder but didn't fix it.

[Kuler087]

pip install opencv-python colour-science scikit-image matplotlib numpy colour opencv-python-headless PyQt5

[ac777103]

thanks for quick reply, working now.

[Kontrarian]

I had those issues before on my Windows 11 PC, and I have no idea how it corrected itself. It seems it is related to the gpu drivers version, Windows update, and cm_analyze...

Hmmm... well that sucks. I'm on a Windows 10 laptop with the latest NVIDIA driver for my GTX1060.
I hope it magically fixes itself sometime soon because right now I can only inject existing RPUs.
Maybe it's time for a new laptop... :)

[Kuler087]

You can use Resolve... It's much faster anyway (but less automated).

[coopzr]

I can't get 3-1 to work for some reason. I tried using the latest beta .bat, I've tried updating the tools, changing the DIR DoVi_Scripts to DoVi.Scripts, etc. It always gets stuck at this line:

Frame 588 complete. Last frame: 0.534 seconds. ETA: 33:47:46.130

But the completed frame will be different.

Here's the full code from my last attempt: https://pastebin.com/LJdD6Pwe

update cm_analyze. Fixed in v5.6.1
Fixed hanging bug when using newer CUDA cards.

P.s. RESET, maybe you can post a note in your GitHub release saying to update cm_analyze? I think this is the 3rd or 4th user to report this issue (myself included)

[dkangel]

hello
i've just try 3.1.1 version with a P7 FEL file and i get this message

Traceback (most recent call last):
File "X:\Utils\DoVi_Scripts\tools\Measure_HDR.py", line 1, in <module>
import cv2
ModuleNotFoundError: No module named 'cv2'

any idea ?

thanks

[Kuler087]

https://forum.doom9.org/showthread.php?p=2028286#post2028286

[dkangel]

oh shit, the answer was on the same page, sorry for that and thank you