[DoVi_Scripts] Multi-Function Scripts for Dolby Vision processing and a lot more...

[ac777103]

thanks, reason I've been running 6-2 first is so that I can look at the plot to give me an informed decision regarding which MDL to pick in 3-1, otherwise I rely on Mediainfo. If there is a more efficient or accurate way of doing this within the workflows, happy to take advice.

[Kuler087]

Then yes, using 6-2 prores cropping (with values of 0) is what you want.

[Kontrarian]

Any idea why WEB DV only has a 100 nits L2 and nothing for 600 or 1,000 nits?

[Kuler087]

Most movies are trimmed only at 100 nits. 600 and 1000nits manual trims are very rare.
The 600-1000nits trims you see on cmv2.9 bluray are just an automatic conversion of the L8 100nits from the cmv4.0 workflow.

Watch my DV trims breakdown video to understand the auto-generated trims on bluray discs: https://www.youtube.com/watch?v=15HTaR0hRvI

[ac777103]

Then yes, using 6-2 prores cropping (with values of 0) is what you want.

thanks. So there are ways of getting GPU support through all stages of 6-2 and 3-1, but I saw a note that disabling indexing is not recommended as it's needed for frame accurate reading. Not sure how big a deal that is.

With all this complexity, I'm beginning to reconsider Resolve, now that I have a low level nvidia GPU to try it on. I'll rewatch your demo video. Can I replicate 6-2 to give me a plot via Resolve? At least now that I have a GPU, 6-2 has sped up a lot so doesn't take long to run it.

[Kuler087]

Not sure how big a deal that is.
I'm not sure too, I always feel safer with indexed input but if the script doesn't complain about framecount mismatch at injection, you know that everything went well.

I'm beginning to reconsider Resolve
definitely worth it. Sometimes, the DV analysis in 3-1 can take hours, even with my RTX3070. In resolve, it's always constant: about only 10min for the scene cuts and another 10 min for the DV analysis.

Can I replicate 6-2 to give me a plot via Resolve?
No. 6-2 madVR is the most accurate way of getting an HDR plot.

[SamuriHL]

Resolve is worth it for SO many reasons. This is just icing on the cake. Now that I have my windows resolve issues fixed, I need to get back to learning this DV process myself. I have some movies I'd like to generate DV for.

[ac777103]

your demo video of Resolve recommends conversion to prores as a first step. Does that recommendation still apply?

[Kuler087]

Yes, Resolve hevc decoding is terrible; I-frames prores codec is a must.

[Kuler087]

Resolve is worth it for SO many reasons. This is just icing on the cake. Now that I have my windows resolve issues fixed, I need to get back to learning this DV process myself. I have some movies I'd like to generate DV for.

What issues did you have?

[SamuriHL]

What issues did you have?

This was a few months ago. When I installed the DoVi Tools, it requires a version of Python to be installed. I installed it from the Windows Store as was recommended. At the time, it installed a beta version. Surprisingly, even to Davinci, after several DAYS worth of troubleshooting with them, it turned out that the version of Python that was installed broke Resolve. It wouldn't even start. It'd just crash over and over. Since removing that version of Python, Resolve has been working fine. I need to get a new version of Python and update DoVi Tools again. I never got to the point where I could use it because of this issue. So I still have a lot to learn.

[B0b]

I'm having an issue installing headless. I've tried installing in Safe Mode, but no luck. I've even reinstalled Windows.

C:\Windows\system32>py -m pip install headless
Collecting headless
Using cached headless-1.12.4.tar.gz (40 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting canonical>=0.24.1 (from headless)
Downloading canonical-1.0.0.tar.gz (17 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [23 lines of output]
Traceback (most recent call last):
File "C:\Program Files\Python313\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 389, in <module>
main()
~~~~^^
File "C:\Program Files\Python313\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 373, in main
json_out["return_val"] = hook(**hook_input["kwargs"])
~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Program Files\Python313\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 143, in get_requires_for_build_wheel
return hook(config_settings)
File "C:\Users\Win\AppData\Local\Temp\pip-build-env-jqmm1jtm\overlay\Lib\site-packages\setuptools\build_meta.py", line 334, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Win\AppData\Local\Temp\pip-build-env-jqmm1jtm\overlay\Lib\site-packages\setuptools\build_meta.py", line 304, in _get_build_requires
self.run_setup()
~~~~~~~~~~~~~~^^
File "C:\Users\Win\AppData\Local\Temp\pip-build-env-jqmm1jtm\overlay\Lib\site-packages\setuptools\build_meta.py", line 522, in run_setup
super().run_setup(setup_script=setup_script)
~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Win\AppData\Local\Temp\pip-build-env-jqmm1jtm\overlay\Lib\site-packages\setuptools\build_meta.py", line 320, in run_setup
exec(code, locals())
~~~~^^^^^^^^^^^^^^^^
File "<string>", line 25, in <module>
FileNotFoundError: [Errno 2] No such file or directory: 'VERSION'
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

C:\Windows\system32>

[Kuler087]

This is only required for the heatmap and gamut workflow, IIRC. This should suffice: pip install opencv-python-headless

[SeeMoreDigital]

For anyone who's interested. When I de-muxed the secondary stream for 'The Fifth Element' and ran it through MediaInfo, it reported the following: -

Video
ID : 4117 (0x1015)
Menu ID : 1 (0x1)
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5.1@High
HDR format : Dolby Vision, Version 1.0, Profile 8.1, dvhe.08.03, BL+RPU, extended metadata compression, HDR10 compatible / SMPTE ST 2086, Version HDR10, HDR10 compatible
Codec ID : 36
Duration : 2 h 6 min
Bit rate : 146 kb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 24.000 FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.003
Stream size : 132 MiB (92%)
Writing library : x265 3.2+20-a4e060a44839:[Windows][GCC 6.3.0][64 bit] 10bit
Encoding settings : cpuid=1111039 / frame-threads=5 / numa-pools=16,16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1920x1080 / interlace=0 / total-frames=3854 / level-idc=51 / high-tier=1 / uhd-bd=1 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / aud / hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=1 / keyint=24 / gop-lookahead=0 / bframes=3 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=24 / lookahead-slices=6 / scenecut=40 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=1 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / selective-sao=4 / early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=cbr / bitrate=500 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=500 / vbv-bufsize=550 / vbv-init=0.9 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=1 / chromaloc-top=2 / chromaloc-bottom=2 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1) / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.01 / no-opt-cu-delta-qp / no-aq-motion / hdr / hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / no-scenecut-aware-qp
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0001 cd/m2, max: 1000 cd/m2


I've never seen the video streams 'encoding settings' before...

[B0b]

This is only required for the heatmap and gamut workflow, IIRC. This should suffice: pip install opencv-python-headless

That worked. Thanks.

[Kuler087]

It's rare, but I've seen this before, usually with foreign studios or on streaming (MAX and AMZN IIRC). They just didn't use the ''--no-info'' option.
With the Dolby Encoding Engines (DEE) scripts, they hide the encode settings, and I removed it.

I've never seen the video streams 'encoding settings' before...

[foxman679]

2- Encode to prores with workflow 8-2-3. By default, it uses qscale 1 (lossless), to speed things up, you may want to compress more (line 168 -qscale:v 6 )



I might have read somewhere that 3-1 uses qscale=4? Just want to make the comparison vs current workflow fair.

Does it matter in quality for the created DV metadata? So is lower qscale better (=1 - higher output qualiyt & filesize) oder higher (=6 or 8 - lower quality and smaller filesize). Or is there a value at which the quality will not enough for the result?

[Kuler087]

No, it should not matter much, but don't go higher than 6-8

[Kontrarian]

Does it matter in quality for the created DV metadata? So is lower qscale better (=1 - higher output qualiyt & filesize) oder higher (=6 or 8 - lower quality and smaller filesize). Or is there a value at which the quality will not enough for the result?

Maybe worth reading if you are interested: https://trac.ffmpeg.org/wiki/Encode/VFX#Prores


The options used here are standard and are explained in other documents, but let's elaborate a little bit more on the qscale paramater. This parameter determines the quality of the resulting prores movie - both the resulting size and bitrate. 0 means best and it goes up to 32 which is worst. From empirical testing we've found that a qscale of 9 - 13 gives a good result without exploding the space needed too much. 11 would be a good bet, 9 if a slightly better quality is required. When space is not a problem, go with qscale 5 or less, but approaching zero the resulting clip will be extremely large and the bitrate will be so high that it will stop being playable on normal equipment. The "vendor" argument when set to "apl0" tricks quicktime and Final Cut Pro into thinking that the movie was generated on using a quicktime prores encoder.

[SeeMoreDigital]

I've never seen the video streams 'encoding settings' before...It's rare, but I've seen this before, usually with foreign studios or on streaming (MAX and AMZN IIRC). They just didn't use the ''--no-info'' option.
With the Dolby Encoding Engines (DEE) scripts, they hide the encode settings, and I removed it.Hello again,

With regard to Dolby Vision encoded discs, have you seen the 'encoding settings' info appear with the primary stream?

[Kuler087]

Hello again,

With regard to Dolby Vision encoded discs, have you seen the 'encoding settings' info appear with the primary stream?

yes, a couple times

[foxman679]

Maybe worth reading if you are interested: https://trac.ffmpeg.org/wiki/Encode/VFX#Prores

thanks - i will look into it ;)
But for me it will be interessting if it makes a noteable difference in resulting dv metadata when using less qulity of prores instead of higher quality.

Is there an (easy) way to check this after creating dv metadata?

[Kuler087]

just plot the metadata with 6-4

[foxman679]

Another short question: in the dv xml created with CM-Analyzer-Tool i see this under the Video-Tag:

<ColorSpace>ycbcr_bt2020</ColorSpace>
<SignalRange>video</SignalRange>


But in the xml created with resolve i see this under the Video-Tag:

<ColorSpace>rgb</ColorSpace>
<SignalRange>computer</SignalRange>


Its the difference relevant in any way or can i ignore that? i follwed your video tutorial generating dv xml metadata with resolve.

[Kuler087]

You can ignore that.

[B0b]

I'm receiving this error or something similar on all my files now. It's worked correctly for quite some time. I've tried re-installing, but still the same thing. Is this a script error from something not configured correctly or something from Resolve? I use Resolve to build the xml file.

Extracting track 0 with the CodecID 'V_MPEGH/ISO/HEVC' to the file 'D:\DoVi.Scripts\temp.folder\BL.hevc'. Container format: HEVC/H.265 elementary stream
Progress: 100%

Error: Found invalid RPU: Index 27, error: CM v2.9: Invalid metadata block. Block level 1 should have length 5

Editing L6... MDL_Min=1, MDL_Max=1000, MaxCLL=, MaxFALL=
Error: expected value at line 5 column 35
Press any key to continue . . .
The system cannot find the file specified.
Done.

Error: The system cannot find the file specified. (os error 2)
Missing operand.
Missing operand.

Video/RPU framecount mismatch. Duplicating frame =, -= times...
Error: expected value at line 4 column 15
Done.

[Kuler087]

I dont know; give me your XML or RPU file and the complete log of the workflow, not just part of it.

[B0b]

This time it completed quite a bit more. Attached are the files.

[Kuler087]

Sorry I meant with ''@echo on'' (line two)
no problem on my end with your XML.

PS: if you don't want to mux, you can disable it at line 58 set MUX=NO instead of using the filename trigger.

https://i.ibb.co/G3bqGG2j/Windows-Terminal-UMugp4a7-MI.gif

[B0b]

If the xml is good, I guess I can wipe the drive and reload Windows again. After this started to happen, I reinstalled Windows and still have the issue.

[Kuler087]

I don't know; the problem happens here, and I have no idea what it means... first time I see this.

E:\>"D:\DoVi.Scripts\tools\dovi_tool.exe" info -s "D:\DoVi.Scripts\temp.folder\rpu.bin" 1>"D:\DoVi.Scripts\temp.folder\sum.rpu.json"

thread 'main' has overflowed its stack

@quietvoid, if you read this. Do you know what could be causing this for him?

[Kontrarian]

Would there be any negative outcome from taking the RPU from Iron Man (2008) IMAX Enhanced and applying it to the US UHD with modified L5 to match the normal UHD ratio?
Have you ever tried anything like that to see what happens?
If not, I might test it out myself.

[quietvoid]

I don't know; the problem happens here, and I have no idea what it means... first time I see this.

E:\>"D:\DoVi.Scripts\tools\dovi_tool.exe" info -s "D:\DoVi.Scripts\temp.folder\rpu.bin" 1>"D:\DoVi.Scripts\temp.folder\sum.rpu.json"

thread 'main' has overflowed its stack

@quietvoid, if you read this. Do you know what could be causing this for him?

Needs a debug enabled build and to run the command with RUST_BACKTRACE=1.
I don't know where this could be happening otherwise.

It might just be using too much stack overall with a lot of RPUs parsed.

[B0b]

Needs a debug enabled build and to run the command with RUST_BACKTRACE=1.
I don't know where this could be happening otherwise.

It might just be using too much stack overall with a lot of RPUs parsed.

Since this hasn't really been seen, not worth the effort. I'll wipe Windows and reload again. Maybe a bad component on the hardware side.

[quietvoid]

You could maybe try using this version: https://github.com/quietvoid/dovi_tool/releases/tag/2.1.1
Since 2.1.2 stack usage might be higher.

Either way I need to analyze on my side how much it's actually using.

[B0b]

I've been using the script for many months and didn't have an issue until now. Wiped the drive 3 times with 2 different downloads of Windows 10. The error is still occurring. The script runs fine on Windows 11 with the exception of Dolby Tools. Might be something with Windows 10 and my hardware. The attached files are the latest error using the replacement dovi_tool file.

[Kuler087]

No idea; sorry. I have Windows 10 and 11 PC, and everything is fine on both systems.
Do you have a missing DLL warning when you launch the script?

[quietvoid]

I've been using the script for many months and didn't have an issue until now. Wiped the drive 3 times with 2 different downloads of Windows 10. The error is still occurring. The script runs fine on Windows 11 with the exception of Dolby Tools. Might be something with Windows 10 and my hardware. The attached files are the latest error using the replacement dovi_tool file.

In these it is failing to edit due to JSON but the `info` command works fine.
So maybe it is caused by more stack being used in the recent versions.
It shouldn't be related to hardware at all.

I don't know which version you've been using without problems, I don't manage the tools that come with the script.

[B0b]

No warnings when it launches. One day, it just gave me this error and after reinstalling Windows I still get the same error. I've updated with each version of the script as it's updated. I've always received a panic on Windows 11 with the Dolby Vision Tools. The scripts will run ok, but Dolby Vision Tools will cause a panic along the way. I've ran the script on Windows 10 since.

I appreciate the assistance. I'll try working backwards through the versions of the script and software and see what happens.

[Kuler087]

a panic? What do you mean?
make sure you have the Windows redistributable installed. Here are the ones (https://ibb.co/zWhdj57P) I have installed on my PC.

[quietvoid]

I appreciate the assistance. I'll try working backwards through the versions of the script and software and see what happens.

Can you generate the RPU and try the info command manually to get a backtrace?
Normally it's only like this in a terminal:


dovi_tool.exe generate -x "Nanny DV.xml"
RUST_BACKTRACE=1 dovi_tool.exe info RPU_generated.bin -s


It would be useful to know where the panic comes from.

I made a debug build you can use: https://github.com/quietvoid/dovi_tool/actions/runs/13884143713
You can download the Windows artifacts and extract from "dovi_tool-2.2.0-x86_64-pc-windows-msvc.zip"

I'm not sure if the debug symbols will work, though. Windows builds have them separate.
The alternative is for you to build it, I'm not sure it'll have any useful info otherwise.

[TR-9970X]

make sure you have the Windows redistributable installed.

https://github.com/abbodi1406/vcredist/releases

[B0b]

Sorry, disconnected for a while. I reinstalled Win11. Only drivers, script and required software. This is what it does. It didn't even get to the part where is shows a panic with the Dolby Vision Tools.

[sunday46ma]

Thank you for providing such a great tool, but I have a question: Does this tool support VapourSynth?

[Kontrarian]

Thank you for providing such a great tool, but I have a question: Does this tool support VapourSynth?

Yes it does.

[Kuler087]

It used vapoursynth a couple of versions ago but now the encoder workflows only use avisynth.

[foxman679]

Another question regarding resolve dv export. Which value should i sue at "coding primary Color Correction"? P3-D65 or Rec.2020?
I set the right mastering color profile in the project settings, so i'm wondering, which one i have to choose here.

[Kuler087]

for the XML/RPU injection in 1-1, it doesn't matter.
For DEE profile 5 encoding, rec.2020 is the correct colorspace. P3 messes up the colors but if you feed a P3 XML in 8-2-6(DEE workflow), the script will correct it and use rec2020.

[foxman679]

thanks @Kuler087 for explanation ;)

[Kontrarian]

So if I have a P5 WEB DL and my remux I first compare the HDR (6-2) of both videos to make sure they are similar, the I check sync with (9-1).
If I get an RPU from the google drive then I can check the HDR in the Dolby Vision stuff spreadsheet, BUT how do I check sync?
All I need to do is have the script output the RPU scene cuts to the text file then open up the remux in avisynth and compare, but I don't see that in the script menu, just comparing two video files.
Right now I;m just injecting then doing (2-1) afterwards, would be nice to be able to do something like that first though.