[DoVi_Scripts] Multi-Function Scripts for Dolby Vision processing and a lot more...

[Kuler087]

only minor changes so far, no changelog yet

[guest]

only minor changes so far, no changelog yet

I think I've just figured out what's going on with the Beta's, you're always changing or adding, and you save it as a Beta, there.

It's 1kb bigger than the other day :)

[Kuler087]

right, the beta file is automatically synced to Google Drive as soon as I make any changes.

[Scaramanga]

I've got a setup of Shield 2019 through to LG TV utilising a Plex server and it can happily play native DV rips (Dune) and standard UHD rips (Sicario) from MKV
no issue.
When I have ran Sicario through the script (3-1) the film does trigger DV on the TV but there are blocky aspects when it comes to dark colours on light background and a few frames that are obviously awry.

Just wondered if I am doing something wrong in prepping the scripts, or do I need to invest in a Nvidia GFX card or a SSD to carry out the conversion work?

Thanks

[guest]

I've got a setup of Shield 2019 through to LG TV utilising a Plex server and it can happily play native DV rips (Dune) and standard UHD rips (Sicario) from MKV
no issue.
When I have ran Sicario through the script (3-1) the film does trigger DV on the TV but there are blocky aspects when it comes to dark colours on light background and a few frames that are obviously awry.

Just wondered if I am doing something wrong in prepping the scripts, or do I need to invest in a Nvidia GFX card or a SSD to carry out the conversion work?

Thanks

Just thought I'd add my 10 cents worth. (I'm still learning)

What are the spec's of your PC ??

Which .bat are you using, .mp4 or .mkv version ??

Is the original video good, without these blocky aspects, etc ??

As you haven't got an nVidia GPU, you can't be using NVENC, so that's not an issue.

There are a lot of "behind the scenes" settings, but default should be fine.

An SSD or Nvme would only speed up certain parts of the process.

[Kuler087]

I've got a setup of Shield 2019 through to LG TV utilising a Plex server and it can happily play native DV rips (Dune) and standard UHD rips (Sicario) from MKV
no issue.
When I have ran Sicario through the script (3-1) the film does trigger DV on the TV but there are blocky aspects when it comes to dark colours on light background and a few frames that are obviously awry.

Just wondered if I am doing something wrong in prepping the scripts, or do I need to invest in a Nvidia GFX card or a SSD to carry out the conversion work?

Thanks

3-1 just generates metadata, it doesn't touch the video in any way. So it's in your source and DV maybe makes it more obvious?

Also, it might be a cmv4.0 vs cm2.9 thing, on my 2023 Hisense TV, cmv4.0 looks wrong/bad. So you could try to remove cmv4.0 with 2-1-5 and if it is the source of the problem, you can configure the script to always remove it at line 101

[Balthazar2k4]

Out of curiosity, what is the best method of dealing with L5 on something like Interstellar with changing aspect ratios when using 3-1?

[Kuler087]

Choose the largest AR but your min and avg pq metadata won't be 100% accurate.

[thirdmoon]

Reset can we get folder processing for 6-7 in future release? Like a folder containing multiple png or tiff and the script generates heatmap and gamut map for all of them?

[Kuler087]

png/tiff input is just for when you want to check a specific frame.
If you want more frames, input a video file and the script will output as many frames as you have configured at line 125 (set frame_number=50)

[thirdmoon]

But those frames are random. I usually take 10-15 screenshots manually and then analyze them. This isn't super important, maybe in some future release if possible.

[Gatorman3385]

I often crop my encodes. Sometimes the original video will have L5 values that are odd, e.g. 0,0,143,143. Can cropping the BL to 0,0,144,144 negatively affect the DV presentation?

If so, would it be better to crop to one row less than the original L5 values, 0,0,142,142 then edit the L5 to 0,0,1,1?

[Kuler087]

Can cropping the BL to 0,0,144,144 negatively affect the DV presentation?


no

If so, would it be better to crop to one row less than the original L5 values, 0,0,142,142 then edit the L5 to 0,0,1,1?
that should be fine but I'm no encoding expert. Maybe leaving a 1-pixel row is a problem, I don't know, others can respond.

[guest]

I've been wanting to ask you if certain workflows are, or could be multi threaded ?

Also, would some workflows benefit from having multiple nVidia GPU's ?

With all the video's you post on Youtube, you must be processing a lot of stuff, a lot of the time.

Which leads me to also ask, I'd be interested in what spec's your PC(s) (or Mac's) are. ?

I've currently got a 7950X & a 13900KF, pretty much the top of the line "normal" desktop CPU's

But I haven't got any hi end 4000 series nVidia GPU's, hence the question about multi cards.

You can never have enough processing power & "speed" :)

[Kuler087]

You can run more than 1 workflow at the same time. I do it all the time.
My PC is similar to yours. I have an i9 13900ks + RTX3070 + 32gb ram.

What allows me to do all the screenshot comparisons and dv plots so quickly is the very fast nvme pci-4 drives (https://www.amazon.com/WD_BLACK-SN850X-Internal-Gaming-Solid/dp/B0B7CQ2CHH/ref=sr_1_6?crid=1LTR4525CXAHQ&dib=eyJ2IjoiMSJ9._RMR3uLmhBd_QCrWsipAZnqEWFxprb3CwkUXYxfsziGvneFxpB2XyGutynJh2ai90RJoTrExy4R0yz001ubQ53fNr8faBUqg-bDTR_9LjrOaozKlbvi_MYrjEIjFBv2trdr5GXfIc1W6Q9xC4vzS0FmQgXXNsouWgOxnQK3jd8zy59k0W49eSZP2NS0EiQhscepu6F0joiStN1g5CCeIbfcz67FtlT_-1ySvXunv48Y.WK9Nf-txr2P0bYvek4z7lVAUR28zGyJN4Gh-l1VMCwk&dib_tag=se&keywords=pci-e%2B4%2Bnvme&qid=1710933233&sprefix=pci-e%2B4%2Bnvme%2Caps%2C70&sr=8-6&th=1) I have. I even have one dedicated to my temp folder so any remuxing/demuxing is crazy fast (1-2gb/s)

[guest]

You can run more than 1 workflow at the same time. I do it all the time.

Now that IS good to know :), but I have to ask, if you're "feeding" 2 or 3 jobs into the same temp folder, is there going to be any naming conflicts ?

My PC is similar to yours. I have an i9 13900ks + RTX3070 + 32gb ram.

So you've got the "Big" 13900, nice. I have a 3050 on the i9, and a 3060 on the 7950X, yet to detemine which system will do things the fastest, with my RipBot encoding the 7950X is noticeably faster.

What allows me to do all the screenshot comparisons and dv plots so quickly is the very fast nvme pci-4 drives (https://www.amazon.com/WD_BLACK-SN850X-Internal-Gaming-Solid/dp/B0B7CQ2CHH/ref=sr_1_6?crid=1LTR4525CXAHQ&dib=eyJ2IjoiMSJ9._RMR3uLmhBd_QCrWsipAZnqEWFxprb3CwkUXYxfsziGvneFxpB2XyGutynJh2ai90RJoTrExy4R0yz001ubQ53fNr8faBUqg-bDTR_9LjrOaozKlbvi_MYrjEIjFBv2trdr5GXfIc1W6Q9xC4vzS0FmQgXXNsouWgOxnQK3jd8zy59k0W49eSZP2NS0EiQhscepu6F0joiStN1g5CCeIbfcz67FtlT_-1ySvXunv48Y.WK9Nf-txr2P0bYvek4z7lVAUR28zGyJN4Gh-l1VMCwk&dib_tag=se&keywords=pci-e%2B4%2Bnvme&qid=1710933233&sprefix=pci-e%2B4%2Bnvme%2Caps%2C70&sr=8-6&th=1) I have. I even have one dedicated to my temp folder so any remuxing/demuxing is crazy fast (1-2gb/s)

I have also dedicated an nvme to both the temp & encodes folders.

So, you didn't mention if the workflows might benefit from multiple GPU's ?

Thanks.

[Kuler087]

Now that IS good to know :), but I have to ask, if you're "feeding" 2 or 3 jobs into the same temp folder, is there going to be any naming conflicts ?

each workflow uses a different temp folder. You cant run the same workflow at the same time though, use the batch mode instead.

So, you didn't mention if the workflows might benefit from multiple GPU's ?
I don't know

[guest]

each workflow uses a different temp folder. You cant run the same workflow at the same time though, use the batch mode instead.

Thanks for clarifying that...

I just thought of something, if you had multiple dovi_script folders that have all the tools in them, but you sent them to different temp folders, you might be able to do multiples of the same workflow then...

I don't know

Well, I might have to test that, then.

[CineManiac]

Thanks for clarifying that...

I just thought of something, if you had multiple dovi_script folders that have all the tools in them, but you sent them to different temp folders, you might be able to do multiples of the same workflow then...



Well, I might have to test that, then.

I believe you’d need to use different drives since some workflows write some temp files to the root of the drive, which could potentially lead to conflicts. Checking for positive lifts writes .trim.offset.txt, etc., to the root of the drive, par exemple.

[Kuler087]

Only the L2 L8 L5 L6 plotter does that so as long as you don't run two instances of 6-4 or 6-5, it will be fine.

This is something I want to revisit though. This is also the reason why these workflows are disabled when your output path is on a C:\ drive.

[speedy]

it might be a cmv4.0 vs cm2.9 thing, on my 2023 Hisense TV, cmv4.0 looks wrong/bad.
Does CMv4 always look bad on the U8K?
Any idea what is wrong with it?

[guest]

I thought I had to demux the audio from my videos before using 8-1-1.

I had a folder of 22 videos, and just for "kick's" I loaded that folder into 8-1-1, and lo & behold it demuxed & converted only the audio tracks :)

That's going to save me a LOT of time, going forward.

The only issue is, I then have to rename them from the .mkv container, to .eac3 so that RipBot264 recognizes them. (could that be changed?)

So I'm guessing that 8-2-x might do the same to the video tracks.

[Kuler087]

I thought I had to demux the audio from my videos before using 8-1-1.

I had a folder of 22 videos, and just for "kick's" I loaded that folder into 8-1-1, and lo & behold it demuxed & converted only the audio tracks :)

That's going to save me a LOT of time, going forward.

The only issue is, I then have to rename them from the .mkv container, to .eac3 so that RipBot264 recognizes them. (could that be changed?)

So I'm guessing that 8-2-x might do the same to the video tracks.

Yes the script batch modes are very useful. You can batch demux the files with
https://www.videohelp.com/software/Inviska-MKV-Extract

But remember that any delay present will be gone.


Does CMv4 always look bad on the U8K?
Any idea what is wrong with it?


I don't know, I did a couple of tests when I bought it a while ago and it looked strange(could be the internal player) but I haven't rechecked because I got this TV only for monitoring my SDR to HDR conversion in Resolve since it can do 850nits at 100% and 2000nits at 25%.

[guest]

Yes the script batch modes are very useful. You can batch demux the files with
https://www.videohelp.com/software/Inviska-MKV-Extract

But remember that any delay present will be gone.

Thanks, I will give that a try, never heard of it, TBH.

I've been using gMKVExtractGUI for years, it sort of has a batch function, by adding multiple files.

Extra:- Just installed Inviska, and it's almost exactly the same as gMKVExtractGUI.

[Scaramanga]

I've got a setup of Shield 2019 through to LG TV utilising a Plex server and it can happily play native DV rips (Dune) and standard UHD rips (Sicario) from MKV
no issue.
When I have ran Sicario through the script (3-1) the film does trigger DV on the TV but there are blocky aspects when it comes to dark colours on light background and a few frames that are obviously awry.

Just wondered if I am doing something wrong in prepping the scripts, or do I need to invest in a Nvidia GFX card or a SSD to carry out the conversion work?

Thanks
Just an update on this as I appreciate the time and effort people put into helping others on this thread.

I tried the original rip and that too has the slight distorted blocky look on at least two of the scenes in the film (Sicario) so nothing to do with the DoVi scripts.

After a lot of trial and error debloating the Shield, tweaking the WiFi 6 card on the Plex server it's almost unnoticeable now and the effect doesn't occur on any of the other movies I've processed so I assume it's an issue with the disc itself or MakeMKV handling the rip?

[Gatorman3385]

When would it be necessary to edit L6 values when injecting DV to HDR?

[Kuler087]

editing L6 from retail DV is not necessary.
When you generate it in Resolve and use 1-1 to inject the XML, the script uses the maxcll/fall of the generated metadata.

Btw, I tested different L6 values during DV playback and it has no effect.

[Gatorman3385]

With different versions of DV available for some releases, how do they rank amongst themselves? Is it worth generating DV from Resolve if the UHD release has DV MEL, or is converting from DV5 to DV8 the better path?

[guest]

With different versions of DV available for some releases, how do they rank amongst themselves? Is it worth generating DV from Resolve if the UHD release has DV MEL, or is converting from DV5 to DV8 the better path?

Metadata
Dolby Vision metadata include:[15]

L0 (static): Mastering and target display characteristics
L1 (dynamic): Automatically generated
L2 trims (dynamic): Manually generated per frame or per scene
L3 trims (dynamic): Manually generated per frame or per scene (since CMv4.0)
L8 trims (dynamic): Manually generated per frame or per scene (since CMv4.0) (equivalent of L2 trims)
L5: Timeline aspect ratio description
L6 (static and optional): MaxCLL and MaxFALL (required for HDR10)
L9 (dynamic): Mastering display color primaries (since CMv4.0)
Dolby Vision 4.0 introduces new secondary trims for hue and saturation adjustment.[18]

Profiles
Dolby Vision profiles[19][20]
Profile Codec BL:EL resolution Backward compatibility
4 10-bit HEVC 1:1/4 SDR
5 10-bit HEVC No enhancement layer None (It uses proprietary IPTPQc2)
7 10-bit HEVC 1:1/4 for UHD
1:1 for FHD

Ultra HD Blu-ray
8 10-bit HEVC No enhancement layer HDR10 or SDR or HLG
9 8-bit AVC High or High Progressive Profile SDR
10 10-bit AV1 None or HDR10 or SDR or HLG
20 10bit MV-HEVC for 3D or HEVC for 2D None (It uses proprietary IPTPQc2)
Profiles not supported for new applications[19][20]

Profile Codec BL:EL resolution Backward compatibility
0 AVC 1:1/4 SDR
1 AVC 1:1 None
2 8-bit HEVC 1:1/4 SDR
3 8-bit HEVC 1:1 None
6 10-bit HEVC 1:1/4 HDR10
8 10-bit HEVC No enhancement layer

[Kuler087]

With different versions of DV available for some releases, how do they rank amongst themselves? Is it worth generating DV from Resolve if the UHD release has DV MEL, or is converting from DV5 to DV8 the better path?

The only times I generate DV for a movie that has retail DV are:

1- static L1 and L2. (Static L1 + dynamic L2 is fine.)
2- P5 web-dl is a different grade and cant be injected into the bluray
3- The original DV was done in cmv2.9(2.5nits avg floor) without any manual trims
4- if the bluray is cmv2.9 MEL and web is 4.0, I injected the web metadata like in: Aquaman and the Lost Kingdom, Migration.2023, Wonka.2023, The.Hunger.Games.The.Ballad.of.Songbirds.and.Snakes.2023
Why? because cmv2.9 content has crushed/clipped black and cmv4.0 does not. It also has better tone mapping.


PS Ugoos AM6B+ has arrived and looks VERY promising: https://www.youtube.com/watch?v=4QUURfLy5ew

[Hellboy.]

I can use DoVi_Scripts to extract HDR10+ from a REMUX and then add the HDR10+ data to another REMUX? How?
Thanks.

[Kuler087]

Yes, 6-6 to export and then 1-1 to inject/resync.
Obviously, you have to make sure its the same HDR grade first.

[jayper]

The only times I generate DV for a movie that has retail DV are:

1- static L1 and L2. (Static L1 + dynamic L2 is fine.)
2- P5 web-dl is a different grade and cant be injected into the bluray
3- The original DV was done in cmv2.9(2.5nits avg floor) without any manual trims
4- if the bluray is cmv2.9 MEL and web is 4.0, I injected the web metadata like in: Aquaman and the Lost Kingdom, Migration.2023, Wonka.2023, The.Hunger.Games.The.Ballad.of.Songbirds.and.Snakes.2023
Why? because cmv2.9 content has crushed/clipped black and cmv4.0 does not. It also has better tone mapping.


PS Ugoos AM6B+ has arrived and looks VERY promising: https://www.youtube.com/watch?v=4QUURfLy5ew

Appreciate your sharing your findings on the Ugoos, I am very close to picking one up.

What container type are you using for playback? mp4? I've been using an Oppo for the last few years, so I invested heavily in ts. Do you see any sort of variance in playback or stability? I could always remux, but was just curious.

[Kuler087]

The test files used in this video are MP4, but I use the P7/P8/P5 MKV container for movies/tv shows.
I only watched one P7 ST-DL FEL MKV movie(primal fear) yet and it played flawlessly with DV+ TrueHD from start to end. FF/RW, seeking and resume work very well too.

I haven't tried TS yet but it should be fine. It doesn't support DT-DL DV rip but Mkvtoolnix can mux them to ST-DL just fine.

I'll do more comparisons this weekend...

[jayper]

Thanks very much. Always appreciate your time and insights. Sounds promising, it looks like I may need to place and order :)

[Hellboy.]

Yes, 6-6 to export and then 1-1 to inject/resync.
Obviously, you have to make sure its the same HDR grade first.

I use 6-6 and get the .json and .png. Then use the 1-1 to inject the .json file without problems.

But in 1-1 ask: "Do you want to re-sync the metadata? If yes, enter a frame number. E.G.: -24 or 24 and/or press enter..."

I just press enter. How i can check in what frame the HDR10+ start in the old video to use the same in the new video.

The 2 videos are remuxes from the same source, but in the HDR10+,
"Maximum Content Light Level and Maximum Frame-Average Light Level"
are modify.

[Kuler087]

Check the P8 Hybrid maker tutorial. I explain how to find the frame difference between two sources and how to verify if your final video is in sync with the metadata.

https://www.youtube.com/watch?v=hVWZpat34oc

[frandan]

Hi to everyone,
I tried a 6.2 workflow (Measure MadVR HDR) but i obtained a "test.mkv.measurements.incomplete" file and a related alert on cmd, where test.mkv is my BL.
From this point, the script stopped working corretcly not founding the expected " test.mkv.measurements" file.
Something wrong in my setup?
Thanks in advance for your replies

[Kuler087]

madvr will fail to measure some files. It usually happens with the TS/M2TS container and remuxing to MKV usually fixes the issue but your input is already MKV so unless you get the same result for all your files, there's nothing you can do about it unfortunately.
Is it an untouched bluray rip or some encode?

[frandan]

First of all thank you for the quick answer.
It's LOTR Fellowship of the Rings part 1 disc, demuxed with dgdemux and incapsulated in mkv with mkvtoolnix.
I'll try with a different disc, probably Back to the future that is in your black list :).
About it i understood that ,for this kind of movies, dovibaker is necessary for basic HDR10 rip too (without DoVi metadata).
Is it correct?
Thanks in advance

[MoonKnight]

Hey Kuler

Any reason why we must use balanced algorythm now ? I thought most highlight most details was the best ?

[Kuler087]

for this kind of movies, dovibaker is necessary for basic HDR10 rip too (without DoVi metadata).
Is it correct?
Thanks in advance

The movies that expand brightness requires a player that fully supports FEL P7. If not , you can use the dovi_baker to map the BL brightness to the EL which is what does a Profile 7 player.

For HDR10 playback, this is not required.

[Kuler087]

Hey Kuler

Any reason why we must use balanced algorythm now ? I thought most highlight most details was the best ?

You can use any tuning you prefer (line 107). I changed the default mode to match Davinci Resolve default. The tunings are user preference and none of them is wrong.

https://professionalsupport.dolby.com/s/article/Dolby-Vision-analysis-tuning-in-CMv-4-0-FAQs?language=en_US

[frandan]

The movies that expand brightness requires a player that fully supports FEL P7. If not , you can use the dovi_baker to map the BL brightness to the EL.

For HDR10 playback, this is not required.

Thank You Kuler.
Excuse me but i don't understand a point: If i compare BL only rip (HDR10) and BL+EL rip obtained using Dovibaker but without DoVi metadata injection (i suppose HDR10), do they looks the same on HDR device?

[Kuler087]

do they looks the same on HDR device?

No. if FEL expands the brightness, the baked HDR10 file will look much brighter and possibly clipped depending on your TV capabilities.

[frandan]

So it should be correct using dovibaker also for "simply" HDR rip , obviously when FEL modifies brightness, or no?

[Kuler087]

It depends on your TV capabilities.
If you have a 2000nits+ TV, yes but if you have a 1000nits or less TV, no you will just introduce more tone mapping/clipping.

[guest]

I read on another thread that the latest dovi_tool supports AV1, but looking thru the changelog of older builds, it looks like it might have supported it for some time...

Do you have any plans on using AV1 ??

I have no experience with it, not sure if it's a different way to process a .hevc, or if it's simply AV1 ??

[Boulder]

Regarding AV1 and DoVi, is it actually possible to use the original FEL and encoded AV1 to create a working DoVi mux? That would make things very interesting now that there are Android boxes that are very close to handling things properly with CoreELEC.

[Kuler087]

I'm not familiar with the av1 codec and I don't see any documentation on the dovi_tool page... What's the correct file extension? avif? ivf? av1? Does the RPU need to be edited? Does anyone have a sample?

dovi_tool.exe inject-rpu -i D:\1.ivf --rpu-in D:\1.bin -o D:\injected.ivf
Error: Invalid input file type.

the original FEL and encoded AV1 to create a working DoVi mux?

I doubt it since the EL would need to be av1 too I suppose and it's already not recommended to re-encode the FEL hevc stream.