[DoVi_Scripts] Multi-Function Scripts for Dolby Vision processing and a lot more...

[Korbus]

Just did that. Now I was able to get past the LAV error, but still not past the madvr error... I have redownloaded the tools and installed madvr through install.bat as administrator.

[Kuler087]

https://forum.doom9.org/showthread.php?p=2010369#post2010369

[Korbus]

I have tried with changing prores setting to yes, still no luck, same error as above. Guess I'm giving up on this workflow, although I'm very frustrated that it worked the first two times. Something has changed and I can't pinpoint it. In the 2 successful cases, measurements were done by prores, I guess, which created a temp .mov file several hundred GB in size. Now it's dxva (copy-back) doing the measurement, which reaches 100% and then the madvr error follows...
Thanks for your help anyway! Sent a donation and got the link for the rpu's, thank you for this and your amazing work!

[Kuler087]

The problem is your input. I've seen this happen only once or twice with my own mkv rip and I processed thousands of files.
You can convert it first to prores with workflow 8-2-3 and then input the prores file in 3-1, it should bypass your problematic MKV rip

[Korbus]

Tried that too. Still no luck... Before that, I have tried 3-1 (mkv, not prores) with 2 different files (Pacific Rim and Blade Runner Final Cut). The same error as in my previous post.

[Kuler087]

oh, I forgot to tell you... You have to put ''transfer=hdr.matrix=2020'' in the prores filename...

Anyway, I added a workaround for those badly hevc muxed MKVs in the latest beta (https://drive.google.com/file/d/128gq8aDUTKA_aT7SQsM9dkjA1EP1sosR/view?usp=sharing). If madvr fails to measure, it will now convert to prores and then generate the scene cuts with madvr.
If you already know a specific input will fail, you can force the workaround at line: 211 set madvr_workaround=YES

[Korbus]

Thank you, it worked by renaming the file. After that it finally initiated the cm-analyze.exe.
L.E. Meanwhile, I have completely uninstalled and removed madvr entries from registry, redownloaded and installed it again and now a file that was previously failing 3-1 (Blade Runner Final Cut) successfully completed the workflow with madvr. So I guess something has corrupted its settings or Windows registry entries, but now everything seems to be working ok again.

[ac777103]

thanks for the updated script with the prores workaround for 3-1. I have been using 6-2 to generate an HDR plot and decide on which setting to use for the mastering display luminance/primaries when starting 3-1, as I've seen cases where MediaInfo stats on MDL aren't that sensible for the content. Is there a workaround that will generate the HDR plot before starting 3-1 ?

[Kuler087]

Is there a workaround that will generate the HDR plot before starting 3-1 ?

the workaround for files that won't work in madvr is only for 3-1.

In 6-2, if you have a file that doesn't work, you can enable prores encoding and crop (even you if use 0,0,0,0)

:: choose if you want 6-2 to convert to prores when you opt to crop. This may be faster depending on your CPU (default = NO)
set crop.prores62=YES

[ac777103]

thanks, I updated the bat file to force prores for 6-2 but got the following error:
18797

[Kuler087]

Oops, this was intended for when you crop the letterbox. I forgot to test it when you don't crop.
This should fix it. (https://drive.google.com/file/d/128gq8aDUTKA_aT7SQsM9dkjA1EP1sosR/view?usp=drive_link)

[ac777103]

Many thanks, making progress but not quite there. I ran 6-2 with the new script, having modified the line to force prores for cropping, and set the crop to 0,0,0,0. I obtained an HDR plot but as you can see the stats on %MaxCLL and %MaxFALL haven't been derived. Nevertheless, the plot appears useful.

18798

18799

[Kuler087]

it looks like your tool folder is not up to date. These files specifically: https://drive.google.com/drive/folders/18403rSHIaVFkHIFVkO1ZYoIZmZrRKwag?usp=sharing

[ac777103]

thanks, did those get updated at some point? I've never had this stage of 6-2 fail before.

[Kuler087]

In 3.0.4 (https://github.com/R3S3t9999/DoVi_Scripts/releases/tag/3.0.4) yes. I added a lot of new stats and these py scripts were updated.
I always put a link to the updated files in the release note so if you update only once in a while, you probably should update the tools too.

[ac777103]

thanks, yes I was on a version pre 3.0.4. Will check the notes from now on per your suggestion. I was able to get a stats complete plot by just plotting the RPU once I'd downloaded the new .py files. Although Pacific Rim is by no means a favourite of mine (you could say it's so dumb it's good), it's niggled me that it's the only film in my collection that failed on 3-1, so glad to have it in DV now (as dumb as the film is, it certainly looks spectacular).

[Kuler087]

so glad to have it in DV now (as dumb as the film is, it certainly looks spectacular).
yes and DV helps a lot in this one. (unless you have a 2000nits TV)

[Korbus]

Sorry to pester, but now I'm getting this error while running workflow 8-2-1, which until now I have used successfully dozens of times...

[Kuler087]

I dont know. Works fine for me, and nothing changed in this workflow since I added 2-pass support...

https://justpaste.it/ahew2

[Korbus]

The first pass is not done at all, after demuxing/indexing/metadata ordering it jumps almost instantly to pass 2, which throws the error. I have recopied mkvtoolnix afresh from the tool pack, reinstalled avisynth+, rebooted and still get the error. It does it on each 4k remux i try.

[Kuler087]

I dont know and can't reproduce...

Set echo to ''on''(line2) and post the complete log on https://justpaste.it/

[Korbus]

Done https://justpaste.it/gt9gi

[Kuler087]

I dont see anything wrong but if you say that it worked before, it must be your input...
You can try to Set the script to keep the temp folder and run the avs script in avspmod.
"D:\Downloads\DoVi.Scripts\temp.folder31\script.avs"

:: choose if you want the temp folder to be deleted after each workflow YES or NO (default=YES)
set DEL_temp=YES

you can also try this sample: https://drive.google.com/file/d/1AflJT_C8Emocm4uyNDw67ZVnl-AC1Gqz/view?usp=drive_link

[Korbus]

... run the avs script in avspmod.
"D:\Downloads\DoVi.Scripts\temp.folder31\script.avs" ...


How do I do that? I got that in order to keep the TEMP folder I need to input NO to set DEL_temp=, but what do I have to do to run the avs script in avspmod?

[Kuler087]

open "D:\Downloads\DoVi.Scripts\temp.folder31\script.avs" in avspmod and try to play it

[Korbus]

With your sample, 8-2-1 worked. I will now try again with my remux (which will surely fail) and then recover the avs script and play it in avspmod.
L.E. This is what happens when I play the script:

[Kuler087]

it means the RPU has more or fewer frame than the video layers.

You can try to extract and inject the rpu back into the P7 ST-DL rip. This should fix the framecount mismatch.

[Korbus]

As usual, you're right. Did what you recommended and now it's running pass one as it should. Thank you!

[Gatorman3385]

Is there a reason to change the default (125) Target Nits when tone mapping to SDR? Is this personal preference or based on the device being used to view the media?

[Kuler087]

You can raise it for a brighter HDR grade if you notice clipping.
125 should be fine for most content.

[Kuler087]

it looks like the latest LG C2 firmware update broke profile 8 DV MP4 support for files with codec ID DVHE which is needed for older TV/hardware (EG C8).
Now all my dvhe P8 samples no longer work on my TV SMH... :(

profile 5 still work...

[SamuriHL]

Oh man. That's not good news!

Sent from my SM-S928U1 using Tapatalk

[-QfG-]

it looks like the latest LG C2 firmware update broke profile 8 DV MP4 support for files with codec ID DVHE which is needed for older TV/hardware (EG C8).
Now all my dvhe P8 samples no longer work on my TV SMH... :(

profile 5 still work...

You can mux "Fake P5" profiles in MP4, that will work.


Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main 10@L5@High
HDR format : Dolby Vision, Version 1.0, Profile 5.0, dvhe.05.07, BL+RPU, no metadata compression / SMPTE ST 2086, Version HDR10, HDR10 compatible
Codec ID : hev1
Codec ID/Info : High Efficiency Video Coding
Duration : 58 s 975 ms
Bit rate : 26.6 Mb/s
Maximum bit rate : 33.3 Mb/s
Width : 3 840 pixels
Height : 1 920 pixels
Display aspect ratio : 2.000
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0 (Type 2)
Bit depth : 10 bits
Bits/(Pixel*Frame) : 0.151
Stream size : 187 MiB (95%)
Tagged date : 2025-01-11 20:52:41 UTC
Color range : Limited
Color primaries : BT.2020
Transfer characteristics : PQ
Matrix coefficients : BT.2020 non-constant
Mastering display color primaries : Display P3
Mastering display luminance : min: 0.0050 cd/m2, max: 1000 cd/m2
Maximum Content Light Level : 366 cd/m2
Maximum Frame-Average Light Level : 105 cd/m2
Codec configuration box : hvcC+dvcC

This is a P8 Hybrid file with P5 header.

[Fuso]

it looks like the latest LG C2 firmware update broke profile 8 DV MP4 support for files with codec ID DVHE which is needed for older TV/hardware (EG C8).
Now all my dvhe P8 samples no longer work on my TV SMH... :(

profile 5 still work...

I have LG C2 but I don't have P8 with dvhe ID. What exactly is broken - can't be played, fallback to HDR10?

[-QfG-]

all mp4 muxer write dvhe instead hev1 in the header. (Codec ID). mp4muxer has a switch to set CODEC ID:

--dvh1flag = Set the elementray stream index to set HEVC track's sample entry name to 'dvh1',
default sample entry box name is 'dvhe' for non-cross compatible stream.
--hvc1flag = Set the elementray stream index to set HEVC track's sample entry name to 'hvc1',
default sample entry box name is 'hev1' for cross compatible stream.

But during my tests the switch doesn't work, don't know why. Only FFMPEG mux the files with Codec ID hvc1. And with Codec ID dvhe mp4 files will not work on older SAMSUNG/LG TVs.

[Kuler087]

You can mux "Fake P5" profiles in MP4,

Oh I know but that's not standard and I don't have to do that since the "hev1" codec id(new mp4muxer) "hvc1" codec id (new mp4box) work just fine on all the TVs made after 2019.
For the TV from before that, you don't even need the ''fake p5'' headers, they just need the "dvhe" codec id which was used in the very first mp4muxer version and also used in the old mp4box version I use in DoVi_Scripts.

So now since the latest update, my C2 plex app fallback to HDR10 with the codec id ''dvhe'' profile 8 (profile 5 still works). I always used the dvhe ID because it worked on older and newer TVs so the patterns and test files (https://drive.google.com/drive/u/1/folders/1yAq-jgsb8pYa92PnGZkxyEV0E3VVkhiC) I did were supported on more devices. Now I have hundreds of files shared everywhere that no longer work on my C2 and possibly all the newer TV.

Kodi probably still works though but since I got my Ugoos, I no longer use my tv internal player and my C2 dev mode expired so I lost my kodi installation and I did not try the native lg app, only the PLEX app.
The TS container still works fine on older and newer TVs so I guess I'll have to remux all my files and upload them again.


My script has two modes for the mp4 muxing

OLD: (supported on older TVs)
use a custom mp4muxer version with dvhe id
use an old mp4box version that keep the id to dvhe

NEW: (supported only on 2019+ TVs)
use the latest mp4muxer with codec id hev1
use the latest mp4box with codec id hvc1

https://i.ibb.co/VmBq3qz/Media-Info-6f-LJtd-NKET.png

[Kuler087]

all mp4 muxer write dvhe instead hev1 in the header. (Codec ID). mp4muxer has a switch to set CODEC ID:

--dvh1flag = Set the elementray stream index to set HEVC track's sample entry name to 'dvh1',
default sample entry box name is 'dvhe' for non-cross compatible stream.
--hvc1flag = Set the elementray stream index to set HEVC track's sample entry name to 'hvc1',
default sample entry box name is 'hev1' for cross compatible stream.

But during my tests the switch doesn't work, don't know why. Only FFMPEG mux the files with Codec ID hvc1. And with Codec ID dvhe mp4 files will not work on older SAMSUNG/LG TVs.

only the first version of mp4muxer(the ones in my tool pack) use the dvhe id and that's the only version that worked on the LG C8 in profile 8

[-QfG-]

Oh no, do you think LG has capped the P8 support for the TVs? Files with P8 and hev1 still working?

[Kuler087]

I'll try the native app later and i hope it's just a PLEX issue but since HEV1 and HVC1 still work, it doesn't look good for ''dvhe'' support on my TV and possibly the newer ones.

My script is using a modded version of the first mp4muxer though, I probably should try the original mp4muxer.
FYI, the modded version (https://drive.google.com/file/d/14fAbUrg8BWRCtb9aRcmznWfMV_PJyh2M/view?usp=sharing)speed up the muxing and adds the compatibility ID parameter which was not present in the very first mp4muxer. (https://drive.google.com/file/d/1FUqXlGoXA7vIFOLnnun_3fsjsCRZEeot/view?usp=sharing)

[Fuso]

Oh I know but that's not standard and I don't have to do that since the "hev1" codec id(new mp4muxer) "hvc1" codec id (new mp4box) work just fine on all the TVs made after 2019.
For the TV from before that, you don't even need the ''fake p5'' headers, they just need the "dvhe" codec id which was used in the very first mp4muxer version and also used in the old mp4box version I use in DoVi_Scripts.

And with Codec ID dvhe mp4 files will not work on older SAMSUNG/LG TVs.

They will play on older LGs. The reason for the fake P5 was to work also on non-DoVi TV (in my case Samsung).

- P8 dvhe plays on both old and new LGs (until the last update apparently);
- P8 hev1 plays DoVi on on new LGs and HDR10[+] on Samsung, on older LGs fallbacks to HDR10, but as a matter of playback it works on all tvs;
- P5 hev1 plays DoVi on both new and old LGs, and HDR10[+] on Samsung.

[Kuler087]

They will play on older LGs. The reason for the fake P5 was to work also on non-DoVi TV (in my case Samsung).

- P8 dvhe plays on both old and new LGs (until the last update apparently);
- P8 hev1 plays DoVi on on new LGs and HDR10[+] on Samsung, on older LGs fallbacks to HDR10, but as a matter of playback it works on all tvs;
- P5 hev1 plays DoVi on both new and old LGs, and HDR10[+] on Samsung.

that's correct.

[Fuso]

I just tested some of your P8 dvhe test videos and yes they fallback to HDR when played from PLEX. But from the internal played they don't play at all.
P8 hev1 and Fake P5 hev1 from PLEX are working fine.

[Kuler087]

Also, the plex app and the tv menu navigation are noticeably slower and sometimes lag. I hate this update!!!

[-QfG-]

I have a Dune Homatics 4k R Plus Box, with a CoreELEC CPM build and full FEL support. I can only recommend it to everyone.

Also, the plex app and the tv menu navigation are noticeably slower and sometimes lag. I hate this update!!!

God damned, i have a C8 and no OTA update still yet.

[dapope]

Thank you again Kuler087 for the awesome tools and research. Do you or anyone else know if Snake Eyes 1998 or Andor S01 UHD are safe to convert to 8.1 without baking fel etc. ? I don't see them on the spreadsheet. Thanks!

[Kuler087]

I dont remember. Run the files in workflow 7-1 and find out.

[Kuler087]

If anyone is interested in testing a new function? I did a new workflow(7-4) that can create DV metadata comparison videos with 10 frames in loop.
it takes 10 different frames (auto or manual) and makes a loop of 69 frames alternating No metadata/no tone mapping vs L1(L3) only vs L1+ artistic trims

The whole process is very fast, about 5 minutes on my PC and supports batch processing.
So comparing the DV tone mapping is now very easy and effortless.

latest beta: https://drive.google.com/file/d/128gq8aDUTKA_aT7SQsM9dkjA1EP1sosR/view?usp=drive_link
new files to put in the tools folder: https://drive.google.com/drive/folders/18403rSHIaVFkHIFVkO1ZYoIZmZrRKwag?usp=drive_link
I did a couple of samples: https://drive.google.com/drive/folders/1g5I-z_sJmVu-SAIPNiiSlcdMiy2ka0mf

(7-5) I'll also add another similar workflow that compares DV metadata with FEL baked and without FEL. This workflow is not completed yet.

[DMD]

If anyone is interested in testing a new function? I did a new workflow(7-4) that can create DV metadata comparison videos with 10 frames in loop.
it takes 10 different frames (auto or manual) and makes a loop of 69 frames alternating No metadata/no tone mapping vs L1(L3) only vs L1+ artistic trims

The whole process is very fast, about 5 minutes on my PC and supports batch processing.
So comparing the DV tone mapping is now very easy and effortless.

latest beta: https://drive.google.com/file/d/128gq8aDUTKA_aT7SQsM9dkjA1EP1sosR/view?usp=drive_link
new files to put in the tools folder: https://drive.google.com/drive/folders/18403rSHIaVFkHIFVkO1ZYoIZmZrRKwag?usp=drive_link
I did a couple of samples: https://drive.google.com/drive/folders/1g5I-z_sJmVu-SAIPNiiSlcdMiy2ka0mf

(7-5) I'll also add another similar workflow that compares DV metadata with FEL baked and without FEL. This workflow is not completed yet.
Thank you it's great like your other tools.

[Korbus]

@R3S3T

If I bake FEL using workflow 8-2-1 to get a 1080p rip, then from that rip I extract the 8.1 RPU with workflow 1, can I inject the said RPU to a fresh 1080p BL/HDR10 rip of the same movie and preserve the FEL information as the original rip did?

[Kuler087]

only if the other 1080p encode has FEL baked in.