[DoVi_Scripts] Multi-Function Scripts for Dolby Vision processing and a lot more...

[guest]

Hi Reset,

I did a 3-1 with the latest everything, and I'm not sure if this is important, or what it means:-

Input #0, avisynth, from 'D:\Dovi_Temp_MP4\temp.folder77\script.avs':
Duration: 00:58:48.07, start: 0.000000, bitrate: N/A
Stream #0:0: Video: rawvideo (Y3[11][10] / 0xA0B3359), yuv420p10le(tv, bt2020nc/bt2020/smpte2084, progressive), 3840x2160, 23.98 fps, 23.98 tbr, 23.98 tbn
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> prores (prores_ks))
Press [q] to stop, [?] for help
Output #0, mov, to 'D:\Dovi_Temp_MP4\temp.folder77\Game of Thrones S01 E01_prores.422.mov':
Metadata:
encoder : Lavf60.16.100
Stream #0:0: Video: prores (HQ) (apch / 0x68637061), yuv422p10le(tv, bt2020nc/bt2020/smpte2084, progressive), 3840x2160, q=2-31, 200 kb/s, 23.98 fps, 24k tbn
Metadata:
encoder : Lavc60.31.102 prores_ks
[out#0/mov @ 0000013ad8ed2600] video:34326861kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.002049%
frame=84589 fps= 71 q=-0.0 Lsize=34327565kB time=00:58:48.02 bitrate=79707.9kbits/s speed=2.98x
Copyright (c) 2013-2024 Dolby Laboratories, Inc. All Rights Reserved
03/05/2024/16:51:51.728000000 CPU: INFO 13th Gen Intel(R) Core(TM) i9-13900KF (GenuineIntel) [ mmx sse sse2 sse3 ssse3 sse41 sse42 avx avx2 fma ]
03/05/2024/16:51:51.755000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:51.813000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:51.899000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:51.937000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:51.956000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:51.995000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:52.109000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:52.204000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:52.365000000 plugin_manager: WARNING plugin does not contain dlb_plugin_file_io_init_v1
03/05/2024/16:51:52.459000000 cm_analyze: INFO Shot cut detection is disabled when shot list is used
03/05/2024/16:51:52.460000000 cm_analyze: INFO Source: 'file:///D:/Dovi_Temp_MP4/temp.folder77/Game of Thrones S01 E01_prores.422.mov'
03/05/2024/16:51:52.460000000 cm_analyze: INFO Master Display: 1000-nit, BT.2020, D65, ST.2084, Full
03/05/2024/16:51:52.460000000 cm_analyze: INFO Src Info: '3840x2160x3 u16 interleaved 422 le tight 0-84588 23.976fps top_left pq ycbcr_bt2020 video bt2020'
03/05/2024/16:51:52.460000000 cm_analyze: INFO Frames: 0 - 84588
03/05/2024/16:51:52.460000000 cm_analyze: INFO Frame rate: 24000/1001
03/05/2024/16:51:52.460000000 cm_analyze: INFO Algorithm Version: 4
03/05/2024/16:51:52.460000000 cm_analyze: INFO XML Version: 4.0.2

I'm going to have to change what drives I do this on, NVME's preferably, doing in on old "spinner's" takes forever :(

[Kuler087]

Yes I've noticed the warning when using the latest CM-Analyze, I don't think it matters. The output is the same as before.

[pancserzso]

Hi, thanks for these scripts. I have two questions:

1. Can you tell me what do I need to run to convert a P7 mkv file to a P8 mkv?
2. Do I need to use MakeMKV to crate the source MKV from a BDMV folder, or I can use mkvmerge with the biggest m2ts file in STREAM? Is there any difference, other than the chapters?

[Kuler087]

Hi, thanks for these scripts. I have two questions:

1. Can you tell me what do I need to run to convert a P7 mkv file to a P8 mkv?
2. Do I need to use MakeMKV to crate the source MKV from a BDMV folder, or I can use mkvmerge with the biggest m2ts file in STREAM? Is there any difference, other than the chapters?

Hi,

For any P7 source(mkv or bdmv) to Profile 8, use workflow 4-1.
For BDMV input, you should input the main playlist file and not the largest M2ts.

see tutorial: https://www.youtube.com/watch?v=sBUL2gjhqEE

[pancserzso]

Hi,

For any P7 source(mkv or bdmv) to Profile 8, use workflow 4-1.
For BDMV input, you should input the main playlist file and not the largest M2ts.

see tutorial: https://www.youtube.com/watch?v=sBUL2gjhqEE

Thanks a lot, video is perfect! About CM4, if my TV supports it (LG G2), then I should use rem_cmv4=NO right?

[Kuler087]

You should not remove cmv4.0 unless maybe if you use the AppleTV 4K with Infuse (non-beta).
Cmv4.0 has superior tone mapping to the old cmv2.9 algo

[giyo279]

Any thoughts on NVENC vs x265 for dovi_baking? I know encoding settings can get pretty advanced, but are the default HDR_settings set for mostly transparent encodes?

[Kuler087]

I'm no HEVC encode setting expert but as far as I can tell, if you use x265 with the script default settings, it should produce something transparent but depending on how much FEL adds, you might end up with a file larger than your source.
For NVENC, I don't know but from the quick tests I did, it seems to produce decent quality.

[CineManiac]

Hey, have you had a chance to check if the Infuse beta implementation of cmv4.0 has the same issues as the Plex version on ATV?

[Kuler087]

Not yet, I have to get in the beta test group first but I'll definitely check it out.

[CineManiac]

Oh man, I'll have to re-inject everything with cmv4

[Kuler087]

Oh man, I'll have to re-inject everything with cmv4

haha, yes but it's worth it.

[Gatorman3385]

It seems my madVR install is broken. I've tried running "install.bat" as admin. 6-2 gives me the error,

Connecting LAV Video Decoder to madVR failed.

MPC-HC has stop using madVR as well. Any thoghts?

[Kuler087]

Yes, it happened to me on 3 different PCs a couple of days ago, and I don't understand how this is possible. It must be a Windows update that broke something, idk...
Re-download the tool pack and overwrite the madvr folder, which should fix it.

[Gatorman3385]

Yes, it happened to me on 3 different PCs a couple of days ago, and I don't understand how this is possible. It must be a Windows update that broke something, idk...
Re-download the tool pack and overwrite the madvr folder, which should fix it.

That worked. Thanks.

[pancserzso]

I'm using a M1 mac and the only way to run the scripts is through a Windows ARM install in UTM, +1 more layer inside Windows to do the CPU emulation. The files can also only be shared by a loopback network share, which is super super slow, about 12 MB/s only!

The native SSD under macOS on the other hand is super fast, a 100 GB demux only takes a few seconds.

So I've spend a few hours trying to understand what's the heart of P7 -> P8 conversion.

Do I understand it correctly that it's basically these 3 commands?


ffmpeg -i $MKVFILE -c:v copy -bsf:v hevc_mp4toannexb -f hevc - | dovi_tool demux -

dovi_tool -m 2 mux --bl BL.hevc --el EL.hevc --discard -o P8.hevc

mkvmerge --output $MKVFILE-P8.mkv --compression 0:none --default-duration 0:24000/1001p P8.hevc --no-video --compression 1:none $MKVFILE


And do I understand it correctly that the last command can be simplified into:

mkvmerge --output $MKVFILE-P8mod.mkv P8.hevc --no-video $MKVFILE


Is this correct?

[Kuler087]

Seems right but you should have a look at @nekno script for MAC

https://github.com/nekno/DV7toDV8

[guest]

Yes I've noticed the warning when using the latest CM-Analyze, I don't think it matters. The output is the same as before.

Can confirm that this IS the case :)

v5.4.1 does not display those WARNINGS.

v5.5.0 does :(

[guest]

Has something changed with the audio conversions ??

I was sure that I had set it that when I encode audio, regardless of it's original bitrate, to would come out either as 1024 or 1536, but it's unchanged, keeping it's original bitrate.

I'll have to dig deeper.

EDIT:- I just figured it out, I wasn't adding the IGNOREBITRATE in the file name https://www.cosgan.de/images/smilie/konfus/a080.gif

Forgive me if I've asked this before (I have a very short term memory, sometimes).

I thought that 3-1 would encode any audio to EC3 at the bitrate set in the .bat.

[speedy]

You should not remove cmv4.0 unless maybe if you use the AppleTV 4K with Infuse (non-beta).
Cmv4.0 has superior tone mapping to the old cmv2.9 algo

Hey, have you had a chance to check if the Infuse beta implementation of cmv4.0 has the same issues as the Plex version on ATV?

Not yet, I have to get in the beta test group first but I'll definitely check it out.

Oh man, I'll have to re-inject everything with cmv4

haha, yes but it's worth it.

Infuse 7.7.2 beta support for CMv4 has been great in my early testing. The only downside is that I'm now likely going to want to go back through a number of my favorite releases and convert re-inject CMv4 metadata :scared:

[Kuler087]

Hey, have you had a chance to check if the Infuse beta implementation of cmv4.0 has the same issues as the Plex version on ATV?

Ha this sucks, the beta group is not available to users with monthly subscriptions :(

I'll have to wait for a public release because there's no way I buy a year subs of something I might never use.

https://i.ibb.co/Ws6BKK5/Capture-d-cran-2024-03-06-114710.png

@@ speedy did you test if the color bars cmv4.0 pattern looks as bad as on PLEX ?
https://slow.pics/c/VGBhNeQU

https://drive.google.com/file/d/1tv6bF-Pj8SCJjQGXz__11dKfo-vDTMgF/view?usp=drive_link

[pancserzso]

Seems right but you should have a look at @nekno script for MAC

https://github.com/nekno/DV7toDV8

Thanks for checking it, I started with nekno script, however the differences confused me.


it does the following:

mkvextract "$mkvFile" tracks 0:"$BL_EL_RPU_HEVC"
dovi_tool demux --el-only "$BL_EL_RPU_HEVC" -e "$DV7_EL_RPU_HEVC"
dovi_tool --edit-config "$jsonFile convert --discard "$BL_EL_RPU_HEVC" -o "$DV8_BL_RPU_HEVC"
dovi_tool extract-rpu "$DV8_BL_RPU_HEVC" -o "$DV8_RPU_BIN"
mkvmerge -o "$mkvBase.DV8.mkv" -D "$mkvFile" "$DV8_BL_RPU_HEVC" --track-order 1:0


It does demux + convert + extract-rpu, which I don't really understand.

[Kuler087]

yeah thats a waste of time. your ffmpeg pipe to dovitool approach is better

[pancserzso]

Thanks! I think the DoVi_Scripts demux + mux approach can be simplified into one command, then it's half the processing time, isn't it? Or you made it like this for special use cases which need the results from demux?

[Kuler087]

Thanks! I think the DoVi_Scripts demux + mux approach can be simplified into one command, then it's half the processing time, isn't it? Or you made it like this for special use cases which need the results from demux?

my script (MKV version) already use the ffmpeg pipe to dovi_tool approach unless you decide to set the following to NO:

set mux_all_audio=NO
set mux_all_sub=NO

I don't know why nekno script doesn't do that as well.


https://i.ibb.co/6NKVCdn/Windows-Terminal-HS9rxegi3p.gif

[pancserzso]

No, I mean your script has this in 2 lines:


"%ffmpeg_path%" -i "%filepath%%filename%%fileext%" -c:v copy -bsf:v hevc_mp4toannexb -f hevc - | "%dovi_tool_path%" demux - & set HDR="%TEMP%BL.hevc"& set EL="%TEMP%EL.hevc"& goto :nodemux



"%dovi_tool_path%" %dropH% %remove.4.0% -m 2 mux --bl %HDR% --el %EL% --discard -o "%output_path%%filename%_P8.hevc"


I believe these two could be a simple command, doesn't it?

[Kuler087]

i dont think you can demux and mux at the same time?? Tell me how please if its possible

[pancserzso]

i dont think you can demux and mux at the same time?? Tell me how please if its possible

I just asked quietvoid:
https://github.com/quietvoid/dovi_tool/discussions/290#discussioncomment-8697502

[Kuler087]

thanks, I'll look into it.

[speedy]

Ha this sucks, the beta group is not available to users with monthly subscriptions :(

I'll have to wait for a public release because there's no way I buy a year subs of something I might never use.

https://i.ibb.co/Ws6BKK5/Capture-d-cran-2024-03-06-114710.png

@@ speedy did you test if the color bars cmv4.0 pattern looks as bad as on PLEX ?
https://slow.pics/c/VGBhNeQU

https://drive.google.com/file/d/1tv6bF-Pj8SCJjQGXz__11dKfo-vDTMgF/view?usp=drive_link
Yes, I tested color bars with the CMv4 pattern on Infuse 7.7.2b1, and they look great: https://youtu.be/D4WJow-iOA0 (the steps are more visible IRL on the gradient than in the video)

I'd recommend throwing out those results that tested Dolby Vision with Plex on Apple TV. Plex for Apple TV doesn't have any support for Dolby Vision and instead just triggers the Apple TV to switch to Dolby Vision but then only plays back the HDR10 base layer, which is problematic for obvious reasons. This is well documented (https://www.reddit.com/r/PleX/comments/13cedvi/is_fake_dolby_vision_on_apple_tv_fixed_somethings/), and the Plex team continues to ignore it. It would be much better if they just triggered HDR10 instead so the Base Layer could be played back as intended.

[Kuler087]

I'd recommend throwing out those results that tested Dolby Vision with Plex on Apple TV. Plex for Apple TV doesn't have any support for Dolby Vision and instead just triggers the Apple TV to switch to Dolby Vision but then only plays back the HDR10 base layer,.

That is not true. All my results with plex are accurate otherwise all of my target brightness results would be the same

And I tested the RPU reaction before doing any test, see for yourself: https://drive.google.com/file/d/1Idl3gocvc2jm6aw53GdbQvGt2Oet5P_P/view?usp=drive_link

[pancserzso]

thanks, I'll look into it.

This is the solution
ffmpeg -i $MKVFILE -c:v copy -bsf:v hevc_mp4toannexb -f hevc - | dovi_tool -m 2 convert --discard - -o P8.hevc

[Kuler087]

Yes, I tested color bars with the CMv4 pattern on Infuse 7.7.2b1, and they look great: https://youtu.be/D4WJow-iOA0


I should have mentioned that my pics are from LLDV and the issue is a lot worse. In tv-led it looks better but still not right and I can see the same in your video, unfortunately.

[speedy]

That is not true. All my results with plex are accurate otherwise all of my target brightness results would be the same

And I tested the RPU reaction before doing any test, see for yourself: https://drive.google.com/file/d/1Idl3gocvc2jm6aw53GdbQvGt2Oet5P_P/view?usp=drive_link
Were you using the "Old Video Player" (aka AVPlayer) and a Profile 5 MP4 to get those results?

Even Plex Staff states they only support the Profile 5 in MP4 with AVPlayer currently: https://forums.plex.tv/t/plex-fireside-in-the-forums/851849/37

[speedy]

I should have mentioned that my pics are from LLDV and the issue is a lot worse. In tv-led it looks better but still not right and I can see the same in your video, unfortunately.
Are you certain?

It looks significantly different (better as in gradient steps are more clear) to my eyes than it does to the camera? I have no idea what I'm doing when it comes to taking pictures of a screen.

[Kuler087]

Were you using the "Old Video Player" (aka AVPlayer) and a Profile 5 MP4 to get those results?

Even Plex Staff states they only support the Profile 5 in MP4 with AVPlayer currently: https://forums.plex.tv/t/plex-fireside-in-the-forums/851849/37

I don't remember if it was the old or new player but my file tests were ALL in profile 8. I linked it, just try you will see that the RPU is indeed used ( L1+L8+L3)

[Kuler087]

Are you certain?

It looks significantly different (better as in gradient steps are more clear) to my eyes than it does to the camera? I have no idea what I'm doing when it comes to taking pictures of a screen.
Yes and it looks just like in your video.
I'm not home right now, but I can have a look again tonight.

[CineManiac]

Quick question, if a video's Mastering Display primaries are in BT.2020, rather than the usual P3, do we export the DV XML in Resolve with Rec.2020 Color Encoding Primaries?

[speedy]

Yes and it looks just like in your video.
I'm not home right now, but I can have a look again tonight.
I think I see what you're seeing more now... 1000 & 4000 nits color bar patterns look perfect, but the gradient doesn't look right after 4000 nits. I think Apple has basically ignored properly rolling off content beyond 4000 nits (I assume this is what you're getting at).

This is definitely annoying in pursuit of a perfect device, but I think this has minimal impact on real world content compared to being able to support TV-led, CMv4, Chroma Weight, accuracy under 4000-nits, etc.

[Kuler087]

Quick question, if a video's Mastering Display primaries are in BT.2020, rather than the usual P3, do we export the DV XML in Resolve with Rec.2020 Color Encoding Primaries?

Yes, always match your base layer primaries.

[Kuler087]

I think I see what you're seeing more now... 1000 & 4000 nits color bar patterns look perfect, but the gradient doesn't look right after 4000 nits. I think Apple has basically ignored properly rolling off content beyond 4000 nits (I assume this is what you're getting at).

This is definitely annoying in pursuit of a perfect device, but I think this has minimal impact on real world content compared to being able to support TV-led, CMv4, Chroma Weight, accuracy under 4000-nits, etc.


yes that makes sense and I agree

[CineManiac]

Thank you, that's what I assumed but there's so much about primaries in MediaInfo hah

[guest]

I think my simple question got lost in the wave of posts, that happened last night.

Forgive me if I've asked this before (I have a very short term memory, sometimes).

I thought that 3-1 would encode any audio to EC3 at the bitrate set in the .bat.

[Kuler087]

yeah we went offtopic a bit... sorry

only the MP4 version will encode any lossless audio to EC3.
For the other version, you have to opt in and the best lossless audio will be selected to encode EC3.

If you meant for Lossy audio to EC3, only 8-1-1 can do that and I don't think I will not add lossy audio encoding to the other workflows.

[guest]

yeah we went offtopic a bit... sorry

only the MP4 version will encode any lossless audio to EC3.
For the other version, you have to opt in and the best lossless audio will be selected to encode EC3.

If you meant for Lossy audio to EC3, only 8-1-1 can do that and I don't think I will not add lossy audio encoding to the other workflows.

Thats fine, it's good to see more ppl posting here :)

I will ALWAYS be using the MP4 workflow.

Yeah, I just thought that 3-1 did, in earlier builds, but like I said, I have short term memory loss, sometimes.

Seems like a bit more "manual" manipulation is required, but lossy audio in 3-1 would be very handy for me, as that will be the workflow I use more than any others.

Cheers.

[Kuler087]

lossy audio in 3-1 would be very handy for me,

maybe, I'll think about it. I don't personally use the MP4 container and because it always encode any lossless audio, it has its own audio converter script unlike other versions which use all the same code from 8-1-1.

[guest]

maybe, I'll think about it. I don't personally use the MP4 container and because it always encode any lossless audio, it has its own audio converter script unlike other versions which use all the same code from 8-1-1.

Not just for me, but all the LG TV owners that would be able to use it.

I went thru the procedure yesterday to get better audio in a 3-1 conversion...its a lot of steps outside your workflows :(

Maybe I should be using the MKV workflows, then use the MP4 muxer as the final step.....

[benwaggoner]

yes that makes sense and I agree
Vanishing little real-world content actually has pixels much brighter than 1000 nits in any case. Content with MaxCLL of 4000 generally indicates it was graded on a Dolby Pulsar, which was capable of going that high. Rarely will a statistical analysis find any pixels close to that. Pretty much everything graded in the last five years was done on a X300 or X310, which only go up to 1000 nits.

[speedy]

Vanishing little real-world content actually has pixels much brighter than 1000 nits in any case. Content with MaxCLL of 4000 generally indicates it was graded on a Dolby Pulsar, which was capable of going that high. Rarely will a statistical analysis find any pixels close to that. Pretty much everything graded in the last five years was done on a X300 or X310, which only go up to 1000 nits.
@benwaggoner, Do you think Amazon is working or open to fixing the issues we've identified with the Fire TV?

[Kuler087]

Not just for me, but all the LG TV owners that would be able to use it.

I went thru the procedure yesterday to get better audio in a 3-1 conversion...its a lot of steps outside your workflows :(

Maybe I should be using the MKV workflows, then use the MP4 muxer as the final step.....

Well the mp4 and ts version already makes LG compatible files. TVs can play lossy AC3/AAC audio so there's no need to convert them to EC3. This is why I'm not sure I want to add lossy to lossy encoding in the workflows. 8-1-1 suffice IMO and converting them to EC3 doesn't make them better.

P-S, you can install Kodi on WEBos( (https://kodi.wiki/view/HOW-TO:Install_Kodi_for_webOS)your tv might be too old though), it supports DV MKV. I've been using this for a couple of months now.

Vanishing little real-world content actually has pixels much brighter than 1000 nits in any case. Content with MaxCLL of 4000 generally indicates it was graded on a Dolby Pulsar, which was capable of going that high. Rarely will a statistical analysis find any pixels close to that. Pretty much everything graded in the last five years was done on a X300 or X310, which only go up to 1000 nits.

Thank's for the info Ben :)
I always appreciate industry insider Info/opinion.