[DoVi_Scripts] Multi-Function Scripts for Dolby Vision processing and a lot more...

[en6ads]

Shield has color issues with P8 P7 DV. you should encode to P5 with 8-2-6 (support FEL baking too)

I don't own a shield, I was being facetious ;). I do own the latest gen apple tv, a zidoo Z9X PRO (not my favorite), and Ugoos AMB6+.

Based on your testing, youtube videos, I see the ATV has some color issues with DV. Shame, I would have used that with baked FEL as it's the one I use for streaming content. Watching color accurate baked FEL on apple TV would be awesome - and would allow me to manage one less box to maintain.

[Kuler087]

Yeah but the ATV is much better than the Shield and also 10 times more stable. The Shield drop or dup at least 1 or 2 frames per hour(tested many times with a capture card).
But there is definitely something going on with the ATV DV colors too, if you look at the Spears and Munsil colorspace pattern, only the X800m2/X700/Oppo and the Ugoos are perfect. I don't know about the zidoo but it's not an option since this device can't do True TV-LED.
colorspace pattern comparison: https://slow.pics/c/NBQVuXQw (my C2 internal player also looks wrong)

Ugoos + CE is simply the best device if you want it all and the latest update seems to have fixed the OOS audio when you seek or FF.

[en6ads]

Yeah but the ATV is much better than the Shield and also 10 times more stable. The Shield drop or dup at least 1 or 2 frames per hour(tested many times with a capture card).
But there is definitely something going on with the ATV DV colors too, if you look at the Spears and Munsil colorspace pattern, only the X800m2/X700/Oppo and the Ugoos are perfect. I don't know about the zidoo but it's not an option since this device can't do True TV-LED.
colorspace pattern comparison: https://slow.pics/c/NBQVuXQw (my C2 internal player also looks wrong)

Ugoos + CE is simply the best device if you want it all and the latest update seems to have fixed the OOS audio when you seek or FF.

Maybe a Amlogic S928X-J device such as Ugoos AM8 PRO might be worth testing as truely a 1 box solution [for streaming and local color correct baked FEL DV playback; if 3D is not needed].

Would love a simple docker, or PC script that auto rips disk with chosen language and options, auto bakes FEL into an 8.1 file. That is probably the end goal.

[Kuler087]

newer DV SOCs do not allow profile 7 decoding.
only these devices can work with FEL: https://discourse.coreelec.org/t/ce-ng-dolby-vision-fel-for-dv-licensed-socs-s905x2-s922x-z-s905x4/50953

[en6ads]

newer DV SOCs do not allow profile 7 decoding.
only these devices can work with FEL: https://discourse.coreelec.org/t/ce-ng-dolby-vision-fel-for-dv-licensed-socs-s905x2-s922x-z-s905x4/50953

Yes, that's why I use an Ugoos AMB6+ for local file playback, and an Apple TV for streaming playback.

However, If I bake FEL using workflow 8-2-1 or 8-2-6, then FEL playback requirement using the Ugoos is no longer necessary.

So is there a one TX box that: 1) can play all the streaming services at highest 4k quality, is Netflix/Disney/Prime etc 4k certified; 2) can play baked-FEL local files with correct color (which ATV and Shield can't do)?

Baking FEL takes many hours, but run on a home server overnight is not a problem.

If I can get my setup down to just 1 box would be ideal.

[Kuler087]

Yes, the firecube 2nd gen. IIRC you can install in dual-boot the coreelec build.

There's a user on avsforum that uses it, ask him: https://www.avsforum.com/posts/63702599/

[benwaggoner]

newer DV SOCs do not allow profile 7 decoding.
only these devices can work with FEL: https://discourse.coreelec.org/t/ce-ng-dolby-vision-fel-for-dv-licensed-socs-s905x2-s922x-z-s905x4/50953
Are there any scenarios that use Profile 7? I don't think I have ever seen one in the wild.

[Kuler087]

Yeah, on paper, profile 7 should be only for bluray players and bluray discs so it makes sense for Dolby to offer only Profile 5 & 8 compatible SOC for the streaming boxes.
The older 2018-2019 SOCs support P7 (LG C8/C9 internal player + all the devices listed in the discourse/coreelec link) but I guess they didn't expect that people would be able to reverse-engineer the format and allow lossless MKV single-track dual-layer Profile 7 rip compatible with these media players.

[Korbus]

Hello everyone! First, let me just say thanks to @R3S3t9999 for this great tool!
I also have a quick question, since even after searching the thread I could not find the answer. For my own quirky reasons and lack of HDD space, I prefer to rip my UHD's to 1080p files, with a fixed bitrate, using the 2-pass process, in order to better alocate the bitrate depending on the needs. I do not like the CRF encoding, since neither the avg. bitrate, nor file size are predictable. Hence, my question:
How do I modify the Encoders section of the .bat file to do the 2-pass encoding (full 1st-pass) with a fixed value for avg. bitrate (meaning what command line argument should I use - I am a perfect newbie in command line, so far I have used RipBot to do my encodes)? I prefer X265 encodes, not nVidia ones (which are, by default, 2-pass, I guess).
Thanks in advance for any help!

[Kuler087]

I would have to add 2-pass support, right now it only supports CRF or 1-pass bitrate. I can have a look later

[Korbus]

Thank you for the answer, that clears it. I thought I was doing something wrong.

[Black Ops]

Ordered the firecube 2nd gen and looking forward towards running corelec on it.

Ive just realised that windows seems to not report the proper max luminance of the panel via the edid and thus defaults to a peak brightness of 1499. Whats the best way to determine peak luminance for appropriately setting LLDV edid.

[Kuler087]

which TV/monitor? you could look at the measurements on rtings

https://www.rtings.com/

[Black Ops]

Yes its the qn90c 50 inch. The measurements were taken with a 55 inch model. Samsung states that 55 inch and above utilise "Neo quantum HDR+" panel which is "double the peak brightness". They reported 2000 ish nits

I have set the qn90c to HGIG mode previously and ran thru the XBOX HDR calibration and it seems the image clips at 1100 nits. Weather this brightness translates whilst the TV is not in game mode is a mystery to me.

[Kuler087]

You can try this app. On my Samsung monitor it reports 400nits which is the same as the edid extracted with hdfury vertex
https://apps.microsoft.com/detail/9pgn3nwpbwl9?hl=en-US&gl=US

otherwise, the cheaper way would be to export the edid with a Raspberry Pi and Pi Generator.


You can also try these patterns via USB and see when it clips but it might not be a proper way since the TV does static tone mapping so I don't know.
https://drive.google.com/drive/u/1/folders/1EJm_GAiNEXsaleXGmQenxzRiSeEAg72W
https://drive.google.com/drive/u/1/folders/1jE0WBNaF9JlgxeQsa7-_axh0zRiGA8pC

[Black Ops]

The app also has 1499 but when running the HDR calibration Store app, it updates accordingly so i dont think its pulling direct from EDID. The patterns cut off at 1500.

Looks like custom resolution utility can export the EDID. No mention of peak luminance tho for HDR

https://pastebin.com/49yx9Rvr

I suspect 1000 nits is indeed the peak. HGIG effectively disables any tonemapping and tells the TV to effectively hard clip at peak brightness of the display. Looking at Rtings, the values for game mode didnt really change

[Kuler087]

With the cpm's coreelec build you can select different edid anyway so you'll be able to tell if the patterns are clipped or not with a 1500nits edid.

[Black Ops]

Is this the luminance? I had a skim through the edid again and spotted this. So 800 nits?

Maximum luminance: 128 (800.000 cd/m^2)
Minimum luminance: 1 (0.000 cd/m^2)
Maximum luminance (without local dimming): 107 (507.620 cd/m^2)
Minimum luminance (without local dimming): 15 (0.018 cd/m^2)
Checksum: 0xff

[Kuler087]

yep, looks like it.

[Black Ops]

Looks like i have rtings to thanks since they stated that the 50 inch panel only differs in terms of higher refresh rate and it being a VA panel failing to mention that the peak brightness is effectively half of the 55 inch+

[Korbus]

I would have to add 2-pass support, right now it only supports CRF or 1-pass bitrate. I can have a look later

Thank you so much for adding two-pass bitrate encoding! Now time to rip those pesky P7 FEL UHD's :)!

[lemaireus]

If a blu-ray has six lossless audio tracks, and I wish to use DoVi_Scripts (workflow 8-1-1) to encode, for example, the fourth audio track. How do I get DoVi_Scripts to encode track 4 while excluding the others?

[Kuler087]

If a blu-ray has six lossless audio tracks, and I wish to use DoVi_Scripts (workflow 8-1-1) to encode, for example, the fourth audio track. How do I get DoVi_Scripts to encode track 4 while excluding the others?

There's an option to select only one language.

line 60
::choose if you want to mux all the audio from the input or just the main audio track. This is valid only if "set MUX=YES" (YES or NO default = YES )
set mux_all_audio=YES

line 221
:: set desired audio language (YES or NO default = english) If all set to no, it will get you the first best audio regardless of the language. (otherwise the audio selected is preferred by default).


otherwise, you'll have to mux your file with only the audio/subs you want first and then run the script.

[Kuler087]

Thank you so much for adding two-pass bitrate encoding! Now time to rip those pesky P7 FEL UHD's :)!

you're welcome.

[lemaireus]

There's an option to select only one language.

line 60
::choose if you want to mux all the audio from the input or just the main audio track. This is valid only if "set MUX=YES" (YES or NO default = YES )
set mux_all_audio=YES

line 221
:: set desired audio language (YES or NO default = english) If all set to no, it will get you the first best audio regardless of the language. (otherwise the audio selected is preferred by default).


otherwise, you'll have to mux your file with only the audio/subs you want first and then run the script.

Thank you very much, Kuler, for the helpful guidance. Unfortunately, in my case the movie is of the silent era and its Criterion blu-ray now comes with four different orchestral scores (and two commentary tracks that do not need re-encoding). I ended up extracting each of the four audio tracks as a separate file and then encoded each of those separately with 8-1-1 workflow in DoVi_Scripts.

Your tip will come in handy with blu-rays with audio tracks of multiple languages. Thank you. For other scenarios, such as the one I outlined above, perhaps an option to select one particular audio track on the basis of track ID or audio track number--track 2, or 3, or 4, or audio 1, 2, or 3 . . .--could be added as a feature in one of the future versions of DoVi_Scripts.

[lemaireus]

Each time I do a 7.1 encode using EAE, the resulting track has .mkv as its file extension. Is there any way I can get DoVi scripts to produce a file with the correct .eac3 file extension?

The ec3 file is muxed to mkv to preserve any delay in the source. You can't just change it manually to ec3, this would make a file with an invalid extension so you have to demux it(mkvextract) if you want it to have ec3 extension.

This is with reference to a conversation thread--cited above--on p. 37:

As a matter of fact, 'eac3' would be the correct file extension of the encoded EAC3 audio. The extension '.mkv' is incorrect because it makes the operating system believe that the file is a video file, when it is in fact an audio file.

Is there any way in which I can tinker with the settings of 'DoVi_Scripts_MKV.bat' in order to get it to output encoded EAC3 audio with the .eac3 file extension rather than .mkv?

[Korbus]

Is there a workflow that would allow me to use an already made 1080p HDR10 rip, to which to add the DV metadata extracted from a P7 FEL UHD/remux and get an P8 mkv with the FEL metadata baked in, similar to the result i would get with the 8-2-1 workflow? I'm asking this because getting a BL HDR10 rip is much faster with RipBot, due to Distributed Encoding (twice as fast, actually). If I could then use DoviScripts to bake in that rip the P7 metadata, it would speed up the whole process, I guess.

[Kuler087]

I ended up extracting each of the four audio tracks as a separate file and then encoded each of those separately with 8-1-1 workflow in DoVi_Scripts.

if you want to encode each track, you can set all the language options to ''NO''. I think this should enable the encoding of every track present.

As a matter of fact, 'eac3' would be the correct file extension of the encoded EAC3 audio. The extension '.mkv' is incorrect because it makes the operating system believe that the file is a video file, when it is in fact an audio file.
yes, I remember but I never did it because it's just simpler to deal with the mkv container when the input has a delay. And it really takes 2-sec to demux with mkvextract (https://www.videohelp.com/software/Inviska-MKV-Extract) anyway.

Is there any way in which I can tinker with the settings of 'DoVi_Scripts_MKV.bat' in order to get it to output encoded EAC3 audio with the .eac3 file extension rather than .mkv?

I think the TS version doesn't have this mkv container muxing for ec3 encoding for a reason I forgot. So I guess you could use it for audio encoding.

[Kuler087]

Is there a workflow that would allow me to use an already made 1080p HDR10 rip, to which to add the DV metadata extracted from a P7 FEL UHD/remux and get an P8 mkv with the FEL metadata baked in, similar to the result i would get with the 8-2-1 workflow? I'm asking this because getting a BL HDR10 rip is much faster with RipBot, due to Distributed Encoding (twice as fast, actually). If I could then use DoviScripts to bake in that rip the P7 metadata, it would speed up the whole process, I guess.

No, to merge the FEL into the BL, it requires re-encoding.

But ripbot can probably accept avisynth script for input? I know staxrip does so you can use workflow 8-2-1 to create the script, cancel the encode right away and use the avisynth script from the TEMP folder.

[Korbus]

Well, apparently it does (it loads it), but when executing it it does not use distributed encoding (which was the only reason to use RB), uses my predefined encoding preset, not the whatever it is defined in the script and the result is a bare HDR10 file. So I'll continue to use Dovi Scripts, exercise patience and accept it'll be some months until I'll convert all the relevant movies in my library.

[Kuler087]

not the whatever it is defined in the script and the result is a bare HDR10 file
Yes, that's normal, the dovibaker output is fel+BL merged into an RGB HDR16 stream. You have to specify the P8 rpu path and the profile in ripbot(if there's an option)

it does not use distributed encoding
Are you talking about dual PC encoding?
If so, just run the script on both PC with two movies, and in the end, it will be the same speed as ripbot dual PC encoding.

[Korbus]

Are you talking about dual PC encoding?


No, distributed encoding, as implemented in RB, can also be used with a single PC, if you have two network interfaces (such as ethernet and wifi, which most PC's do nowadays). With the same encoding settings and video file, the speed basically doubles, but the PC resources are heavily taxed (meaning you can't do much while there's an encode running, but... it's half the time). Also, you can pause the conversion at any time and resume it whenever you like, which is very convenient.
But... no DoVi metadata of any kind are preserved when encoding, nevermind P7 FEL baking...

[Kuler087]

yes, but it's the same speed as using two pc to encode two movies at the same time. (unless one of the two PC is very old/slow)
The PC cannot go faster than 100% CPU usage, distributed or not.

so two movies being encoded by two PC at the same time is the same speed as encoding 2 movies in a row in Ripbot with distributed encoding.

[Korbus]

I don't have two PC's, so for me that gimmick was very convenient. I would rip a DV movie to HDR10 and then use DDVT Tool or your script to inject the RPU and get a P8 mkv, but now that there's a way to preserve the FEL information, of course I prefer that. So I'll just take the slower path :)

[Gatorman3385]

Encoding FEL to SDR I'm getting very little CPU usage, ≈15%. Is this expected?

[Kuler087]

I don't know. On my end, I get 90-100% cpu usage when I encode anything:

https://i.ibb.co/pvRx6hK/Windows-Terminal-haq-Omx-Hz8-S.png

[Kuler087]

oh never mind... I misread that you were talking about x264...
Indeed it's not very fast, I get about 40% usage only... not sure why

https://i.ibb.co/5rR3R0q/SLx-Z6-Q6pdh.png

[lemaireus]

I think the TS version doesn't have this mkv container muxing for ec3 encoding for a reason I forgot. So I guess you could use it for audio encoding.

I just tried it and even the TS version outputs an .eac3 file as .mkv. Anyway, this is not really a big deal; all I need to do is to change the file extension and that does the job.

Thank you for the link to mkvextract. It works well in extracting each track. Mkvtoolnix also pretty much does the same job in about the same time.

Thank you once again for making available what, to the best of my knowledge, is the only 7.1 audio encoder that there is at the moment.

[Gatorman3385]

oh never mind... I misread that you were talking about x264...
Indeed it's not very fast, I get about 40% usage only... not sure why

Adding "Prefetch(8)" after the DoViBaker(bl, el, rpu="I:\Working\Dolby-Vision-Tools\temp.folder31\RPU_FEL.bin") call, significantly increased my CPU usage (60%). Adding "Prefetch(2)" at the end of the script increased CPU usage again (90%-100%). Encoding FPS went from 2fps to 12fps. Is this a reasonable fix or could this affect the encode negatively?

DoViBaker(bl, el, rpu="I:\Working\Dolby-Vision-Tools\temp.folder31\RPU_FEL.bin")
Prefetch(8)
z_ConvertFormat(pixel_type="YUV420P10",colorspace_op="rgb:st2084:2020:full=>2020ncl:st2084:2020:limited",dither_type="error_diffusion",resample_filter="spline36",resample_filter_uv="spline36",chromaloc_op="left=>top_left")
ConvertBits(16)
libplacebo_Tonemap(src_csp=1, dst_csp=0, dst_max=125, dynamic_peak_detection=false, smoothing_period=20.0, percentile=99.995, tone_mapping_function="bt2390", gamut_mapping_mode="perceptual")
ConvertToYUV420(matrix="709")
ConvertBits(10)
Crop(0, 264, -0, -264)
Spline36Resize(Width / 2, Height / 2)
Prefetch(2)

[Kuler087]

I dont know, it should be fine. thanks

added in latest beta: https://drive.google.com/file/d/128gq8aDUTKA_aT7SQsM9dkjA1EP1sosR/view?usp=drive_link

[Kuler087]

I just tried it and even the TS version outputs an .eac3 file as .mkv. Anyway, this is not really a big deal; all I need to do is to change the file extension and that does the job.

Thank you for the link to mkvextract. It works well in extracting each track. Mkvtoolnix also pretty much does the same job in about the same time.

Thank you once again for making available what, to the best of my knowledge, is the only 7.1 audio encoder that there is at the moment.

For 7.1, the TS version should not use the mkv container but If you want to use the mkv version, you can edit line 17191. Replace ''TEMP'' by ''output_path'' and delete lines 17195 and 17196

"%PLEX%" -eae_root "%output_path%\EAE" -c:a pcm_s24le -i "%TEMP%1.wav" -eae_root "%output_path%\EAE" -b:a %DDP_bitrate%k "%output_path%%temp.name%_EAE%DELAYB%.eac3"


For 5.1, change the output extension to .ec3 at line 17185
"%ffmpeg_path%" -i "%temp.path%%temp.name%%temp.ext%" -map 0:%FID% -c:a eac3 -b:a %DDP_bitrate%k %ffmpeg_channels% -ar 48000 -c:s copy -map_chapters -1 -metadata:s:a:0 title="DDP @ %DDP_bitrate%kbps" -y "%output_path%%temp.name%_EC3.ec3"

[lemaireus]

For 7.1, the TS version should not use the mkv container but If you want to use the mkv version, you can edit line 17191. Replace ''TEMP'' by ''output_path'' and delete lines 17195 and 17196

"%PLEX%" -eae_root "%output_path%\EAE" -c:a pcm_s24le -i "%TEMP%1.wav" -eae_root "%output_path%\EAE" -b:a %DDP_bitrate%k "%output_path%%temp.name%_EAE%DELAYB%.eac3"


For 5.1, change the output extension to .ec3 at line 17185
"%ffmpeg_path%" -i "%temp.path%%temp.name%%temp.ext%" -map 0:%FID% -c:a eac3 -b:a %DDP_bitrate%k %ffmpeg_channels% -ar 48000 -c:s copy -map_chapters -1 -metadata:s:a:0 title="DDP @ %DDP_bitrate%kbps" -y "%output_path%%temp.name%_EC3.ec3"

Thank should fix it, I'm sure. Will give that a shot and let you know whether it worked or not. Thanks a lot!

[lemaireus]

For 7.1, the TS version should not use the mkv container

Yes, this worked! On the previous occasion I had tried encoding a 2.0 DTS-HD MA track to EAC3 with the TS version and the output was in .mkv format. The moment I encoded a 7.1 track with TS.bat, the encoded file came out as .eac3, exactly as it should have. Thank you!

If you want to use the mkv version, you can edit line 17191. Replace ''TEMP'' by ''output_path'' and delete lines 17195 and 17196

"%PLEX%" -eae_root "%output_path%\EAE" -c:a pcm_s24le -i "%TEMP%1.wav" -eae_root "%output_path%\EAE" -b:a %DDP_bitrate%k "%output_path%%temp.name%_EAE%DELAYB%.eac3"

I was almost certain that this would work--since the file extension at the end of line 17191 is .eac3 (and I had deleted lines 17195 and 17196)--but it didn't. I wonder why.

In any case, the TS version is happily encoding 7.1 tracks with .eac3 extension so the job is done. Thank you!

[Gatorman3385]

Am I correct that 8.2.1 FEL to Baked HDR, encodes the RPU with the video or does x265 inject the RPU at the end? I wonder if this uses up bitrate that could be used by the video encode if the P8 RPU were injected after the process finishes.

I should say that I've modified my x265_HDR settings to limit bitrate (--vbv-bufsize 24000 --vbv-maxrate 12000). This is for remote playback not local viewing.

[guest]

Hi Kuler,

Just a quick question, does the AV1 codec support Dolby Vision ??

Can AV1 be used in an .mp4 container ??

And then can DV, AV1 & .mp4 "live" together ??

If so, do you have any plans for AV1 support ??

Cheers

[Kuler087]

Am I correct that 8.2.1 FEL to Baked HDR, encodes the RPU with the video or does x265 inject the RPU at the end?



It injects the RPU on the fly while encoding the new HEVC stream.

I wonder if this uses up bitrate that could be used by the video encode if the P8 RPU were injected after the process finishes.
I don't understand what you mean. Bitrate is not related to the RPU.

I should say that I've modified my x265_HDR settings to limit bitrate (--vbv-bufsize 24000 --vbv-maxrate 12000). This is for remote playback not local viewing.
You probably should use a low two-pass bitrate for these values.

[Kuler087]

Hi Kuler,

Just a quick question, does the AV1 codec support Dolby Vision ??

Can AV1 be used in an .mp4 container ??

And then can DV, AV1 & .mp4 "live" together ??

If so, do you have any plans for AV1 support ??

Cheers

Sorry, I have no experience with this codec but yes it supports DV.

[thirdmoon]

Sorry, I have no experience with this codec but yes it supports DV.

I have done some testing with AV1. It's not good for high quality encoding especially when film grain is present.

Its better at high CRF values with low/no grain.

[Korbus]

While using workflow 3-1, I have encountered the error from the attached screenshot - basically LAV Splitter Source not found and madvr failing to make the scene cuts. I have used the same workflow successfully two times before, with no problems whatsoever, with two different video files.
Nothing has changed since. I have reinstalled both LAV filters and madvr and rebooted, to no avail. I have tried with three different video files, same error. I don't understand what could have happened...

[Kuler087]

Looks like you broke your madMeasureHDR.exe, download the tools pack again.
It happened to me one time when I was using the beta time-limited madvr builds.